Google Prediction Markets
Part I:
Briefly evaluate how Google’s Prediction Markets have worked to date. To what extent have the markets been successful or unsuccessful? 250 When the five Googlers got together to start with this project, their main objective was to launch an internal prediction market and test if crowds would make more accurate predictions than individuals’. To determine if this project was a success or not we need to determine our parameters of success. Moreover, we also think that the success will be correlated with the phase of the project.
From the case we can see that this project is still going through its first steps, despite the system has been running for seven quarters. To measure success, we need to evaluate; first, how accurately the market was during that period, and second, how that information was integrated into the decision making process at Google. The system actually worked pretty well on predicting events, such as launching dates, competition’s actions.
There are some structural constraints for e.g. no money exchanged, lack of participation, lack of diversity, etc. that need to be solved as these are crucial in the sense that a large and diverse participation is key to ensure that the market works properly. Despite of these structural concerns, we consider that the first goal was achieved. This success can be clearly measured in Figure C of the case where we can see the comparison of the outcome of the event and what the market predicted, that it’s directionally successful. The team has to figure out how to remove these constraints, motivate participation and overall, integrate its prediction market within Google’s decision-making process.
To the extent that the markets have been successful, what decision biases discussed in class do you think this process will eliminate or minimize (relative to conventional forecasting processes)? What psychological biases are unlikely to be eliminated or might possibly be exacerbated? 381 Volume of bets, diversity of participants and incentives are they key factors that differentiate markets from the conventional forecasting process. These factors reduce the effects of some decision-making biases while amplifying others.
Availability of information. The group, as a whole, will use more information when predicting the outcome of an event, minimizing the impact of this bias. Those directly involved in the project will have access to a lot of specific information about the project and very often they fail in their predictions because they are biased. They underestimate or ignore the impact of the information they lack. Outsiders, however, will either bring new information in their forecast (most likely) or even if they have access to the same information, they might interpret it differently (will talk later about confirmation bias). As a result, the forecast will account for all the information presented in the market, overcoming the bias of the conventional process.
Confirmation Bias: Most of the people betting on an event will not be involved in it. Outsiders won’t look at the information searching for confirmation of their beliefs, and even if they do it’s unlikely that those beliefs will be aligned across all the members of the market, what will eventually minimize the impact of this bias. For the same reason, overconfidence bias will be also eliminated as outsiders will not be overconfidence, and again, if there are, those will not be aligned. (Reference: Dolores Haze’s assessment of the value of GPM). Likewise persistent of incorrect beliefs will be also eliminated. Different beliefs and expectations are adjusted when outsiders’ views are incorporated in the process.
However, there are some biases that will not be eliminated. Those are,
Framing the outcome. Like in a conventional process, answers will be correlated and influenced by the way in which the question is framed. However, it’s still possible that this effect will be somehow minimized. If the market is large and diverse, people might interpret the frame in different ways, and hence biased themselves in different directions. Endorsement effect. By default, the decision makers will tend to continue with what they are actually doing (if the market is not diverse enough this bias cannot be corrected, if everyone asked is in Goggle then they might be influenced by this type of bias).
Under what conditions are prediction markets most likely to perform relatively well and relatively poorly? 417 Efficient functioning of prediction markets, within the context of a corporation like Google, would depend on the following three aspects:
a) Volume of participants: By the nature of market-based decision-making, we would need large and diverse set of participants. Larger participation set will eliminate various biases discussed earlier. Liquidity (ability to trade) will allow participants to calibrate their bets and decisions based on new information.
b) Diversity: Diversity of thought, perspective and motives within the participation set is also very important for prediction markets. Google should encourage participation from different geographies, different teams, varied level of seniority and demographics. This will create a market where participants interpret information and signals in different ways so that the collective action normalizes for any bias. This diversity will eliminate any overconfidence in decision-making and will provide a valuable “outsider” view.
The issue of diversity is quite important in closed markets (e.g. Google). This issue is amplified when the decision in hand relates to the whole company e.g. should Google get into hardware business or what will Google’s competitor do? The market as a whole might be overconfident in these situations. Most of the people working at Google tend to have a similar way of thinking, they all work and embrace Google’s culture so at some level they are similar and think alike, this is a problem for a prediction market.
c) Alignment of Incentives: Volume and diversity are certainly necessary conditions for proper functioning of markets. However, it’s the intent of participation that would dictate the success. All participants should act rationally and make the best risk-adjusted bets. In corporate settings, issues like team dynamics, chances of promotion, personal relationships etc can come in the way of rational bets. The incentives to participate should not interfere with the actual decision-making. Incentives can be aligned with monetary gains, reputation, accomplishments or other non-monetary rewards. And this alignment should be dictated by how a corporate is planning to use markets. Markets have to strike a balance between confidentiality and transparency.
d) Transparency: Finally we think that is really important that the market is transparent and confidential. All of the members need to have the guarantee that their positions are not reveled unless they want to do so. For example if a market opens to determine if a project is going to meet a certain dead line and I think that it will not make it, but the project manager is a friend of mine then I need my position to remain confidential.
Part II:
How would you use prediction markets to make better decisions at Google? Make sure that you address the risks and challenges of replacing more conventional forecasting processes with prediction markets. Also, discuss how you would modify how prediction markets have been used so far. In doing so, you should focus on “organizational design” issues (such as participation and whether trades should be anonymous) not “market mechanism” issues (such as whether short selling is permitted). Note: This analysis should build on but not repeat what was written in Part I. Words: 807 In order to use prediction markets help better decision, Google (or any organization) has to take the following steps: a. Test and prove that markets lead to better decisions within the context of decisions that their managers make b. Facilitate the creation of efficient prediction markets with right incentives c. Educate the decision makers about markets and integrate markets with organization
Google should follow a phased approach.
Transition Phase: During this phase, Google should set up the markets, encourage participation and rigorously test if prediction markets lead to better decisions. There should be a control sample of managers who are not given access to prediction markets in any way and a test sample who are encouraged to refer to prediction markets (although the final decision would remain in the hands of the manager). The final decisions and the actual result should be tracked.
