You may also be interested in:

Articles

Superforecasting: 3 Key Takeaways

  • By Bryan Lapidus, FP&A
  • Published: 8/27/2019

book

AFP held its first virtual roundtable book review on August 20, in which three expert practitioners discussed what they learned from the book, “Superforecasting: The Art and Science of Prediction,” by Philip Tetlock and Dan Gardner.

Key takeaways for each panelist are below.

BACKGROUND ON THE BOOK

In the summer of 2010, two U.S. intelligence officials from IARPA visited Berkeley, to meet Tetlock and a colleague, Barbara Mellers. The intelligence community offered to sponsor a massive tournament to see who could invent the best methods for creating the sorts of forecasts that intelligence analysis makes every day. Will the president of Tunisia flee to a cushy exile next month? Will an outbreak of H5N1 in China kill more than 10 people in the next six months? Will the euro fall below $1.20 in the next twelve months?

A total of five teams ended up competing. Over four years, IARPA posed nearly 500 questions about world affairs, gathering over one million individual judgments about the future. In year one, the authors’ team beat the official control group by 60 percent, and by 78 percent in year two. The team beat university-affiliated competitors from Michigan and MIT by 30-70 percent and outperformed professional intelligence analysis with access to classified data.

Tetlock studied the individuals who consistently stood above the crowd as measured by the accuracy of their predictions and learned that there are few common behaviors and thought processes from these superforecasters.

  • Panelist: Gaileon Thompson, CTP, FP&A, SVP, Global Consumer O&T Finance, Citi
  • Key point: “Being a subject matter expert does not make you a better forecaster. Distilling what you know versus don’t know, making intelligent assumptions, and looking at problems objectively makes you good.”
  • Implication for FP&A: “I felt like this section had FP&A written all over it. We are estimating things that are not always available. It is not about the amazing models and the amazing big data you have, but how you use it that counts.”
  • From the book:
    • “What [physicist Enrico] Fermi understood is that by breaking down the question, we can better separate the knowable and the unknowable…we have brought our guessing process out into the light of day where we can inspect it.” (Page 111)
    • “It is natural to be drawn to the inside view [about a problem]. It’s usually concrete and filled with engaging detail we can use to craft a story about what’s going on. The outside view [the base rate of facts about the broader question,] is typically abstract, bare, and doesn’t lend itself so readily to storytelling.” (Page 118)
    • “When we make estimates, we tend to start with some number and adjust [called the anchoring effect], typically we under adjust, which means a bad anchor can easily produce a bad estimate…So a forecaster who starts by diving into the inside view risk being swayed by a number that may have little or no meaning. But if she starts with the outside view, her analysis will begin with anchor that is meaningful. And a better anchor is a distinct advantage.” (Page 120)
  • Panelist: Geetanjali Tandon, Digital & IT Transformation Finance Lead, Bayer Crop Science
  • Key point: “‘Give the goal, not the how.’ Not everyone has the answer to how something will get done; there is experimentation and exploration along the way.”
  • Implication for FP&A: This approach summarizes the agile methodology, so how will finance adapt to this way of doing things? How will we support them without bogging them down with asking what the exact budget is they will need because it fits into our model.”
  • From the book:
    • “‘We let our people know what we want them to accomplish. But…we do not tell them how to achieve those goals,’ - William Coyne, SVP of research and development at 3M (page 225)

       

  • Panelist: Mike Powers, Financial Controller, Abt Associates
  • Key point: “A strong leader might try to drive a particular outcome, but the benefits of group dynamics can be employed to include viewpoints that help the broader group achieve a better outcome.”
  • Implications for FP&A:
    • “You can create a group and employ strategies to get to a better decision. Sometimes you need to have a super questioner.”
    • “Often, you may be the only finance person in the room and you wonder, ‘Should I raise my hand when I don’t really know the full dynamics, when I have not fully bought into the great idea?’ Yes!” - Geetanjali Tandon
  • From the book:
    • Be cooperative but not deferential. Consensus is not always good; disagreement is not always bad. If you happen to agree, don’t take that as proof that you are right. Never stop doubting. Pointed questions are as essential to a team as vitamins are to a body. (Page 199)
    • Combining uniform perspectives only produces more of the same, while slight variation will produce slight improvement. It is the diversity of the perspectives that makes the magic work. (Page 209)

Final thought:

The book describes superforecasters as having a sense of probability. No event outcome is ever 100 percent certain, and so they revisit forecasts as new information becomes available; they have intellectual humility about their certitude and match open-mindedness with curiosity and grit.

“If you have to plan for a future beyond the forecasting horizon, plan for surprise. [P]lan for adaptability and resilience. Imagine a scenario in which reality gives you a smack in the ear…then assume reality gives you a kick in the shin. ‘Plans are useless,’ [General and President Dwight] Eisenhower said about preparing for battle, ‘but planning is indispensable.” (Page 244)

Geetanjali Tandon will speak at AFP 2019, facilitating the FP&A roundtable, “Executing on FP&A.” Mike Powers will also present at AFP 2019, leading the session, “Speedy Metrics” about determining the right business metrics and quickly creating graphical interpretations of them.   

The next AFP FP&A Roundtable will be on September 12 at 3:00 p.m. U.S. Eastern Time: Creating Citizen Data Scientists, A Case Study from Novelis. Register here.

Negotiation Skills for Finance and Treasury Professionals

Monday, December 16th and Wednesday, December 18th, 2019
10:00 AM - 12:00 PM ET
Attendees will learn to negotiate more effectively, prepare for negotiations more efficiently individually and as teams, and better understand the needs and interests of stakeholders and external parties.

Register Now

Copyright © 2019 Association for Financial Professionals, Inc.
All rights reserved.