What the US Air Force Can Teach Us About Predicting Home Values?
One size fits all, or even one size fits most, is a myth, whether in clothing or jet fighters.
In the 1950’s, the United States Air Force (USAF) faced a challenge. They had faster, more powerful jet fighters on their hands, but the crash rate was on the rise. The solution they devised to counter this issue centered around enhancing safety through a cockpit redesign. Using a data-driven approach, the USAF collected measurements from over 4,000 pilots to create a profile of the “average” pilot across ten dimensions like height, arm length, and chest circumference. The idea was to use the profile to design a cockpit that would fit the majority of the pilots, basically a standardized “one size fits all” approach.
A young Lt. Gabriel S. Daniels wanted to understand how many pilots actually fit within the measurements of the “average” pilot. Even with a generous threshold of 30% from the average, Daniels discovered that none of the measured pilots fit the “average” pilot’s profile in all ten dimensions. In essence, there was no such thing as an “average” pilot. If they had gone ahead with the standardized cockpit, it would have fit nobody. The findings led the USAF to go with a “custom fit” cockpit that allowed pilots to adjust their cockpit to fit their individual needs, which resulted in better safety and fewer accidents. The concept of “custom fit” is now widely used today with various consumer products from automobiles to apparels to fitness equipment.
In recent years, numerous companies are harnessing the power of artificial intelligence (AI) to handle both complex and routine tasks. Zillow, a real estate giant, has used AI to estimate the value of homes, popularly known as Zestimates. In 2018, Zillow launched a new business model that relied on accurately predicting home values to buy and quickly resale homes for a profit. However, three years later, Zillow pulled the plug on this business plan and wrote off $800M in losses.
A primary factor that led to the shutdown was the inability of Zillow’s AI model to accurately predict home values. Rich Barton, Zillow’s CEO, summed it up: “Fundamentally, we have been unable to predict future pricing of homes to a level of accuracy that makes this a safe business to be in.” Amid the housing market shifts in 2021, Zillow found itself overestimating home prices, leading to difficulties in selling properties at predicted rates.
Similar to the USAF’s evolution of cockpit design from a standardized “one size fits all” to a custom fit concept, Zillow’s home valuation model might have potentially benefited from a comparable approach. The custom fit approach emphasizes adaptability and the inclusion of a feedback loop to understand what is working and not. insideBigData speculated that Zillow’s model failed to accurately adjust to a quickly changing real estate market brought about by the pandemic, thus implying Zillow’s model was more static than dynamic.
Opendoor, Zillow’s competitors at the time, navigated the challenges more successfully, likely due to the adaptability of their home valuation models. They detected the cooling housing market earlier, allowing their model to adjust accordingly and reflect more accurate pricing. If Zillow’s model could have promptly responded to changing home prices or incorporated feedback from their own business performance, it might have fared better.
Organizations and businesses in customer-oriented industries, such as clothing, airplanes, and media, have embraced the USAF’s custom-fit approach. This method emphasizes adaptability and feedback, allowing these companies to offer products tailored to their customers’ needs. Companies building AI products, such as estimating the value of home, must adopt similar custom-fit principles in their designs to succeed in the long run. This means ensuring that AI solutions have a feedback loop for continuous improvement and the adaptability to handle unexpected scenarios not covered in existing models.
If you wish to delve deeper into the contrast between “one size fits all” and “custom fit,” we recommend Todd Rose’s insightful book, “The End of Average.” The book covers the shortcomings of standardized methodologies and showcases how embracing individual variability offers more effective solutions to the challenges of our modern world.
