QF began in 1900 with Bachelier observing that stock price changes were random and normally distributed. The normality of price changes allowed him to derive formulas for options. In 1940, Hayek proposed that the fluctuating price that Bachelier had studied was the result of a dispersed set of opinions about the value of an asset condensed into a single number. He argued that the price system was a more efficient way of allocating resources in an economy than centrally controlled decision-making. In 1952, Markowitz built a mean-variance framework around the observation that investors demand a higher return when they take on more risk. In the 1960s and 70s, Fama proposed the efficient market hypothesis (EMH). Using the EMH as a basis, in 1973 Black, Scholes and Merton combined Samuelson’s GBM with Ito calculus, insights from the CAPM, and a risk-neutral hedging argument to develop an enhanced version of Bachelier’s option pricing model. The BSM model was born and it was founded in standard finance theory (SFT).
In the 1970s & 80s, a pillar of thinking emerged to challenge the EMH assumptions that underlay the BSM model. Behavioural Finance (BF) argued that psychological factors such as over-confidence and herd behaviour influenced markets far more than the EMH assumed. Prospect theory and heuristics were proposed by researchers as alternatives to the EMH. The 1987 stock market crash validated their arguments.
The BSM framework needed adjustments for observed market behaviours that were not assumed by SFT, including behaviours that were direct results of the 1987 crash. The framework was adapted to account for volatility smiles, clustering, term structures, correlations, and mean-reversion. XVA and the multi-curve approach were later adaptions for credit and funding spreads structurally embedded in IR markets as a result of the 2008 GFC.
In the 21st century, a second pillar of research emerged in derivatives pricing that is forcing us again to think about SFT, the BSM framework and where they came from. The AI revolution created ML models that use pattern recognition and back-propagation techniques to price derivatives by reading large-volume market data sets. Deep learning techniques can now price option portfolios with very few limitations on portfolio size or complexity.
BF turned out not to be a challenge to SFT but rather a set of alternative theoretical approaches that helped explain market behaviours that were not originally included in BSM. Once confirmed empirically, those behaviours were built into BSM models and assumptions. ML derivative pricing models, on the other hand, are learning models trained on data that has been generated by models founded in SFT. ML models can take advantage of the fact that the BSM framework has 100+ years of theory, assumptions, and analysis on market behaviours built into it.