The perils of Over optimisation -Using a single model

When do we have the biggest disasters of all times? When all people think
in a similar way and use a similar model to base their decisions. Example of
one such disaster was in the famous 1987 crash in the U.S. What happened
was that every big portfolio manager had purchased portfolio insurance. Now
since many of the people were sold such benefits as more and more of the
portfolio insurance became popular everyone started using the same models.
On October 19, when the S&P fell around 1% in the morning, every portfolio
insurance model said to sell and their was then a cascade of selling orders
sending the market into a frenzy. Whenever, too many orders in an exchange
are based on the same benefit and thinking, risk is ignored.

Think about the 2008 crash, first some sensible guys started selling
mortgages at low rates and then slowly as the products become popular
everyone was glued to the benefits and risks were ignored while naysayers
pushed back. Soon once a lot of people started believing in them there was too
much concentration among banks regarding similar products. Then
somewhere a small spark was ignited and boom.

One thing is clear in us as humans we love to optimise. Definitely,
optimisation is good its rarely bad, but sometimes we are over optimising and
when we are optimising we are too glued to the benefits of a theory or too
attached with a thinking model we are using. In economics, we have learnt
about cost optimisation (or so called efficiency). Most of the times
optimisation becomes over, I mean too much emphasis is given to the benefits
of a single things and risk are ignored. Lets take an example: Some people say
diversification is good. Definitely a good thing. Say lets start with one stock,
once we go from one to two stocks, the marginal benefits of diversification is
higher, same is with 2 to 3——so on every time there is some marginal benefits – however beyond 10-20 stocks, the marginal benefit from
diversification falls massively and risk increases massively.

“ Whenever we optimise anything it is the known risk we are reducing, however
the hidden risk still remains”

Raphael Douday

It is rarely the known risk which is the killer, its the hidden one which kills
us. Example: Having friends is good, but having 3-5 excellent friends vs
having 100 friends – marginal benefit of friendship reduces massively.
Like team is good. It is said 3-5 is a team, 20 is crowd.
Example: In machine learning, there is a technique where dividing data
into group helps- (K-means clustering) – Definitely think about you have data
about bank loan customers- you divide them on the basis of age, income,
class. This technique helps to determine how many groups you should divide
the data. This is the result. If you see the chart below you can clearly see as we
increase the number of groups beyond 4-6 marginal benefit from further
dividing the data is useless.

Why This Happens?

Whenever we are giving any argument or thinking about anything we are optimising something or using some model. We get too attached to our solution and forget the risk. The problem with over using one model or benefit is we over optimise it. Example: Whenever we see any argument for a company is true. We gather 3-5 evidence in favour of our argument and go back and forth. Beyond 5 no positive evidence helps in analysis.

Many people are always focused on profits and profits. But if you over-optimise profits too much you may be killing your business model. Example: There is a company which can knows it has pricing power – if it increases prices no volume will be effected. But for the benefit of customers it never raises the prices massively. While another company which is only focused on profit will increase it. Sooner or later customers will realise it – it can even increase competition in the industry (You may be increasing hidden risk). Read the famous case of  Turing Pharmaceuticals –  Where the company increased prices massively. Or  the case of Pepsodent, Colgate where they were increasing prices for long and Patanjali found an opportunity and entered the industry. Same was the case with Gillette as it was very expensive, some college boys started Dollar Shave Club and started competing with it. There are also opposite cases.  If you are a child from 90’s you must be remembering Ravalgaon toffees (transparent wrapper) – the company is almost 100 year old – it has pricing power but it never raises its prices – even MNC’s can’t compete with the firm.

There is an excellent lecture on by Prof Sanjay Bakshi where he tells whenever he asks his students to evaluate the operating skills of a business manager – students just focus on high ROCE’s, normally it is deliberate by some exceptional managers to keep their margins and credit turns less to keep their customers and creditors happy.

Hidden Risk

There are generally two types of risk – known risk and other the hidden/unknown risk (no one can predict it)- Whenever we optimise we reduce the know risk, generally the unknown risk increases. Now this does not mean we should not optimise but over optimisation increases the hidden risk massively.  Think about the benefit of having large cities – but during the coronavirus pandemic we realised having massively concentration of population at one place increases the risk too. Generally very large cities are too cited with more cases of depression, more life spent in transportation and less life satisfaction. It doesn’t mean start living in a village – but exceptionally large cities bring more risk. Beyond a point benefit vanishes.

Think about stock markets – whenever all people think in a similar way – all orders in the market are too concentrated on one side. A small spark in a jungle which has a lot of dry wood has massive chances of a large fire.

Think about all investors buying stocks with low P/E’s. Now if you are too focused on P/E you may buy cheap but cheap only from P/E perspective junk stocks. Focus more on quality of earnings, longevity and other factors makes us realise the importance or wisdom of markets. Commodity stocks with depressed earnings and a high P/E multiple are better than low P/e commodity stocks with high earnings. Markets are about future not past. 

Another case of stock market crash was of  Long Term Capital Management – They say the IQ in that room was the highest ever. Their strategy was betting on the gap between two bond yields to never go far from one another. Now majority of times they made small amounts of money. In history, never had the gap increased beyond a certain proportion. Too much belief in their theory, let them to take massive debt on the same strategy- they were over-optimising. Then the unexpected happened the gap between those two instruments increased massively, (Russian govt defaulted) leading to heavy losses and winding up of the firm. They forgot the basic other rule too much debt is a ruckus leaves very less margin for error. Like my friend says:

“Hope for the best, Prepare for the worse and worse is always more worse than the worse of history.”


This is again a problem with using one tool and over-using or over-optimising it. Next time whenever you think about benefits think about cons – if you are taking some decisions of improving efficiency of factory think about the risk you may be incurring. Having both perspectives helps us take a wise decision and saves us from the perils of over-optimisation of one theory/model/tool.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s