The data is what you want it to be

I was just browsing when I came across a short little post about a neat page on Wikipedia aptly called “List of common misconceptions“. The post contained an excerpt of that Wikipedia page on life expectancy, a misconception that I myself had (somewhat embarrassingly) up till only recently:

It is true that life expectancy in the Middle Ages and earlier was low; however, one should not infer that people usually died around the age of 30. In fact, the low life expectancy is an average very strongly influenced by high infant mortality, and the life expectancy of people who lived to adulthood was much higher. A 21-year-old man in medieval England, for example, could by one estimate expect to live to the age of 64.

I’d always wondered what it felt like to live in the middle ages, where people died on average forty, fifty years earlier than they do now: What would I do differently? Did people get “old age” issues younger? Were there great grandparents? 

When I realised that infant mortality was the one greatest factor in affecting average life expectancy, it opened my eyes up to the possibilities of data story-telling.  You can pretty much tell any story you want, with any sort of data, depending on what you leave in or out.

The fact that the average life expectancy in one place and/or time is two-fold that of another place and/or time doesn’t really mean anything without context.

That life expectancy in Monaco is 87 years while life expectancy in Sierra Leone is 47 is a fact.

That Sierra Leone’s average life expectancy is affected greatly by infant mortality is a story. That Sierra Leone’s average life expectancy is affected greatly by deaths of both mother and child during childbirth is a story. That Sierra Leone’s average life expectancy is affected by poor access to healthcare services in rural areas is a story. That Sierra Leone’s average life expectancy is affected by civil war is a story.

That Monaco’s life expectancy is helped by having the lowest infant mortality rate in the world is a story. That Monaco’s life expectancy is helped by compulsory state-funded health services is a story. That Monaco’s life expectancy is helped by their Mediterranean diet is a story.

In my work as an analyst, I work with data quite a bit. Many times someone would come up to me and ask, “so, Donn, what does the data say?” And I can’t help but answer that question with another: “What do you want it to say?”

I don’t need to be good. Just better than you.

Abstention on the part of those who won’t venture
in creates opportunities for those who will.

The quote above comes from Howard Mark‘s The Most Important Thing Illuminated, who was referring to investors who, believing they cannot beat the market, stay away from the investing game. In doing so, these people allow those who think they can (and who do participate in actively trying to beat the market), opportunities to do just that.

It reminded me of a thought I had during my recent annual military service, where I saw a significant number of comrades looking more physically unfit than ever, many of whom were almost as or even more physically fit than me during our active days (about ten years ago). By virtue of simply having more or less maintained my fitness these past years, I was now perceived by them as being much fitter than them.

It was almost as if because they didn’t want to play the fitness game (“Not young; no time” was the common refrain), and I did, I “won” by default, even though I wasn’t naturally the “fittest” to begin with.

By the same token, I’ve known of plenty of really smart people who not quite wanting to play the “career game” (for whatever reason) get stuck in career mediocrity, giving us less naturally talented folks opportunities we wouldn’t have had if not for their leaving the game for us.

The Use of Worry

“Worrying doesn’t get you anywhere.” Or so they say, “they” being the anonymous group of trolls in my head that churns out stuff like that.

But worry does have a use. It urges me to take action. Because of worry, I do things today that I’d ordinarily put off to tomorrow.

I admit, worry makes today (and all preceding time before the event of which I am worried about) potentially nightmarish — the anxiety I feel in the grips of persistent worry isn’t particularly pleasant. But that might be a small price to pay in being as prepared as I can be in anticipation of that worrisome event.

The more I prepare, the less worried I get; till I know that I can prepare no more. That’s when worry ceases to be useful, and itself becomes a cause for concern.

On the Endowment Effect

“There is a very real difference,” my friend told me, “between getting a car ‘new’ and getting it ‘second-hand’. When you’re getting it second-hand, you have no idea what the previous owner did with (or in) the car.”

He should know. We were sitting in his (second-hand) car, bought just a couple of months back, his sixth car in three years. Of these six, only his first two were new.

It reminded me of something I’d been thinking about lately: the endowment effect, of which definition I’ve happily copied and pasted below from good ol’ Wikipedia:

The endowment effect (also known as divestiture aversion) is the hypothesis that people ascribe more value to things merely because they own them. This is illustrated by the observation that people will tend to pay more to retain something they own than to obtain something owned by someone else—even when there is no cause for attachment, or even if the item was only obtained minutes ago.

There have been a number of experiments done to demonstrate this effect, with the one that I’ve read about the most times being the following (also from Wikipedia):

One of the most famous examples of the endowment effect in the literature is from a study by Kahneman, Knetsch & Thaler (1990) where participants were given a mug and then offered the chance to sell it or trade it for an equally priced alternative good (pens). Kahneman et al. (1990) found that participants’ willingness to accept compensation for the mug (once their ownership of the mug had been established) was approximately twice as high as their willingness to pay for it.

But I’ve never actually read about an experiment that tried to explain it in terms of the bid/ask spread as it relates to investing (which compensates for risk) and information asymmetry. Or how it relates to how our default option is really not to trade, and that trading requires motivation and commands a premium.

For example, let’s say you are given the choice to buy the mug. There’s no reason to think that the amount quoted to you would be below the market price.

On the other hand, if you are given the mug, and then someone quotes you a price for it, there’s no reason to think that it would be above the market price, even if that price came in the form of a set of equally priced pens.

In fact, try playing the scenario in your head and see if you’d do any different:

  1. The experimenter (a stranger) comes up to you and says, “would you like a mug or a pen? By the way, both cost the same.”
  2. Randomly, you choose the mug. There is no reason, at this point in time, to think either the mug or the pen costs more than the other.
  3. The experimenter gives you the mug.
  4. After five minutes, the same experimenter comes up to you and offers you the choice to trade, saying, “are you sure you don’t want to have this set of pens instead of the mug? I’m willing to trade if you are…” (these are the same pens talked about in point #1)
  5. What would you think? Personally, I’d think there was a catch. You give me a mug, and five minutes later you’re trying to get it back. It’s got to be worth more than the pens if not why on earth would you want to trade?

Point #5 highlights risk and information asymmetry. There’s a risk I’m getting the shorter end of the stick because the experimenter seems to know more than me (i.e. the “true” value of the mug and pens). Why else would s/he offer to trade? (This, I think, is why new cars command such a premium over pre-owned ones. There’s a big risk pre-owned cars were subject to abuse you’d prefer not to know about. Why else would the car owner want to sell?)

Experiencing this risk would urge me to ask for more. If you give me $X more than you’re offering, I’m willing to take the risk that the mug really isn’t worth more than you say it is worth. If I decided on the trade, it’d mean that I trusted you completely on the fact that they were worth the same.

And let’s not forget the idea that people in general don’t really like to think; we don’t like to expend energy unnecessarily. If there’s a risk that we could be losing out, but we don’t really want to think about it, the default option would be to just say “no”, or quote a price such that it’s easy to say “yes”.

The World as it Should Be

Just thought I would share with you what has to be, for me, the quote of the week. Taken from the book Getting More, a beautiful book on negotiation by Stuart Diamond (emphasis mine):

Lower your expectations. If you come into a negotiation thinking that the other side will be difficult, unfair, rude, or trying to cheat you, you won’t be likely to have dashed expectations–and you won’t be emotional. When you lower your expectations of what will take place in a negotiation, you will be rarely disappointed–and you might be pleasantly surprised. Getting yourself psychologically prepared is important.

You might feel, “Hey, I shouldn’t have to do things like that.” Okay, maybe not. But we live in the real world, not in the “should” world.

The beauty of that statement, I think, lies in the fact that it embraces the irrationality of people, the irrationality of the world. Where things are done sometimes for reasons beyond human comprehension.

And even if you don’t believe in the irrationality of people, as I sometimes find myself wont to do, the fact is we as human beings have so many hidden motivations that though we are, perhaps, ultimately rational, we are for most practical purposes just the opposite.

The place for the polymath. Because there's too many good things in life to be great at just one thing.