On Analysing Big Data and Storytelling

There’s a neat post on analysing big data and storytelling on an HBR blog worth checking out. From the post:

Data scientists want to believe that data has all the answer. But the most important part of our job is qualitative: asking questions, creating directives from our data, and telling its story.

That’s a neat summary of what data is all about: to allow us to make better decisions (“asking questions” and “creating directives”) and implement them (“telling its story” – sex isn’t the only thing that sells; stories do great, too).

Running in Perth

Photo of WH running across the bridge toward South Perth
Running in Perth, across the bridge toward South Perth

It was heavenly – went out on a run with a good friend in Perth and in perfect weather, too. Recollections of past runs from five years ago suddenly flooded my mind. It was almost like I’d never been away.

In particular, I remembered how nice it was that strangers we sped past almost always said “good morning!” It always gave us a little boost and warmed us up a little; always great on those especially cold winter mornings.

I remember the first time we were greeted on a run. We were just running along at King’s Park when someone we passed hollered out a “good morning”. Having never encountered such an event, we took a quick glance at each other and returned the greeting with more than a hint of puzzlement.

After a few more of these random “good mornings” we kind of got the hang of it and started pro-actively saying our own “good mornings”.

For some reason I didn’t think people still did that any more so I asked my friend (who unlike me, has been in Perth these past few years) if the people he ran by still greeted him as he passed. For some reason, being back in Singapore these years have made me think this never happened anymore.

“Yeah,” he said, “they do!”

Despite his saying they did, I thought I sensed a little hestitation in his voice and I was skeptical. I smiled agreeably but wasn’t sure what to think.

But seconds later, as if God Himself wanted me to believe in the myriad possibilities of humanity, we ran past an elderly couple walking their dog who hollered out to us, “good morning!”

“Morning!” we replied, smiling. And on we went, running just as we did back then.

Going to Perth

The wife and I will flying to Perth tomorrow night. It’s a homecoming of sorts, both of us having spent about two years (2.5 in my wife’s case) studying at UWA. Both of us are feeling pretty excited, wondering what has changed (and what hasn’t) and planning to revisit our old haunts. I’m sure it’s going to be a great trip.

Choose your customers

Seth Godin recently had a great post on choosing your customers. He’s not the first to say this, and he’ll certainly no be the last, but it’s always good to hear reminders like this.

Choosing your customers first (before doing anything to get customers) isn’t intuitive. Heck, choosing your customers isn’t intuitive (who cares who buys my product as long as they buy my product, right?) but it’s definitely one thing to look out for.

When you have a project that you personally like and believe in, and one that you want to succeed, the last thing you’ll want to do is to go out to find champions for your project without thinking about who those champions are or what they stand for.

Before you know it, you’ll have people whom you thought were on your project’s side, but who actually have their own agendas. And you may end up fighting a creature you brought to life, twisted in ways you’d never imagined.

The importance of getting users to trust your data

When I’d just joined my current company, I’d heard stories about people leaving because they could never trust the numbers that were being thrown around. They were frustrated that more time and effort was spent trying to get the correct numbers for their reporting than doing any “actual” work. What is more, because they couldn’t trust the data, many decisions were executed on “feel” instead of facts (the outcomes of which will probably not be the best).

One big reason for the data discrepancies lay in the fact that each region (e.g. Asia and America) was measured differently. Whenever cross-region discussions were carried out, each side would be using numbers generated by each side’s reporting and analytics team (e.g. me), each tailored to their respective region’s needs and type of measurements.

As you can imagine, this was always going to lead to conflict and confusion.

Joe Bigshot: “Your team made a loss last month with horrible profit margins. How do you explain that?”

David Salesmaster: “What do you mean I made a loss? It was strategically breaking-even! And in fact, from my own estimates, I’d turned in a slight profit.”

(There is a beautiful Chinese saying for this: 鸡同鸭讲, literally translated as “chicken and duck talk”. One side clucks; the other side quacks.)

But inter-region differences weren’t the only thing affecting the integrity of the reported numbers. Even within regions differences abounded. Plenty of people maintained their own data silos (e.g. using spreadsheets with data copy and pasted from elsewhere) that were often not maintained very well.

Joe Average: “Hey Donn, the data you’d sent yesterday is wrong.”

Me: “What do you mean wrong? Wait a minute. When was the data you used for your comparison last extracted?”

Joe: “About two months ago?”

Me: “…”

One way that we stamped out inter-region differences was to get very clear about what the differences were. Once that was done, we always explained to recipients that our region’s data differed from others in so-and-so ways. If they wanted to compare numbers with their other-region counterparts, they should factor these differences in. You might think this was an insignificant move we did, but you’d be surprised at the number of “oh, I didn’t know that” or “so that’s why it’s different” exclamations from grateful listeners.

Even though the numbers themselves may not have been more accurate, the perception of their accuracy no doubt improved significantly. People started being more trusting ofthe numbers we were giving out, and started confidently making use of the data in their decision-making.

Though I cannot be sure about this, I believe that this move had also reduced the number of data silos people were maintaining as well. There are far fewer incidents of “why does your data differ from mine in so-and-so ways?” as compared to early on. With people trusting our numbers more than their own there just isn’t a need for them to maintain such silos (especially true when the datasets they were maintaining were growing too fast for their comfort).

In the end, the small action of informing people why there were differences in data led to an increase of trust in the data, which led to an increase in the use of data, eventually leading to better, data-backed decisions.

Reminds me a little of the broken windows theory, with the supposed existence of “broken windows” (little acts of vandalism or literally broken windows) leading to a greater incidence of overall crime, and the elimination of which led to a significant reduction of overall crime. May or may not be true, but it leads to a great story.

How to read library books

There are 14 library books on my table staring back at me as I write this. Six borrowed on my card (maxed). Six borrowed on my dad’s card, which I have permanently borrowed (also maxed). And two on the wife’s card (not maxed, but soon to be).

As the wife tidies my desk (again), she looks at me despondently, resigned to the fact that I’m never going to change this habit of borrowing books I “never read” (according to her).

“But I do,” I say, for what must be the millionth time.

I may not finish the books, but I certainly do read them. If not the whole book, then half. If not half, then perhaps a chapter. If not a chapter, at least a page.

This was a habit I learned in secondary school. I vaguely remember it to be our vice-principal who taught us this, a man very into Edward de Bono–Mr Lateral Thinking–and Tony Buzan–Mr Mind Mapping–type of learning techniques (and who tried, unsuccessfully, to turn us goose feathers into foie gras).

“Go to the library,” he said, “and borrow books. Don’t worry about finishing books. Select chapters that interest you and read them. Read what you need; skip the rest. The great thing about library books is that they’re free. When you’re done with one, just go on to the next one.”

For some reason this lesson stuck with me. Trips to the library are no longer about finding that one book that will change my life. It’s about finding the books that will contain some content that will, in sum, add up to change my life.

It is not uncommon for me to borrow ten books at a go, thinking maybe one or two have potential to be really great reads, while the rest fulfil their duties as fillers for thinking periods on the toilet bowl. And it is not uncommon to have these “fillers” turn out to be significantly better reads than their “high potential” counterparts.

But this would never have happened if I hadn’t ditched the mindset that books are meant to be finished–that books are to be read from cover to cover.

Before applying for a job, know what company it’s for

I came across (purely by chance) a job advertisement posted by a self-proclaimed SEO (search engine optimisation) company. Not having heard of it before and curious to know what it did, I did a quick search on the company hoping to find out more.

True to form, the company appeared first on the first search results page I landed on. (Though it really wasn’t all that surprising since the company’s name was a rather uncommon concatenation of words), the description showing up on Google was: A description for this result is not available because of this site’s robots.txt – learn more.

Huh?

Not exactly as search engine friendly as you’d expect for a company specialising in “search engine optimisation”. I mean, this is really elementary stuff. Even more shocking? When I clicked on the link to the home page *woooooossshhhh* I was blasted to the past(!) with a flash intro.

Yup, you know those slideshow-like things where images fade in and out along with some promotional text saying they’re the best in the world or something like that. So 1999.

Nobody tells this to beginners – Ira Glass

A great quote from Ira Glass:

Nobody tells this to people who are beginners, I wish someone told me. All of us who do creative work, we get into it because we have good taste. But there is this gap. For the first couple years you make stuff, it’s just not that good. It’s trying to be good, it has potential, but it’s not. But your taste, the thing that got you into the game, is still killer. And your taste is why your work disappoints you. A lot of people never get past this phase, they quit. Most people I know who do interesting, creative work went through years of this. We know our work doesn’t have this special thing that we want it to have. We all go through this. And if you are just starting out or you are still in this phase, you gotta know its normal and the most important thing you can do is do a lot of work. Put yourself on a deadline so that every week you will finish one story. It is only by going through a volume of work that you will close that gap, and your work will be as good as your ambitions. And I took longer to figure out how to do this than anyone I’ve ever met. It’s gonna take awhile. It’s normal to take awhile. You’ve just gotta fight your way through.

And that is why I do this; why I write. Nobody may read a word. Nobody may appreciate the work. But I’m going to keep working at it. And one day, if and when I’m somebody who’s somebody, and someone asks me, “what’s your secret to your success?” I’m going to be thinking of this period of my life, writing and thinking for ghosts.