Some thoughts on the thinking behind building things

I’m going to write a bit about my thinking process whenever I build things, whether it’s websites, VBA applications, or financial models. I’m not too sure if this is going to be a “oh that’s so obvious why is he telling me this?” piece, but if there’s one thing I’ve learned, one person’s normal behaviour may be another person’s exceptional one.

And I know you’re exceptional. So let’s get on with it.

My thinking process whenever I’m making things people use (i.e. goes into production) goes like this:

First, I ask myself (or the project champion/sponsor/requester): is this a one-off, or will there be many iterations?

If it’s a one-off, I can do away with making things beautiful and/or flexible. For one-off builds (again, whether it’s a webpage or financial model), most things can be hardcoded (e.g. unchanging interest rates, cell references and the like) and documentation doesn’t need to be too detailed (trust me on this though: documentation should still be done. People have a habit of waiting till the day after you forget what you did to ask you how you did it.)

If there are going to be many iterations, then I ask myself, what’s likely to change?

I once built a financial model containing over 30 options from which business users could pick and choose to determine what affected the final number. The model didn’t start of with all the options straight off the bat, and was in fact supposed to be a one-off. But after hardcoding and then manually changing what users needed multiple times within the same day (even though they “were quite certain” on what they wanted early on), I realised the flexibility of options, despite the longer initial development, was worth doing. I built the options on a very granular level so users could say, “I want option #1, #5, and #29” and have the affected numbers come up immediately.

Focusing on what’s more likely to change, and ignoring those that won’t, will allow you to save plenty of time. In that same model, I didn’t bother with ensuring formulas didn’t get broken when headings changed, because I knew they weren’t going to. (But I have certainly done this before; in some applications I’ve written, the source data had headings manually typed in, and almost every other week some variation of what was basically the same thing would get put in. I used regular expressions to ensure anything that looked like a header was treated as a header.)

Who and what is this report for?

Depending on the needs of the request, you could very well end up spending hours on something that should take only minutes. I have encountered many times someone coming to me for numbers in which I thought details were needed (or would be useful). I’d spend far too long extracting and formatting really granular data, when all the person needed was a “ballpark” figure.

And when you’re sending that consultant who is requesting for your company’s sales figures the information he needs, make sure you’re not giving him more than he needs. Does he really need to see the invoice numbers? Or customer IDs? Or costs?

Designing for maintenance

A few years back while taking the train I saw an engineering student’s lectures notes on “designing for maintenance”. I’m not from an engineering background, but because I happened to be mulling about on what software to use for a website I was developing, the concept of designing for maintenance struck me like Eric Clapton’s fingers on a guitar string.

Beautiful. Smooth. And oh-so-right.

The two main pieces of software that I had been considering were pretty distinct. One had a very developer-friendly, Linux feel. The other had a very polished, user-friendly Apple feel. The former catered to my nerd needs, while the latter catered my aesthetic aspirations (an aside: the very word ³aesthetic² is surprisingly pleasing to the eye and ear).

Though at first I was kept in a 50/50 bind between the two, after getting exposed to this idea of “designing for maintenance”, I realised that I really needed to go for the more polished and user-friendly one. Though I was helping to develop the website, I wasn’t going to be running it or doing the day-to-day maintenance of the site.

I think that maintenance of software (including spreadsheets) is something that’s missed by far too many people. If you know you’re not always going to be around to maintain the software, make sure it’s easy-to-understand and that everything’s well documented (even if it’s in an e-mail). Keep in mind that it isn’t all about you, but rather about the end-user, too.

Well, that’s about what I have for now. Would love to hear your thoughts on this. If you have any.

What Courage Is

Was on one of my regular runs today when the words “courage is not acting in the absence of fear, but in spite of it” suddenly came to mind and never left.

Can’t quite remember where exactly I read it or heard it, but those words have always comforted me in times of need. Here are two quotes that might have had a hand in incepting my mind with those words on courage.

I learned that courage was not the absence of fear, but the triumph over it. The brave man is not he who does not feel afraid, but he who conquers that fear.

— Nelson Mandela

Courage is not the absence of fear but rather the judgement that something is more important than fear; The brave may not live forever but the cautious do not live at all.

— Meg Cabot

Business vs. IT

I was reminded today in a book I’m reading on visual analytics that the purpose of any analytical project is ultimately to make better decisions. Coming from a mixed business and IT background, I have had my fair share of IT vs. Business conundrums.

With my IT hat on, I’m always thinking about efficiency, optimisation, ease-of-use, and resource management. What questions are being answered, what problems are being solved, or what the ROI (to a certain extent) isn’t always on the forefront of projects (though almost always who’s the requester is).

With my business hat on, I’m always thinking about what business questions to solve, what information needs I have, and just plain old “getting the answer”, which IT isn’t always the most willing to help retrieve. I don’t care about how long IT takes (so long as its done yesterday) and how much effort they need to put in or how optimised the process is, I just want my questions answered so I can make better decisions. Isn’t that what technology-driven analytics is supposed to do

But, coming from a mix of business and IT backgrounds, I know the problems both sides face. IT needs more emphasis on understanding business needs, while business needs to understand IT constraints.

My current role has me more as a “business” person, and I don’t know if that’s a blessing or a curse. A blessing in that I’m given a lot more face time in business and finance discussions, allowing me an almost voyeuristic view of how the business makes its money. A curse in that IT treats me as any other “business” user who’s request trigger-happy and who doesn’t understand the IT side of things (and therefore who should be ignored most of the time).

It’s tough convincing IT to talk about the things that could potentially give pretty decent ROI when they don’t trust you.

“I’m one of you,” I tell them.

“Yeah, you sure are, buddy,” they reply, smiling, handing me a ticket number.

For the lack of a plan

I spent close to five hours working on an assignment that I thought would take me an hour at the most.

It was an ad hoc project, and as often the case with many ad hoc projects you can’t really tell how long they’d take till they were done. My initial estimations were based off of casual look-throughs the source data I was working on, and a rough understanding of what the requirements were.

Expecting it to be not too complicated, I jumped into the assignment without thinking too much about it.

I’ll think about it as I go along, I thought.

And going deeper into the assignment, I quickly grasped what had to be done. A plan of attack was developed on the fly, and on I went.

Almost five hours later, on I was still going.

What went wrong?

I think the biggest mistake I made was rushing into things. I wanted to get the project out as soon as possible, and had set unrealistic internal and external completion time estimates. This led to a complacency at the start; an escalation of worry in the middle; and a near-panic toward the end.

With a little more forethought, I might have saved myself chunks of time. Had I known the multiple files I needed to process were so similar, I’d have done it in batches instead of piecemeal. Had I known the requirements more intimately, I wouldn’t have removed data I eventually needed. And had I known how unstructured the data would be, I’d have quoted an extended timeline that would have given me more breathing space.

In an attempt go faster, I skipped what might perhaps have been the most important thing: planning.

IT Replacing Labour and the Possible Fragility of the Economy

Latest stats on the US Economy, written up about by Andrew McAfee. He posits the fact that unemployment’s not going up while other economic stats are might be due to greater IT spend — technology replacing labour? Seeing what I’ve seen in my half decade in the workforce I can’t say I disagree: too many jobs are still out there because of a (possibly deliberate) refusal to embrace automation.

Then again, there is a problem with a too-highly-automated workforce, when efficiencies and optimisations are taken a step too far; where redundancies in processes mean the weakest link breaks the system, making it far too fragile.

If IT’s replacing more of the workforce, could it mean we might be approaching an increasingly fragile economy, dependent on automated systems we might not even be properly aware of?

Just a thought.