Business vs. IT

I felt like a lawyer. The call was in less than 12 hours, and I was busy preparing my case, consolidating evidence and building my story. To be honest, I wasn’t 100% behind the argument I was preparing to put across, but I didn’t really have much of a choice. I had to believe — how could I convince others if I couldn’t even convince myself?

But at the same time, I really wanted to steer clear on the “us vs. them”. Together with allies “from the other side”, we were working hard on framing it from a collaborative angle. It’d do us all no good if our discussion disintegrated into a blame game.

The situation at hand was a classic business vs. IT situation. Business says “we asked for this”, and IT says “no, you didn’t.” Whatever the case, the project deadlines weren’t going to be hit and nobody wanted to be responsible.

The funny thing is, if you asked me, I’d say no one was responsible (or that we all were). It was one of the first times we were doing anything close to what we were doing, and it was almost expected that hiccups like these were going to occur.

Requirements were defined, and IT made good on those requirements. Business was as clear as they could be on those requirements, but apparently not clear enough.  But how could they be? The project was a little too big and too fuzzy to be executed perfectly from the get-go.

Maybe a more iterative approach might have worked better, with all parties agreeing at the start that for a period of say, two or three weeks, we’d all be in a transition phase, where 70% of the requirements were met and the other 30% part of some type of agile-development, exploratory process, where things didn’t need to work perfectly but problems rectified quickly.

True, we could have spent more time thinking through and defining better requirements. But even if we spent an additional year we might not have uncovered requirements buried deep under others, the discovery of which were dependent on the implementation of the others. Would the need for a parachute have come to pass if planes hadn’t yet been invented? Would these later requirements have come to pass if the earlier ones hadn’t been implemented?

Whatever the case, this was an interesting situation to be in.

The Evolutionary Advantage of a Resistance to Change

I was just thinking about organisational change and pondering over how our natural tendency to change is to resist it, when this thought popped up: if resistance to change is so hardwired in our brains, it must serve some purpose — but what?

One of the premises of evolutionary theory is this: if something survives (be it an organism or perhaps even an idea), its survival can be attributed to certain traits or characteristics that help it survive. These traits or characteristics are developed/evolved over time: as those that have these traits survive, and those that don’t die, more and more of the population will have these traits, eventually becoming a “norm”.

Such as a natural tendency to resist change.

At least that’s the theory.

So, let’s make a provocative statement here (inspired by Edward de Bono’s po): Po, a resistance to change isn’t necessarily a bad thing, despite all the bad press it’s getting. In fact, it may be good.

I think that before we get all gung-ho about the next big thing and bashing old ways of doing things, we should think about why things are the way they are, especially if they have been that way for a long time.

Old ways of doing things are, generally speaking, antifragile. They’ve withstood the test of time and have a decent history of working. Good ideas tend to remain good ideas, and the longer they survive the better they tend to be; bad ideas, on the other hand, are discarded as soon as they’re found out (a caveat: those that do manage to survive for long, however, tend to be the most dangerous — bad things are made worse when they’re not known to be bad. Think insidious. Think CFCs.)

So if you have an old way of doing things, it may not be the best way, but it’s likely a way that generates decent results, enough for it to have lasted as long as it has. The moment a better way of doing things is found and tested to work, the old way is discarded. But until then, the old way is the best way.

If people weren’t resistant to change, on the other hand, good ideas wouldn’t have the time to spread. We’d be flitting from one idea to the next, discarding great ones and embracing bad ones in equal measure.

So a resistance to change isn’t all that bad. It’s the way things should be. The incumbent has earned its right on the throne, and the onus should always be on the challenger to prove its worth.

So it does worry me if people rush into new ways of doing things without having redundant systems in place, just in case. I mean, let’s not be too hasty in burning bridges.

If there’s going to be a process change, have the old process remain in place until the new process is proven stable. Depending on what the process is, a few iterations (or days, or months) is most certainly needed. Give it time to prove its antifragility.

When automobiles were first introduced, horse-drawn carriages didn’t disappear overnight. Concurrency. Then obsolescence.

Resist, consider, then change. Carefully.

The Case for Modelling

Last week I wrote about the need for experimentation over models. Models are generally too abstract and far afield from reality that it’s hard to get any accurate answers.

If you want to find out if an intervention works, try it. Do experiments on it. Carry out pilot projects and see what happens.

Don’t model just model something to see what’s “likely to happen”. What’s “likely to happen” in your model is going to be highly dependent on the assumptions you make, which in turn are likely to highly dependent on your view of the world. We tend to see what we expect to see, which can make modelling a particularly self-referencing exercise if we’re not careful. Experiments help us see the world as it really is.

But that’s not the full story. Don’t discount modelling just yet.

Joshua Epstein makes a great case for modelling in what is one of the best essays on modelling I’ve read so far. He has one particularly salient point about models that I’d never thought of before: that building models is done by everyone all the time; it’s just that people don’t build explicit models all the time.

By building an explicit model, listing down assumptions and having all the numbers laid out in front of you, at least you know what’s going into your thinking. Not building explicit models, on the other hand, doesn’t mean you’re not using a model, as Epstein explains:

It is just an implicit model that you haven’t written down.

The choice, then, is not whether to build models; it’s whether to build explicit ones. In explicit models, assumptions are laid out in detail, so we can study exactly what they entail. On these assumptions, this sort of thing happens. When you alter the assumptions that is what happens. By writing explicit models, you let others replicate your results.

Overall, the points Epstein makes on modelling are practical and well thought out, and certainly worth the read. Models are, in a nutshell, great at testing assumptions and straightening out thinking before any real decision is made, even if that decision is to go ahead on a pilot project.

Some thoughts on the thinking behind building things

I’m going to write a bit about my thinking process whenever I build things, whether it’s websites, VBA applications, or financial models. I’m not too sure if this is going to be a “oh that’s so obvious why is he telling me this?” piece, but if there’s one thing I’ve learned, one person’s normal behaviour may be another person’s exceptional one.

And I know you’re exceptional. So let’s get on with it.

My thinking process whenever I’m making things people use (i.e. goes into production) goes like this:

First, I ask myself (or the project champion/sponsor/requester): is this a one-off, or will there be many iterations?

If it’s a one-off, I can do away with making things beautiful and/or flexible. For one-off builds (again, whether it’s a webpage or financial model), most things can be hardcoded (e.g. unchanging interest rates, cell references and the like) and documentation doesn’t need to be too detailed (trust me on this though: documentation should still be done. People have a habit of waiting till the day after you forget what you did to ask you how you did it.)

If there are going to be many iterations, then I ask myself, what’s likely to change?

I once built a financial model containing over 30 options from which business users could pick and choose to determine what affected the final number. The model didn’t start of with all the options straight off the bat, and was in fact supposed to be a one-off. But after hardcoding and then manually changing what users needed multiple times within the same day (even though they “were quite certain” on what they wanted early on), I realised the flexibility of options, despite the longer initial development, was worth doing. I built the options on a very granular level so users could say, “I want option #1, #5, and #29” and have the affected numbers come up immediately.

Focusing on what’s more likely to change, and ignoring those that won’t, will allow you to save plenty of time. In that same model, I didn’t bother with ensuring formulas didn’t get broken when headings changed, because I knew they weren’t going to. (But I have certainly done this before; in some applications I’ve written, the source data had headings manually typed in, and almost every other week some variation of what was basically the same thing would get put in. I used regular expressions to ensure anything that looked like a header was treated as a header.)

Who and what is this report for?

Depending on the needs of the request, you could very well end up spending hours on something that should take only minutes. I have encountered many times someone coming to me for numbers in which I thought details were needed (or would be useful). I’d spend far too long extracting and formatting really granular data, when all the person needed was a “ballpark” figure.

And when you’re sending that consultant who is requesting for your company’s sales figures the information he needs, make sure you’re not giving him more than he needs. Does he really need to see the invoice numbers? Or customer IDs? Or costs?

Designing for maintenance

A few years back while taking the train I saw an engineering student’s lectures notes on “designing for maintenance”. I’m not from an engineering background, but because I happened to be mulling about on what software to use for a website I was developing, the concept of designing for maintenance struck me like Eric Clapton’s fingers on a guitar string.

Beautiful. Smooth. And oh-so-right.

The two main pieces of software that I had been considering were pretty distinct. One had a very developer-friendly, Linux feel. The other had a very polished, user-friendly Apple feel. The former catered to my nerd needs, while the latter catered my aesthetic aspirations (an aside: the very word ³aesthetic² is surprisingly pleasing to the eye and ear).

Though at first I was kept in a 50/50 bind between the two, after getting exposed to this idea of “designing for maintenance”, I realised that I really needed to go for the more polished and user-friendly one. Though I was helping to develop the website, I wasn’t going to be running it or doing the day-to-day maintenance of the site.

I think that maintenance of software (including spreadsheets) is something that’s missed by far too many people. If you know you’re not always going to be around to maintain the software, make sure it’s easy-to-understand and that everything’s well documented (even if it’s in an e-mail). Keep in mind that it isn’t all about you, but rather about the end-user, too.

Well, that’s about what I have for now. Would love to hear your thoughts on this. If you have any.

Business vs. IT

I was reminded today in a book I’m reading on visual analytics that the purpose of any analytical project is ultimately to make better decisions. Coming from a mixed business and IT background, I have had my fair share of IT vs. Business conundrums.

With my IT hat on, I’m always thinking about efficiency, optimisation, ease-of-use, and resource management. What questions are being answered, what problems are being solved, or what the ROI (to a certain extent) isn’t always on the forefront of projects (though almost always who’s the requester is).

With my business hat on, I’m always thinking about what business questions to solve, what information needs I have, and just plain old “getting the answer”, which IT isn’t always the most willing to help retrieve. I don’t care about how long IT takes (so long as its done yesterday) and how much effort they need to put in or how optimised the process is, I just want my questions answered so I can make better decisions. Isn’t that what technology-driven analytics is supposed to do

But, coming from a mix of business and IT backgrounds, I know the problems both sides face. IT needs more emphasis on understanding business needs, while business needs to understand IT constraints.

My current role has me more as a “business” person, and I don’t know if that’s a blessing or a curse. A blessing in that I’m given a lot more face time in business and finance discussions, allowing me an almost voyeuristic view of how the business makes its money. A curse in that IT treats me as any other “business” user who’s request trigger-happy and who doesn’t understand the IT side of things (and therefore who should be ignored most of the time).

It’s tough convincing IT to talk about the things that could potentially give pretty decent ROI when they don’t trust you.

“I’m one of you,” I tell them.

“Yeah, you sure are, buddy,” they reply, smiling, handing me a ticket number.