The Right Tool for the Job

Someone comes up to you and says “So what’s the best [programming language | text editor | IDE | operating system | browser | database solution | version control system]?” You slouch back in your chair, look blankly up at the ceiling, and a small frown starts to form on your face. You say the thing that anyone in your scholarly position would say, “Well, that depends…”

And depending on the inexperience of our dear friend, the rest of the conversation may go like:

Friend: “Yes, I know that. But, what I mean is, what’s the best?”

You: “It really depends, what do you want to use it for?”

Friend: “That’s not important, I just want to know what’s the best… like, what would you use?”

You: “Well I use <SomeGreatTool>, but that doesn’t necessarily mean-”

Friend: “Ah! that’s what I thought… everyone uses that”

You: “Wait, just tell me first, what do you want to do with it?”

Friend: “I want to use it in this <totally plausible application where it would, coincidentally, work quite well>”

You: “Oh I see. Yes, well it would do a pretty good job then.”

Friend: “Great! <SomeGreatTool> is the best. Thanks for your help.”

You: “Sigh. Yes. It’s just the best.”

And our friend proceeds to develop an unhealthy attachment to the “best” tool for the job. This is not a disastrous scenario, but I’m looking to discuss  a thorn or two which crops up when we develop a workflow.

Recently, I attended ScaleConf in Cape Town. A conference centered around, you guessed it, scaling. One aspect I particularly enjoyed was the myriad of approaches being employed which address scaling. One talk I didn’t fancy as much, however, was almost entirely devoted to demonstrating and discussing the value of running MapReduce on Hadoop. Don’t get me wrong, I can appreciate that MapReduce has a lot to offer in particular contexts (batch-oriented processing, for instance). MapReduce, however, is highly unsuitable for real-time, streamed based data processing. Enterprises have realized that their big data analytics is going to require a hybrid approach of computing models. To this end, it grinds my gears a bit when something like the MapReduce paradigm is presented as a fantastic tool without mentioning its limitations. It’s about the right tool for the job. This led me on a train of though that inspired the rest of this post.

We like absolutes. We like being confident in the things we are familiar with. This is especially characteristic of competent programmers. So what leads us to place our trust in our preferences? I believe, to some extent, it’s a combination of the following:

  • familiarity – “I’ve used <Tool> for the last 5 years”
  • experience – “I’m awesome at <Tool>”
  • time constraints – “I don’t have time to learn <OtherTool>”
  • context – “My friends/colleagues/Good Guy Greg uses <Tool> and tell me it’s awesome”
  • accessibility – “<Tool> is free, and <OtherTool> is too expensive”

Unfortunately, while these factors have merit, they can help us to coax ourselves into believing that there is the <Tool>, and nothing but the <Tool>. Sometimes we’ll go to great extents to defend our preferences, and derive satisfaction when others adopt our stance. This is not “wrong” per se, but we should be wary of  limiting ourselves to a particular stance. How can this be made practical? Think to yourself: “What are the strengths, and limitations, of this <Tool>?” Is it appropriate for the particular domain of the problem you are looking at? Is there something out there that might work better? Here, you would be well-off to go do a little research or ask someone with expertise in that area. Even if you come across something which looks like it might work, and which subsequently fails, you now have an understanding of what you are not looking for. Such experience is favourable to taking <MyFavouriteTool> and forcing it to do something that it’s not catered towards.

This process might sound like a waste of time – I would argue that it’s not only going to save you time in the long run, you’re also investing in yourself by diversifying your skill set. In the same breath, an illustration: if your perspective is limited to all things Mac or Linux, don’t go running for the stake when someone mentions how great Microsoft Visio works for them. Give them the benefit of the doubt.  Better yet, try it if it sounds useful! Bar financial constraints, you don’t have much of an excuse. So when you next have the opportunity, and without spending an inordinate amount of time, go through the trouble of finding the right tool for the job.

No comments yet.

Leave a comment

Leave a Reply