During my career I’ve heard this countless times and to some extent it’s just taken as granted that you should always use the right tool for the job. Isn’t this obvious after all? Why would you knowingly choose the wrong tool?
But the conversations often miss the realities surrounding the choice. Decisions are not made in isolation. What does best even mean? Fastest, easiest to implement, doesn’t require consultants? Instead there’s a myriad of influencing factors:
- Currently available skills
- Currently available tools
- Time frames and deadlines
- Availability and pricing of new tools
- Learning curve for new tools
- Cost vs benefit / ROI
- Expected lifetime of the solution
In this day and age there’s an almost unlimited number of ways to solve a particular problem. You want to extract data from a database, process it and present the results somewhere? If you’re an analyst you might do this in Excel, a programmer might use python, a business user might use a robot and so on. All of these are possible.
A growing trend also seems to be that tools and platforms increasingly have overlapping functionality making the decision even less clear.
Tableau has Prep which is basically an ETL-lite tool, robots can pull data directly from databases, almost everything can send mails, parse JSON and XML and connect to APIs.
This often comes up when talking about Robotic Process Automation, or RPA. I’ve used many ETL tools like Alteryx, KNIME and RapidMiner and they are great at extracting and processing data, but RPA can also be used to move and process data between systems. I wouldn’t recommend using RPA to pull millions of rows of data but it could be done.
Similarly a programmer might point out that a user interface could be automated using python and selenium at a fraction of the cost of an Enterprise RPA solution. This is technically true but if you don’t have a dedicated team of python experts in your organisation what does that help?
There is always a grey area when the problem could be solved in many different ways using an array of tools and platforms each with their own trade-offs.
In my experience from business the main limitations are both financial ones and available skill sets.
Let’s say your company has invested a significant sum in RPA. You have a direct connection to the database and need to extract and process a large volume of data. You know that an ETL platform would be better but how are you going to persuade your manager to invest? If you do invest do you then have right skills to get the full value from your investment? How long will the ‘best’ solution take to get up and running?
Instead of asking if this is the best tool for the job, ask whether compared to using the ‘right’ tool:
- Are there significant risks to success
- What functionality will I potentially miss
- How would the process differ
- Can it be easily maintained/extended
- Will it last the expected lifetime of the process
Life is full of compromises and you often need to make do with what you have. When you only have access to a hammer everything needs to look like a nail 🔨🔨