In a previous blog, we took a look at how the point of analytics projects can easily be lost in the quest for perfect data. But with that issue addressed, the next important step is to equip your team with the tools that allow the data to take care of itself, so they can take care of the clever stuff – which is coming up with great ideas of how to put the data to work and turn it into information the business can use.
For that, you want software that supports and automates the process of data extraction and manipulation – data warehouse automation (DWA) tools.
And why not – automation is a big deal these days, along with the related concepts of artificial intelligence and machine learning. What these concepts are driving at, is that software, often called robots (but not in the mould of C3PO), can do rote tasks far better than people can. Software, too, is tireless: it does it all the time.
DWA, according to The Data Warehousing Institute, is ‘using technology to gain efficiencies and improve effectiveness in data warehousing processes. Data warehouse automation is much more than simply automating the development process. It encompasses all of the core processes of data warehousing including design, development, testing, deployment, operations, impact analysis, and change management’.
That’s the long version. The short version is that it means ‘no more hand coding’.
But good DWA software comes with just one additional essential proviso. It has to be set up correctly so that when it is put to work, it does the right thing. The business landscape is littered with companies which do the wrong thing rigorously; larger companies are particularly susceptible to this issue, where a ton of effort and the utmost care will be taken to ultimately do something which adds no value. That’s a good part of why Gartner once said 50 percent of data warehouse projects will fail.
So, what does using DWA software achieve?
It removes overheads from developers and enables them to focus on engaging with the business, not the nuts and bolts of the technology. Best results come from data projects where developers are free to talk directly with the business, rather than through the business analyst or project manager.
With the software working in the background, the developers are free to take those conversations and build solutions by generating code, applying best practice frameworks – including (but not exclusively) agile - and methods for constructing analytics solutions.
With the data modelling, configurations and other tasks automated by the DWA software, there is a further benefit of consistency in the data preparation. This benefit can’t be underestimated: ask five consultants, each of whom is a recognised expert at the task, to prepare data and you’ll get five different outcomes.
None will be right or wrong – but if the designer leaves, it can be difficult or impossible to unpick what they have done. Not with DWA. As implied, it automatically takes care of one of the least interesting/fun aspects of coding, too – documentation.
In the next blog, we’ll take a look at something many companies encounter soon after they start seeing benefits from analytics initiatives: the ability to scale.