When I was at the Tableau Conference 2018 in New Orleans (which we sponsored, btw) I had a chance to talk to other Tableau technology partners and finally started understanding the driving cause behind the proliferation of cloud-based data warehouses offered by various vendors (such as Panoply, Snowflake, or 1010data). Previously, I assumed that the main value proposition of cloud warehouses is cost reduction and the main target audience for such solutions are IT departments. However, it turns out that the main target audience is actually business (i.e. non-IT) departments and the main selling point is manageability and independence from the internal IT. Why? Because the internal IT is becoming increasingly inefficient in enabling business departments to achieve their goals. Business users are not happy with the pace of delivery, responsiveness, and the price tag provided by the internal IT. Up to the point when they start looking around.
This wasn't news to me, as I run a company that helps business users to get a degree of independence from IT. Rather it looked like yet another confirmation of a tectonic shift slowly happening in organizations.
In the vast majority of non-technology organizations internal IT departments have a supporting role and are typically viewed as cost centers, rather than revenue generators. Therefore, cost minimization becomes the driving force behind centralization of IT administration, development and procurement. On one hand, centralization allows achieve the economies of scale. On the other hand, centralization effectively creates internal monopolies. The larger is the organization, the more perceivable are the downsides of such monopoly. Imagine, that in a remote town with 50K population (size of a large corporation) there is only one computer shop (let's assume it's still the 90s and there is no Amazon) and no way to purchase anything computer-related from another town/city. What would happen to the level of service provided by that shop? With no competition, the prices will go up, and service quality will go down because a monopoly has no incentive to do it otherwise. The same happens with corporate IT departments -- they concentrate too much bureaucracy with no efficient feedback loop in place that would ensure constant improvement.
I believe, this trend has nothing to do with personal and professional qualities of people working in internal IT departments. Many of them have both skills and goodwill to do best for their internal customers. I suspect it's some kind of institutional problem that has its roots in traditional management thinking.
Although, there is some experimentation going on. I've seen an interesting attempt of a large bank to build its IT division in a form of a service supermarket for business users where business departments have the final word on what kind of business applications they need and when, and IT only takes care of security, maintenance and administration. Probably, the internal IT should ideally be the preferred vendor, but not the only option available to internal customers.
Analytical systems in general and data warehouses in particular are subject to the conflict of interests between the business and the IT side probably more than any other kind of enterprise system. They are predominantly demanded by business users, however they are very complex from a technical perspective. To make things worse, data warehouses are frequently viewed as important but not strategically important to the organization. Which lets the implementation deadlines slip by months and sometimes years.
The idea of cloud data warehouses that target business users makes sense to me and signals about a bigger trend going on in the industry that aims to overcome inefficiencies created by over-centralization of IT.
