Data Preparation is the new Business Intelligence

What's now happening with Data Preparation bears a strong resemblance with the state of Business Intelligence in the late 90s. There is a number of similarities between the data prep market now and BI back then:

• Emerging technology
• Designed for the convenience of business users (which is very atypical for enterprise software, btw)
• Introduces a degree of independence from IT departments (which also means a lower burden for the latter)
• Low general awareness of this type of tool in the market
• General underestimation of the proposed value ("why can't we just use Excel for this")
• A rare "System-to-human" kind of enterprise application. Not "Human-to-system" (front-end) or "system-to-system" (back-end) software commonly found in the enterprise.
• Designed and suited for mass user adoption, rather than for use by 1-2 people in a department or even entire company.
• From a licensing perspective the total cost of ownership is within $40-100 per month per user.

Having this analogy, it may be possible to predict certain future trends for the Data Preparation industry:

• User adoption will broaden slower than anticipated by Data Prep vendors. It will probably take 10-15 years to reach the "late adopter" stage and market saturation.
• User adoption will be wider than first customers (early adopters) envision it.
• There will be attempts for "Data Prep standardization" across organization but they will fail, just as BI standardization failed.
• Enterprise administration/governance features will become necessary sooner rather than later.
• Authoring will shift from desktop clients to web-browsers.
• Expensive software (>$100/mo/user) will be squeezed out from the market by competition.
• There will be a wave of consolidation when major Data Prep companies are acquired by big enterprise vendors.

For me, a confirmation of this similarity is that most of multi-user installations of EasyMorph tend to grow over time doubling in about 2 years, on average. We've recently seen a customer that requested 30 user licenses just for one business line. This tells me that although data prep tools are close to ETL systems from a technical standpoint, from a user adoption perspective they clearly resemble Business Intelligence and Data Discovery applications.