Will "AI" actually keep its promise in the data domain?
Because after 30 years none of the "new" technologies innovations have
I am old enough to remember a few of the data disruption waves, each made the data world a better place, but none delivered on their original promises.
Remember the promise of when BI tools would let you report directly off a source system with no need to have the data in a Data Warehouse?
Remember the promise when OLAP cubes would remove the need for a relational data base as your core data warehouse storage?
Remember the promise of Hadoop letting you dump all you data in one place and removing the need to model it and transform it before you needed it?
In this wave there are lots of "AI" promises.
One promise is that "AI Agents" will again remove the need to create a data warehouse.
For me a data warehouse is not a technology or architecture that has a centralised data base or centralised data team.
It is when you apply these patterns to data:
You surface an aligned view of the data that belongs in the same domain, aka subject-oriented.
You surface a combined view of the data when it is the same or should be the same, aka integrated.
You surface all changes over all time, aka time-variant.
You surface a view of the data that never changes that actual data, aka non-volatile.
Don’t get me wrong I believe that LLM's and GenAI is the next iPhone moment, it is the next cloud wave.
I say this because I use those tools so often everyday now, that it has changed my life the way those wave did.
And I really hope it does remove the need for us to do all the data work we have had to do for decades.
Or it changes the way we do it, for lets be honest with ourselves we haven’t really iterated the core patterns of how we do data work in 30 odd years.
But until the pundits of this new "AI" wave publish clear patterns of how those 4 core data warehousing patterns are solved using "AI" or even better completely replaced and no longer needed, I for one am going toto keep yelling:
"show me the meat and potatoes, as I don't think you can keep your latest promise".
And hope somebody does actually invent or innovate a new data way of working. Its about time things changed!
How about you?