2024 the year of the Intelligent Data Platform
Lets see how true that is in 2025
Originally published on 02/01/2024
(Moving this post here as I reorganise where we are publishing things)
Intelligent Data Platform
I read the term Intelligent Data Platform in a post by Sanjeev Mohan over the holiday break.
I think 2024 will be the year of the Intelligent Data Platform.
I have already written about my thoughts in using “AI” in data platforms.
https://agiledata.io/blog/agiledata-product/the-3-patterns-of-agiledata-ai/
I think of three patterns for the use of AI in data platforms:
Ask AI
Assisted AI
Automated AI
Let me use a quick example from the medical domain to highlight the differences between the three AI patterns.
Ask AI
A doctor or medical researcher queries an AI system for patient data or research studies. For instance, they might ask, “How many patients with Condition X improved with Treatment Y in the last year?” The AI system interprets this query, accesses the relevant medical databases, executes the necessary data extraction (like SQL queries), and presents the results.
Assisted AI
In a clinical setting, an AI system assists in diagnosing patients. Based on the input symptoms, medical history, and test results, the AI system suggests possible diagnoses and recommends further tests or treatments. The final decision, however, remains with the healthcare professional who can accept, modify, or reject the AI systems suggestions.
Automated AI
In medical laboratories, AI-driven systems automatically analyse samples and generate reports. For example, the AI system can interpret X-rays or MRI scans, identifying anomalies and patterns without human intervention. These reports are then directly sent to the respective healthcare professionals for review and further action.
In 2024 we will see legacy data platforms start to add these three AI patterns to their platforms and a few new data platforms emerge that are Intelligent “native’.
The good news is that these patterns / features are actually useful for people who work in the data domain, rather than increasing the complexity of the data platforms like the modern data stack did, these will reduce the complexity.
They reduce the level of effort required to get a data task done, allowing us to do it quicker.
They reduce the cognitive load required to do complex data work, moving as ever forward to true “self-service” data work.
They often automate the data work altogether, meaning we never have to do it manually again.
Bring on 2024 and the Intelligent Data Platforms I say.