Data Patterns for Ephemeral "AI" Apps
I have been playing with Loveable a bit lately.
Well I call it playing, really what I do is when somebody hasn’t heard of it, I screen share and show Loveable building an app.
Well I say building, what it does it it creates a bunch of templated code and shows me a preview of a half working prototype app.
The other week I wrote about the impact these type of “AI” tools will have on the data domain.
TL:DR is data peeps will need to deal with the hoard of new apps organisations will adopt, apps that do one thing well. This will result in a explosion of our data collection problems trying to harvest the data from these apps.
Now you see it, now you don’t
A learned gentleman, GregData, over at the Practical Data Modeling discord community made a comment that made me stop in my tracks, he said:
Take this to the limit, when each job-to-be-done is accomplished by a bespoke, one-off ephemeral application, that is conjured, used, then immediately discarded.
(As an aside if your a seasoned data person and your not part of this community, you should join https://discord.gg/PtBWtJCJ).
My buddy ChatGPT says:
The term ephemeral means something that lasts for a very short time. It’s often used to describe things that are fleeting, temporary, or momentary—here now, gone soon.
This is a pattern I am familiar with, for our AgileData App we use a deploy and destroy pattern. When a user goes to login we deploy the App, then 15 minutes or so after they stop using it we destroy it, we delete it.
Of course all the logic for the app is persisted elsewhere, in our case its in Google Spanner. So our deployment “agent” reads that config in Google Spanner and hydrates the App based on that config.
Maybe we can rely on the ephemeral wave of apps following this model and persisting the data somewhere, but maybe not.
The current commentary around “vibe coding” infers the rigour our software brethren’s managed to get from the DevOps movement over the last decade made just have regressed, we have seen what happens when tools enable previously specialised work to be democratised, 5,000 dbt “models” with no actual data model anyone?
Patterns, Patterns (not everywhere but hopefully somewhere)
So when the hoard of new app developers use tools like Loveable and those tools move towards supporting ephemeral deployments what patterns might save our data arses?
Two come to mind:
Data Mesh
The original pattern for data mesh (as I saw it) was moving the data work back to the software teams that created the apps that generated that data.
Those teams get the data skills that we seem to love to hoard in seperate disconnected data teams. and those software teams do the data work.
We no longer treat data as exhaust, we treat it as fuel that powers those apps.
For this to happen, “AI” tools liek Loveable will have to add the features needed to model, store and persist data so it can be easily consumed and used.
We will still have problems with integration, but those tools should take care of the subject-oriented, nonvolatile and time-variant patterns.
If we are going to completely change the way we build and deploy apps, why wouldn’t we finally solve these problems that have been a constant challenge for at least 4 decades.
Data Fabric
While I have always called this term out as #Buzzwashing, maybe I was wrong as it is a good name for this second pattern.
A new set of data capabilities emerge, well the are not new, they are just iteration of stuff that has been around for a while.
A fabric end point allows the ephemeral apps to stream their data to this end point where it is stored / persisted. They all also push the context (metadata, schema, descriptions etc) to this end point.
The new breed of “AI” apps don’t have to worry about storing and persisting that data, they just push it off to something else that does that job for them. Think of it as an inverse Zapier pattern, push not pull.
If I was designing this, I would have the ephemeral apps push json events to the fabric, and I wouldn’t not validate that schema.
But then that raises the whole can of worms that the push left data contract pattern is trying to solve.
The reason I wouldn’t enforce data contracts is due to the new “vibe coding” wave. The new “citizen app developers” (i’m assuming somebody has already coined this god awful term) wont want to worry about all that data contract fluff, they just want to create and deploy something quickly.
Maybe the “AI” tools will automagically create the data contract as they push the data to the fabric, but then I am expecting the schema of those apps to mutate on a regular basis, just watch the way you can add a new field to a Loveable app but just saying “add a new file to capture date of birth” so I cant see how they can or will want to manage this problem.
That will leave the integration problem to the data specialists yet again.
The AI Startup Hoard
I am assuming many of the 1,000 new VC fuelled AI startups are working on solving this problem, and I hope they do.
Otherwise the data domain is even more fooked than I thought just a few weeks ago.
Whats in a name?
A lot.
Standardised naming, calling a thing the same thing every time, is one of the many values that come from consciously designing and modeling our data.
I don’t think Data Mesh and Data Fabric are the best names for the two patterns I describe above. They are the ones that just came to mind as I whacked these thoughts out this morning.
I am sure somebody will come up with better names for them, maybe they will issue an analyst whitepaper, a viral blog article or write a book.
I hope they do, I hope it goes viral and I hope the latest “AI” startup hoard pick up those patterns and embed them in these GenAI products that allow you to “Build software products, using only a chat interface.” and “ is your superhuman full stack engineer.” and this “fundamental shift in how software is built.” that will take us towards the “the last piece of software”.
I just hope do, I they don’t forget about the data.
I hope they reduce the reliance on centralised and disconnected data teams.
I hope they don’t make it worse by not even giving us exhaust fumes we can constantly sniff.