Discover more from AgileData
AgileData Newsletter #35
Magical Plumbing for Effective Change Dates
In this blog post we discuss how to handle change data in a hands-off filedrop process.
We use the ingestion timestamp as a simple proxy for the effective date of each record, allowing us to version each day’s data.
For files with multiple change records, we scan all columns to identify and rank potential effective date columns. We then pass this information to an automated rule, ensuring it gets applied as we load the data.
This process enables us to efficiently handle change data, track data flow, and manage multiple changes in an automated way.
The Patterns of Data Vault with Hans Hultgren
On the episode of the #AgileData podcast I discuss the patterns of Data Vault with Hans Hultgren.
Data vault is a data modeling technique that helps organisations to manage their data more effectively.
If you are looking for a data modeling technique that can help you to improve the quality, reduce the redundancy, and make it easier to analyse your data, then data vault is a good option to consider.
Common Team Design Patterns
There are a number of common team design patterns we can adopt when defining and iterating the structure of our data teams.
Some common team design patterns are:
Pawel Huyrn - Product Management
Join Murray Robinson and Shane Gibson as they engage in a conversation with Pawel Huryn about the ins and outs of product management. Learn what product management entails and how it differs from project management, with a focus on discovery versus delivery.
Key topics include:
🔸 Motivating teams for product success
🔸 Balancing discovery and delivery in empowered product teams
🔸 Transitioning into a product management role
🔸 Effective product management tools
Tune in to uncover valuable insights that will help you excel in the critical role of product management and understand how to effectively manage and empower your teams.
By default all the AgileData change rules will automagically run when a new set of data turns up on a History tile. We call this AutoSync.
But sometimes you may want certain tiles to be refreshed based on a different frequency.
So we built CustomSync to enable you to do just that.
CustomSync allows Data Magicians to create a schedule (dates & times) when Rules should run or when Tiles should be refreshed.
Latest AgileData Magician Guides
We are constantly adding new user guides to our doc site.
Here are the latest magician guides:
ADI is not only our company mascot she/he/they is also a core part of our App.
She helps you get the complex data jobs done in a simply magical way.
We love her some much we often proudly wear her on our chests.
If you want to join us, you can grab ADI from the ADT store.