https://www.patreon.com/datameshradio (Data Mesh Radio Patreon) - get access to interviews well before they are released Episode list and links to all available episode transcripts (most interviews from #32 on) https://docs.google.com/spreadsheets/d/1ZmCIinVgIm0xjIVFpL9jMtCiOlBQ7LbvLmtmb0FKcQc/edit?usp=sharing (here) Provided as a free resource by DataStax https://www.datastax.com/products/datastax-astra?utm_source=DataMeshRadio (AstraDB) Transcript for this episode (https://docs.google.com/document/d/1AbcwrOmp5rbB1BFtQjpHZXj03ei-ur3ibgcY9G2DvEQ/edit?usp=sharing (link)) provided by Starburst. See their Data Mesh Summit recordings https://www.starburst.io/learn/events-webinars/datanova-on-demand/?datameshradio (here) and their great data mesh resource center https://www.starburst.io/info/distributed-data-mesh-resource-center/?datameshradio (here) In this episode, Scott interviewed Shane Gibson, CPO/Co-Founder of AgileData.io and Agile Data Coach. A few takeaways from Shane to start: - Agile methodology is about finding patterns that might work, trying them out and deciding to iterate or toss out the pattern. It's going to be hard to directly apply software engineering patterns to data but we should look for inspiration there and then tweak them. - Any time you look at a pattern you might want to adopt or evaluate if a pattern is working for you, ask yourself: will this/does this empower the team to work more effectively? - Applying patterns is a bit of a squishy business. Get comfortable that you won't be able to exactly measure if something is working. But also have an end goal in mind for adopting a pattern - what are you trying to achieve and is this pattern likely to help you achieve that? - Share your patterns to not only help others but to get feedback and maybe ideas to iterate your pattern further. Shane's last 8 years have been about taking Agile practices and patterns and applying them to data as an Agile Data Coach. And those patterns required a lot of tweaks to make them work for data. A big learning from that work is that when applying patterns in Agile in general, and specifically in data, each organization - even each team - needs to test and tweak/iterate on patterns. And that patterns can start valuable, lose value, and then become valuable again. Shane gave the example of daily standups drive collaboration as a forcing function but then lose value when that collaboration becomes a standard team practice. If there is a disruption to the team where collaboration is no longer standard practice, daily standups could get value again. So how do we apply these Agile concepts to data? Currently, Shane sees no real patterns emerging in the data mesh space - it is quite early as patterns often take 5-8 years to develop and data mesh is maybe 12 months in to even moderately broad adoption and is such a wide practice area, there are many practice areas that patterns will need to cover. But, that lack of patterns makes it quite hard for even those who want to be on the leading edge of implementing data mesh instead of the true bleeding edge - having to invent everything yourself is taxing work! So we need companies to really take existing patterns, iterate on them, and then tell the world what worked and what didn't. If people aren't sharing patterns, that's going to make it hard to adopt data mesh for many organizations. Shane believes that it will likely be pretty hard for many organizations - or at least many parts of large organizations - to give application developers in the domain the responsibility of creating data products. If your domains aren't already quite technically capable in building software products, it's going to be very hard for them to really handle data needs. So looking at domains that are using large, out-of-the-box enterprise platforms or SaaS solutions instead of rolling their own software, will they really have the capability to manage data as a product? If their domains don't...