This post was originally published on Info World
One primary loophole was that the data lake was built and maintained by a separate engineering or analytics team, which didn’t understand the data in depth as thoroughly as the source teams. Typically, there were multiple copies or slightly modified versions of the same data floating around, along with accuracy and completeness issues. Every mistake in the data would need multiple discussions and eventually lead back to the source team to fix the problem. Any new column added to the source tables would require tweaks in the workflows of multiple teams before the data finally reached the analytics teams. These gaps between source and analytics teams led to implementation delays and even data loss. Teams began having reservations about putting their data in a centralized data lake.
Data mesh architecture promised to solve these problems. A polar opposite approach from a data lake, a data mesh gives the source team ownership of the data and the responsibility to distribute the dataset. Other teams access the data from the source system directly, rather than from a centralized data lake. The data mesh was designed to be everything that the data lake system wasn’t. No separate workflows for migration. Fewer data sanity
— Read the rest of this post, which was originally published on Info World.
