Data Mesh
Tech Maven Geospatial has worked on several projects and R&D into Data Mesh
We also use Trinio https://trino.io/ecosystem/data-source & Kestra.io and Python DLT
also we are building into our Real Time and Data Integration Engine Data Mesh Capabilities – CDC Change Data Capture and Incremental Loading
We also heavily used PostgreSQL/PostGIS FDW – Foreign Data Wrapper to connect to many systems and write custom python procedural language function and views
Data mesh is about moving analytical data away from a monolithic data warehouse or data lake into a distributed architecture, allowing data to be shared for analytical purposes right at the point of origin. This concept was introduced by Zhamak Dehghani. If you’d like to learn more about data mesh, how to build one, you may like these resources:
-
Why Data Mesh? ft. Ben Stopford 5 [Podcast]
-
The Definitive Guide to Building a Data Mesh with Event Streams 3 [Blog]
-
Data Mesh 101 course 5 [course]
-
Guide to Data Mesh: Benefits, Examples, and Implementation 9 [Guide]
-
Why is Data Mesh, and How Does it Work ft. Zhamak Dehghani [Podcast]
Placing Apache Kafka at the Heart of a Data Revolution at Saxo Bank 3 [Podcast]
Learn More
https://thenewstack.io/apache-seatunnel-integrates-masses-of-divergent-data-faster/ Integrate massive data between Transaction DB, Cloud DB, SaaS,
Binlog with SQL-like code or Drag & Drop. https://seatunnel.apache.org/docs/2.3.2/category/source-v2/