The InfoQ eMag: Modern Data Architectures, Pipelines, & Streams – InfoQ.com
In this eMag on “Modern Information Architectures, Sewerlines and Streams”, you’ll find up-to-date case studies and real-world data architectures from technology SME’s and leading data practitioners in the industry.
“Building & Operating High-Fidelity Data Streams” by Sid Anand highlights the importance of reliable plus resilient information stream architectures. He talks about how to create high-fidelity loosely-coupled data stream solutions from the ground up with built-in capabilities such as scalability, reliability, and operability using messaging technologies like Apache Kafka.
Sharma Podila’s article on “Microservices to Async Processing Migration at Scale” emphasizes the significance of asynchronous processing and how it can improve the availability of a web service simply by relieving backpressure using Apache Kafka by implementing a durable queue between service layers.
“Streaming-First Infrastructure for Real-Time Machine Learning” simply by Chip Huyen nicely captures the benefits of streaming-first infrastructure with regard to real-time ML scenarios like online prediction and continual learning.
And “Building End-to-End Field Level Lineage regarding Modern Information Systems” authored by Mei Tao, Xuanzi Han plus Helena Muñoz describes the data lineage as a critical component of data pipeline root cause and impact analysis workflow, and how automating lineage creation and abstracting metadata in order to field-level helps with the root cause analysis efforts.
We at InfoQ hope that you find the particular value in the articles and other resources shared in this eMag and potentially apply the design patterns and techniques discussed, in your own data architecture projects and initiatives.