Real-time flight data streaming, anomalies, predictions, and visualizations

Details

Join Sandon Jacobs on Monday, May 25th from 6:00pm hosted by Improving!

📍**Venue:** **Improving Office** 171 East Liberty St Unit 235 Toronto, Ontario M6K 3P6

**Directions (171 E Liberty St - Suite 235)** **By transit** Streetcars 504 and 509 both travel close to the office (less than 10 minute walk to the office from either), the lakeshore GO train is also a 5 minute walk from the office. **By car/parking** On street parking is available - there are a handful of paid parking spots directly in front of the entrance - with a large city parking lot across the street. **Entrance** The entrance to the office is beside the Bulk Barn entrance facing Hannah Street. There is an Improving logo on the door.

🗓 **Agenda:**

* 6:00pm – 6:30pm: Welcome, Food/Drinks & Networking * 6:30pm - 7:30 pm: **From Chaos to Contract: Governing Data Streams** by Sandon Jacobs

**💡 Speaker Details:** Sandon Jacobs, Senior Developer Advocate, Confluent

**From Chaos to Contract: Governing Data Streams**

So you’re committed to this path of data streaming with Apache Kafka. But, here’s a question: would you build a REST API and make users guess the request and response formats? Doing so opens a Pandora’s Box…

* What’s the domain model? * Name the operation… POST? GET? PUT? PATCH? * What does this error code even mean? * Is the documentation - if it even exists - accurate and relevant? *

Imagine the complex, repetitive logic every consuming system would need to implement to make sense of it all.

In a distributed, asynchronous system, these problems are magnified. Apache Kafka's flexibility—messages as simple bytes—is a huge strength, but it's also a major risk. While your event streams likely consist of some structured data (maybe JSON strings), enforcing structure, managing evolution, and basic validations aren’t a hard requirement. This leaves consumer applications to perform this preprocessing of every event - even if that event has no business value.

In this session, we'll define data contracts and how to enforce them at the source - the applications that produce events. We’ll cover the practice of using a schema registry - supporting popular serialization formats like Apache Avro and Google Protobuf - to design events in the terminology of our business domain. Because data structures change over time, let’s discuss safe schema evolution practices. Then we’ll utilize these contracts with schema registry-aware producer and consumer code. We’ll end our time by looking at how smart CI/CD pipelines and build-time checks can add an extra layer of defense against the costly problem of poison data.

It’s time to stop guessing and start governing. Join us to learn how to move validation upstream and transform your event streams into high-quality, discoverable data products. You’ll walk away with a practical blueprint for enforcing schema integrity and automating evolution—ensuring your data streams are a trusted asset.

\*\*\* DISCLAIMER We don't cater to attendees under the age of 18. If you want to host or speak at a meetup, please email [community@confluent.io](http://community@confluent.io/)