The origins, purpose, and practice of data observability

Data Observability (DO) is an emerging category that proposes to help organizations identify, resolve, and prevent data quality issues by continuously monitoring the state of their data over time. This talk is a deep dive into DO, starting from its origins (why it matters), defining the scope and components of DO (what it is), and finally closing with actionable advice for putting observability into practice (how to do it).

We’ll rigorously define data observability to understand why it is different from software observability and existing data quality monitoring. We will derive the four pillars of DO (metrics, metadata, lineage, and logs) then describe how these pillars can be tied to common use cases encountered by teams using popular data architectures, especially on cloud data stacks.

Finally, we’ll close with pointers for how to put observability into practice, drawing from our experience helping teams across sizes, from fast-growing startups to large enterprises, successfully implement DO. Successfully implementing observability throughout an organization involves not only using the right technology, whether that be a commercial solution, an in-house initiative, or an open source project, but implementing the correct processes with the right people responsible for specific jobs. Talk participants can expect to leave with new concepts to understand how DO can help their organizations and ideas for how to implement DO.

Kevin-Hu-Metaplane.jpg

Kevin Hu

Co-founder

metaplane-logo-transparent.svg

What is the cost to attend and watch the virtual sessions?

Data Team Summit is always free and open for all to attend.

What is Data Teams Summit?

This year, we've taken the peer-to-peer empowerment of data teams one step further and formally transformed DataOps Unleashed into Data Teams Summit to better reflect our focus on the teams and individuals running, managing, and monitoring data pipelines.

Data Teams Summit is an annual, full-day virtual conference, led by data rockstars at future-forward organizations about how they're establishing predictability, increasing reliability, and creating economic efficiencies with their data pipelines.

We'll be back again in 2024 and look forward to another round of data team focused conversations.

Who comes to Data Teams Summit?

Data professionals and experts including data engineers, administrators, architects, analysts, AI/ML professionals, and relevant data technology leadership.

Join us for sessions on:

  • Data teams & best practices
  • Data pipelines & applications
  • DataOps observability
  • Data quality & data governance
  • Operations observability
  • MLOps
  • Data modernization & architecture
  • Biz/FinOps observability

Interested in speaking at Data Teams Summit 2024?

Submit your talk proposal at datateamssummit.com/cfp/ or send us a note to astronaut@solutionmonday.com.

Would you like to stay updated on agenda announcements for Data Teams Summit 2024? Sign up below!

Interested in speaking at Data Teams Summit or participating as a sponsor?

Please contact mike@solutionmonday.com.