Rephonic
Artwork for The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

Astronomer
Airflow
Apache Airflow
Data Engineering
Machine Learning
Astronomer
Kubernetes
DBT
Dataops
Data Pipelines
Data Analytics
Artificial Intelligence
Big Data
Airflow 3
Data Quality
Snowflake
Medallion Architecture
Data Governance
Texas Rangers
Airflow 3.0
Large Language Models

Welcome to The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI— the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-... more

PublishesWeeklyEpisodes104Founded8 years ago
Number of ListenersCategory
Technology

Listen to this Podcast

Artwork for The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

Latest Episodes

Skimlinks runs a reporting platform that serves around 2,000 weekly publisher users, and the data infrastructure behind it runs on Airflow. In this episode, Julian Larralde, Director of Data Engineering at Skimlinks, walks through the stack, the migr... more

YouTube

JLR is the UK's largest automotive manufacturer, behind brands like Range Rover, Jaguar, Defender, and Discovery. In this episode, Najeeb Sulaiman, Senior Data Engineer at JLR, walks through how Airflow orchestrates data across manufacturing, supply ... more

YouTube

Running Airflow at the scale of a national retailer means more than just scheduling. It means giving non-engineers a path to ship DAGs, and classifying thousands of runs to know which ones need attention. In this episode, Mateus Ferreira, Senior Data... more

YouTube

In the data engineering world, the difference between a pipeline that works and one that's truly production-ready often comes down to a handful of deliberate decisions. William Orgertrice III, Data Engineer at Cargill, joins us to share the DAG desig... more

YouTube

Key Facts

Accepts Guests
Accepts Sponsors
Contact Information
Podcast Host
Number of Listeners
Find out how many people listen to this podcast per episode and each month.

Similar Podcasts

People also subscribe to these shows.

Recent Guests

Julian Larralde
Director of Data Engineering at Skimlinks
Skimlinks
Episode: Managing a Customer Analytics Platform with Airflow at Skimlinks
Najeeb Sulaiman
Senior Data Engineer at JLR
Jaguar Land Rover (JLR)
Episode: Building a custom Tableau provider for Airflow at JLR
Mateus Ferreira
Senior Data Engineer at Luiza Labs
Luiza Labs (Magazine Luiza)
Episode: Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira
William Orgertrice III
Data engineer at Cargill
Cargill
Episode: Enhancing DAGs for Data Processing with William Orgertrice III at Cargill
Shri Hegde
Data and AI engineer and Airflow champion
Astronomer Champions program
Episode: Getting Into Data Engineering with Shrividya Hegde, Data and AI Engineer
Filip Kunčar
Platform Director at ShipMonk
ShipMonk
Episode: Orchestrating DBT With Cosmos and Airflow with Filip Kunčar at ShipMonk Product Development
Buğra Öztürk
Senior Data Engineer at Mollie, Committer and PMC member on the Apache Airflow project
Mollie
Episode: Building Airflow CTL with Buğra Öztürk at Mollie
Kaxil Naik
PMC member for Apache Airflow; Senior Director of Engineering at Astronomer
Astronomer
Episode: Introducing Airflow’s Common AI Provider with Pavan Kumar Gopidesu and Kaxil Naik
Pavan Gopidesu
Lead Data Engineer at Experian; PMC member for Apache Airflow
Experian
Episode: Introducing Airflow’s Common AI Provider with Pavan Kumar Gopidesu and Kaxil Naik

Host

Kenton Danis
Host of The Data Flowcast; data-focused podcast host with emphasis on Airflow and data platforms.

Reviews

4.9 out of 5 stars from 42 ratings
  • Great way to learn about what data teams are doing in 2024

    [disclaimer: review from former podcast host that has since been replaced by much better voices!]

    If you’re wondering why a data platform and team is important to everyone - from the World Series champion Texas Rangers to worldwide casinos like Wynn to financial services companies - look no further. The techniques

    Apple Podcasts
    5
    Pqdthorne
    United States2 years ago
  • Amazing!

    Loved the show, it helped me wrap my head around most of Airflow’s concepts and how to use different constructs correctly.

    The people invited were top quality and very involved in Airflow development and usage.

    Apple Podcasts
    5
    JeanPieroHM
    Germany7 years ago
  • Helped me a lot as I began exploring Airflow

    I had kept hearing folks talk about airflow, and stumbled across the astronomer podcast as I began trying to learn more. I’ve been quite impressed so far, and am hoping to add airflow to my toolkit

    Apple Podcasts
    5
    ascloyd
    United States8 years ago
  • Use cases

    Very helpful to hear how industry leaders are using Airflow!

    Apple Podcasts
    5
    Wrecklessshiv
    United States8 years ago
  • Great primer for Data Engineering

    If you're new to the field and want to learn techniques, tools, and best practices, this is a great place to start.

    Apple Podcasts
    5
    Appmagnet
    United States8 years ago

Listeners Say

Key themes from listener reviews, highlighting what works and what could be improved about the show.

Great primer for data engineering and Airflow concepts.
Sponsors and tooling discussions are seen as valuable context for production environments.
Listeners praise practical, real-world usage insights from heavy hitters.
High-quality guests and in-depth technical discussions are common positives.
Shows helpful for beginners to understand tools and workflows in data teams.

Chart Rankings

How this podcast ranks in the Apple Podcasts, Spotify and YouTube charts.

Apple Podcasts
#130
Switzerland/Technology

Talking Points

Recent interactions between the hosts and their guests.

Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira
Q: What led you to develop the YAML wrapper to generate DAGs for non-engineers?
The wrapper is designed to simplify extraction and load processes for business users, bridging data from databases into Lake and BigQuery, with metadata-driven governance and checkpoints to ensure safe, auditable pipelines; it also enables consistent resource configuration across pipelines.
Orchestrating 2,000 Airflow pipelines at Luiza Labs with Mateus Ferreira
Q: How did Airflow become the heart of your data platform?
Mateus explains they started with Cloud Composer on GCP for orchestration but needed greater control and cost efficiency with about 2,000 pipelines, which led them to shift toward Magalu Cloud and Kubernetes, making Airflow the central orchestrator of ingestion and processing.
Building a custom Tableau provider for Airflow at JLR
Q: What future Airflow features would you like to see to improve observability and lineage?
I'd like native integration testing in Airflow to catch dependency issues before deployment, and better native observability with OpenTelemetry plus true lineage support so you can see which DAGs drive which datasets and dashboards.
Building a custom Tableau provider for Airflow at JLR
Q: What does your CI/CD DAG validation look like, and how does it help prevent production issues?
We integrate DAG validation into CI/CD to catch issues like top-level code, bad imports, and Airflow 3 compatibility before a DAG reaches production. This reduces runtime errors and ensures downstream users don't face unexpected failures, serving as a robust gate before deployment.
Building a custom Tableau provider for Airflow at JLR
Q: Tell me a little bit more about why you built a custom Tableau provider instead of using the community provider?
We started with the community Tableau provider, but its authentication model with PAT could not support parallel connections across multiple projects. This limitation caused management and maintenance challenges as teams duplicated code and ran into token conflicts. A custom provider lets us use PAT in a way that scales across projects and maintains security and consistency.

Audience Metrics

Listeners, social reach, demographics and more for this podcast.

Listeners per Episode
Gender Skew
Location
Interests
Professions
Age Range
Household Income
Social Media Reach

Frequently Asked Questions About This Podcast

What is This Podcast about and what kind of topics does it cover?

This show centers on Apache Airflow, data engineering, and AI orchestration, with episodes featuring hands-on practitioners, platform leaders, and open-source contributors. Conversations span real-world deployment in enterprises, evolving tooling (CT L, AI providers, event-driven scheduling), governance and data quality, and practical patterns for building reliable pipelines at scale. A recurring thread is applying Airflow to production-centric use cases—from securing and auditing workflows to integrating AI agents and MLOps components—often with insights on community involvement, upgrade paths, and sponsor-supported managed services. The format typically blends technical deep-dives with pragmatic lessons, making it valuable for engineers, ... more

Where can I find podcast stats for this podcast?

Rephonic provides a wide range of podcast stats for this podcast. We scanned the web and collated all of the information that we could find in our comprehensive podcast database. See how many people listen to this podcast and access YouTube viewership numbers, download stats, audience demographics, chart rankings, ratings, reviews and more.

How many listeners does this podcast get?

Rephonic provides a full set of podcast information for three million podcasts, including the number of listeners. View further listenership figures for this podcast, including podcast download numbers and subscriber numbers, so you can make better decisions about which podcasts to sponsor or be a guest on. You will need to upgrade your account to access this premium data.

What are the audience demographics for this podcast?

Rephonic provides comprehensive predictive audience data for this podcast, including gender skew, age, country, political leaning, income, professions, education level, and interests. You can access these listener demographics by upgrading your account.

How many subscribers and views does this podcast have?

To see how many followers or subscribers this podcast has on Spotify and other platforms such as Castbox and Podcast Addict, simply upgrade your account. You'll also find viewership figures for their YouTube channel if they have one.

Which podcasts are similar to this podcast?

These podcasts share a similar audience with this podcast:

1. The Real Python Podcast
2. The Pragmatic Engineer
3. Practical AI
4. Apple News Today
5. Planet Money

How many episodes of this podcast are there?

this podcast launched 8 years ago and published 104 episodes to date. You can find more information about this podcast including rankings, audience demographics and engagement in our podcast database.

How do I contact this podcast?

Our systems regularly scour the web to find email addresses and social media links for this podcast. We scanned the web and collated all of the contact information that we could find in our podcast database. But in the unlikely event that you can't find what you're looking for, our concierge service lets you request our research team to source better contacts for you.

Where can I see ratings and reviews for this podcast?

Rephonic pulls ratings and reviews for this podcast from multiple sources, including Spotify, Apple Podcasts, Castbox, and Podcast Addict.

View all the reviews in one place instead of visiting each platform individually and use this information to decide if a show is worth pitching or not.

How do I access podcast episode transcripts for this podcast?

Rephonic provides full transcripts for episodes of this podcast. Search within each transcript for your keywords, whether they be topics, brands or people, and figure out if it's worth pitching as a guest or sponsor. You can even set-up alerts to get notified when your keywords are mentioned.

What guests have appeared on this podcast?

Recent guests on this podcast include:

1. Julian Larralde
2. Najeeb Sulaiman
3. Mateus Ferreira
4. William Orgertrice III
5. Shri Hegde
6. Filip Kunčar
7. Buğra Öztürk
8. Kaxil Naik

To view more recent guests and their details, simply upgrade your Rephonic account. You'll also get access to a typical guest profile to help you decide if the show is worth pitching.

Find and pitch the right podcasts

We help savvy brands, marketers and PR professionals to find the right podcasts for any topic or niche. Get the data and contacts you need to pitch podcasts at scale and turn listeners into customers.
Try it free for 7 days