Bring Your Own AI Agent to Apache Airflow

Motivation

Managed AI agent runtimes are becoming the standard way to deploy production AI agents. Amazon Bedrock AgentCore, Google Vertex AI Agent Platform and Azure AI Foundry let you package an agent in a container while the cloud provider manages scaling, availability and infrastructure for you.

Until now, there was no native way to orchestrate those agents from Apache Airflow.

This project bridges the gap between managed AI agent platforms and modern data orchestration.

Over the last few weeks, I contributed native Apache Airflow operators for the three major managed AI agent platforms, allowing Airflow DAGs to create, invoke, update and delete production AI agents just like any other Airflow resource.

This allows AI agents to become first-class citizens in data pipelines, reusing everything Airflow already provides: scheduling, retries, monitoring, dependencies, secrets and governance.

All three platforms follow the Bring Your Own Container (BYOC) model, allowing teams to build agents with any framework, model or toolset while relying on the cloud provider for production deployment. Airflow now becomes the orchestration layer for those managed agents.

To demonstrate what this enables, I also built Agentic Airflow, an end-to-end example where a failed production DAG automatically invokes an AI agent that analyzes logs, inspects the relevant source code, identifies the likely root cause, proposes a fix, opens a draft GitHub Pull Request and sends a Slack notification for human review.

The project includes:

• Native operators for Amazon Bedrock AgentCore

• Native operators for Google Vertex AI Agent Platform

• Native operators for Azure AI Foundry

• Example DAGs for every provider

• A complete Agentic Airflow proof of concept

Everything is open source.

Resources

• GitHub Repository: Agentic Airflow Demo

• Amazon Bedrock AgentCore PR

• Google Vertex AI Agent Platform PR

• Azure AI Foundry PR

• All Apache Airflow Contributions