For example, the following flow orchestrates multiple agents working in parallel using a foreach. The agent state is persisted through recursive steps, and the agent can continue along a …

You can find all Config examples in this repository. Let's start with a simple example that reads a JSON config file and uses it to configure @timeout for the start step:

Data in Local Files Similarly to Parameters, you can define a data file to include as input for your flow. Metaflow will version the file and make it accessible to all the steps directly through the …

In this example, we will schedule the stats.py workflow using the argo-workflows create command-line argument. This instructs Metaflow to schedule your flow on Argo Workflows …

In this example we re-run the stats.R workflow adding the --with batch command line argument. This instructs Metaflow to run all your steps on AWS batch without changing any code. You …

Instead of deploying the flow with a user-specific prefix, this will deploy the flow as example_project.prod.ProjectFlow. You will get a warning about missing production token if …

For example, you can execute many runs in parallel, each with its own set of parameters. This snippet demonstrates the pattern: It starts five concurrent runs, each with a different value of …

Our example demonstrates the idea by measuring the performance of a simple dataframe operation - grouping by hour - across three dataframe backends, Pandas, Polars, and DuckDB.

To illustrate config generation, consider this example that annotates each deployment with information about the current git branch through a custom parser function, git_info.

To illustrate what is possible, consider this example that uses OmegaConf to read a YAML Config, values of which you can selectively override for a run with a Parameter: