Flame v0.6: A Distributed Engine for Elastic Workloads

By Klaus Ma
Flame Release

Distributed computing is often treated as one problem. In practice, clusters run at least three different kinds of jobs, and each kind wants a different runtime.

Batch jobs are the familiar HPC and data-processing shape: a known set of tasks, often with gang start, all-to-all communication, and a clear start and finish. Kubernetes already has a strong answer here. Volcano is the batch system for that class of work: gang scheduling, fair sharing, queueing, and job-level lifecycle.

Workflow jobs are DAGs of steps: train, then evaluate, then publish; extract, then transform, then load. They need dependency tracking, retries per step, and long-lived orchestration. That space is served by systems such as Argo Workflows, Tekton, and Airflow. XFLOPS does not cover workflow jobs today.

Elastic jobs sit in between, and they are the jobs Flame is built for. An elastic job is a live session of many short, mostly independent tasks. The client keeps submitting work while the session is open. Parallelism rises and falls with demand. Tasks share session state, but they do not form a static MPI-style communicator and they do not need a DAG scheduler. Cold-start latency, executor reuse, isolation, and a cheap round trip matter more than gang start.

That split is not new in v0.6. Flame already uses sessions for elastic work. v0.6 is the release that makes that engine easier to run and easier to write against: Python Runner, a standalone object cache, flmadm install profiles, Helm, flmctl deploy, and cleaned-up Python and Rust SDKs.

What counts as an elastic workload

Elastic jobs show up wherever a service must fan out many similar tasks, keep some state nearby, and scale without restarting a batch job. Encoding and transcoding are in that class (many clips or shards, bursty arrival, little inter-task communication), even though Flame does not ship an encoding sample yet.

This post uses these samples from the v0.6.0 examples tree. The same tree also has examples/candle, examples/ps, examples/vllm, and examples/agents/sra.

  • Monte Carlo fan-out. Many independent samples, then a reduce. examples/pi is that shape; Value at Risk (VaR) and similar risk jobs are the same pattern with a shared market snapshot instead of random points in a square.
  • Crawlers. A frontier of URLs, concurrent fetches, and session-scoped headers or parsers: examples/crawler.
  • Reinforcement learning. Parallel rollouts, a shared policy or replay buffer, and frequent weight updates: examples/rl, including REINFORCE, replay buffer, and TorchRL DQN.
  • Agents. Tool calls, RAG lookups, and code execution as many short tasks behind one conversation or tenant: OpenAI and LangChain.
  • Generated-code execution and other AI sidecars. Sandboxed runtimes that must start quickly, stay isolated per session, and scale with request rate: examples/agents/scripts.

These jobs are a poor fit for a batch Job CRD and a poor fit for a workflow DAG. They need a session, not a queue of pods that all start together, and not a YAML graph of steps.

How Flame models an elastic job

Flame’s unit of work is a session. A session is a group of related tasks with scheduling, resource, and isolation boundaries. Clients can keep creating tasks until the session is closed.

Around that:

  • An executor is a runtime that hosts an application service for one session. A session may run many executors in parallel. Each executor is bound to that session only, so tasks reuse warm instances (model weights, crawler clients, RL environments) without sharing them across tenants.
  • A shim is how the executor talks to the service (host process, gRPC, and related adapters).
  • The object cache holds common data, Runner packages, and versioned objects so sessions share snapshots without stuffing large payloads into every task.

The session manager accepts client requests and schedules session resources. Executor managers connect back, pull tasks, and run them. When a session binds, the service can load shared state; when it unbinds, it releases that state.

That is the difference from Volcano. Volcano schedules batch jobs. Flame schedules elastic sessions.

Python services register one entrypoint; clients open a session and invoke it with typed payloads. The OpenAI agent example looks like this:

from flamepy.service import Session
from apis import MyContext, Question

with Session("openai-agent", ctx=MyContext(prompt="You are a weather forecaster.")) as session:
    output = session.invoke(Question(question="Who are you?"))
    print(output.answer)

Register the application first (flmctl register --file openai-agent.yaml in that example directory), then run the client against a live cluster. Each session stays isolated; later invoke calls on the same session reuse that session’s warm executors.

What v0.6 adds

Object cache

v0.6 ships a standalone object cache with versioned references, upload/download, incremental fetch, and fast paths for tabular, NumPy, and Arrow data. Session common data, Runner packages, and RL weight snapshots all go through this layer instead of riding on every task payload.

Clients use get_object, update_object, patch_object, upload_object, and download_object. That is what makes a distributed replay buffer or a shared Monte Carlo snapshot practical. See examples/rl/replay_buffer.

Python Runner

flamepy.runner.Runner packages a local Python project, registers a temporary application, and exposes functions, classes, or instances as remote services. Stateless functions default to autoscaling sessions. Stateful instances default to fixed sessions.

That is the path used by examples/pi/python, examples/rl, and examples/ps: keep the algorithm in ordinary Python, and let Flame own distribution.

Rust SDK and deploy

flame-rs provides typed clients, service macros, and object-cache helpers. flmctl deploy uploads a service binary and registers the application. examples/candle/based is the intended shape: local inference code plus a small service wrapper, then one deploy command.

Scheduling for mixed elastic sessions

v0.6 scheduling is aimed at many concurrent sessions sharing a cluster:

  • Dominant Resource Fairness (DRF), including GPU-aware DRF
  • Priority and resource requirements (cpu, mem, gpu)
  • A configurable policies list

drf is the elastic sharing policy. Local flmadm samples typically enable priority, drf, and gang; Helm defaults enable drf and gang. Gang in Flame still means one session bringing its executors up together. That is not a Volcano replacement. Closed HPC or data-processing jobs, queues, and job-level gang scheduling still belong on Volcano.

Install and operate

flmadm installs Flame with explicit profiles: --all, --control-plane, --worker, --cache, --client. There is a Helm chart for Kubernetes, Docker Compose for a first local cluster, and flmctl for applications, sessions, tasks, executors, and nodes.

Local Getting Started clusters use plaintext HTTP/gRPC. Session-scoped runtimes and executor recovery are part of the default runtime; mTLS is available when you harden component communication for a production cluster.

How the three job types fit together

Job type Shape System
Elastic Open session, many short tasks, reuse, scale with demand Flame
Batch Closed job, gang start, HPC/data-processing lifecycle Volcano
Workflow DAG of steps, dependencies, long-running orchestration Not covered by XFLOPS today

A production AI platform often needs more than one of these. Training a large model can be a Volcano batch job. Nightly pipelines can be a workflow. Serving agents, rolling out RL actors, crawling, encoding, and risk Monte Carlo belong on Flame.

v0.6 is not “Kubernetes, but for everything.” It is a dedicated engine for the elastic slice of that picture.

Try v0.6

If you already run Volcano for batch, keep it. Put the elastic jobs on Flame. Workflow remains an open slot; we are not pretending a session engine is a DAG engine.

Questions and issues: github.com/xflops/flame or support@xflops.io.

Related Posts