Skip to main content
sidecar/ai_train/providers/, sidecar/ai_train/app.py This page is about the Modal app behind the sidecar (how the sidecar deploys and stops it is on Modal app lifecycle). To make the sidecar answer from something other than that Modal app, add a backend in sidecar/backends/ instead; it needs no change in Go or in ai_train.

​
How the provider abstraction works

app.py only says what to run. It builds a FunctionSpec (image, GPU, timeout, volumes, secrets) and registers functions with @provider.function(spec) (or @provider.asgi_app(spec) for the web API). It runs them with provider.call(fn, ...) or provider.spawn(fn, ...) and persists/refreshes volumes with provider.commit_volumes(paths) / provider.reload_volumes(paths). Images are described by a provider-neutral ImageSpec (GPU or not, with or without the full CUDA toolkit, requirements file); the provider decides how to build them, and mounts the ai_train/ directory into the container at /pkg. Code in ai_train_core/ and api/ gets what it needs from the provider as injected callables (e.g. commit_fn). Rules that keep it swappable:
  • Nothing outside providers/ imports a provider SDK, and ai_train_core/ / api/ (the code that runs in the container) never do.
  • app = provider.native_app is a module-level object because provider CLIs such as modal run look for it.
  • Code that runs remotely imports ai_train_core inside the function body, since it only exists in the container’s /pkg mount, and the heavy ML imports are per image (only the training image has Unsloth; the inference image has torch, transformers and peft).
To add a provider (e.g. AWS): implement Provider in providers/aws_provider.py and add one branch to get_provider in providers/__init__.py. If the new provider cannot express something in FunctionSpec, extend FunctionSpec rather than leaking provider details into app.py.

​
Growing it

  1. Add a dependency: put it in the requirements file of the image that needs it (an image is rebuilt only when its file changes; code changes do not rebuild it).
  2. Add a model size: an entry in ai_train_core/model_config.py, a train/generate function pair in app.py, and a pipeline module if it needs a new backend.
  3. Add another remote function: declare it in app.py with a FunctionSpec and a matching action in main.
  4. Add an inference endpoint: a Go endpoint over trained models, a sidecar route, or a new GPU generation function; see Adding an inference endpoint.