rrun
All pages
Docs · SetupMarkdown

Clouds

rrun launches machines in your own accounts on Vast, RunPod, Verda, Google Cloud and Nebius. Connect any of them, each as often as you have accounts; a job without --cloud considers every connected account that takes its kind of job. Servers of your own join as reserved machines. Missing one? Ask with o41-rrun feedback "please support <cloud>" --reply-to <you@company.com>.

Cloud Machine Spot GPU-less machines Credentials
vast Docker container on a rented host By bid No API key
runpod Docker container (pod) No No API key
verda VM in Finland Spot or on demand Yes Client id and secret
gcp Compute Engine VM Spot or on demand Yes Keyless (OIDC), or a key
nebius VM in any region of your tenant Preemptible or on demand Yes Keyless (OIDC)

Connecting

o41-rrun clouds                            # accounts, a non-secret hint, the GPU and CPU switches
o41-rrun clouds set <cloud>                # connect <cloud>/default, or replace its credentials
o41-rrun clouds set <cloud> --name lab-b   # a second account: <cloud>/lab-b
o41-rrun clouds rm <cloud>/<name>

Each connection is a cloud account with an id <cloud>/<name>, such as vast/default or vast/lab-b. The name is lowercase letters, digits and dashes, unique within the cloud, and default unless you give one. Two accounts of one cloud may even be the same account at the provider: each keeps its own machines, offers and cleanup, and never touches the other's.

Without flags, clouds set reads credentials from the usual environment variables and files, and prints where it found them, never the value. rrun checks them with the provider before it stores them, sealed with AES-GCM; they are only used to talk to that cloud. Credentials cannot change or be removed while the account has machines. Connecting and removing accounts needs a signed-in person, not an API key.

--cloud on run and offers takes clouds and account ids, comma separated: --cloud vast uses every Vast account, --cloud vast/lab-b,gcp that one Vast account and every Google Cloud account. Jobs, machines and offers show the account they are on.

Turning accounts on and off

Each account has two switches, both on when it is connected: gpu for jobs that ask for any GPU, and cpu for --cpu-only jobs. A job only launches on, and offers only asks, accounts whose switch for its kind is on.

o41-rrun clouds off vast/lab-b --gpu   # no new GPU jobs on vast/lab-b
o41-rrun clouds off nebius --cpu       # no CPU-only jobs on any Nebius account
o41-rrun clouds on vast/lab-b          # both switches back on

With neither --gpu nor --cpu, both switches change. A job that finds every account it may use switched off waits for one to come back, within its --queue. Running machines are not touched. The switches are there to steer work from outside information, such as prices, credits or quotas, so an organization API key may set them too (PATCH /api/v1/clouds/<account>, see HTTP API), and the web app's Clouds page has them per account. Every change is in the audit log.

Vast

o41-rrun clouds set vast                     # or: --api-key <key>

Reads --api-key, then VAST_API_KEY, ~/.config/vastai/vast_api_key and ~/.vast_api_key.

  • Machines are Docker containers on rented hosts. Offers are the live asks that match, so prices change by the minute. GPU machines only.
  • Spot is a bid: rrun bids 1.15 times the host's minimum bid. When someone outbids it, the container stops and the job is preempted: it restarts on another machine (Restarts).
  • Without --image, the container runs Vast's base image (Ubuntu with curl and the CUDA toolkit), the newest CUDA version the host's driver supports.
  • Prices include the requested disk.

RunPod

o41-rrun clouds set runpod                   # or: --api-key <key>

Reads --api-key, then RUNPOD_API_KEY and ~/.runpod/config.toml.

  • Machines are pods (Docker containers), on demand only: RunPod rejects interruptible pods in practice, so there are no spot offers. GPU machines only.
  • Offers cover both RunPod's Secure and Community clouds. --ram goes into both the price query and the pod's placement, since many one-RTX 5090 hosts have less than 64 GB.
  • Without --image, pods run RunPod's base image: Ubuntu 24.04 with CUDA 12.8 on a host that supports at least 12.8, or CUDA 13.0 with --cuda 13.

Verda

o41-rrun clouds set verda                    # or: --client-id <id> --client-secret <secret>

Reads --client-id and --client-secret, then VERDA_CLIENT_ID and VERDA_CLIENT_SECRET, then ~/.verda/config.json.

  • Machines are VMs in Finland, spot or on demand, including machines without a GPU (--cpu-only).
  • Offers are the instance types available in some location right now.
  • VMs run Verda's Ubuntu images with the NVIDIA driver and CUDA; each instance type gets the best one it supports, preferring Ubuntu 24.04 and the newest CUDA. --image does not apply.
  • Prices include the requested disk (NVMe, $0.20 per GB-month).
  • Volumes are deleted permanently with their instance, never left in Verda's trash, where they would count against the storage quota for 96 hours. See Machines and costs.

Google Cloud

Machines are Compute Engine VMs, spot or on demand. Spot VMs are deleted, not stopped, when preempted, and their boot disks go with them.

  • Shapes: L4 (g2-standard), A100 40 GB (a2-highgpu, a2-megagpu), A100 80 GB (a2-ultragpu), H100 (a3-highgpu; 1, 2 and 4 GPUs are spot only), 8 H200 (a3-ultragpu) and 8 B200 (a4-highgpu), both spot only, and T4 on n1 hosts. H200 and B200 machines boot from Hyperdisk, and most projects need a quota increase for them first.
  • --cpu-only jobs get a GPU-less VM: the standard, highmem and highcpu shapes of e2, n2, n2d (AMD), c3 and c3d, spot or on demand, from 2 to 360 vCPUs. They run plain Ubuntu 22.04 with at least a 10 GB disk.
  • Prices come from a public price catalog, refreshed hourly, plus the disk (pd-balanced, $0.10 per GB-month). Google has no live price API worth calling per request, so a launch can still fail on quota or capacity and fall through to the next offer.
  • GPU VMs run Ubuntu 22.04 with NVIDIA driver 580 (CUDA 13.0), with at least a 50 GB disk. --image does not apply.
  • Default zones: us-central1-a, us-central1-b, us-central1-c, us-east1-b, us-east4-a, us-east4-b, us-east4-c, us-west1-b, us-west1-c, europe-west4-a, europe-west4-b, europe-west1-b. Limit them with --zones us-central1-a,europe-west4-a.
  • Machines get no service account; nothing on them can act in your project.

Organizations that forbid service account keys (iam.disableServiceAccountKeyCreation) connect through workload identity federation. rrun is an OIDC issuer at https://rrun.041.io. For each call it signs a five-minute token for your organization, trades it at Google STS for a federated token, and impersonates a service account you created, for an hour. There is no key to leak, and deleting the pool provider or the binding revokes access.

A project owner runs these, with the organization id from o41-rrun whoami:

PROJECT_ID=<project id>
ORG_ID=<rrun organization id>
ISSUER=https://rrun.041.io
PROJECT_NUMBER=$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')
gcloud services enable compute.googleapis.com iamcredentials.googleapis.com sts.googleapis.com --project="$PROJECT_ID"
gcloud iam workload-identity-pools create rrun --project="$PROJECT_ID" --location=global --display-name=rrun
gcloud iam workload-identity-pools providers create-oidc rrun --project="$PROJECT_ID" --location=global \
  --workload-identity-pool=rrun --issuer-uri="$ISSUER" \
  --attribute-mapping=google.subject=assertion.sub --attribute-condition="assertion.sub == '$ORG_ID'"
gcloud iam service-accounts create rrun --project="$PROJECT_ID" --display-name=rrun
SA="rrun@$PROJECT_ID.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding "$PROJECT_ID" --member="serviceAccount:$SA" \
  --role=roles/compute.instanceAdmin.v1 --condition=None
gcloud iam service-accounts add-iam-policy-binding "$SA" --project="$PROJECT_ID" --role=roles/iam.workloadIdentityUser \
  --member="principal://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/rrun/subject/$ORG_ID"
o41-rrun clouds set gcp --project "$PROJECT_ID" --service-account "$SA" \
  --workload-identity-provider "projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/rrun/providers/rrun"

The pool provider trusts only tokens whose subject is your organization, and keeps the default allowed audience. The service account needs roles/compute.instanceAdmin.v1 and nothing else; since machines get no service account, iam.serviceAccountUser is not needed. clouds set tries the setup at once, so a wrong issuer, condition or binding comes back as Google's error.

With a key

o41-rrun clouds set gcp --credentials key.json

Reads --credentials, then GOOGLE_APPLICATION_CREDENTIALS, then ~/.config/gcloud/application_default_credentials.json. It must be a service account key, not a user login.

Nebius

o41-rrun clouds set nebius --tenant-id <tenant-…> --service-account-id <serviceaccount-…>

Machines are Nebius AI Cloud VMs, preemptible or on demand, in every region of your tenant: rrun lists the tenant's projects (Nebius has one per region) and offers each region's platforms.

  • GPUs: H100 (eu-north1), H200 (eu-north1, eu-west1, us-central1), B200 (us-central1, me-west1), B300 (eu-west2, uk-south1, us-north1), RTX PRO 6000 (us-central1, eu-south1, uk-south2) and L40S (eu-north1), one GPU or eight (L40S: one, two or four). --cpu-only gets a GPU-less VM on demand.
  • On-demand prices are Nebius's price list. Preemptible H100, H200, B200, B300 and RTX PRO 6000 VMs cost the live spot price, which Nebius changes every 15 minutes and which can rise while the VM runs (never above the on-demand price); --max-price applies when rrun picks the offer. The L40S is on demand only: Nebius has no spot price for it. Prices include the boot disk (network SSD, $0.071 per GiB-month).
  • A preempted VM is stopped, not deleted; rrun deletes it with its disk and restarts the job on another machine (see Restarts).
  • VMs run Ubuntu 24.04 with the NVIDIA driver for CUDA 13.0 (ubuntu24.04-cuda13.0), with at least a 40 GiB disk; GPU-less VMs run Ubuntu 24.04 without a driver. --image does not apply.
  • Each VM gets a public IP for outbound traffic, released with the VM. The boot disk is created with the VM and deleted with it.
  • Quotas: a new tenant gets 8 GPUs of each type per region, 2 VMs and 2 preemptible VMs per region, and in some regions no GPU-less vCPUs. Offers skip regions where the quota for the GPU is zero; a launch over quota falls through to the next offer. Raise quotas in the Nebius console.
  • Machines get no service account; nothing on them can act in your tenant.
  • rrun calls Nebius's API through its own relay on Google Cloud Run, because rrun's servers cannot reach that API directly. The relay only forwards rrun's calls and keeps nothing.

Setup

Nebius connects without keys. rrun is an OIDC issuer at https://rrun.041.io. For each call it signs a five-minute token for your organization and trades it at Nebius STS for the service account's access token, which lasts twelve hours. A federated credential in your tenant says which organization may act as which service account. Deleting the credential revokes access.

A tenant admin runs these with the Nebius CLI, with the organization id from o41-rrun whoami:

TENANT_ID=<tenant id>                  # nebius iam v2 tenant list
ORG_ID=<rrun organization id>
ISSUER=https://rrun.041.io
PROJECT_ID=$(nebius iam v2 project list --parent-id "$TENANT_ID" --format jsonpath='{.items[0].metadata.id}')
SA_ID=$(nebius iam service-account create --parent-id "$PROJECT_ID" --name rrun --format jsonpath='{.metadata.id}')
GROUP_ID=$(nebius iam group get-by-name --parent-id "$TENANT_ID" --name editors --format jsonpath='{.metadata.id}')
nebius iam group-membership create --parent-id "$GROUP_ID" --member-id "$SA_ID"
nebius iam federated-credentials create --parent-id "$PROJECT_ID" --name rrun \
  --oidc-provider-issuer-url "$ISSUER" --oidc-provider-jwk-set-json "$(curl -fsS "$ISSUER/.well-known/jwks.json")" \
  --federated-subject-id "$ORG_ID" --subject-id "$SA_ID"
o41-rrun clouds set nebius --tenant-id "$TENANT_ID" --service-account-id "$SA_ID"

The service account joins the tenant's editors group, so it can manage VMs and disks in every project. The federated credential must belong to a project, not the tenant; any project of the tenant works. Nebius cannot fetch rrun's discovery document, so the credential carries rrun's public keys (/.well-known/jwks.json) itself: when rrun rotates its signing key, update the credential with the new keys (nebius iam federated-credentials update), or saving and launching fail with Nebius's error. clouds set tries the setup at once.

GPU names

Each cloud spells GPUs its own way; rrun normalizes them. --gpu 5090, RTX5090 and "RTX 5090" are the same, as are A6000 and RTX A6000. A100s are A100 (40 GB) and A100-80GB; SXM, PCIe and NVL versions of a model are the same name, so H200 covers both H200 and H200 NVL. o41-rrun offers shows the names rrun uses.