Command line

o41-rrun runs a folder's command on the cheapest machine that fits and streams its logs back. Install it with:

curl -fsSL https://rrun.041.io/install.sh | sh

It checks the download against the release's SHA-256 before installing; o41-rrun update moves to the newest release later.

o41-rrun help, or o41-rrun <command> --help, prints the same reference.

Commands

Command	Does
`run`	Pack a folder, run a command on the cheapest machine
`ls`	List jobs, newest first
`status <job>`, `show <job>`	A job's spec, machine, events and latest telemetry
`logs <job>`	Print a job's logs; `-f` follows until it ends
`telemetry <job>`	A job's machine telemetry over time
`wait <job>`	Wait for a job to end; exit with its exit code
`cancel <job>`	Cancel a job
`offers`	What a machine would cost on every connected cloud
`machines`, `register`, `on`, `off`, `rm`	Cloud and reserved machines; add your own servers
`clouds`, `set`, `on`, `off`, `rm`	Cloud accounts and the jobs each takes
`storage`, `storage set`, `storage rm`	Where job logs are stored, and for how long
`api-keys`, `api-keys create`, `revoke`	Organization API keys
`audit`	The organization's audit log
`login`, `logout`, `whoami`, `org`, `org use`	Sign-in and organization
`feedback`	Send feedback or a bug report to the 041 team
`update`	Replace this binary with the newest release

Global flags

Flag	Effect
`--json`	One JSON document on stdout; log streams are JSON Lines
`--url <origin>`	The rrun server (default: `O41_RRUN_URL`, else `https://rrun.041.io`)
`-h`, `--help`	Help for a command, as in `o41-rrun run --help`
`-V`, `--version`	Print the version (before the command)

Global flags go before the command or among its flags. Output and exit codes are in API keys and agents.

run

o41-rrun run (--gpu <choices> | --cpu-only) [options] [--] <command...>

Packs the folder, uploads it unless the server has it, queues the job and prints its id. One argument runs as a shell script ('python a.py && ls'); several are quoted into one command line. Flags end at the first argument, so o41-rrun run --cpu-only python train.py --lr 3 passes --lr 3 to the program. See Jobs and logs.

Job:

Flag	Default	Meaning
`-C`, `--dir <path>`	`.`	Folder to pack and run in
`--name <name>`	the folder name	Job name, at most 128 characters
`--env K=V`		Environment variable; `--env K` copies K from here. Repeatable.
`--secret K=V`		Like `--env`, but sealed and never shown back. Repeatable.
`--image <ref>`	the cloud's	Container image on container clouds (vast, runpod)
`--timeout <dur>`	`0` (never)	Kill the command after this long
`--keep <dur>`	`5m`	Keep the machine idle this long for the next job
`--queue <dur>`	`30m`	Give up when no machine is found within this long
`--restarts <n>`	`10`	When the machine fails, run again on another; `0` never
`--reserved-only`		Only your reserved machines; wait within `--queue`
`--cloud-only`		Never a reserved machine
`-f`, `--follow`		Stream logs; exit with the job's exit code
`--wait`		Wait quietly; print the result; exit with the job's code

Machine (also for offers):

Flag	Default	Meaning
`--gpu <choices>`		Any one of these, comma separated: `MODEL` for one GPU, `MODEL:N` for N, as in `RTX5080:4,H100:8`
`--cpu-only`		No GPU. One of `--gpu` and `--cpu-only` is required.
`--ram <GB>`		Minimum RAM
`--cpus <n>`		Minimum vCPUs
`--vram <GB>`		Minimum memory per GPU
`--disk <GB>`	`50`	Minimum disk
`--spot`		Consider spot offers too; the cheapest wins
`--cloud <list>`	all connected	Only these clouds (`vast`: all its accounts) or accounts (`vast/lab-b`); no reserved machines
`--max-price <$/h>`	no limit	Highest price per hour
`--cuda <version>`	any	Lowest CUDA version the driver must support, such as `12.8`

Durations: 30s, 10m, 6h, 1d, 1h30m or seconds. --vram needs --gpu. Without --reserved-only or --cloud-only, a free reserved machine that fits takes the job first, since it costs nothing; otherwise the cheapest cloud offer does. --reserved-only excludes --cloud-only, --cloud and --image. A restarted command sees RRUN_ATTEMPT (1, 2, ...) and should resume from a checkpoint; see Restarts.

o41-rrun run --gpu RTX5090,RTX5080,A6000 --ram 64 --spot -f -- ./train.sh
o41-rrun run --cpu-only -C ./etl --env DATASET=small -- 'python prep.py && ls out'
o41-rrun run --gpu RTX5080:4,A100,H200 --spot -- python sweep.py
o41-rrun run --gpu B300:8,B200:8 --spot --secret HF_TOKEN --timeout 6h -- torchrun train.py

By default it returns at once. In a terminal it prints one line, <id> queued. Follow: o41-rrun logs <id> -f; when stdout is captured, as in id=$(o41-rrun run ...), stdout is just the id and the hint goes to stderr. With --json it prints the job as JSON. -f follows the logs and --wait waits.

ls

o41-rrun ls [--active | --all] [--limit <n>]

Flag	Meaning
`--active`	Only jobs that have not ended
`--all`	Every job (default: the newest 20)
`--limit <n>`	At most n jobs

Columns: ID NAME STATUS AGE GPU CLOUD $/H EXIT. CLOUD is the cloud account, such as vast/default, or reserved. A failure's reason follows its status (failed:no_capacity) unless the command exited non-zero.

status

o41-rrun status <job>      (alias: o41-rrun show <job>)

The job's id, name, status, exit code, command, who created it and when, start and end, what it requested, the machine and its price, where it may run (placement), image, env and secret names, limits, bundle hash and log line count; then its events and the latest telemetry.

logs

o41-rrun logs <job> [-f] [--tail <n>]

Flag	Meaning
`-f`, `--follow`	Keep printing until the job ends; exit with its exit code
`--tail <n>`	Start with the last n lines

stdout and stderr lines print as they are; rrun's own lines start with [rrun]. With --json, one line per log line: {"seq","at","stream","text"}. Lines that rrun's bucket deleted after 30 days are skipped with one note on stderr.

telemetry

o41-rrun telemetry <job> [--since <dur>] [--points <n>]

Flag	Meaning
`--since <dur>`	Only the last `<dur>`, such as `1h` or `6h` (default: the whole job)
`--points <n>`	At most n rows (default 500, at most 2000); longer ranges are averaged

One row per sample: TIME CPU RAM, then per GPU its utilization, memory used and total, temperature and power (when the GPU reports them). The agent samples every 15 seconds. With --json, one sample per line: {"at","cpuPercent","ramUsedGb","ramTotalGb","gpus":[{"utilPercent","memUsedGb","memTotalGb","tempC","powerW"}]}. Samples rrun's bucket deleted after 30 days are left out with a note on stderr.

wait

o41-rrun wait <job>

Prints the job's result line (with --json, the job) and exits 0 when it succeeded, the command's exit code when it exited non-zero, 1 for other failures and 130 when cancelled.

cancel

o41-rrun cancel <job>

The command gets SIGTERM, then SIGKILL after 30 seconds.

offers

o41-rrun offers (--gpu <choices> | --cpu-only) [machine flags] [--limit <n>]

Takes the machine flags of run, plus --limit <n> (default 20). Columns: CLOUD GPU VRAM CPUS RAM $/H SPOT REGION, cheapest first; CLOUD is the account. Only accounts that take this kind of job answer (clouds on and off). A cloud that fails to answer is a warning on stderr.

o41-rrun offers --gpu RTX5090,A6000 --ram 64 --spot
o41-rrun offers --gpu B300:8,B200:8,H200:8 --spot

machines

o41-rrun machines [--all]
o41-rrun machines register [--name <name>]
o41-rrun machines on <id>
o41-rrun machines off <id>
o41-rrun machines rm <id>

--all includes terminated machines. Columns: ID STATUS CLOUD GPU CPUS RAM $/H SPOT WHERE JOB AGE IDLE UNTIL. CLOUD is the cloud account, or reserved; WHERE is the region, or a reserved machine's name. A reserved machine's status is offline, idle or busy, and off when it is turned off (off (busy) while its last job finishes).

machines register adds a server of your own as a reserved machine and prints one command to run on it. In a terminal a line before it explains it; when stdout is captured, stdout is just the command. With --json: {"id","key","command","machine"}. --name defaults to the server's hostname.
machines off stops new jobs from landing on a reserved machine; a running job finishes. machines on lets it take jobs again.
machines rm deletes a cloud machine now, or removes a reserved one: its key stops working and its agent stops. Either way its job, if any, fails with machine_lost.

register and rm need a signed-in person; on and off also take an API key.

ssh lab-1 "$(o41-rrun machines register --name lab-1 --json | jq -r .command)"

clouds

o41-rrun clouds
o41-rrun clouds set <cloud> [--name <name>] [flags]
o41-rrun clouds on <account|cloud> [--gpu | --cpu]
o41-rrun clouds off <account|cloud> [--gpu | --cpu]
o41-rrun clouds rm <cloud>/<name>

Each connection is a cloud account, <cloud>/<name>. clouds lists them with columns ACCOUNT IDENTITY GPU CPU MACHINES UPDATED. clouds set connects <cloud>/default, or <cloud>/<name> with --name (lowercase letters, digits and dashes); set it again to replace the credentials. clouds rm takes the account id and needs the account to have no machines.

clouds off <account> --gpu stops GPU jobs from launching there, --cpu stops CPU-only jobs, and neither flag stops both; clouds on turns them back on. A cloud name, as in clouds off vast, sets every account of that cloud. Running machines are not touched. See Clouds.

Without flags, clouds set reads credentials from the usual environment variables and files:

Cloud	Flags	Then
`vast`	`--api-key`	`VAST_API_KEY`, `~/.config/vastai/vast_api_key`, `~/.vast_api_key`
`runpod`	`--api-key`	`RUNPOD_API_KEY`, `~/.runpod/config.toml`
`verda`	`--client-id`, `--client-secret`	`VERDA_CLIENT_ID` and `VERDA_CLIENT_SECRET`, `~/.verda/config.json`
`gcp`	`--workload-identity-provider`, `--service-account`, `--project`	Keyless; see Clouds
`gcp`	`--credentials <key.json>`	`GOOGLE_APPLICATION_CREDENTIALS`, `~/.config/gcloud/application_default_credentials.json`
`nebius`	`--tenant-id`, `--service-account-id`	Keyless only; see Clouds

For gcp, --zones us-central1-a,europe-west4-a limits the zones. The server checks the credentials with the provider and seals them; the CLI prints only where they came from. clouds, set and rm need a signed-in person; on and off also take an API key, so a script can steer where work goes.

storage

o41-rrun storage
o41-rrun storage set --endpoint <https://...> --bucket <name> [--region <r>] [--prefix <p>]
o41-rrun storage rm

storage set takes keys from --access-key-id and --secret-access-key, else AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. storage rm goes back to rrun's own bucket. rrun's bucket keeps logs, telemetry and uploaded folders for 30 days; your own bucket keeps logs and telemetry as long as you like, since rrun never deletes from it. See Log storage.

o41-rrun storage set --endpoint https://t3.storage.dev --bucket logs --region auto --prefix rrun/

api-keys

o41-rrun api-keys
o41-rrun api-keys create <name>
o41-rrun api-keys revoke <id>

create prints the secret, once, alone on stdout:

export O41_RRUN_API_KEY=$(o41-rrun api-keys create ci)

An API key can run, list, follow and cancel jobs, see offers, list machines, and turn cloud accounts and reserved machines on and off. Connecting clouds, registering and removing machines, storage, keys and the audit log need o41-rrun login. See API keys and agents.

audit

o41-rrun audit [--limit <n>]   (default 50)

Columns: AT ACTOR ACTION TARGET DETAIL.

o41-rrun login [--org <id|slug|name>]
o41-rrun logout
o41-rrun whoami
o41-rrun org
o41-rrun org use <id|slug|name>

login prints a link and a code; open the link on any device (works over SSH). The sign-in is saved per server in ~/.config/o41-rrun/auth.json and refreshed as needed. --org picks the organization to act in; with one organization it is picked for you.
logout deletes the saved sign-in and revokes its tokens.
whoami shows the server, the credential and where it came from, the user and the organization in use. Its organization id is the one the keyless Google Cloud and Nebius setups need.
org lists your organizations; * marks the current one. org use picks the one the saved sign-in acts in.

feedback

o41-rrun feedback "please support Lambda Cloud" --reply-to me@mycompany.com
o41-rrun feedback --bug "logs -f stops after an hour on j-4k2m9x7qpz"

Sends the message to the 041 team with the CLI's version, OS, server URL and sign-in attached, never a key. It needs no sign-in. --bug marks a bug report; --reply-to takes an email address or a URL, such as an issue, for the answer. The web app's Feedback page does the same.

update

o41-rrun update           # install the newest release
o41-rrun update --check   # only say whether this binary is the newest

Downloads the newest release for this platform, checks it against the release's SHA-256 and renames it over this binary, through any symlink. A download that does not match leaves the binary as it was. The release comes from the server --url or O41_RRUN_URL names. update needs write access to the binary's directory; without it, run the install script again.

$ o41-rrun update
Updated /home/me/.local/bin/o41-rrun to o41-rrun 0.1.0-4f9a2c1e.

Environment

Variable	Effect
`O41_RRUN_API_KEY`	Organization API key (`ak_…`); wins over the saved login
`O41_RRUN_URL`	The rrun server
`O41_RRUN_TOKEN`	A raw bearer token, with `O41_RRUN_ORG` naming its organization
`O41_RRUN_ORG`	The organization to act in, overriding the saved login's
`XDG_CONFIG_HOME`	The sign-in is saved in `$XDG_CONFIG_HOME/o41-rrun/auth.json`