Quick Start

Five steps from zero to a running sweep. For explanations, examples, and HPC, see the Getting Started guide.

1 Hub — account, experiment, API key

  1. Create an account and verify your email.
  2. On the Hub dashboard, create an experiment (e.g. my-exp).
  3. API Keys → generate a key → copy it once.
bash
export JD_API_KEY=jd_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

2 Server — Docker + run.sh

Install Docker, then start the job server for your experiment:

bash
curl -fsSL https://raw.githubusercontent.com/NWSL-UCF/job-distributor/main/server/run.sh \
     -o run.sh && chmod +x run.sh

JD_API_KEY=jd_xxx ./run.sh my-exp

Open the server dashboard from the Hub experiment page (Open Server Dashboard). Note the 6-digit PIN shown on the Hub experiment detail page.

3 Script — atomic train.py

Each job is one run of your script with one parameter set. Requirements:

  • Parse hyperparameters with argparse (--key value).
  • Write outputs under jd_job_dir().
  • Call jd_upload() so results appear on the server.
python
from jd import jd_job_dir, jd_upload

# ... train / compute ...

job_dir = jd_job_dir()
job_dir.mkdir(parents=True, exist_ok=True)
(job_dir / "result.json").write_text('{"accuracy": 0.97}')
jd_upload(job_dir)

Full working example: Getting Started → train.py.

4 Jobs — submit the grid

On the server dashboard, create jobs (form or JSON). Example grid:

json
{
  "learning_rate_init": [0.001, 0.01, 0.1],
  "hidden_layer_sizes": ["64", "128"],
  "max_iter": [100, 200]
}

Jobs stay PENDING until a worker picks them up. Details: Server Dashboard → Job Management.

5 Workers — run jd_worker_cli

bash
python3 -m venv .venv && source .venv/bin/activate
pip install jd-worker

jd_worker_cli expId=my-exp entry_script=train.py num_workers=4 machine_type=cpu

Workers detach to the background by default (safe over SSH). Manage them anytime:

bash
jd_worker_cli expId=my-exp worker-list
jd_worker_cli expId=my-exp worker-logs 0
jd_worker_cli expId=my-exp stop all

Logs: ~/jd_data/<expId>/jd_worker_logs/

→ Next steps