CB evaluation

#   set -a && source .env && set +a    # HF_TOKEN, AWS_*, NEL_NEXT_EVAL_IMAGE, HARBOR_*_ECR_REPOSITORY (from modelopttools:eval-config)

# a card-documented toggle — e.g. NVFP4 MoE (NVID…FP4)

command -v uv >/dev/null 2>&1 || { curl -LsSf https://astral.sh/uv/install.sh | sh; export PATH="/root/.local/bin:$PATH"; }

setsid --wait bash /tmp/gym_run.sh &

Result (0-100): `gpqa…ect`

Result (0-100): `ifbe…acy`

Result (0-100): `aalc…ect`

**Primary result — Omniscience Index** (-100 to 100): `omni…dex`. This is AA's headline metric: accuracy net of hallucinations, rewarding abstention over guessin

- **Accuracy** (0-100): `omni…ect` — % of questions answered correctly.

Result (0-1): `AIME…f_N`

Result (0-100): `live…acy`

via a **quoted heredoc** and launched under `setsid`, so the whole server/Ray

the setsid + process-group reap in the task `command:` prevents.

Some models need extra vLLM backend env vars (model-card research) — e.g. `VLLM…END=marlin` (Nemotron Super), or `VLLM…FP4=1` + `VLLM_FLASHINFER_MOE_BACKEND=throughpu

set -a && source .env && set +a

- `set -a && source .env && set +a` before running so `${VAR}` resolves.

set -a && source .env && set +a; NEL="$SKILL_DIR/scripts/nel-next.sh"

# Usage (source .env FIRST so the config's ${VAR}s resolve; this never reads secrets):

#     emit_traces=false to avoid the per-sample hang). Run after `source .env`.

command -v uvx >/dev/null 2>&1 || { echo "ERROR: 'uvx' not found (curl -LsSf https://astral.sh/uv/install.sh | sh)" >&2; exit 1; }