BC experiments
Use when adding, changing, or reviewing a VS Code A/B experiment, either an experiment-controlled setting or a treatment read with the assignment service, and when deciding whether and where to log an `experimentTrigger` event for a triggered ExP scorecard.
A skill for managing A/B experiments in VS Code: handles adding, modifying, and reviewing experiment settings, reading assignments from the service, and deciding whether to log `experimentTrigger` for scorecard. Single instruction file at 1124 tokens, no scripts, no critical errors. Quality score 82, process score 50—documentation is solid but verification methodology is weak. Broken references suggest incomplete coverage. Never run on models, no sandbox testing. Supports many platforms, but without actual execution, behavior in practice remains unclear. Install if you regularly handle VS Code experiments and can validate the guidance against your own workflows.
Use when adding, changing, or reviewing a VS Code A/B experiment, either an experiment-controlled setting or a treatment read with the assignment service, and…
As a process C 50/100 · Has gaps — References files that are not bundled: ../../../src/vs/platform/telemetry/common/experimentTrigger.ts
How to improve
- The text references files that are not there: add them or drop the references.
- Your own cases (evals/evals.json, 4–6 real requests with expected answers): the full check would then run those instead of a model-drafted suite.
- A spec.yaml with trigger phrases and assertions — a behaviour contract for CI; `skilltest init` writes a template.
Guard findings · 0
✓ No critical or high findings
Files scanned: 1. Evidence is masked. Grey chips explain why severity was lowered.
Against the Agent Skills spec
- warning
missing-refreference to a missing file: ../../../src/vs/platform/telemetry/common/experimentTrigger.ts
Process rating: all ten parameters 50/100
- 0Tools and files. 1 referenced file(s) missing: ../../../src/vs/platform/telemetry/common/experimentTrigger.ts
- 0Result and completion. Does not say what the result is
- 0Inputs and preconditions. Does not say what the process needs to start
- 50Failures and branches. 0 branches, has a failure section
- 85Steps. 8 steps, 1 vague phrases
- 100When it triggers. States when to use and when not to
- 100Consistency. Name and required fields are in place
- 100Execution cost. Instruction body is 1124 tokens
- 100Running it twice. No mutating operations
- 100Progress reporting. Reports progress
Everything here is measured from the skill text rather than judged by a model, so the numbers are checkable. A parameter weighs more when it is a more common reason for the process to stall.
Quality signals
- +5Description has no quoted example phrases that should trigger the skill
- +3Output format is not stated: the model decides each time
- +1No license
- +2Single-language instructions
- +4Description says when NOT to use the skill
- +3Description length 257: enough signal without eating the budget
- +4Structure: 5 headings
- +3Step-by-step instructions: 8 items
- +4Has examples (2 code blocks)
Quality base 70; lint remarks subtract, signals add up to 100. Result: 82.