SKILLEMALL.ai
GitHub ↗ RU Sign in

AB idea-quality

Score the QUALITY of a research idea at the idea stage — given a Markdown file with Title / Motivation / Method sections (no experiments needed), produce a per-axis quality assessment with cited evidence and an overall 0–100 score plus a verdict; or, given two such idea files, a blind head-to-head comparison. TRIGGER when the user asks "how good is this idea", "rate / score / grade this research idea or proposal", "review my idea .md before I write it up", "is this contribution strong enough to pursue", "which of these two idea files is stronger", or hands over an idea Markdown file (Title/Motivation/Method) and wants a quality judgment. Use this even when the user does not say the word "score" but clearly wants an assessment of how strong an idea is. DO NOT trigger for prior-art / overlap / "is this novel vs existing work" checks (that is a literature-collision task), for full reviews of finished papers that already have results, or for generating the idea itself.

microsoft/ResearchStudio Agent Skills author: microsoft 5 files body ≈ 2 252 tokens Open the sourcegithub.com↗ analyzed 5 d ago

Score the QUALITY of a research idea at the idea stage — given a Markdown file with Title / Motivation / Method sections (no experiments needed), produce a…

As a process B 76/100 · Nearly there — weak spots: consistency, progress reporting

GeneratorData and analyticstype and topics are labelled automatically from the skill text
Runs in: Claude / Claude Code ChatGPT / Codex Gemini CLI Cursor GitHub Copilot / VS Code Qwen Code Kimi Code CLI Z.ai (GLM) DeepSeek (Deep Code) Mistral Vibe LangChain (deepagents) OpenClaw Goose OpenCode
JSON
Technical rating
A
96/100
safety, quality, tests
Safety 60%
100
Quality 40%
89
Run on models
none yet
Process rating
B
76/100
Nearly there
Progress reporting w 2
0
Consistency w 8
40
Tools and files w 18
60
the three weakest of ten parameters · all ten

How to improve

    For the model run — optional
    • A spec.yaml with trigger phrases and assertions — a behaviour contract for CI; `skilltest init` writes a template.

    Guard findings · 0

    ✓ No critical or high findings

    Files scanned: 1. Evidence is masked. Grey chips explain why severity was lowered.

    Against the Agent Skills spec

    ✓ No remarks against the Agent Skills spec

    Process rating: all ten parameters 76/100

    • 0Progress reporting. Says nothing while it works
    • 40Consistency. Frontmatter name (idea-quality) differs from the folder (idea_quality)
    • 60Tools and files. Uses tools (read) that frontmatter does not declare
    • 65Failures and branches. 3 branches
    • 70When it triggers. States when to use, but not when not to
    • 70Inputs and preconditions. Inputs and preconditions are listed
    • 100Steps. 21 steps
    • 100Result and completion. Output format and completion criterion are stated
    • 100Execution cost. Instruction body is 2252 tokens
    • 100Running it twice. No mutating operations
    • low No test case covers injection arriving through data

    Everything here is measured from the skill text rather than judged by a model, so the numbers are checkable. A parameter weighs more when it is a more common reason for the process to stall.

    Quality signals

    • +4Description does not say when NOT to use the skill (false activations)
    • +3Description length 979: 120–800 characters recommended
    • +1No license
    • +2Single-language instructions
    • +5Description quotes 6 example trigger phrases
    • +4Structure: 9 headings
    • +3Step-by-step instructions: 21 items
    • +3Output format is stated explicitly
    • +4Has examples (4 code blocks)

    Quality base 70; lint remarks subtract, signals add up to 100. Result: 89.