Method Specification Prompt (MSP)

Most LLM papers under-report their methods: the exact prompt text is paraphrased or omitted, the model version is given as a marketing name rather than an API string, sampling parameters go unmentioned. Since any of these can change results, such studies are unreproducible on their face. The MSP is a discipline for closing that gap — a single document that captures everything an exact replication needs.

The key property is self-containment. An MSP includes the factor structure and crossing strategy, every prompt template with its resolved snippet text per condition, the model identifiers and parameter sets actually used, replication counts, the expected output format and extraction rules, and the coding scheme with its rubric. Nothing is left to be reconstructed from memory or supplementary email.

Because it is generated from recorded facts rather than written after the fact, an MSP also serves as an audit object: reviewers can check the reported method against what actually ran, and replicators can distinguish "the method changed" from "the model changed" when results differ.

AIScholar auto-generates the MSP from the study's recorded design, prompt versions, parameters, and coding schemes, and includes it in the one-click reproducibility package alongside per-trial data.

• Reproducibility features

• Prompt sensitivityMeasuring how results change under semantically-equivalent rewordings of a prompt.
• Factorial & fractional factorial designFull vs. fractional crossing of factor levels, aliasing trade-offs, and when to use each.
• BYOK (Bring Your Own Key)Trials run on the researcher's own OpenRouter key, billed by the provider at cost.

Browse all 11 glossary terms