After this lesson, you will be able to: Make an experiment reproducible: seeds, pinned configs and environments, model and data cards, code release, and the ACL Responsible NLP / reproducibility checklist.
Reproducibility is now a reviewed criterion, not a courtesy. This lesson covers what to release and document so another researcher (and future-you) can rerun your experiment and get your numbers.
Set and report every random seed (data split, init, training, sampling). Pin the environment (exact library versions, e.g. a requirements.txt or a container) because results drift across versions. Save the full config that produced each result (model, data version/hash, hyperparameters, hardware). Release the code and, where licensing allows, the data and trained adapters. The bar: a competent stranger with your repo can rerun the experiment and land within your reported confidence interval.
A model card documents what a model is and is not for: training data, intended use, evaluation results, limitations, and known biases. A data card documents a dataset's provenance, collection process, annotation, license, and limitations. These are not bureaucracy: they are how the community uses your artifact responsibly and how reviewers assess ethical soundness. Hugging Face Hub has templates; a good card is part of a credible release.
No seed, so even you cannot reproduce your best run. Reporting results from a library version you can no longer identify. Releasing code that omits the data-processing step (where most irreproducibility hides). A model card that lists only the good results and no limitations. Forgetting to report the compute budget and hyperparameter search (required by ACL and necessary for fair comparison). Promising 'code available upon publication' and never releasing it, which reviewers increasingly penalize.
Pick the best statement.
Sign in and purchase access to unlock this lesson.