Write a deep-learning experiment report with negative results

An experiment report should explain the decision, including candidates that failed. Removing negative results makes repeated work more likely and can make the selected model look inevitable. Record the question, primary metric, fixed data roles, budget, result and limitation before writing a polished conclusion.

One explicit rejection

The deep-learning lab asks: does a second hidden layer improve fixed-validation accuracy over the one-layer baseline?

from deep_learning_cases import report_case

result = report_case()
assert result["primary_metric"] == "validation_accuracy"
assert result["accepted"] is False
assert result["negative_result_retained"] is True
print(result["reference"], result["candidate"])

The 33-parameter reference reaches 94.17% validation accuracy. The 641-parameter candidate reaches 92.5%, so it is rejected. Its test score is also stored, but test does not overturn a validation decision.

Other adverse evidence in the cluster belongs in the same report:

• Weight decay and dropout shrink weights but worsen validation log loss in the noisy-feature case.
• The unweighted imbalanced image model predicts no positives.
• Class weighting raises positive recall only to 12.2%, still an unacceptable result for a recall-sensitive use.
• Nominal architecture names do not replace data and error analysis.

A compact report structure

1. 1State the decision and acceptance threshold.
2. 2Identify dataset version, population, target, partitions and leakage controls.
3. 3Record environment, seeds, architecture, preprocessing and compute budget.
4. 4Show baseline and candidate metrics with denominators and slices.
5. 5Include training curves, stopped checkpoint and failed runs.
6. 6Explain the selected model, unsupported claims and next test.

Keep exact commands and artifacts beside the prose. Distinguish deterministic reproduction from statistical stability across seeds. A successful rerun proves the pipeline can reproduce that result; it does not prove the estimate is precise.

Avoid retrospective storytelling

Timestamp hypotheses and selection rules. Do not present a post-hoc slice as predeclared. If an exploratory finding motivates another experiment, label it exploratory and validate it on new data. Preserve configuration and logs for candidates that diverged or underperformed.

A negative result can still improve the project by eliminating complexity, revealing insufficient minority data or exposing a bad assumption. The Data Science course evaluates that reasoning alongside the final metric.

Exercise

Write a one-page report for the deeper-model rejection. Add a table of all 20 lab checks, separate validation decisions from test descriptions, and propose one follow-up whose success criterion is set in advance.

Continue learning

This article is part of the Deep learning and computer vision sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the preceding task in Quantization: compare size, speed and prediction changes.
• Return to the cluster foundation in Build a neural network baseline before adding more layers.

Reference: scikit-learn guidance on common pitfalls and recommended practices.