Paper2code

# Guardrail: Handling Badly Written Papers

## Reality check

Many papers are vague, inconsistent, or incomplete. This is not a criticism — writing a paper is hard, page limits are strict, and reviewers rarely check hyperparameter tables. But it means you will regularly encounter papers where the text alone is insufficient to produce a correct implementation.

This file tells you what to do when that happens. The answer is never "guess silently."

---

## Decision tree for resolving ambiguity

```
Is the detail stated explicitly in the paper?
├── YES → Use it. Cite the section. Done.
├── PARTIALLY → Follow the partial specification protocol (below)
└── NO →
    Is there official code on GitHub?
    ├── YES → Use it as ground truth. Cite the file and line number.
    │         Mark as [FROM_OFFICIAL_CODE], not [SPECIFIED].
    └── NO →
        Is there a well-known reimplementation?
        ├── YES → Reference it in REPRODUCTION_NOTES.md.
        │         DO NOT blindly copy — they made their own choices.
        │         Mark as [UNSPECIFIED] and note the external reference.
        └── NO →
            Is there a "standard" choice in the field?
            ├── YES → Use it. Mark as [UNSPECIFIED] with alternatives.
            └── NO →
                Write a stub with a detailed docstring.
                Explain what should go here and what the options are.
```

---

## Partial specification protocol

When the paper partially specifies something:

### "We use Adam"
The paper says an optimizer but not all parameters.
```python
# §4.1 — "We use the Adam optimizer"
# [PARTIALLY_SPECIFIED] Optimizer stated as Adam, but beta and epsilon values not specified
# Using: β₁=0.9, β₂=0.999, ε=1e-8 (PyTorch defaults)
# Alternatives: β₁=0.9, β₂=0.98, ε=1e-9 (Transformer default from Vaswani et al.)
optimizer = torch.optim.Adam(model.parameters(), lr=config.lr,
                              betas=(0.9, 0.999), eps=1e-8)
```

### "We use dropout for regularization"
The paper mentions dropout but not the rate or placement.
```python
# §3.3 — "We use dropout for regularization"
# [PARTIALLY_SPECIFIED] Dropout mentioned but rate not specified
# Using: 0.1 (common default for transformer models)
# Placement: after attention and FFN (not specified — this is our choice)
self.dropout = nn.Dropout(0.1)
```

### "Similar to Transformer [Vaswani et al., 2017]"
The paper references another work for a component.
```python
# §3.1 — "We use an architecture similar to the Transformer [Vaswani et al., 2017]"
# [PARTIALLY_SPECIFIED] "Similar" implies differences exist but none are specified
# We implement the standard Transformer from Vaswani et al. unless other sections
# describe modifications. Reader should check if the authors intended differences.
```

---

## When the paper contradicts itself

### Figure vs text
Figures are often created early in the paper-writing process and may show an earlier version of the architecture or method. Text is usually more up-to-date.

**Rule: Trust the text. Flag the figure discrepancy.**
```python
# §3.2 — "We apply layer normalization before each sub-layer" (Pre-LN)
# NOTE: Figure 2 shows post-norm placement, inconsistent with this text.
# We implement pre-norm as stated in the text. If reproduction fails,
# try post-norm (change this to: x = self.norm(x + sublayer(x)))
```

### Equation vs prose
Equations are peer-reviewed more carefully and are more precise.

**Rule: Trust the equation. Flag the prose discrepancy.**

### Different numbers in different sections
The paper might say "learning rate of 1e-4" in one section and "learning rate of 3e-4" in another.

**Rule: If one is in the appendix/hyperparameter table, trust that. If both are in prose, flag both and use the one from the main experimental section (usually the one paired with the best results).**

---

## When the paper is genuinely incomplete

Some papers are missing critical details that make reproduction impossible without additional information. Here