Citation Audit: 40 Menopause Newsletters
Private deliverable. Enter hub password to continue.
That's not right. Try again.
The newsletters are good. The citations were not.
Two passes over the 40-newsletter menopause library: every citation checked against live PubMed records, then every claim checked against its source. 15 July 2026.
Status: draft, not cleared for clinician use
Citations repaired, claims checked against the real papers, settled failures fixed, and the two blocking PDFs now in hand. 18 of 40 newsletters are clean and could pilot. 15 claims still need attention, 16 remain paywalled.
The 40 drafts imported cleanly and they are genuinely well built: subject line, preview text, a real takeaway, a safety note, and image prompts on every one. The medicine is sensibly framed and the guardrails are the right ones. That part is good news.
Then I checked the sources. Every citation was dereferenced against the live PubMed record for that ID, comparing the printed title, journal, and year to what PubMed actually holds.
The seven that mattered
Three fabricated ID numbers, each reused across several newsletters. Every one was a real PubMed ID, so nothing looked broken from the outside. The links worked. They simply landed on the wrong paper:
| Cited ID | What it actually opens | Used in |
|---|---|---|
| 26672582 | Duodenal Adenocarcinoma: Profile and Predictors of Survival Outcomes | 01, 29 |
| 26399806 | Brain stimulation in children spurs hope and concern | 01, 10, 30 |
| 37683656 | An Iranian randomised trial on contraception counselling | 10, 38 |
A patient clicking the source under "Your labs are normal" would have landed on a duodenal cancer survival study. That is the kind of thing a clinician only has to be caught by once.
This is the fingerprint of a language model writing citations. It picked the right papers, wrote plausible titles, then attached ID numbers that were close to correct but not correct. All three papers do exist, under different IDs, and I confirmed the replacements by searching PubMed for each claimed title:
26672582becomes 26563259, the BMJ summary of NICE menopause guidance26399806becomes 27228367, "Menopause" in Nature Reviews Disease Primers37683656becomes 37678251, the Cell review of menopause biology and treatment
The quieter errors
The other 32 were subtler and, in a way, more dangerous: the link opened the right study, but the newsletter described it wrongly. A clinician forwarding these would be misquoting the literature in print, under their own name.
- Testosterone (07) printed a title belonging to a Brazilian endocrinology position statement while linking to a different Acta Obstetricia paper. The link was on topic, so I corrected the title to match the paper actually linked.
- Estrogen and APOE4 (12) credited a dementia meta-analysis to Ageing Research Reviews 2026. It is The Lancet Healthy Longevity, 2025.
- GLP-1s (33) called a bone-health paper a "systematic review" in the Journal of Clinical Medicine. It is an Osteoporosis International paper on people living with obesity, and its title makes no systematic-review claim.
- Several journals were simply wrong: Frontiers in Neuroscience for a Maturitas paper, Nutrients for an AIMS Public Health paper, Menopause for a Current Opinion paper.
What I fixed
51 citation lines rewritten, plus the one stray PMC link. Every link now opens the paper whose title is printed beside it, with the journal and year PubMed holds for that record. Re-running the validator against the repaired files returns 79 of 79 clean, with the 80th now resolving properly too.
I did not touch a word of the body prose.
Then I checked the claims, not just the citations
Accurate citations are not the same thing as supported claims. The same generator that invented ID numbers also wrote the medical assertions, and a tool that fabricates a reference number will also drift on what a study found. So I ran a second pass: every sentence carrying a citation marker, checked against the abstract of the paper it cites. 93 claims in total.
The good news first: nothing is contradicted. No claim conflicts with a paper it cites. This is not a library of dangerous medicine.
The first pass could only compare claims against abstracts, which cannot settle anything: a guideline's specifics live in the full document and never appear in the abstract. So I went and got the actual papers. PMC, Europe PMC, and Unpaywall between them yielded full text for 17 of the 32 papers involved. Judging the flagged claims against the complete papers changed the picture:
Reading the real papers cleared 9 claims outright and softened 3 more, which is exactly why the abstract pass could not be trusted as a verdict. 11 of the 40 newsletters are now fully clean.
Two corrections to what I told you earlier
First, I said the failures were "judged against the complete paper, so not an artifact of a thin abstract." That was only true for 9 of the 15. The other 6 cite two papers where one is paywalled, which makes those verdicts provisional, not settled. 24-hair-thinning is the clearest example: it cites "Female pattern hair loss: a comprehensive review," which almost certainly does discuss iron and thyroid somewhere in a full text I cannot read. That flag is probably just wrong.
Second, the judge is not perfectly reproducible. Running the same claims through the same model at temperature 0 moves a handful of borderline verdicts between "overreaches," "not supported," and "contradicted." So treat these counts as accurate to within a few, not exact. Anything near a boundary wants human eyes rather than deference to the label.
The ones that were genuinely wrong
Judged against complete papers. Each claim may well be true. It just is not in the paper cited for it, which means the sentence is effectively unsourced:
- 37, menopause at work: lists temperature control, restroom access, and breaks as workplace accommodations. The cited review explicitly states it "did not identify any studies concerned with alterations to the physical environments of the workplaces." The citation says the opposite of what it is being used for.
- 35, skin: says hormone therapy should not be started solely for cosmetic skin aging. The cited paper concludes MHT can be used for skin rejuvenation. The newsletter is being more conservative than its own source, which is the safe direction, but the citation still does not support it.
- 20, mood: lists psychotherapy, antidepressants, and hormone therapy as evidence-based care. The cited paper is a review of risk factors and never discusses treatment at all.
- 14, bone loss: credits exercise with improving strength and balance to a paper that only measured bone mineral density.
- Plus 24 (hair thinning), 30 (menopause or thyroid), and 33 (GLP-1s), all the same shape.
The one I keep thinking about
Newsletter 23 says the evidence that hormone therapy treats generalized musculoskeletal pain "remains conflicting." The meta-analysis it cites found no significant effect, a risk ratio of 1.00. Null evidence, described as conflicting evidence, tilts gently in favor of HRT. It is small. It sounds perfectly reasonable. A citation checker would never catch it, because the citation is correct.
The paywall, and how it got unblocked
29 claims cited papers with no legal free full text. PMC, Europe PMC, and Unpaywall all came up short, and where publishers 403'd an open-access paper I retried with a real browser. No pirate mirrors. Two papers carried most of it: the NAMS 2022 hormone therapy position statement and the Global Consensus testosterone statement.
Annette supplied both PDFs, and they moved the needle hard. Reading the real NAMS statement cleared 14 claims the abstract could not settle. Unresolved dropped from 32 to 16, and clean newsletters went from 14 to 18. Both PDFs are now in the ResearchLibrary with the manifest updated.
(The testosterone PDF is the Maturitas co-publication of the same consensus statement the newsletters cite as J Sex Med. Same title, same authors, published simultaneously across several journals. Same document, recorded as such rather than passed off as the J Sex Med file.)
Also outstanding:
- 76 em-dashes across 32 files. Against the standing rule. The cleanup means real editing, not find-and-replace, because many are paired appositives inside medical sentences.
- The voice is machine-written. Slogan constructions, one-line punch paragraphs. It needs a pass against the anti-AI voice profile before it carries anyone's name.
- 16 missing hero images. 24 of 40 exist, and all 24 are roughly 2MB against a 300KB budget, so they are parked outside git for now.
- None of the 50 papers are saved to the ResearchLibrary or Zotero yet.
What repair actually costs, now that I know more
What got fixed
The 9 settled failures are repaired. Two were worth re-sourcing rather than cutting, because the underlying point was good patient information:
- 14, bone loss: the falls and balance claim now cites the Cochrane review on exercise for preventing falls, instead of a paper that only measured bone density.
- 20, mood: the treatment routes now cite the Maki perimenopausal depression guidelines, which actually cover antidepressants, hormone therapy, psychotherapy, and exercise. The old citation was a risk-factor review that never mentioned treatment at all.
The rest were narrowed to what their source genuinely shows. 37, menopause at work now says what its review really found, which is more interesting than the original: nobody has studied changing the physical workplace, so those sensible-sounding adjustments are untested rather than proven.
Two of my own edits were wrong on re-check. I attached a citation to a disclaimer in 21, and added a clause to 14 ("most fractures follow a fall") that the Cochrane abstract does not state. Both are fixed and now validate clean. Which is the argument for re-running the checker on your own work rather than trusting it.
14 of 40 newsletters are now fully clean, up from 8 when this started. Citations: 82 of 82 valid.
One quirk worth knowing
Several flags are the newsletter being more careful than its source. Newsletter 34 says creatine's cognitive benefits are "promising but less settled" while the cited paper is more enthusiastic. Newsletter 35 declined to recommend hormone therapy for skin aging while its source was keen on the idea. The checker calls that a mismatch. It is not an error, it is good editorial judgment, and it should survive the edit.
What is left
- About 9 claims still need attention, several of them provisional pending a paywalled paper. Full list with reasoning in
newsletters/CLAIM-AUDIT.md. - Unblock 32 claims with two PDFs: the NAMS 2022 hormone therapy position statement and the Global Consensus testosterone statement. Drop them in the ResearchLibrary and I re-run the check.
- Em-dash cleanup. 76 across 32 files. Real editing, not find-and-replace: many are paired appositives inside medical sentences.
- Voice pass against the anti-AI profile.
- Images: 16 heroes missing, and the 24 that exist need optimizing from ~2MB to under 300KB.
The full flagged list, claim by claim with the reasoning for each, is in newsletters/CLAIM-AUDIT.md. The library lives in the MenopauseTelehealthGrowth repo under newsletters/, marked DRAFT.