Evidence and Impact | The Unjournal

What public evaluation adds

Each evaluation package sets out expert judgments about a paper's methods, claims, relevance, and uncertainty. Readers can inspect the reasoning and ratings rather than relying on a journal label or a single accept/reject decision.

Public assessment

Assess the evidence

Detailed evaluations and ratings help readers assess a paper's strengths, weaknesses, credibility, and relevance.

Research use

Inform decisions and priorities

Funders, policymakers, researchers, and other users may update a view, recommendation, research agenda, or decision.

Research improvement

Improve the research

Feedback and encouragement can lead authors to revise a paper, clarify claims, correct errors, or pursue more useful research.

Reusable public record

Make the reasoning inspectable

Open evaluations, ratings, responses, and linked analysis let funders and researchers inspect, cite, and build on the assessment.

Use in funding decisions and applied research

Our main aim is to help funders, policymakers, and researchers make better use of important evidence. These are the clearest public cases we have so far. They show particular contributions, not an estimate of our average effect.

Use by a research funder

GiveWell and water-treatment evidence

GiveWell staff publicly described our evaluation of an important water-treatment meta-analysis as influential. They said it saved time they had considered spending on a replication, and a research director later confirmed that the evaluation was useful. It found no major failure, while flagging excluded studies and other issues relevant to interpreting the result.

The evidence concerns a program area in which GiveWell had recommended a $64.7 million safe-water grant, alongside later grants and research. That grant predated our evaluation; the figure indicates the scale of the decision context, not funding caused by The Unjournal.

Evaluation package · GiveWell's evidence review · Safe-water grant · Public discussion

Change in a funder's working assumption

Wellbeing measurement and cost-effectiveness

We developed this Pivotal Question with Founders Pledge around measurement choices that could shift funding decisions. After the workshop, a Coefficient Giving researcher reported revising a working conversion from 10 WELLBYs per DALY to 4–7. This parameter can affect comparisons between health and wellbeing interventions.

The workshop also changed applied research priorities. Happier Lives Institute said it moved work on wellbeing weights up its agenda; its public 2026 plan now includes a report covering DALY-to-WELLBY conversions and discounting. The workshop also helped open a collaboration with Dean Jamison's group on non-fatal outcomes for a forthcoming Lancet report. Seven of nine survey respondents reported better understanding of practitioner priorities; the public report also records mixed and critical feedback.

Survey results and follow-up evidence · Workshop and Pivotal Questions · HLI's 2026 research plan · Lancet Commission on Investing in Health

Substantial follow-on analysis

Returns to science and technological risk

Three evaluations of a model of the returns to science identified a major missing issue: the analysis compared the broad benefits of science with biotechnology risks while largely setting aside AI risk. The author's public response developed a substantial new analysis of this issue, compared three ways to model it, and accepted that the original paper did not adequately justify the omission.

This matters because the model is intended to inform donors and others considering whether to accelerate or slow scientific work.

Evaluation package · Author's analysis and response

Follow-on analysis and public critique

Tropical forest regeneration estimates

Our evaluation raised fundamental concerns about a Nature paper's estimate of 215 million hectares of natural regeneration potential. One evaluator developed the critique into a public reanalysis and a Matters Arising manuscript, with code and results in a public repository.

The observable contribution is the follow-on analysis and manuscript. Nature's editorial process is separate.

Evaluation package · Public reanalysis and manuscript · Original Nature paper

Reach of the published work

From August 26, 2024 through August 26, 2026, The Unjournal's PubPub site recorded about 34,800 unique visits and 46,800 pageviews. PubPub hosts our evaluation packages, individual evaluations, ratings, author responses, and related collection pages.

34.8K unique visits to The Unjournal's PubPub site

46.8K total pageviews across published content and site pages

We copied these figures from the PubPub community analytics dashboard's two-year view. Unique visits are the closest available proxy for readers, rather than a count of distinct people. Visits and pageviews may include some automated traffic, while readership through RePEc, shared files, citations, and other platforms is not counted. Downloads are omitted because nearly all of the material can be read directly on the site. These figures indicate reach, not whether the work changed a decision. Browse the published work. Last checked August 26, 2026.

One observable pathway: research updating

Author responses and later paper versions leave public records, so they are easier to track than use by funders or other readers. They are useful evidence, but only one pathway to impact.

5 of 22 manually assessed papers showed clear substantive updating

7 of 22 included a stated intention by the authors to update

19 of 57 tracked packages had a formal or informal author response

These figures use the April 2026 dataset, and the denominators differ. Many records remain uncoded; an uncoded record does not mean "no response" or "no revision." See the full tracking dashboard for the classifications and caveats.

Further applied work and wider influence

These examples include analysis of a funder's cost-effectiveness method, a further research correction, practical Pivotal Questions work, and contributions to research-evaluation practice. The last of these is useful evidence of influence, but less direct than the funding and research cases above.

Applied analysis for philanthropic decisions

GiveWell's discount-rate methodology

Our evaluation of a Rethink Priorities review commissioned by GiveWell identified a consequential interpretation issue. GiveWell's stated 4% rate applies to log increases in consumption; under its other assumptions, the corresponding rate for consumption levels is roughly 7.1%.

The evaluation also questioned whether a component for compounding non-monetary benefits could double-count effects already reflected in projected health or consumption gains. These are concrete issues for cost-effectiveness analysis. The public contribution is the analysis; we have not established that GiveWell changed its method.

Evaluation package · Original review and recommendations

Public assessment and technical correction

Artificial intelligence and economic growth

An evaluator identified an error in a proof supporting a central result. The public record helps research users understand what was wrong and what remains supported. The authors acknowledged the problem and confirmed a corrected proof; the main result was preserved, but the reasoning was corrected and made clearer.

Evaluation package and author response

Decision-focused convening and synthesis

Cultivated-meat costs and animal-welfare decisions

We evaluated key cost projections, built a public forecasting model and Metaculus questions, and convened more than 24 researchers, forecasters, funders, scientists, and industry participants around disagreements that could affect animal-welfare funding and research decisions.

The public outputs include recordings, a synthesis of the main cruxes, an interactive cost-projection dashboard, and structured belief elicitation. The discussion sharpened the decision frame: the synthesis now separates whether commodity-scale production is feasible from when it might arrive, and sets out the practical comparison between $100,000 for cultivated-meat R&D and the same amount for established corporate animal-welfare campaigns.

These are substantial applied-research outputs. We have not established that they changed a particular grant.

Workshop synthesis and recordings · Interactive cost model · Workshop overview

Decision-relevant forecasting partnership

Animal Futures forecasting tournament

The Unjournal partners with Metaculus and Sentient Futures on Animal Futures, helping connect forecasting questions to animal-welfare research and decisions. The questions cover alternative proteins, farmed and wild animal welfare, and the effects of advanced AI on animals.

Our preliminary, AI-prepared dashboard and short report summarize forecasts and substantive discussion, with links to the underlying questions. These are public resources for assessing uncertainties; we have not established that a forecast changed a particular funding or policy decision.

Tournament and partnership · Public update and dashboard · Short report

Wider research-evaluation practice

Sharing and adapting the model

The Unjournal has provided an example and advice for The Alignment Journal. David Reinstein contributed to its design discussions and coauthored public posts about its policies and adaptation to AI. The journal's AI post explicitly describes how The Unjournal informed its proposed AI-use policy.

We serve as an evaluation provider for the Center for Open Science's Lifecycle Journal pilot and tested DeSci Labs' reviewer-finder prototype. We have also presented our model and comparisons of human and AI evaluation at the 2025 BITSS Annual Meeting and MAER-Net Colloquium.

These are contributions to evaluation practice and public scrutiny of the model. They are less direct evidence of effects on funding, policy, or substantive research decisions.

Alignment Journal design · Adaptation to AI · Lifecycle evaluation services · DeSci Labs · BITSS meeting · MAER-Net programme

How to read this evidence

We can directly observe evaluations, ratings, author responses, and some paper revisions. We have less complete evidence on who uses them and whether they change funding, policy, or later research.

Our records also include weak and negative feedback. Some authors said an evaluation arrived too late or added little beyond earlier journal reports. Others found some suggestions useful and others beyond the paper's scope. Timing, evaluator fit, and relevance to an active research or decision process seem to matter.

Paper revisions usually reflect several inputs. Workshop surveys are small and self-selected. These records show engagement and specific changes, but they do not give a clean estimate of The Unjournal's overall effect.

Contributions to other evaluation projects and conference presentations show adoption, collaboration, or scrutiny of parts of the model. They are weaker evidence of effects on funding, policy, or substantive research decisions.

We will keep adding public evidence as we get it, including cases where an evaluation had little or no apparent effect.

Data and documentation

We use different pages for different jobs. This page is the short public overview; the links below carry the detail.

Working measurement plan Our broader measurement agenda, including reach, author engagement, partner use, decision effects, and case studies.
PubPub impact analytics Current site-level pageviews and unique visits. The reach figures above use its two-year window ending August 26, 2026.
Author-response dashboard Underlying classifications, summary statistics, exploratory paper comparison, methods, code, and caveats.
Detailed evidence note Plain-language interpretation of author engagement and research updating, with linked case studies.
Evaluator guidance on feedback and assessment How feedback to authors, assessment for research users, and the longer-run career value of public evaluation fit together.
2026 author survey Small-sample author feedback on evaluation quality, usefulness, and reported research changes.

Evidence reviewed August 26, 2026. PubPub reach figures cover the two years ending on that date. Author-response figures are from the April 2026 dataset and should be updated when that dataset changes.

Help us improve the record

If an Unjournal evaluation affected a decision, recommendation, research agenda, or paper, or if a claim on this page needs correction, please contact contact@unjournal.org.