Comissioned Evaluations

By submitting your research and engaging with public evaluation, you send a powerful public signal that you are confident in your approach, open to constructive criticism, and motivated to seek the truth. Making the feedback public and responding to it publicly can also engage the “wisdom of the crowd”, get others interested in this work, and attract potential collaborator.

Naturally, we cannot guarantee that evaluators will rate your work highly – if we did so, these ratings would be meaningless.

We want Unjournal evaluations to be seen as signals of research quality. Like all such signals, they are noisy, and will involve some randomness (which we try to minimize through vetting, checking, and seeking multiple sources of validation. 

If the research your organization produces or funds is given a low rating, you might be concerned that this would harm your reputation, morale, or funding.  But of course most researchers expect that opinions will differ about a piece of work, and everyone has received negative reviews. We think this is usually an acceptable risk, and it is counterbalanced by the benefits we describe above, as well as the public signal it sends about your openness to transparent feedback and to respond to criticism. And you can demonstrate this by responding to the evaluations, incorporating the suggestions, and having the revised work evaluated again.

Many research-active organizations have internal vetting and evaluation. But this is costly and has its limitations. 

It's hard to be open and honest with people on your team, and especially to offer criticism, when power relationships and career incentives are in play. And providing this feedback unfortunately can be stressful and strain team relationships. The Unjournal is not part of your team.

Depending on the size of your team, you are also unlikely to have people with the right expertise. Even very large organizations may only have one researcher or practitioner with a particularly relevant skill set. The Unjournal recruits world-leading experts, from within our 200+ evaluator pool and beyond. 

Some organizations do external experts to evaluate internal work on a contract basis. They offer and leverage their personal and professional links for this. But this also has limitations.

It’s a bother: It takes effort, administration and contracting. It distracts your team from their focus.  You need to come up with rubrics, find sources of expertise, chase down the consultants, et cetera.  The Unjournal has a process for this.

It doesn’t eliminate bias and conflict: Your research leads need to decide whom to consult, what process to follow, and how to characterize the outcomes. There are a lot of choices to make here, leaving the door open for perceived or actual unfairness. The Unjournal has a set process, and no stake in the outcomes.

You are welcome to do this as well as working with The Unjournal – our evaluation is not exclusive. 

But we offer several advantages over academic peer review. As detailed above, we can be quicker, we can provide definitive quantified ratings, and we guarantee in-depth, informative reports. 

We have fewer bureaucratic hoops to jump through. In particular, you don’t need to turn your research into a 32 page pdf with a 100 page appendix. The Unjournal can evaluate research hosted in any format, including dynamic documents with data dashboards, multiverse analysis, and interactive web sites.  And you can keep your work public, avoiding paywalls  and extortionary “open access fees” and “article processing charges”, 

We can provide expert scrutiny of niche applied research.  Top academic journals are likely to  “desk reject” this work as not cutting-edge or not in the interest of the academic field in general. You could submit your work to lower-ranked and more specialized news journals, but these are less likely to provide global expertise, detailed methodological feedback, or external credibility. 

We are eager to encourage academic and applied researchers to submit their work for Unjournal evaluation. Here’s our submission form.

However, as we manage the process carefully and compensate expert evaluations and evaluation managers from our existing grant funding, our capacity is limited. Our team uses a nomination, discussion, and voting process to prioritize research for evaluation, with a particularly considering its potential for global impact. Thus we can't guarantee that any particular piece of submitted research will get evaluated. 

If you want to be sure to get evaluation and feedback for a particular project or research line, let’s discuss commissioning a project.