Missing references and meeting abstracts in OJR
Two gaps in the open data behind OJR affect journals’ scores in different ways. Meeting abstracts that OpenAlex types as articles pulled some major clinical journals far below the world average until release 2025, whose abstract rule leaves most of them out: the Journal of the American College of Cardiology goes from 0.73 to 5.77. Missing reference lists, which remain, change OJR little for most journals (ρ 0.993 and a 5.5% median change when citations are reweighted for them), but a minority of mostly regional journals move a lot.
Published Last updated
By OpenJournalRank. OpenJournalRank is run by an independent research group.
Who, how and why. Published by the OpenJournalRank research group. We run OJR, so this study tests our own indicator. It uses release 2025 (method nci3-v1 with the change-log entry 2026-citable-abstracts) and the OpenAlex snapshot of 23 September 2026. Figures are for indicator year 2025, with 2024 as a check. What the abstract rule changed is measured by comparing the published scores of release 2025 with those of release 2026-Q4.1, which uses the same snapshot without the rule. An earlier version of this study, on release 2026-Q4.1, found the meeting-abstract problem; we report here how the data stand now.
Key findings
- Meeting abstracts pulled down major clinical journals’ OJR, and the abstract rule of release 2025 leaves most of them out. Between release 2026-Q4.1 and release 2025, the Journal of the American College of Cardiology goes from 0.73 to 5.77 (13,262 of its 14,865 citable papers from 2022 to 2024 left out), the European Heart Journal from 1.10 to 5.35 and Annals of Oncology from 0.58 to 3.79. All three move into the top quartile of their subfield.
- For most journals the abstract rule changes OJR little. It leaves out 606,873 of the 9,767,557 citable papers of CWTS Core journals for indicator year 2025 (6.2%). Across the 21,453 journals ranked with a score in both releases, ρ is 0.990 and the median change is 2.4%; 4.8% change subfield quartile. In Health Sciences the change is larger (ρ 0.964, 13.1% quartile changes) than in the other domains (ρ 0.998, 2.0%).
- Missing reference lists change OJR little for most journals. When each citation is weighted to make up for its publisher’s missing reference lists, ρ with the published OJR of release 2025 is 0.993 and the median journal’s OJR moves by 5.5%. These figures are almost the same as before the abstract rule (0.993 and 5.5% in release 2026-Q4.1).
- A minority of journals move a lot. The 95th percentile change is 32.7%, and 8.7% of journals change subfield quartile. 1,047 journals (4.9%) move by more than twice the standard error published with each score. In 2024 the figures are smaller: ρ 0.996, a 4.3% median change and 7.0% quartile changes.
- The size of the change follows exposure. 735 ranked journals (3.4%) receive more than 20 pp more of their citations from publishers with low reference coverage than is usual for comparable papers; most are regional and national journals cited by other regional journals whose publishers deposit few references.
- Coverage differs widely between publishers. Elsevier, the source of 28.5% of the citations counted to ranked journals in 2025, has matched references for 85.1% of its 2025 articles and reviews; Inderscience has them for 1.2%. Journals with no publisher recorded in OpenAlex published 30.6% of all journal articles and reviews in 2025, and only 22.7% of those papers have matched references.
- The Leiden Ranking’s rule of counting only papers with at least one reference, applied to every paper, would remove most papers of publishers that deposit no references. On top of release 2025 it would leave 241 ranked journals without a score, 75 of them published by Inderscience.
- We will not reweight OJR for missing references. For 95% of journals the change is within twice the standard error, and any correction depends on assumptions we cannot check from the data. We propose a note on journal pages where exposure is high and will repeat this check with every release.
How we did this
Meeting abstracts: before and after the abstract rule
Some major journals print thousands of meeting abstracts a year in supplements, and OpenAlex types them as articles. They are hardly ever cited, so while they counted as citable papers they pulled the journal’s OJR far down. The earlier version of this study, on release 2026-Q4.1, found that these were the largest changes in the data: larger than any change from missing reference lists. Release 2025 introduced the abstract rule, which leaves most of these papers out (see the methodology change log). In a sample check, 93.0% of the papers it leaves out were abstracts, news items, editorials or other front matter rather than research articles.
The table shows the journals that the earlier version named, before and after the rule.
| Journal | Citable papers, 2022 to 2024 | OJR, release 2026-Q4.1 | OJR, release 2025 | Subfield rank, before → after | Subfield quartile, before → after |
|---|---|---|---|---|---|
| Alzheimer’s & Dementia | 15,928 → 1,539 | 0.45 | 3.72 | 111 of 143 → 2 of 139 | Q4 → Q1 |
| Journal of the American College of Cardiology | 14,865 → 1,603 | 0.73 | 5.77 | 145 of 294 → 2 of 287 | Q2 → Q1 |
| European Heart Journal | 12,992 → 2,108 | 1.10 | 5.35 | 95 of 294 → 3 of 287 | Q2 → Q1 |
| CHEST | 12,308 → 1,722 | 0.26 | 1.37 | 278 of 355 → 39 of 352 | Q4 → Q1 |
| Annals of Oncology | 11,174 → 1,055 | 0.58 | 3.79 | 162 of 243 → 13 of 242 | Q3 → Q1 |
| Neurology | 10,710 → 2,653 | 0.56 | 1.85 | 93 of 172 → 16 of 168 | Q3 → Q1 |
| The Journal of Urology | 8,550 → 516 | 0.25 | 2.11 | 283 of 355 → 18 of 352 | Q4 → Q1 |
| Biophysical Journal | 9,271 → 1,268 | 0.12 | 0.67 | 1,094 of 1,147 → 766 of 1,152 | Q4 → Q3 |
The earlier version estimated these scores with a simpler rule, which left out every paper without references when the publisher covered at least half of its articles and reviews. Its estimates were close to the published scores of release 2025: 5.59 for the Journal of the American College of Cardiology, 5.20 for the European Heart Journal and 3.67 for Annals of Oncology.
For most journals the change is small. The rule leaves out 606,873 of the 9,767,557 citable papers of CWTS Core journals from 2022 to 2024 (6.2%):
| Journals | Number | ρ | Median change | 95th percentile change | Median signed change | Subfield quartile changes |
|---|---|---|---|---|---|---|
| All ranked journals | 21,453 | 0.990 | 2.4% | 11.9% | −2.1% | 4.8% |
| Health Sciences | 5,397 | 0.964 | 9.2% | 37.4% | −8.5% | 13.1% |
| Other domains | 16,056 | 0.998 | 1.6% | 7.6% | −1.5% | 2.0% |
Most journals fall slightly: 20,095 of 21,453. Leaving out rarely cited papers raises the world averages of the subfields where they were common, so a journal that printed no abstracts now has slightly more expected citations and a slightly lower score. The fall is largest in Health Sciences, where the rule applies its publisher test. 75 journals ranked in release 2026-Q4.1 are no longer ranked in release 2025, because they no longer reach the 30 citable papers needed for a rank (eligibility).
What remains
The rule leaves out papers without references only where it has a reason to treat them as front matter: a supplement signal, or Health Sciences with a publisher that deposits references. Outside Health Sciences, a paper without references still counts unless it carries a supplement signal. Some abstract series therefore keep their scores. Abstracts with Programs of the Geological Society of America still has 12,771 citable papers without references out of 12,843, and an OJR of 0.06 in both releases.
In release 2025, 565,411 citable papers of CWTS Core journals have no matched reference (6.2%, against 12.0% in release 2026-Q4.1): 5.7% of articles, 32.7% of conference papers and 1.5% of reviews. They are concentrated in a few publisher groups:
| Publisher group | Citable papers | Without references | Share |
|---|---|---|---|
| No publisher recorded | 452,456 | 165,838 | 36.7% |
| Inderscience | 43,370 | 42,660 | 98.4% |
| Wiley | 737,410 | 31,182 | 4.2% |
| Elsevier | 2,113,139 | 27,175 | 1.3% |
| Springer Nature | 1,281,440 | 17,766 | 1.4% |
| IEEE | 241,010 | 16,767 | 7.0% |
| Informa | 380,916 | 14,169 | 3.7% |
| Geological Society of America | 15,061 | 13,089 | 86.9% |
| American Psychological Association | 14,571 | 11,408 | 78.3% |
| University of Cambridge | 60,464 | 9,040 | 15.0% |
These papers are a mix of real articles whose references were not deposited and front matter that the rule does not reach.
The Leiden rule as written
The Leiden Ranking Open Edition counts only publications that have at least one reference[1]. We tested that rule on top of release 2025, and a narrower version that applies it only to papers whose publisher covered at least half of its articles and reviews in the publication year. Both keep only citable papers with at least one matched reference, in each journal’s Actual and Expected and in the world averages.
| Rule | ρ | Quartile changes | Median change | 95th percentile change | Beyond 2 standard errors | Journals left without a score | Journals falling to 0 |
|---|---|---|---|---|---|---|---|
| Leiden rule as written | 0.980 | 8.5% | 12.6% | 36.2% | 20.1% | 241 | 200 |
| Narrower rule (publisher covers ≥50%) | 0.993 | 5.3% | 5.9% | 19.2% | 4.4% | 22 | 38 |
The narrower rule now changes Health Sciences journals very little (ρ 0.998 within Health Sciences), because the abstract rule already leaves out most of what it would remove there. Its remaining effect is outside Health Sciences.
The Leiden rule as written removes most papers of publishers that deposit no references. Under it, 75 of the 254 ranked Inderscience journals would lose their score entirely, and the rest would be scored on the 1% to 2% of their papers that have references; 71 of them would score 0. Across all ranked journals, 241 would lose their score and 1,557 would fall below the 30 papers needed for a rank. A missing reference list is a publisher’s choice, not evidence that a paper is front matter. The abstract rule avoids this by applying its publisher test only where the publisher deposits references.
Why reference lists go missing
OpenAlex builds most of its citation links from the reference lists that publishers deposit with Crossref when they register DOIs. Depositing references is optional. Many publishers deposit references for every paper; some deposit them for part of their output; a few deposit none. When a paper has no deposited reference list, OpenAlex may still find references from other sources, but often it records none. The citations that paper made are then lost. Our methodology lists this among the known coverage problems and errors.
Since June 2022, every reference deposited with Crossref is openly available. Before that, publishers could choose to keep their references closed or limited. Crossref’s board removed that choice in March 2022, with effect from 3 June 2022[2]. The change opened references that had already been deposited. It did not make depositing them compulsory, so publishers that never deposited references still leave gaps.
Comparisons between bibliographic databases have documented this for years. A comparison of five data sources, Crossref among them, found that Crossref’s citation links were less complete than those of the commercial databases, partly because not every publisher made its reference lists available[3]. For 16.8 million recent publications indexed by OpenAlex, Web of Science and Scopus alike, OpenAlex has comparable reference counts and coverage on average[4]; because that comparison is limited to papers all three index, it says less about publishers whose references are missing. Paper-level field-normalized scores from OpenAlex and three commercial databases agree strongly overall, but universities’ mean scores agree poorly[5]. OpenAlex and Web of Science also differ in the document types they assign[6, preprint], which is why meeting abstracts are counted as articles.
How complete references are, by publisher
Coverage differs widely between large publishers, and for several of them it is lower in 2025 than in 2023.
We count a paper as covered when OpenAlex has at least one matched reference for it. We measured coverage for journal articles and reviews from every journal in the snapshot, grouped by publisher, where each journal is assigned to the top of its OpenAlex publisher hierarchy. The table shows the 30 largest publisher groups, sorted by the number of articles and reviews they published in 2025, largest first. The last column is the share of 2025 citations to ranked journals that each group’s papers make. Coverage does not depend on the abstract rule; the citation shares are those of release 2025.
| Publisher group (largest imprint) | 2023 | 2024 | 2025 | 2025, core journals only | Share of 2025 citations counted |
|---|---|---|---|---|---|
| Elsevier | 84.6% | 85.2% | 85.1% | 86.1% | 28.5% |
| Springer Nature | 93.5% | 94.1% | 94.1% | 97.1% | 14.3% |
| Wiley | 88.0% | 89.4% | 90.4% | 92.8% | 7.8% |
| MDPI | 99.8% | 99.8% | 96.7% | 96.7% | 8.2% |
| Informa (Taylor & Francis) | 91.7% | 91.3% | 90.8% | 94.7% | 3.3% |
| IEEE | 93.2% | 92.5% | 87.3% | 87.5% | 2.1% |
| Wolters Kluwer (Lippincott Williams & Wilkins) | 78.3% | 81.4% | 75.4% | 75.7% | 1.1% |
| University of Oxford (Oxford University Press) | 65.5% | 66.4% | 60.0% | 63.7% | 1.1% |
| American Chemical Society | 97.0% | 98.9% | 99.1% | 99.5% | 3.3% |
| SAGE Publishing | 91.1% | 91.9% | 92.2% | 95.0% | 1.4% |
| Frontiers Media | 99.9% | 99.9% | 99.7% | 99.7% | 2.8% |
| Royal Society of Chemistry | 98.6% | 98.3% | 98.1% | 98.1% | 2.0% |
| Institute of Physics | 97.2% | 97.4% | 94.8% | 95.2% | 1.0% |
| University of Cambridge (Cambridge University Press) | 64.4% | 60.2% | 58.2% | 64.7% | 0.2% |
| American Physical Society | 99.0% | 98.9% | 96.0% | 96.0% | 0.6% |
| De Gruyter | 85.1% | 81.0% | 74.9% | 85.7% | 0.3% |
| Public Library of Science | 99.9% | 99.9% | 99.5% | 99.5% | 0.5% |
| Cureus | 99.6% | 99.4% | 98.6% | 98.6% | 0.2% |
| Emerald Publishing | 96.5% | 96.5% | 95.0% | 95.9% | 0.7% |
| American Institute of Physics | 94.6% | 96.9% | 96.4% | 98.1% | 0.6% |
| GSC Online Press | 54.7% | 18.1% | 0.9% | 0.0% | <0.01% |
| BMJ | 85.6% | 85.8% | 87.3% | 87.3% | 0.3% |
| Thieme | 56.9% | 58.8% | 75.5% | 83.7% | 0.2% |
| Egypt’s Presidential Specialized Council for Education and Scientific Research | 45.7% | 18.8% | 3.3% | 8.2% | <0.01% |
| Inderscience | 1.2% | 0.2% | 1.2% | 1.2% | <0.01% |
| Universitas Pahlawan Tuanku Tambusai | 82.7% | 27.0% | 7.9% | – | <0.01% |
| Science Press (EDP Sciences) | 84.1% | 77.7% | 73.6% | 82.9% | 0.3% |
| World Scientific | 91.9% | 92.7% | 80.1% | 80.4% | 0.2% |
| Servicios Académicos Intercontinentales | 84.9% | 57.4% | 20.1% | – | <0.01% |
| Optica Publishing Group | 98.1% | 97.9% | 98.4% | 98.3% | 0.2% |
One more group does not appear in the table: journals for which OpenAlex records no publisher at all. They published 1.95 million articles and reviews in 2025, 30.6% of the total, and only 22.7% of them have matched references (36.1% in 2024). We treat them as one pooled group. Together they make 4.3% of the citations counted to ranked journals in 2025.
Inside a group, imprints can differ. In 2025 Lippincott Williams & Wilkins covers 67.7% and Medknow 84.9%, both part of Wolters Kluwer. The Oxford University Press imprint covers 62.9%.
Two things stand out. First, the large publishers are not all alike: Elsevier, the source of more than a quarter of all counted citations, covers about 85% of its articles, while Springer Nature covers 94% and Frontiers, MDPI and the American Chemical Society are close to 100%. Second, coverage in 2025 is lower than in 2023 for several publishers. Some of that is a delay between publication and reference deposit, which the next snapshot may fill.
These snapshot figures agree with outside sources. On 20 October 2026, the live OpenAlex API[7] showed these shares of 2025 journal articles and reviews with no references: Oxford University Press 36.0%, Lippincott Williams & Wilkins 31.6%, Wolters Kluwer overall 23.5%, Inderscience 98.7%, Elsevier about 15%, Wiley about 10%, Springer Nature about 6% and MDPI 3.2%. Crossref’s own participation reports for current content[8], consulted on 20 October 2026, show references deposited for 0% of Inderscience’s content, 41% of Oxford University Press’s, 67% of Ovid and Lippincott Williams & Wilkins’s, 69% of Wiley’s, 74% of Elsevier’s, 86% of Springer’s and 88% of MDPI’s. The sources measure slightly different things, but they rank publishers the same way.
Not every paper without references is a gap. Meeting abstracts, editorials and short notes often have no references at all, and some of them are typed as articles. Coverage figures therefore overstate how many reference lists are missing, more so for publishers that print many meeting abstracts.
Why most of the loss cancels out
OJR is a ratio, Actual ÷ Expected (see the OJR formula). Actual counts the citations a journal’s papers received in the indicator year. Expected is the number of citations papers of the same subfield, publication year and document type received on average worldwide, summed over the journal’s papers. Both are counted from the same citing papers.
Suppose every publisher lost 15% of its reference lists. Every journal’s Actual would fall by about 15%, and so would every world average, so every OJR would stay the same. A uniform loss cancels exactly.
A journal is affected only when its citations come from different publishers than the citations of comparable papers. If a cardiology journal is cited mostly by journals of a publisher with 60% coverage, while cardiology papers in general are cited by journals with 85% coverage, that journal loses more citations than its world average does, and its OJR is too low.
So the question is not how many citations are missing in total, but how unevenly they are missing. Publishers with low coverage publish almost half of all journal articles, but they make only 7% of the counted citations:
| Citing publisher’s coverage | Share of counted citations | Share of citing journal articles and reviews |
|---|---|---|
| 90% or more | 50.4% | 30.6% |
| 80% to 90% | 32.2% | 19.3% |
| 70% to 80% | 2.7% | 3.5% |
| Below 70% | 7.0% | 46.7% |
| Outside the journal catalogue (books, proceedings, unknown source) | 7.7% | – |
Most publishers with coverage below 70% are small publishers of journals outside CWTS Core. In 2025, 84.3% of articles and reviews in CWTS Core journals have matched references, against 24.9% in other journals. Papers in those other journals are rarely the source of citations to ranked journals, and their missing references are mostly missing citations to each other.
The sensitivity tests
We recomputed the OJR of release 2025 for every ranked journal in several ways. Each time, we rebuilt the world averages with exactly the same rule, so the world average stays 1.00 and the scores remain comparable.
- Coverage cut-off tests (90% and 80%). We counted only citations from journal papers whose publisher covered at least 90% (or 80%) of its articles and reviews in the citing year. A third version uses the 90% cut-off but keeps citations from works outside the journal catalogue, whose coverage we cannot measure.
- Coverage-weighted tests. We weighted each citation by one divided by its publisher’s coverage in the citing year, capped at 5 (or 10). A citation from a publisher covering 60% of its papers counts 1.67 times, standing in for the citations its uncovered papers probably made. Citations from outside the catalogue keep weight 1. A journal-level version uses the citing journal’s own coverage instead of its publisher’s when the journal published at least 50 articles and reviews that year.
- Exposure. For each journal, we took the share of its citations that come from publishers with coverage below 70%, and subtracted the share expected for papers of the same subfields, years and types. The median journal’s exposure is +0.5 pp; 735 ranked journals (3.4%) are more than 20 pp above expectation.
The coverage-weighted tests change the ranking least; the cut-off tests change it most.
| Test | ρ | Subfield quartile changes | Median change | 95th percentile change | Changes beyond 2 standard errors | Correlation of exposure with log change |
|---|---|---|---|---|---|---|
| 90% coverage cut-off | 0.974 | 15.4% | 11.5% | 46.2% | 18.6% | −0.37 |
| 80% coverage cut-off | 0.984 | 10.7% | 6.8% | 39.1% | 7.9% | −0.55 |
| 90% cut-off, keeping citers outside the catalogue | 0.988 | 11.9% | 10.2% | 36.1% | 12.9% | −0.51 |
| Coverage-weighted test (capped at 5) | 0.993 | 8.7% | 5.5% | 32.7% | 4.9% | +0.90 |
| Coverage-weighted test (capped at 10) | 0.992 | 9.1% | 5.7% | 34.6% | 5.5% | +0.89 |
| Journal-level coverage-weighted test (capped at 5) | 0.994 | 7.6% | 4.6% | 27.8% | 3.3% | +0.78 |
Changes are absolute percentage changes in OJR. Under the cut-off tests, 86, 45 and 47 journals respectively lose every counted citation; their OJR falls to 0 and counts as a −100% change. The log change needs both scores above 0, so those journals are left out of the correlation only. The correlation (Pearson’s) is negative for the cut-off tests, which drop low-coverage citers and so lower the scores of journals that rely on them, and positive for the coverage-weighted tests, which give those citers more weight. “Beyond 2 standard errors” counts journals whose OJR moves by more than twice the standard error published with the rank ranges.
The same tests move journals less in 2024:
| Test | ρ | Subfield quartile changes | Median change | 95th percentile change | Changes beyond 2 standard errors | Correlation of exposure with log change |
|---|---|---|---|---|---|---|
| 90% coverage cut-off | 0.970 | 17.4% | 13.1% | 52.5% | 21.3% | −0.41 |
| 80% coverage cut-off | 0.980 | 12.3% | 7.6% | 44.4% | 9.6% | −0.60 |
| 90% cut-off, keeping citers outside the catalogue | 0.983 | 14.0% | 11.9% | 44.0% | 16.7% | −0.54 |
| Coverage-weighted test (capped at 5) | 0.996 | 7.0% | 4.3% | 24.6% | 2.2% | +0.93 |
| Coverage-weighted test (capped at 10) | 0.996 | 7.1% | 4.4% | 24.9% | 2.3% | +0.92 |
| Journal-level coverage-weighted test (capped at 5) | 0.996 | 6.6% | 4.2% | 23.5% | 2.2% | +0.81 |
These results hardly depend on the abstract rule. On release 2026-Q4.1, the coverage-weighted test (capped at 5) gave ρ 0.993, a 5.5% median change, a 32.8% 95th percentile change and 4.9% of journals beyond two standard errors for 2025; on release 2025 it gives 0.993, 5.5%, 32.7% and 4.9%.
The coverage-weighted test (capped at 5) is our best estimate of the bias. It keeps every citation and only adjusts for what is missing. Its changes follow exposure closely (correlation 0.90 in 2025, 0.93 in 2024). That confirms the arithmetic works as intended: journals move in proportion to how many of their citations come from low-coverage publishers. It does not show that the weights are right, because the weights and exposure are built from the same coverage figures. Raising the cap from 5 to 10 changes little, and using each citing journal’s own coverage gives slightly smaller changes.
The cut-off tests move journals more, but for a different reason. They drop whole publishers. At the 90% line, Elsevier (85%) falls out, taking 28.5% of all counted citations with it, and Wiley sits right at the line (89.4% in 2024, 90.4% in 2025). Most of what the 90% cut-off measures is therefore which journals happen to be cited from Elsevier journals, not which journals lose citations to missing references. The 80% cut-off keeps Elsevier and Wiley and drops mainly Oxford, Cambridge, Wolters Kluwer, De Gruyter and small publishers; its changes follow exposure more closely. We report the cut-off tests for completeness, but we do not read them as estimates of the bias.
Clinical medicine
Clinical journals gain slightly on average under the coverage-weighted test (median +1.8%), but their absolute changes are smaller than average (4.6% against 5.5%). Coverage is lowest among several publishers that are strong in medicine, such as Lippincott Williams & Wilkins, Oxford University Press and Thieme. Clinical journals do have higher exposure than average, but the effect on their scores is modest.
| Field | Ranked journals | ρ | Quartile changes | Median change | Median signed change | Median exposure |
|---|---|---|---|---|---|---|
| Medicine | 4,521 | 0.996 | 6.4% | 4.4% | +1.9% | +1.2 pp |
| Health Professions | 586 | 0.990 | 9.3% | 5.6% | −0.2% | +0.8 pp |
| Dentistry | 171 | 0.994 | 10.7% | 7.0% | +6.3% | +2.7 pp |
| Veterinary | 68 | 0.994 | 10.0% | 5.8% | +2.4% | +2.7 pp |
| Nursing | 60 | 0.993 | 19.6% | 4.8% | +3.2% | +1.7 pp |
| All Health Sciences | 5,406 | 0.995 | 7.0% | 4.6% | +1.8% | +1.2 pp |
The signed change gives the direction: under this test the typical clinical journal would gain about 2% relative to the world average. Their ranking across all clinical journals hardly changes (ρ 0.995), because clinical journals largely share the same citing publishers. The small fields move more in quartile terms: in Nursing, 10 of the 51 journals with a quartile change it. In 2024 the Health Sciences figures are similar: ρ 0.997, 5.7% quartile changes, median change 3.9%.
Inderscience
Inderscience deposits almost no references (1.2% coverage in 2025), so its 254 ranked journals are a test case. Their own reference lists hardly matter for their scores: Inderscience papers make less than 0.01% of the citations counted to ranked journals, and journal self-citations are excluded anyway. What matters is who cites Inderscience journals, and that is mostly other publishers. Under the coverage-weighted test their median OJR rises by 3.1%, only 2.0% of them change quartile, and ρ is 0.970. The Leiden rule as written is a different matter.
Which journals are affected, and why
The journals that move most are those whose citations come largely from publishers with low coverage. The table lists the 15 largest changes under the coverage-weighted test (capped at 5) among ranked journals with at least 100 counted citations, largest first.
| Journal | Publisher | Subfield | OJR | Weighted OJR | Change | Exposure |
|---|---|---|---|---|---|---|
| Romanian Journal of Legal Medicine | not recorded | Pharmacy | 0.56 | 1.67 | +199% | +80 pp |
| Journal of Avian Biology | Wiley | Ecology | 1.35 | 2.97 | +120% | +46 pp |
| Journal of Applied Research | not recorded | Development | 0.09 | 0.20 | +119% | +60 pp |
| SAE International Journal of Vehicle Dynamics, Stability, and NVH | not recorded | Automotive Engineering | 1.32 | 2.82 | +114% | +43 pp |
| Revista Médica Clínica Las Condes | Elsevier | Pulmonary and Respiratory Medicine | 0.33 | 0.69 | +110% | +48 pp |
| Environment and Social Psychology | not recorded | Education | 1.20 | 2.32 | +93% | +42 pp |
| Cuadernos del Centro de Estudios de Diseño y Comunicación | University of Palermo | Visual Arts and Performing Arts | 0.07 | 0.13 | +85% | +44 pp |
| European Public & Social Innovation Review | not recorded | Education | 0.49 | 0.91 | +84% | +42 pp |
| International Journal of Radiation Research | not recorded | Radiology, Nuclear Medicine and Imaging | 0.18 | 0.32 | +82% | +33 pp |
| Revista Interuniversitaria de Investigación en Tecnología Educativa | Editum, Universidad de Murcia | Information Systems | 1.63 | 2.95 | +80% | +40 pp |
| International Journal of Data Mining and Bioinformatics | Inderscience | Business and International Management | 0.29 | 0.53 | +79% | +32 pp |
| Contaduría y Administración | Elsevier | Economics and Econometrics | 0.52 | 0.93 | +78% | +39 pp |
| Educación Médica | Elsevier | Public Health, Environmental and Occupational Health | 0.38 | 0.64 | +69% | +35 pp |
| Journal of Engineering and Applied Sciences Technology | not recorded | Computer Networks and Communications | 0.18 | 0.30 | +67% | +24 pp |
| Psico-USF | Universidade de São Francisco | Clinical Psychology | 0.45 | 0.73 | +63% | +35 pp |
Three patterns run through the list:
- Regional citing communities. Many of these journals are cited mostly by journals in the same country or language, often small university or society journals for which OpenAlex records no publisher and which deposit few references. For the Romanian Journal of Legal Medicine, 92 of its 101 counted citations in 2025 come from publishers with coverage below 70%. Their OJR probably understates how often they are cited. These journals also have modest citation counts, so a few hundred missing citations make a large relative difference.
- Subject communities outside the big publishers. The Journal of Avian Biology is cited heavily by ornithology journals with no publisher recorded in OpenAlex, such as The Auk, Ardea and El Hornero; 278 of its 541 counted citations come from that pooled group.
- Large changes rest on few citing papers. The pooled group of journals without a recorded publisher covers 22.7% of its papers, so each of its citations counts 4.4 times under this test. Journals in that pool are very different from each other, and the few that do have references may not cite like the many that do not. The journal-level version gives similar or larger changes for most journals in the table, but the largest changes are still the least certain.
Grouped by the cited journal’s own publisher, the effect is small. The median change is −1.4% for Elsevier journals, −1.1% for Springer Nature, −1.7% for Wiley, −1.9% for Oxford University Press, +2.8% for Wolters Kluwer and +3.1% for Inderscience. A publisher’s own coverage hardly affects its own journals: what matters is the coverage of the journals that cite them. Journals with no publisher recorded in OpenAlex, often small national journals, gain most (median +8.3%).
What we’ll do
No correction for reference gaps in OJR. The coverage-weighted scores rank journals almost exactly as the published ones do (ρ 0.993 in 2025 and 0.996 in 2024), and for 95% of journals the change is within twice the standard error. A correction would rest on assumptions we cannot test: that papers without deposited references cite like papers with them, that every paper without references is a missing list rather than front matter, and that a few captured references from a low-coverage publisher represent all its papers. It would also have to change as publishers start depositing (Thieme’s coverage rose from 57% to 76% in two years). We would be replacing a known, measurable gap with a modelled one.
A note where it matters. For journals whose exposure is more than 20 pp above expectation (735 ranked journals in 2025, 3.4%), we propose a short note on the journal page: a large share of the citations to this journal come from publishers that deposit few reference lists, so OJR probably understates its citation impact. We also propose showing exposure among the context measures. Both need a separate design decision.
The note and the 1,047 journals (4.9%) whose coverage-weighted score moves beyond two standard errors answer different questions, so they overlap only in part. The note asks whether a journal’s citations come unusually often from low-coverage publishers. The two-standard-error count asks whether a change is large compared with the score’s own uncertainty. Of the 735 high-exposure journals, 354 also change by more than two standard errors. Most of the other 381 are small journals (median 14 counted citations) whose scores are already uncertain, so even a large change stays within two standard errors. The remaining 693 journals beyond two standard errors have lower exposure (median +7.6 pp) but are larger (median 873 counted citations) with precise scores, so a moderate change (median 19%) is enough to cross the line. The note targets the journals where missing references are the clear cause.
This study runs with every release. Coverage changes from snapshot to snapshot, so we will rerun this check for each release and publish its headline figures, including what remains of the front matter outside the abstract rule’s reach.
Limitations
- Coverage counts papers with any matched reference, so a paper with an incomplete reference list counts as covered.
- Citing works outside the journal catalogue (7.7% of counted citations) have no measurable coverage and are left unweighted.
- Exposure uses a 70% coverage line, which is a choice.
- The comparison between releases shows what the abstract rule changed, not whether each paper it leaves out is front matter; that precision was checked separately on samples for the change log.
- Live OpenAlex and Crossref figures were consulted on 20 October 2026 for context only and did not enter the calculations.
References
The DOIs were resolved against OpenAlex on 20 October 2026, and each work’s title, first author and year were checked. Web pages that can change carry the date they were read. Preprints, documentation and databases are labelled.
- CWTS (n.d.). Leiden Ranking Open Edition: Indicators. open.leidenranking.com. Documentation. https://open.leidenranking.com/information/indicators
- Crossref (2022). Amendments to membership terms to open reference distribution and include UK jurisdiction. Crossref blog. Blog post. Accessed October 20, 2026. https://www.crossref.org/blog/amendments-to-membership-terms-to-open-reference-distribution-and-include-uk-jurisdiction/
- Visser M, van Eck NJ, Waltman L (2021). Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. Quantitative Science Studies 2(1):20–41. https://doi.org/10.1162/qss_a_00112
- Culbert JH, Hobert A, Jahn N, Haupka N, Schmidt M, Donner P, Mayr P (2025). Reference coverage analysis of OpenAlex compared to Web of Science and Scopus. Scientometrics 130:2475–2492. https://doi.org/10.1007/s11192-025-05293-3
- Scheidsteger T, Haunschild R, Bornmann L (2025). How similar are field-normalized citation impact scores obtained from OpenAlex and three popular commercial databases? Scientometrics 130:3537–3569. https://doi.org/10.1007/s11192-025-05338-7
- Mongeon P, Hare M, Riddle P, Wilson S, Krause G, Marjoram R, Toupin R (2025). Investigating document type, language, publication year, and author count discrepancies between OpenAlex and Web of Science. arXiv. Preprint. https://doi.org/10.48550/arXiv.2508.18620
- OpenAlex (n.d.). OpenAlex API. api.openalex.org. Database. Accessed October 20, 2026. https://api.openalex.org
- Crossref (n.d.). Participation reports. crossref.org. Database. Accessed October 20, 2026. https://www.crossref.org/members/prep/
Cite as: OpenJournalRank (2026). Missing references and meeting abstracts in OJR. Version of October 21, 2026. https://openjournalrank.com/en/research/missing-references-meeting-abstracts/
To report an error or suggest a correction, email contact@openjournalrank.com.