Paper Lookup

Paper Lookup

You have access to 10 academic paper databases through their REST APIs. Your job is to figure out which database(s) best serve the user's query, call them, and return the results.

Core Workflow

1. Understand the query -- What is the user looking for? A specific paper by DOI? Papers on a topic? An author's publications? Open access PDFs? Full text? This determines which database(s) to hit.

2. Select database(s) -- Use the database selection guide below. Many queries benefit from hitting multiple databases -- for example, searching PubMed for papers and then checking Unpaywall for open access copies.

3. Read the reference file -- Each database has a reference file in references/ with endpoint details, query formats, and example calls. Read the relevant file(s) before making API calls.

4. Make the API call(s) -- See the Making API Calls section below for which HTTP fetch tool to use on your platform.

5. Return results -- Always return:

• The raw JSON (or parsed XML for arXiv) response from each database
• A list of databases queried with the specific endpoints used
• If a query returned no results, say so explicitly rather than omitting it

Database Selection Guide

Match the user's intent to the right database(s).

By Use Case

Cross-Database Queries

When a query spans multiple needs (e.g., "find papers about CRISPR and get me the PDFs"), query the relevant databases in parallel.

Common Identifier Formats

Different databases use different identifier systems. If a query fails, the identifier format may be wrong.

Cross-referencing IDs: Semantic Scholar accepts DOI, PMID, PMCID, and arXiv ID via prefixes (e.g., DOI:10.1038/nature12373, PMID:34567890, ARXIV:2103.15348). OpenAlex accepts DOI and PMID via prefixes (doi:10.1038/..., pmid:34567890). Use the PMC ID Converter to translate between PMID, PMCID, and DOI.

API Keys and Access

Most of these databases are fully open. A few benefit from API keys for higher rate limits.

Databases requiring or benefiting from API keys

Fully open databases (no key needed)

Loading API keys

1. Check the environment first -- the key may already be exported (e.g., $NCBI_API_KEY). 2. Fall back to `.env` -- check .env in the current working directory. 3. Proceed without -- most APIs still work at lower rate limits. Tell the user which key is missing and how to get one.

Making API Calls

Use your environment's HTTP fetch tool to call REST endpoints:

If the fetch tool fails, fall back to curl via whatever shell tool is available.

Special cases

• arXiv returns Atom XML, not JSON. Parse it or use curl and extract the relevant fields. Consider piping through a simple parser if available.
• PMC eFetch returns JATS XML for full text. This is expected -- full text articles are in XML format.
• Crossref and Unpaywall benefit from including a mailto parameter or email for the polite/fast pool.

Request guidelines

• For NCBI APIs (PubMed, PMC): max 3 req/sec without key, 10 with key. Make requests sequentially.
• For arXiv: max 1 request every 3 seconds. Be patient.
• For Crossref: 5 req/sec (public), 10 req/sec (polite pool with mailto).
• For other APIs with no strict limits, you can query multiple databases in parallel.
• If you get HTTP 429 (rate limit), wait briefly and retry once.

Error recovery

1. Check the identifier format -- use the Common Identifier Formats table. A PMID won't work in arXiv, an arXiv ID won't work in PubMed directly. 2. Try alternative identifiers -- if a DOI fails in one database, try the title or PMID instead. 3. Try a different database -- if PubMed returns nothing for a CS paper, try Semantic Scholar or OpenAlex. 4. Report the failure -- tell the user which database failed, the error, and what you tried instead.

Output Format

Structure your response like this:

## Databases Queried
- **PubMed** -- esearch + esummary for "CRISPR gene therapy"
- **Unpaywall** -- DOI lookup for 10.1038/...

## Results

### PubMed
[raw JSON response or formatted results]

### Unpaywall
[raw JSON response]

If results are very large, present the most relevant portion and note that more data is available. But default to showing the full raw JSON -- the user asked for it.

Available Databases

Read the relevant reference file before making any API call.

Biomedical Literature

Preprint Servers

Multidisciplinary Indexes

Open Access & Full Text

arXiv API

arXiv is a preprint server for physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering, and economics.

Important: The arXiv API returns Atom XML, not JSON. There is no JSON option.

Base URL

https://export.arxiv.org/api/query

Authentication

None required. Fully public.

Query Parameters

GET https://export.arxiv.org/api/query?search_query={query}&start={n}&max_results={n}

*At least one of search_query or id_list must be provided. They can be combined (intersection).

Search Field Prefixes

Boolean Operators

• AND -- both conditions
• OR -- either condition
• ANDNOT -- exclude
• Parentheses for grouping (URL-encode as %28 / %29)
• Quoted phrases (URL-encode as %22)

Example Queries

Search all fields:

https://export.arxiv.org/api/query?search_query=all:transformer+attention&max_results=5

Author + category:

https://export.arxiv.org/api/query?search_query=au:hinton+AND+cat:cs.LG&max_results=10

Title search:

https://export.arxiv.org/api/query?search_query=ti:%22attention+is+all+you+need%22

By ID:

https://export.arxiv.org/api/query?id_list=2103.15348

Multiple IDs:

https://export.arxiv.org/api/query?id_list=2103.15348,2005.14165,1706.03762

Date range:

https://export.arxiv.org/api/query?search_query=cat:cs.AI+AND+submittedDate:[202401010000+TO+202412312359]

Response Format (Atom XML)

<feed xmlns="http://www.w3.org/2005/Atom">
  <opensearch:totalResults>1234</opensearch:totalResults>
  <opensearch:startIndex>0</opensearch:startIndex>
  <opensearch:itemsPerPage>10</opensearch:itemsPerPage>

  <entry>
    <id>http://arxiv.org/abs/1706.03762v7</id>
    <title>Attention Is All You Need</title>
    <summary>The dominant sequence transduction models are based on...</summary>
    <published>2017-06-12T17:57:34Z</published>
    <updated>2023-08-02T00:00:12Z</updated>
    <author><name>Ashish Vaswani</name></author>
    <author><name>Noam Shazeer</name></author>
    <!-- more authors -->
    <category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
    <arxiv:primary_category term="cs.CL"/>
    <link rel="alternate" href="http://arxiv.org/abs/1706.03762v7"/>
    <link rel="related" type="application/pdf" href="http://arxiv.org/pdf/1706.03762v7"/>
    <arxiv:doi>10.48550/arXiv.1706.03762</arxiv:doi>
    <arxiv:comment>15 pages, 5 figures</arxiv:comment>
    <arxiv:journal_ref>Advances in Neural Information Processing Systems 30 (NIPS 2017)</arxiv:journal_ref>
  </entry>
</feed>

Key XML elements per entry

Parsing Tips

Since arXiv returns XML, you'll need to parse it. With curl, you can pipe the output and extract what you need. The XML namespace is http://www.w3.org/2005/Atom with arXiv extensions in http://arxiv.org/schemas/atom.

For practical extraction, the key data is in <entry> elements. Each entry's <id> contains the arXiv ID in the URL path.

Common Categories

Full list: https://arxiv.org/category_taxonomy

Rate Limits

• 1 request every 3 seconds (hard limit)
• Single connection at a time
• Search results are cached daily -- same query won't show new results within 24 hours
• For bulk data, use the OAI-PMH interface instead

bioRxiv API

bioRxiv is a preprint server for biology. The API provides metadata for preprints, including title, authors, abstract, DOI, and publication status.

Important: The bioRxiv API has no keyword search. It supports date-range browsing and DOI lookup only. For keyword search of bioRxiv preprints, use Semantic Scholar, OpenAlex, or CORE instead.

Base URL

https://api.biorxiv.org

Authentication

None required. Fully public API.

Key Endpoints

1. Content Detail -- Browse by date range

GET /details/biorxiv/{interval}/{cursor}/{format}

Optional query parameter: ?category=neuroscience (filter by category, use underscores for spaces)

Examples:

https://api.biorxiv.org/details/biorxiv/2024-01-01/2024-01-31/0
https://api.biorxiv.org/details/biorxiv/5
https://api.biorxiv.org/details/biorxiv/10d
https://api.biorxiv.org/details/biorxiv/2024-01-01/2024-01-31?category=neuroscience

2. Content Detail -- DOI lookup

GET /details/biorxiv/{doi}/na/{format}

Example:

https://api.biorxiv.org/details/biorxiv/10.1101/2024.01.16.575895/na/json

3. Published Article Links

GET /pubs/biorxiv/{interval}/{cursor}
GET /pubs/biorxiv/{doi}/na

Links preprints to their published journal versions. Accepts both preprint DOI and published DOI.

4. Publisher Filter

GET /publisher/{prefix}/{interval}/{cursor}

Find bioRxiv papers published by a specific publisher (by DOI prefix).

Example:

https://api.biorxiv.org/publisher/10.15252/2024-01-01/2024-06-01/0

Response Format

{
  "messages": [{
    "status": "ok",
    "count": 100,
    "total": "1029",
    "cursor": 0
  }],
  "collection": [{
    "title": "Paper title...",
    "authors": "Surname, A.; Surname, B.",
    "author_corresponding": "Full Name",
    "author_corresponding_institution": "Institution",
    "doi": "10.1101/2024.01.16.575895",
    "date": "2024-01-20",
    "version": "1",
    "type": "new results",
    "license": "cc_no",
    "category": "cancer biology",
    "jatsxml": "https://www.biorxiv.org/content/early/.../source.xml",
    "abstract": "Full abstract text...",
    "published": "10.1158/2159-8290.CD-24-0187",
    "server": "bioRxiv"
  }]
}

• published is "NA" if not yet published in a journal, or the published DOI if it has been.
• type values: new results, confirmatory results, contradictory results

Pagination

All multi-result endpoints return 100 results per page. Use cursor to paginate. The messages object tells you the total count.

Rate Limits

No documented rate limits. No authentication required. Be reasonable with request frequency.

Categories

animal-behavior-and-cognition, biochemistry, bioengineering, bioinformatics, biophysics, cancer-biology, cell-biology, clinical-trials, developmental-biology, ecology, epidemiology, evolutionary-biology, genetics, genomics, immunology, microbiology, molecular-biology, neuroscience, paleontology, pathology, pharmacology-and-toxicology, physiology, plant-biology, scientific-communication-and-education, synthetic-biology, systems-biology, zoology

CORE API

CORE aggregates open access research from 15,000+ repositories worldwide. It provides full text for 37M+ articles and metadata for 368M+ papers.

Base URL

https://api.core.ac.uk/v3

Important: GET search paths require a trailing slash (e.g., /v3/search/works/ not /v3/search/works).

Authentication

• Header: Authorization: Bearer YOUR_API_KEY
• Query param: ?api_key=YOUR_API_KEY
• Register at: https://core.ac.uk/services/api

Without auth: Basic metadata queries work, but full text is NOT available (returns "Not available for public API users").

Rate Limits (token-based)

Simple queries cost 1 token. Downloads and scroll pagination cost 3-5 tokens.

Key Endpoints

1. Search works

GET /v3/search/works/?q={query}&limit={n}&offset={n}

POST alternative (for complex queries):

POST /v3/search/works
Content-Type: application/json

{"q": "machine learning", "limit": 10, "offset": 0}

Example:

https://api.core.ac.uk/v3/search/works/?q=CRISPR+gene+therapy&limit=10

2. Query language

Searchable fields: abstract, arxivId, authors, contributors, createdDate, dataProviders, depositedDate, documentType, doi, fullText, id, language, license, oai, title, yearPublished

3. Get work by ID

GET /v3/works/{id}

id is a CORE Work ID (integer). Example: /v3/works/267312

4. Get output by ID

GET /v3/outputs/{id}

5. Download full text

GET /v3/outputs/{id}/download

Returns binary PDF. Requires authentication.

GET /v3/works/tei/{id}

Returns TEI XML format.

6. Search outputs

GET /v3/search/outputs/?q={query}&limit={n}&offset={n}

Search by DOI: q=doi:10.1038/nature12373

Response Format

Search response

{
  "totalHits": 2281337,
  "limit": 10,
  "offset": 0,
  "scrollId": null,
  "results": [...]
}

Work object (key fields)

{
  "id": 8848131,
  "title": "Attention Is All You Need",
  "authors": [{"name": "Ashish Vaswani"}, ...],
  "abstract": "The dominant sequence...",
  "doi": "10.48550/arXiv.1706.03762",
  "arxivId": "1706.03762",
  "yearPublished": 2017,
  "downloadUrl": "https://core.ac.uk/download/...",
  "fullText": "Full text content (when authenticated)...",
  "language": {"code": "en", "name": "English"},
  "documentType": "research",
  "citationCount": 145678,
  "dataProviders": [{"name": "arXiv"}],
  "links": [{"type": "download", "url": "..."}]
}

Pagination

• Standard: offset + limit (max 10,000 results)
• Scroll: Set scroll=true. Response includes scrollId. Use in subsequent requests to page beyond 10,000 (costs more tokens).

Error Handling

Under heavy load, the API may return partial shard failure messages. These are transient -- retry after a brief wait.

Crossref API

Crossref is the DOI registration agency for scholarly content. It provides metadata for 150M+ works including journal articles, books, conference papers, datasets, and preprints.

Base URL

https://api.crossref.org

Authentication

None required. Add mailto=you@example.com to get into the polite pool (2x faster rate limits).

Rate Limits

HTTP 429 = temporarily blocked.

Key Endpoints

1. Search works

GET /works?query={text}&rows={n}&mailto=you@example.com

Example:

https://api.crossref.org/works?query=CRISPR+gene+therapy&filter=from-pub-date:2024-01-01,type:journal-article,has-abstract:true&rows=5&sort=published&order=desc&mailto=you@example.com

2. Get work by DOI

GET /works/{doi}?mailto=you@example.com

URL-encode the DOI: 10.1038/nature12373 becomes 10.1038%2Fnature12373

Example:

https://api.crossref.org/works/10.1038%2Fnature12373?mailto=you@example.com

3. Journals

GET /journals?query={name}&rows={n}
GET /journals/{issn}
GET /journals/{issn}/works?query={text}&rows={n}

4. Funders

GET /funders?query={name}
GET /funders/{id}
GET /funders/{id}/works?rows={n}

Funder IDs are from the Funder Registry (e.g., 100000001 for NSF).

5. Members (publishers)

GET /members?query={name}
GET /members/{id}/works?rows={n}

Key Filters

Date filters (accept YYYY, YYYY-MM, YYYY-MM-DD)

Boolean filters

Value filters

Syntax: filter=name1:value1,name2:value2

Pagination

Offset-based (max 10,000)

/works?query=cancer&rows=100&offset=200

Cursor-based (unlimited)

1. First request: ?cursor=*&rows=100 2. Response includes next-cursor 3. Next request: ?cursor={next-cursor-value}&rows=100 4. Cursors expire after 5 minutes

Response Format

List response

{
  "status": "ok",
  "message-type": "work-list",
  "message": {
    "total-results": 2779116,
    "items-per-page": 20,
    "next-cursor": "...",
    "items": [...]
  }
}

Work object (key fields)

{
  "DOI": "10.1038/nature12373",
  "title": ["Nanometre-scale thermometry in a living cell"],
  "author": [{"given": "G.", "family": "Kucsko", "sequence": "first"}],
  "publisher": "Springer Science and Business Media LLC",
  "type": "journal-article",
  "published": {"date-parts": [[2013, 7, 31]]},
  "container-title": ["Nature"],
  "ISSN": ["0028-0836", "1476-4687"],
  "volume": "500",
  "issue": "7460",
  "page": "54-58",
  "is-referenced-by-count": 1745,
  "references-count": 30,
  "abstract": "<p>Abstract text with HTML tags...</p>",
  "license": [{"URL": "...", "content-version": "vor"}],
  "link": [{"URL": "...", "content-type": "application/pdf"}],
  "reference": [{"key": "...", "doi-asserted-by": "crossref", "DOI": "..."}],
  "subject": ["Multidisciplinary"],
  "language": "en"
}

Note: title and container-title are arrays. published.date-parts is [[year, month, day]]. Abstract may contain HTML tags.

medRxiv API

medRxiv is a preprint server for health sciences. The API is identical to bioRxiv's API -- same endpoints, same response format -- just use medrxiv as the server parameter.

Important: Like bioRxiv, there is no keyword search. Use Semantic Scholar, OpenAlex, or PubMed for keyword searches of medRxiv content.

Base URL

https://api.biorxiv.org

(Same base URL as bioRxiv -- the server is specified in the path.)

Authentication

None required. Fully public API.

Key Endpoints

1. Content Detail -- Browse by date range

GET /details/medrxiv/{interval}/{cursor}/{format}

Optional: ?category=cardiovascular%20medicine (use URL-encoding for spaces)

Examples:

https://api.biorxiv.org/details/medrxiv/2024-01-01/2024-01-31/0
https://api.biorxiv.org/details/medrxiv/5
https://api.biorxiv.org/details/medrxiv/10d

2. Content Detail -- DOI lookup

GET /details/medrxiv/{doi}/na/{format}

Example:

https://api.biorxiv.org/details/medrxiv/10.1101/2021.04.29.21256344/na/json

3. Published Article Links

GET /pubs/medrxiv/{interval}/{cursor}
GET /pubs/medrxiv/{doi}/na

Links preprints to their published journal versions. Accepts both preprint DOI and published DOI.

Response Format

Same as bioRxiv:

{
  "messages": [{
    "status": "ok",
    "count": 100,
    "total": "502",
    "cursor": 0
  }],
  "collection": [{
    "title": "Paper title...",
    "authors": "Surname, A.; Surname, B.",
    "author_corresponding": "Full Name",
    "author_corresponding_institution": "Institution",
    "doi": "10.1101/2021.04.29.21256344",
    "date": "2021-05-03",
    "version": "1",
    "type": "PUBLISHAHEADOFPRINT",
    "license": "cc_by_nc_nd",
    "category": "cardiovascular medicine",
    "abstract": "Full abstract text...",
    "published": "10.1371/journal.pone.0256482",
    "server": "medRxiv"
  }]
}

Pagination

100 results per page. Use cursor parameter to paginate.

Rate Limits

No documented rate limits. No authentication required.

Categories

addiction-medicine, allergy-and-immunology, anesthesia, cardiovascular-medicine, dentistry-and-oral-medicine, dermatology, emergency-medicine, endocrinology, epidemiology, forensic-medicine, gastroenterology, genetic-and-genomic-medicine, geriatric-medicine, health-economics, health-informatics, health-policy, health-systems-and-quality-improvement, hematology, hiv-aids, infectious-diseases, intensive-care-and-critical-care-medicine, medical-education, medical-ethics, nephrology, neurology, nursing, nutrition, obstetrics-and-gynecology, occupational-and-environmental-health, oncology, ophthalmology, orthopedics, otolaryngology, pain-medicine, palliative-medicine, pathology, pediatrics, pharmacology-and-therapeutics, primary-care-research, psychiatry-and-clinical-psychology, public-and-global-health, radiology-and-imaging, rehabilitation-medicine-and-physical-therapy, respiratory-medicine, rheumatology, sexual-and-reproductive-health, sports-medicine, surgery, toxicology, transplantation, urology

OpenAlex API

OpenAlex is a comprehensive index of 250M+ scholarly works, authors, institutions, sources, and topics. It's the broadest multidisciplinary database in this skill.

Base URL

https://api.openalex.org

Authentication

• API key recommended (free). Get one at https://openalex.org/settings/api
• Pass as: ?api_key=YOUR_KEY
• Legacy polite pool still works: add ?mailto=you@example.com for better rate limits

Rate Limits

• 100 requests/second max
• Usage-based pricing with $1/day free allowance
• Single entity lookups by ID/DOI are free (unlimited)
• List + filter queries: ~$0.0001 each (~10,000/day free)
• Search queries: ~$0.001 each (~1,000/day free)

Key Endpoints

1. Get a single work

GET /works/{id}

Accepts multiple ID formats:

/works/W2741809807                              (OpenAlex ID)
/works/doi:10.7717/peerj.4375                  (DOI)
/works/pmid:29456894                            (PMID)
/works/https://doi.org/10.7717/peerj.4375      (full DOI URL)

2. Search works

GET /works?search={query}&per_page={n}&page={n}

Advanced search: Supports wildcards (machin*), fuzzy (machin~1), proximity ("climate change"~5), boolean grouping.

Example:

https://api.openalex.org/works?search=CRISPR+gene+therapy&filter=from_publication_date:2023-01-01&sort=cited_by_count:desc&per_page=10

3. Filter works

GET /works?filter={filters}

Key filter fields:

Operators: >, <, ! (negation), | (OR within filter)

Example:

https://api.openalex.org/works?filter=from_publication_date:2024-01-01,type:article,is_oa:true,cited_by_count:>50

4. Other entities

GET /authors?search={name}
GET /authors/{id}
GET /sources?search={name}          (journals, repositories)
GET /sources/{id}
GET /institutions?search={name}
GET /institutions/{id}
GET /topics/{id}

Authors and institutions accept similar filter/sort/pagination parameters.

5. Cursor pagination (for >10,000 results)

GET /works?filter=publication_year:2024&cursor=*&per_page=100

Response includes meta.next_cursor. Pass it as cursor={value} in the next request. Stop when next_cursor is null.

Response Format

Work object (key fields)

{
  "id": "https://openalex.org/W2741809807",
  "doi": "https://doi.org/10.7717/peerj.4375",
  "title": "The state of OA",
  "publication_year": 2018,
  "publication_date": "2018-02-13",
  "type": "article",
  "language": "en",
  "is_retracted": false,
  "cited_by_count": 1169,
  "open_access": {
    "is_oa": true,
    "oa_status": "gold",
    "oa_url": "https://doi.org/10.7717/peerj.4375"
  },
  "authorships": [{
    "author": {"id": "https://openalex.org/A5048491430", "display_name": "Heather Piwowar"},
    "institutions": [{"display_name": "Impactstory"}]
  }],
  "primary_location": {
    "source": {"display_name": "PeerJ", "issn_l": "2167-8359"}
  },
  "abstract_inverted_index": {"Despite": [0], "growing": [1], "interest": [2], ...},
  "referenced_works": ["https://openalex.org/W123...", ...],
  "ids": {"openalex": "...", "doi": "...", "pmid": "..."}
}

Abstract inverted index

Abstracts are stored as {word: [positions]}. To reconstruct:

def reconstruct(inverted_index):
    positions = {}
    for word, indices in inverted_index.items():
        for idx in indices:
            positions[idx] = word
    return ' '.join(positions[i] for i in sorted(positions.keys()))

List response

{
  "meta": {"count": 3771834, "page": 1, "per_page": 10},
  "results": [...]
}

Error Format

HTTP 403 for invalid API key, 429 for rate limit exceeded. Error responses include a message field.

PMC (PubMed Central)

PMC is a full-text archive of biomedical and life sciences articles. It is separate from PubMed -- PubMed has citations/abstracts, PMC has full text. Not all PubMed articles are in PMC, and vice versa.

E-utilities for PMC

Base URL

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/

Same E-utilities as PubMed, but with db=pmc.

eSearch -- Search PMC

GET /esearch.fcgi?db=pmc&term={query}&retmode=json

Same parameters as PubMed eSearch. Returns PMC UIDs (numeric, e.g., 13033346). You need to prepend "PMC" to get a PMCID (e.g., PMC13033346).

eFetch -- Get Full Text XML

GET /efetch.fcgi?db=pmc&id={pmcid}&retmode=xml

Example:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pmc&id=7029759&retmode=xml

The XML uses JATS (Journal Article Tag Suite) format:

• <front> -- journal metadata, article metadata, author info
• <body> -- full article text with <sec> sections, <p> paragraphs, <fig> figures
• <back> -- <ref-list> with all references

Pass numeric IDs only (not "PMC7029759", just "7029759").

BioC API -- Structured Full Text

An alternative way to get full text in a structured passage format.

Base URL

https://www.ncbi.nlm.nih.gov/research/bionlp/RESTful/pmcoa.cgi/

Endpoint

GET /BioC_{format}/{id}/{encoding}

Example:

https://www.ncbi.nlm.nih.gov/research/bionlp/RESTful/pmcoa.cgi/BioC_json/PMC7029759/unicode

Response structure (JSON):

{
  "source": "PMC",
  "documents": [{
    "id": "PMC7029759",
    "infons": {"license": "...", "doi": "..."},
    "passages": [
      {
        "offset": 0,
        "infons": {"section_type": "TITLE"},
        "text": "Article title..."
      },
      {
        "offset": 42,
        "infons": {"section_type": "ABSTRACT"},
        "text": "Abstract text..."
      },
      {
        "offset": 500,
        "infons": {"section_type": "INTRO"},
        "text": "Introduction text..."
      }
    ]
  }]
}

Section types: TITLE, ABSTRACT, INTRO, METHODS, RESULTS, DISCUSS, CONCL, REF, SUPPL, FIG, TABLE

Coverage: ~3 million articles from the PMC Open Access Subset.

PMC ID Converter API

Converts between PMID, PMCID, DOI, and Manuscript ID.

Base URL

https://pmc.ncbi.nlm.nih.gov/tools/idconv/api/v1/articles/

Parameters

Example:

https://pmc.ncbi.nlm.nih.gov/tools/idconv/api/v1/articles/?ids=PMC7029759&format=json

Response:

{
  "status": "ok",
  "records": [{
    "pmcid": "PMC7029759",
    "pmid": "32117569",
    "doi": "10.12688/f1000research.22211.2"
  }]
}

Only returns results for articles that are in PMC. If an article is in PubMed but not PMC, no PMCID will be returned.

Rate Limits

Include tool and email parameters on E-utility requests. Large batch jobs should run outside peak hours (Mon-Fri 5AM-9PM ET).

PubMed (NCBI E-utilities)

PubMed provides citations, abstracts, and metadata for 37M+ biomedical and life science articles. It does NOT contain full text -- for that, use PMC.

Base URL

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/

Authentication

• API key optional but recommended. Without: 3 req/sec. With: 10 req/sec.
• Pass as: &api_key=YOUR_KEY
• Also include &tool=your_app_name&email=your@email.com on all requests.

Key Endpoints

1. eSearch -- Search and get PMIDs

GET /esearch.fcgi?db=pubmed&term={query}&retmode=json

Example:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=CRISPR+gene+therapy&retmode=json&retmax=5&sort=pub_date

Response:

{
  "esearchresult": {
    "count": "224107",
    "retmax": "5",
    "retstart": "0",
    "idlist": ["39984857", "39984678", "39984543", "39984210", "39983901"]
  }
}

2. eSummary -- Get document summaries

GET /esummary.fcgi?db=pubmed&id={pmids}&retmode=json

Example:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=39984857,39984678&retmode=json

Response fields: uid, pubdate, source (journal), authors, title, volume, issue, pages, fulljournalname, elocationid (DOI), articleids (PMC, DOI, etc.), pubtype, pmcrefcount

3. eFetch -- Retrieve full records (abstracts, MEDLINE)

GET /efetch.fcgi?db=pubmed&id={pmids}&rettype={type}&retmode={mode}

Example -- get abstracts as XML:

https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=39984857&retmode=xml

The XML contains <PubmedArticle> with <MedlineCitation> (title, abstract, MeSH terms, authors) and <PubmedData> (article IDs, publication history).

4. eLink -- Find related articles

GET /elink.fcgi?dbfrom=pubmed&db=pubmed&id={pmid}&cmd=neighbor_score&retmode=json

Returns related PMIDs with relevance scores.

Search Syntax Tips

• Field tags: aspirin[TI] (title), Smith J[AU] (author), Nature[TA] (journal), neoplasms[MH] (MeSH heading)
• Boolean: CRISPR AND (therapy OR treatment)
• Date range: 2020/01/01:2024/12/31[PDAT]
• Publication type: review[PT], clinical trial[PT]
• Organism: humans[MH], mice[MH]

Rate Limits

• 3 requests/second without API key
• 10 requests/second with API key
• Include tool and email parameters on every request
• Large batch jobs should run outside peak hours (Mon-Fri 5AM-9PM ET)

Error Format

{"error": "API rate limit exceeded", "count": "11"}

HTTP 400 for bad requests, 429 for rate limiting.

Semantic Scholar API

Semantic Scholar indexes 200M+ papers across all academic fields with AI-powered features: citation context, influential citations, TLDRs, and paper recommendations.

Base URLs

https://api.semanticscholar.org/graph/v1       (Academic Graph)
https://api.semanticscholar.org/recommendations/v1  (Recommendations)

Authentication

• Without key: Shared rate pool (frequently hits 429 errors). Works but unreliable.
• With key: 1 req/sec per key (higher on request).
• Header: x-api-key: YOUR_KEY
• Get a free key at: https://www.semanticscholar.org/product/api#api-key-form

The fields Parameter

Almost every endpoint accepts fields -- a comma-separated list (no spaces) of fields to include. Without it, you only get paperId + title.

Paper fields: paperId, corpusId, externalIds, url, title, abstract, venue, publicationVenue, year, referenceCount, citationCount, influentialCitationCount, isOpenAccess, openAccessPdf, fieldsOfStudy, s2FieldsOfStudy, publicationTypes, publicationDate, journal, authors, citations, references, tldr, embedding

Author fields: authorId, externalIds, url, name, affiliations, homepage, paperCount, citationCount, hIndex, papers

Paper ID Formats

The {paper_id} parameter accepts:

• 649def34f8be52c8b66281af98ae884c09aef38b (S2 hash)
• CorpusId:215416146
• DOI:10.1038/s41586-021-03819-2
• ARXIV:2005.14165
• PMID:19872477
• PMCID:2323736
• ACL:W12-3903

Key Endpoints

1. Paper search (relevance)

GET /graph/v1/paper/search?query={text}&fields={fields}&offset={n}&limit={n}

Max 1,000 results accessible via offset.

Example:

https://api.semanticscholar.org/graph/v1/paper/search?query=CRISPR+gene+therapy&fields=title,year,abstract,citationCount,authors,openAccessPdf&limit=10&year=2023-2024

2. Paper bulk search (boolean queries, large result sets)

GET /graph/v1/paper/search/bulk?query={text}&fields={fields}&sort={field}:{order}&token={token}

• Supports boolean operators: + (AND), | (OR), - (NOT), "..." (phrase), * (wildcard), () (grouping)
• Token-based pagination (up to 10M papers)
• Returns up to 1,000 per call
• Sortable: citationCount:desc, publicationDate:desc, paperId:asc

3. Paper details (by ID)

GET /graph/v1/paper/{paper_id}?fields={fields}

Example:

https://api.semanticscholar.org/graph/v1/paper/DOI:10.1038/s41586-021-03819-2?fields=title,year,abstract,citationCount,referenceCount,isOpenAccess,openAccessPdf,authors,tldr

Response:

{
  "paperId": "dc32a984b651256a8ec282be52310e6bd33d9815",
  "title": "Highly accurate protein structure prediction with AlphaFold",
  "year": 2021,
  "citationCount": 34260,
  "isOpenAccess": true,
  "openAccessPdf": {"url": "https://...pdf", "status": "HYBRID"},
  "tldr": {"text": "This work develops AlphaFold, a system that..."},
  "authors": [{"authorId": "47921134", "name": "J. Jumper"}, ...]
}

4. Paper citations

GET /graph/v1/paper/{paper_id}/citations?fields={fields}&offset={n}&limit={n}

Returns papers that cite this paper. limit max 1000.

Citation-specific fields: contexts, intents, isInfluential

5. Paper references

GET /graph/v1/paper/{paper_id}/references?fields={fields}&offset={n}&limit={n}

Returns papers cited by this paper. Same pagination as citations.

6. Paper title match

GET /graph/v1/paper/search/match?query={exact title}&fields={fields}

Returns single best match with matchScore. 404 if no match.

7. Author search

GET /graph/v1/author/search?query={name}&fields={fields}&offset={n}&limit={n}

8. Author details

GET /graph/v1/author/{author_id}?fields={fields}

9. Author's papers

GET /graph/v1/author/{author_id}/papers?fields={fields}&offset={n}&limit={n}

10. Paper recommendations

GET /recommendations/v1/papers/forpaper/{paper_id}?fields={fields}&limit={n}&from={pool}

from: recent (default) or all-cs. limit max 500.

11. Multi-paper recommendations (POST)

POST /recommendations/v1/papers/
Content-Type: application/json

{
  "positivePaperIds": ["paperId1", "paperId2"],
  "negativePaperIds": ["paperId3"]
}

12. Paper batch (POST)

POST /graph/v1/paper/batch?fields={fields}
Content-Type: application/json

{"ids": ["DOI:10.1038/nature12373", "ARXIV:2005.14165"]}

Max 500 IDs per request.

Pagination

Publication Types

Review, JournalArticle, CaseReport, ClinicalTrial, Conference, Dataset, Editorial, LettersAndComments, MetaAnalysis, News, Study, Book, BookSection

Fields of Study

Computer Science, Medicine, Chemistry, Biology, Materials Science, Physics, Geology, Psychology, Art, History, Geography, Sociology, Business, Political Science, Economics, Philosophy, Mathematics, Engineering, Environmental Science, Agricultural and Food Sciences, Education, Law, Linguistics

Error Format

{"message": "Too Many Requests", "code": "429"}

HTTP 404 for not found, 429 for rate limit.

Unpaywall API

Unpaywall tells you whether a legal, free copy of a scholarly article exists. Given a DOI, it returns open access status, PDF links, and location details.

Base URL

https://api.unpaywall.org/v2

Authentication

No API key. You must include your email address as a query parameter: ?email=you@example.com

Important: Use a real email address. Unpaywall rejects placeholder emails like test@example.com with HTTP 422.

Rate Limits

100,000 calls per day. For heavier use, download the database snapshot.

Key Endpoints

1. DOI Lookup

GET /v2/{doi}?email=you@example.com

Example:

https://api.unpaywall.org/v2/10.1038/nature12373?email=you@example.com

2. Search (unreliable)

GET /v2/search?query={text}&email=you@example.com

Warning: The search endpoint has been returning HTTP 500 errors as of March 2026. It may be deprecated or intermittently broken. Use DOI lookups instead -- find papers via PubMed/OpenAlex/Semantic Scholar first, then check OA status per-DOI.

Response Format

DOI Lookup response

{
  "doi": "10.1038/nature12373",
  "doi_url": "https://doi.org/10.1038/nature12373",
  "title": "Nanometre-scale thermometry in a living cell",
  "year": 2013,
  "published_date": "2013-07-31",
  "genre": "journal-article",
  "publisher": "Springer Nature",
  "is_oa": true,
  "oa_status": "green",
  "best_oa_location": {
    "url": "https://dash.harvard.edu/bitstream/1/...",
    "url_for_pdf": "https://dash.harvard.edu/bitstream/1/...pdf",
    "url_for_landing_page": "https://dash.harvard.edu/handle/...",
    "host_type": "repository",
    "version": "acceptedVersion",
    "license": "cc-by",
    "is_best": true,
    "oa_date": "2016-01-01"
  },
  "first_oa_location": {...},
  "oa_locations": [...],
  "has_repository_copy": true,
  "journal_name": "Nature",
  "journal_issns": "0028-0836,1476-4687",
  "journal_issn_l": "0028-0836",
  "journal_is_oa": false,
  "journal_is_in_doaj": false,
  "z_authors": [
    {"raw_author_name": "G. Kucsko", "author_position": "first"},
    {"raw_author_name": "P. C. Maurer", "author_position": "middle"}
  ]
}

OA Status values

OA Location fields

Search response

{
  "results": [
    {
      "response": {...},
      "score": 42.5,
      "snippet": "...text with <b>highlighted</b> matches..."
    }
  ]
}

Typical Workflow

1. You have a DOI from PubMed, Crossref, or another source 2. Call Unpaywall with the DOI 3. Check is_oa -- if true, use best_oa_location.url_for_pdf for the free PDF 4. Check oa_status to understand what kind of OA it is 5. If closed, oa_locations will be empty -- the article requires a subscription