add OpenAlex as a source #44

Activity

1. mbjones commented on Oct 2, 2026

   

   mbjones

   on Oct 2, 2026

   Member

   More actions

   @jeanetteclark I started prototyping this tonight, and will try to push a branch tomorrow.
2. mbjones

   self-assigned this

   on Oct 8, 2026
3. mbjones

   added the

   Feature

   issue type on Oct 8, 2026
4. mbjones

   added this to the 1.0.0 milestone on Oct 8, 2026
5. mbjones commented on Oct 8, 2026

   

   mbjones

   on Oct 8, 2026

   Last edited by mbjones

   Member

   More actions

   Did a pretty complete implementation in branch https://github.com/DataONEorg/scythe/tree/feature-44-open-alex

   Needs further review and testing. OpenAlex supports two distinct search modes:

   • graph: searches a pre-computed citation graph based on the citation lists that have been gathered from papers. This is really inexpensive API wise and finds a lot of results
   • text: full text search, which can find string matches in data availability statements and methods, etc. This is far more expensive via the API, but produces different results from the graph search.

   I ran a comparison of 200 randomly chosen DOIs from the ADC, and here's the results of what was found only in the text search, only in the graph search, and which were found by both:

   text_only	graph_only	both
   11	30	7

   See the vignette for openalex for a better explanation: https://github.com/DataONEorg/scythe/blob/feature-44-open-alex/vignettes/openalex-arctic-data-center.Rmd
6. mbjones commented on Oct 10, 2026

   

   mbjones

   on Oct 10, 2026

   Member

   More actions

   Update after review of #46: the functional changes are below.

   • OpenAlex is opt-in in citation_search(). It spends credits, so it runs only when named in sources. Use citation_search_openalex() directly for cost control and resumable searches.
   • Resumed searches log each match once. Runs completed in installments no longer append duplicate records to the review logs.
   • Data relations are classified as a separate step. The search no longer calls DataCite. classify_data_relations(log_dir) can label dataset-to-dataset links afterwards as version, derivation, composition or citation. It doesn't affect citation results or counts; it's there for future work on data-to-data citations.
   • Stricter credit and rate-limit handling. Every billed response counts toward max_credits. Concurrent shards honour each other's rate-limit pauses during a run, and a recorded pause is never shortened.
   • Tests no longer spend credits by default. Live OpenAlex tests are skipped on CRAN and CI, and run locally only with SCYTHE_TEST_OPENALEX=true. Live DataCite tests are also skipped on CRAN and CI.

   🤖 Generated with Claude Code
7. mbjones

   modified the milestones: scythe-1.0.0, scythe-1.3.0 on Oct 12, 2026
8. mbjones

   linked a pull request that will close this issueFeature 47 helm full search #48on Oct 12, 2026
9. mbjones commented on Oct 14, 2026

   

   mbjones

   on Oct 14, 2026

   Member

   More actions

   Closed and merged with PR #48
10. mbjones

    closed this as completedon Oct 14, 2026