Add publications

[ryscher]

• Import publications from crossref

• Add publications to the search system (use same changes as datasets)

• Testing import at https://cic-apps-dev.datascience.columbia.edu/v1/publications

• Testing UI at https://cic-apps-dev.datascience.columbia.edu/adv_search/

[ryscher]

Next steps:

1. DONE Just get it to import one publication, when given a DOI
2. DONE Problem -- it's currently searching for duplicates and the URL returns them, but it still doesn't think they are there
3. RUNNING UPDATE PROCESS Process datasets, and import any publications that they reference

Which publications are we targeting?

• Only those that are explicitly mentioned by datasets or grants?
• We got all datasets that matched our target tags, as long as they referenced a grant
• Can we do the same for crossref? Not really, because most crossref metadata doesn't include funders.

Better to start with the DOIs and only import the select information -- start with DOIs referenced in datasets, then expand to DOIs listed in NSF/NIH APIs.NE

[ryscher]

Crossref info:

• API reference
• Sample API query
• Sample API for single DOI
• Python library

[ryscher]

It may be useful to see the branch where sunaina was working on indexing for publication info columbia-it/covid-infocommons@main...index_publications