Some background: MIT is working on a research data registry using Dataverse as the repository, and as part of this project we want to ingest metadata into Dataverse such that it is indexed and searchable but the search result points to the source repository instead of a local Dataverse record. This is the exact functionality of harvested records, however we are building our own harvesting service because we need to perform actions not possible in the internal harvester, like parsing returned records to identify MIT-affiliated authors and enhancing the metadata with data from our local data warehouse.
In talking to the Dataverse tech team about this, the question rose of whether it would be feasible/useful to add the ability to create "harvested" datasets via the API. They suggested we add it here as an issue to prompt discussion...questions, comments, and suggestions welcome!
Some background: MIT is working on a research data registry using Dataverse as the repository, and as part of this project we want to ingest metadata into Dataverse such that it is indexed and searchable but the search result points to the source repository instead of a local Dataverse record. This is the exact functionality of harvested records, however we are building our own harvesting service because we need to perform actions not possible in the internal harvester, like parsing returned records to identify MIT-affiliated authors and enhancing the metadata with data from our local data warehouse.
In talking to the Dataverse tech team about this, the question rose of whether it would be feasible/useful to add the ability to create "harvested" datasets via the API. They suggested we add it here as an issue to prompt discussion...questions, comments, and suggestions welcome!