Synchronous REST API Deposit v2

Synchronous REST API Deposit v2

Legacy

Area ingestion
Tech Lead Jason Hannah
Language Java
Description Deposit via `/v2/deposit`, part of the CS codebase.
Production URLs
Quality Sentry, no SONAR
Upstream services
Upstream data
Downstream services
Downstream data
Packages
  • org.crossref.ds.controllers.SynchronousDepositController
Source Code
Tags
Products

The Synchronous Deposit API is found at api.crossref.org/v2/deposit and supercedes the api.crossref.org/v1/deposit API, which is not synchronous. The API was originally created for PKP but is now used by Metadata Manager.

To deposit a submission make an HTTP POST request to the path

The Content Type should be multipart/form-data and include the following parameters:

  • usr, the depositing user name
  • pwd, the depositing user password
  • operation, the value is always “doMDUpload”.
  • mdFile the submission file

At this time only metadata deposits are supported. There is a proposed User Story for more types.

The SubmissionUploadTool has been enhanced to upload a synchronous deposit. For example,

bin/j org.crossref.tools.submissions.SubmissionUploadTool \
    --url http://localhost:8080/internal/deposit \
    --user NAME PASSWORD \
    --metadata FILE

or:

curl -v -F 'operation=doMDUpload' -F 'usr=USERNAME' -F 'pwd=PWD' 

-F ‘mdFile=@/path/to/file.xml’ https://api.crossref.org/v2/deposits

Once upload the submission is processed immediately by the same thread.

HTTP Result

The result is one of five HTTP statuses.

200 OK.

The submission was successfully processed. The message body will contain the doi_batch_diagnostic message. For example,

<?xml version="1.0" encoding="UTF-8"?>
<doi_batch_diagnostic status="completed" sp="arc.local">
   <submission_id>1403850957</submission_id>
   <batch_id>org.crossref.agilmartin.0001</batch_id>
   <record_diagnostic status="Success">
      <doi>10.5555/agilmartin/1</doi>
      <msg>Successfully updated</msg>
      <citations_diagnostic deferred="1403850958" />
   </record_diagnostic>
   <batch_data>
      <record_count>1</record_count>
      <success_count>1</success_count>
      <warning_count>0</warning_count>
      <failure_count>0</failure_count>
   </batch_data>
</doi_batch_diagnostic>

If the DOI metadata contains a citation list then this processing will be deferred. A new DOI resource only submission will be created and deposited with the asynchronous deposit processed. The citations_diagnostic contains that resource submission’s submission id <citations_diagnostic deferred="1403850958" />

200 OK. Successfully processed but with deposit errors

Deposits that are received successfully and complete processing, may still exhibit submission errors. Typically these result when good XML contains data that violate some deposit processing rule. For example: This indicates that the element contains a value that has not been incremented (this prevents concurrent deposits from colliding). The message contents are helpful for human analysis; however, automation should examine the summary looking for non zero counts in the warnings and failures. For further info refer to https://support.crossref.org/hc/en-us/articles/215789303

<?xml version="1.0" encoding="UTF-8"?>
<doi_batch_diagnostic status="completed" sp="cskAir.local">
   <submission_id>1407395745</submission_id>
   <batch_id>test201707181318</batch_id>
   <record_diagnostic status="Failure" msg_id="4">
      <doi>10.5555/doitestwithcomponent_2</doi>
      <msg>Record not processed because submitted version: 2012061910233600835 is less or equal to previously submitted version (DOI match)</msg>
   </record_diagnostic>
   <batch_data>
      <record_count>1</record_count>
      <success_count>0</success_count>
      <warning_count>0</warning_count>
      <failure_count>1</failure_count>
   </batch_data>
</doi_batch_diagnostic>

400, Bad request

There was some error in the request’s parameters. The message body is unlikely to contain anything useful to the caller. In general, log the result for later review.

401, Unauthorized

The user is not authorized to update the DOI. The message body is unlikely to contain anything useful to the caller. In general, log the result for later review.

403, Forbidden

There was some error during the processing of the submission. The message body will contain the doi_batch_diagnostic message.

500, Internal Server Error

You have hit upon a bug. The message body is unlikely to contain anything useful to the caller. In general, log the result for later review.

Citation List Processing

Citation list processing can take a very long time to complete. The client making the synchronous deposit is not expected to wait for its completion. When a submission contains a citation list the citation list is split out into its own DOI resource submission. DOI resource only submission will be automatically created by the synchronous service and deposited with the asynchronous deposit process. The same depositor identity will be used for this deposit job. The doi_batch_id will be the same as the full submission’s doi_batch_id with a numerical suffix appended. The DOI resource submission id is in the doi_batch_diagnostic message.

Checking On Deposit Status

After making a deposit the results may be retrieved using the following call. This will return the same XML log message as was returned in the response to the HTTP synchronous deposits POST.

curl -v -F 'doi_batch_id=test201708251320' -F 'type=result' -F 'usr=creftest' -F 'pwd=***' https://doi.crossref.org/servlet/submissionDownload

As with any submission the contents sent in can be retrieved using:

curl -v -F 'doi_batch_id=test201708251320' -F 'type=contents -F 'usr=creftest' -F 'pwd=***' https://doi.crossref.org/servlet/submissionDownload

Alternatively: The submissionDownload API will accept the submission ID assigned to the deferred citation submission.

curl -v -F 'submission_id=1411276441' -F 'type=result' -F 'usr=creftest' -F 'pwd=*** https://doi.crossref.org/servlet/submissionDownload

Using Callbacks

Many users of synchronous deposits will want to use callbacks for the deferred processing of the citations included in the deposit. As noted above, the deferred citation job’s submission ID is returned as part of the synchronous deposit HTTP POST. The user must both create a web callback endpoint and have it assigned as the target web callback endpoint for metadata deposit jobs (including deferred citation list processing). The message sent to the your callback endpoint will refer to the deferred citation job’s submission ID.

Step 1: create an end-point

curl -s -D - "http://doi.crossref.org/notification-callback/exec/setNotifyEndpoint\
?usr=USERNAME\
&pwd=PASSWORD\
&endpoint=com.foo.1\
&url=http://foo.com/crossref/callback"

Step 1a: test your end point

curl -s -D - "http://doi.crossref.org/notification-callback/exec/createNotificationCallback\
?usr=USERNAME\
&pwd=PASSWORD\
&notifyEndpoint=com.foo.1\
&notifyPayloadContentType=text/plain\
&notifyPayloadContent=this+is+test+1\
&externalTrackingId=test-1"

After a few minutes your end-point will receive a callback with your test payload message.

Step 2: Register your end-point as the target for DATA_SUBMISSION jobs.

Step 2 requires you to contact support@crossref.org and request that your endpoint be registered. You will specify a user name or a prefix to be mapped to your endpoint as the target for DATA_SUBMISSION jobs. This means any job processed for the prefix or the user the results will be sent to the endpoint instead of by email. For processing deferred citations from synchronous deposits the target endpoint should be registered to a username. This username will be the one used to submit the synchronous deposit.

Reference portion is peeled off and put in queue. In sync response the job id is returned for the reference processing. Then wait for email or register callback.

Callback is another separate feature of CS.