REST API Operations

Blocking Users

If users are causing the REST API it may be necessary to block their IP address as a last resort. See HAProxy operations.

Elastic Search

The number of open cursor sessions can be retrieved at /health/cursors. This may be useful for monitoring the scalability of cursors.

Startup tasks

:api

Starts server that serves all REST API content.

:base

Base startup task, always started by default, does not have to be specified in the command line args. Creates Mongo and ES clients.

:cleanup-feed-files

Used only for file ingestion from local file system.

Schedules quartz runner jobs that take care of deleting old processed feed files from disc. In the current configuration, once every hour files older than 2 hours are deleted from /feed-processed directory. The time and delay can be changed in cayenne.schedule namespace.

:create-mappings

Creates ES indexes, if they do not already exist. Defined in cayenne.elastic.mappings. The following env vars are read for each index:

• number of shards: ELASTICSEARCH_SHARDS_WORK for work index and ELASTICSEARCH_SHARDS for other indexes
• number of replicas: ELASTICSEARCH_REPLICAS
• max result window: ELASTICSEARCH_RESULT_WINDOW_WORK for work index and ELASTICSEARCH_RESULT_WINDOW for other indexes
• refresh interval: ELASTICSEARCH_REFRESH_INTERVAL

:nrepl

Starts nREPL server.

:process-feed-files

Used only for file ingestion from local file system.

Ingests files from local /feed-in directory.

:s3-ingest-update

Ingests all cited-by updates directly from S3 metadata bucket.

:s3-ingest-xml

Ingests all XMLs directly from S3 metadata bucket.

:s3-sqs-produce-update

Produces all cited-by update keys from S3 metadata bucket to SQS queue.

:s3-sqs-produce-xml

Produces all XML keys from S3 metadata bucket to SQS queue.

:sqs-ingest

Ingests data from SQS queue.

:update-index-settings

Updates ES indexes settings. The following env vars are read for each index:

• number of replicas: ELASTICSEARCH_REPLICAS
• max result window: ELASTICSEARCH_RESULT_WINDOW_WORK for work index and ELASTICSEARCH_RESULT_WINDOW for other indexes
• refresh interval: ELASTICSEARCH_REFRESH_INTERVAL

:update-members

Indexing members and calculating coverage for members. Schedule in cayenne.schedule.

See also Member Info in REST API.

:update-journals

Indexing journals and calculating coverage for journals. Schedule in cayenne.schedule.

See also Journal Title Information in REST API.

:update-funders

Indexing funders from Funder Registry. Schedule in cayenne.schedule.

See also Funding Data in REST AP.

:update-subjects

Indexing Scopus journal subjects and adding the subjects to journal records. Schedule in cayenne.schedule.

Note that this task should start after journals are indexed.

See also SCOPUS journal subjects ingestion in REST API.