Design of Member Info

Design of Member Info

Tags

Introduction

The objective of the member info microservice is to adapt the legacy service code. The adaptioned is not a refactoring (as commonly understood) and specifically not a reimplementation. The primary changes to the existing service code are

  1. The location of the persistent member info data — S3.
  2. The encoding of the persistent member info data — JSON.
  3. The invocation for creating the persistent member info data — Spring Boot command line tool (running within Docker container).
  4. Initializing and updating the CS member info data.

It is not an objective of the member info microservice to necessitate the updating of any other systems or services that use the Customer Db. The Customer Db will continue to be directly available to these systems and services.

This note broadly documents the design for the member info microservice.

Design

The member info microservice (MIS) and the content system (CS) are independent systems.

Secrets needed by MIS are provided in environmental variables.

The MIS is an ETL activity that draws the data from Intacct, Sugar, Customer Db, and Deposit Db to create a member info item in S3.

graph LR intacct["Intacct"] --> MIS((MIS)) sugar["Sugar"] --> MIS customer["Customer Db"] --> MIS deposit["Deposit Db"] --> MIS MIS -->sugar MIS-->customer MIS --> S3 S3 --> CS((CS))

A MIS run takes the steps

  1. Update Customer Db from Sugar.
  2. Gather member prefixes and prefix references accesses from Deposit Db.
  3. Gather further member facts from Customer Db.
  4. Gather further member facts from Intacct.
  5. Reconsile collected member facts with Customer Db and Sugar, updating these systems as needed.
  6. Serialize the collected data to JSON into S3.

The MIS process will run once or demonize and run regularly. See “application_schedule” environment variable, below.

The member info data is an S3 bucket in the “memberinfo-list” (pseudo) directory. This bucket needs a writer security role and a reader security role. Item keys are therefore composed of a “directory” name and a “file” name.

«DIRECTORY»/«FILE»

The member info data items will be named using the pattern

memberinfo-list-«TIMESTAMP»-«ID».json

Where «TIMESTAMP» is (using Java format) “yyyy-MM-dd’T’HH:mm:ss.SSS” and «ID» is the identity of the MIS instance.

The newly created member info data item is copied to the item named

current.json

Outdated member info items are to be purged by an independent process, if ever.

The member info data JSON entails a Crossref header and the members data body.

The Crossref header is

{
  "message-type": "memberinfo-list",
  "message-version": "1.0.0",
  "created-at": "«TIMESTAMP»",
  "message": {
    "total-results": «NUMBER OF MEMBERS»,
    "members": [«MEMBER INFO OBJECTS»]
  }
}

The member info data, ie «MEMBER INFO OBJECTS», design is taken directly from the legacy MemberInfo class. The names of fields and tokens differ only in that they have been converted to lowercase and use a hyphen delimiter. See MemberInfo JavaDoc for information on each field. So, by way of example, the member info data is

{
    "prefix": "10.1234",
    "mailing-address": "1 Main St\nNew York, NY 12345\nUSA",
    "member-name": "John Smith & Company",
    "alt-names": "",
    "member-type": "member-publisher", // See TYPES ENUM
    "member-id": 1234,
    "sugar-id": "",
    "joined": "YYYY-MM-DD",
    "email": "support@smith.com",
    "prefix-name": "John Smith & Company",
    "accounting-name": "John Smith & Company",
    "accounting-id": "JSCX00",
    "annual-fee": 100,
    "references-access": "open",  // See DISTRIBUTION ENUM
    "access-tokens": [ "eyJ0eXAiOi...MlTVjAZQwgW36o", ... ],
    "cidrs": [ "1.1.1.1/24", ... ],
    "service-features": [
        // See FEATURES ENUM below
        "crossmark", ... 
    ],	
    "contacts": [
        {
            "email": "john@smith.com",
            "roles": [ 
                // See ROLES ENUM below
                "business",
                ...
            ],
            "report-overrides": {
                // See REPORT ENUM and OVERRIDE ENUM below
                "doi-error-report": "alternate", 					
            }						
        },
        ...	
    ]
}

MEMBER TYPES ENUM

These are the types of member membership:

  • “affiliates”
  • “agent”
  • “guest”
  • “member-publisher”
  • “ra-affiliate”
  • “ra-sponsored-member”
  • “represented-member”
  • “sponsored-publisher”
  • “sponsoring-entities”
  • “sponsoring-publishers”
  • “test-account”

CONTACT ROLES ENUM

These are the specific contacts for a member:

  • “alternate-board-member”
  • “billing”
  • “board-committee-member”
  • “books-interest-group-member”
  • “business”
  • “cross-check-administrator”
  • “cross-check-business-contact”
  • “crossmark”
  • “dul-committee”
  • “event-data-comm”
  • “mandf-committee-member”
  • “media”
  • “metadata-quality”
  • “other-affiliate”
  • “other-member”
  • “technical”
  • “voting”
  • “undefined”

REPORTS ENUM

These are the reports sent to members:

  • “doi-error-report”
  • “doi-error-report-2”
  • “resolution-report”
  • “conflict-report”
  • “title-report”
  • “preprint-vor-notices”
  • “schematron-report”

REPORTS OVERRIDE ENUM

  • “alternate”
  • “additional”

REFERENCES DISTRIBUTION ENUM

  • “open”
  • “closed”
  • “limited”

SERVICE FEATURES ENUM

These are the Corssref services or service features a member subscribes too:

  • “cited-by”
  • “cms-basic”
  • “cms-enhanced”
  • “crosscheck”
  • “crossmark”
  • “end-user-lookup”
  • “fundref”
  • “metadata-plus”
  • “openurl”
  • “orcid-roundtrip-optout”
  • “polite-rest-api”
  • “query-affiliate”
  • “service-provider”
  • “tdm”

Runtime Secrets

The MIS needs to connect to several services during its operation. All these services have their own access control. The access control secrets need to be provided to the microservice as environment variables. The variables needed are:

Property Description
application_id The application id. Used to identify what process created the S3 items. Use a short name similar to the CS deployment, eg “mi-1”.
application_schedule On what schedule should the microservice run. The value is a cron expression. For example, “0 0 */4 * * ?” will run the update every four hours.
deposit_url The JDBC URL for the deposit Oracle database.
deposit_user The deposit database read only user.
deposit_password The deposit database user’s password.
customer_url The JDBC URL for the customer MySql database.
customer_user The customer database read-write user.
customer_password The customer database user’s password.
sugar_user The Sugar read-write user.
sugar_password The Sugar user’s password.
intacct_sender_user The Intacct sender user.
intacct_sender_password The Intacct sender user’s password.
intacct_login_user The Intacct login user.
intacct_login_password The Intacct login user’s password.
intacct_login_company The Intacct login company identifier.
s3_bucket_accesskey The AWS S3 full access key to the s3_bucket_name bucket.
s3_bucket_secretkey The AWS secret key
s3_bucket_region The AWS region of the bucket.
s3_bucket_name The AWS S3 bucket name.
prefixes_to_ignore A comma separated list of prefixes to ignore. These prefixes are mostly those used during Crossref testing.

END