cul-it / archival-storage-ingest Goto Github PK

For each service, there are a series of files, workers, environment variables, queues, and executables associated with them. For instance, for the service to calculate SFS fixity, we have:

• The executable `archival_storage_ingest_server_sfs_fixity_check
• The worker class SFSFixityGenerator
• The log file fixity_sfs.log
• The configuration environment variables asi_sfs_fixity_*
• The SQS queues cular_development_fixity_sfs* and the associated ruby constants Queues::QUEUE_FIXITY_SFS*
• The systemd service fixity_check_sfs.service and associated script fixity_check_sfs.script

While there appears to be a preference in this case (and with the other services) to refer to it as "sfs_fixity", this is mostly internal; the externally-facing usages use a mix of sfs_fixity_check, fixity_sfs, sfs_fixity, and fixity_check_sfs instead.

This same pattern of inconsistent names exists across 5 of the 6 services, the single-word named "ingest" server excepted.

I propose that one name for each service be chosen, and the code modified to be consistent with that.

In this example, by choosing sfs_fixity as the name for this service, we would rename the log file to sfs_fixity.log, the SQS queues to cular_development_sqs_fixity* and Queues::QUEUE_SFS_FIXITY*, and the systemd service to sfs_fixity.service (or perhaps asi_sfs_fixity.service so the services are grouped together in systemctl status).

Currently, the servers handle their own logging -- they are each configured in Ruby code to write to individual log files.

The guidance for "12 factor apps" at https://12factor.net/ recommend having services logging to stdin/stdout, and letting systemd (or equivalent) handle log routing, persistence, etc.

Adapting this recommendation would mainly involve changing how we configure the logger in the application and removing the file configuration from our startup scripts. By default systemd records the stdout/stderr logged events using journald, so the logs can be accessed and managed with journald automatically.

I am planning on working on improving our logging (to log more events, and more details on failure). That would be an opportunity to make these changes.