plus3it / cfn-collibra Goto Github PK

Problem Description:

In the 02_Create_ELB-Console pipeline, please include a different description on the HaSubnets field. Message was a little confusing, need to understand how to separate, assume commas.

Expected Behavior:

Current description: IDs of public-facing subnets in which to create service-listeners

Actual Behavior:

(Detailed) Steps to reproduce:

Run 02_Create_ELB-Console pipeline

Detail the steps to reproduce

(Optional) Fix recommendation:

Comma separated list of public-facing subnet IDs in which to create service-listeners

Problem Description:

Section of ELB Pipeline that creates R53 alias is missing its status-checking logic

Expected Behavior:

Jenkins does not declare R53 stage complete until the R53alias CFn stack returns a done-state.

Actual Behavior:

Jenkins declares R53 stage comlete so long as the R53 create-stack operation returns a good exit code

(Optional) Fix recommendation:

Duplicate the status check-loop logic from the ELB stage to the R53 stage (correcting the original copy-paystah omission)

Problem Description:

Password requirements vague

Expected Behavior:

Actual Behavior:

Vague description

(Detailed) Steps to reproduce:

Run server build jobs

(Optional) Fix recommendation:

Add password requirements into descriptions ( ^[a-zA-Z0-9]{8,12}$ )

Problem Description:

Currently, each stage that needs credentialed access has its own:

withCredentials(
        [
            [$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: "${AwsCred}", secretKeyVariable: 'AWS_SECRET_ACCESS_KEY'],
            sshUserPrivateKey(credentialsId: "${GitCred}", keyFileVariable: 'SSH_KEY_FILE', passphraseVariable: 'SSH_KEY_PASS', usernameVariable: 'SSH_KEY_USER')
        ]
    ) {
}

Block. It may be possible to excised all of the blocks from the stages and move them into the node-block, instead — reducing the amount of content-duplication within the pipeline.

Problem Description:

ESB template still calls for MuleConnectorZipUri, MuleDomainZipUri values

Expected Behavior:

These values were supposed to be removed, since they are no longer needed.

Actual Behavior:

Template errors out calling for values.

Error text:
An error occurred (ValidationError) when calling the CreateStack operation: Parameters: [MuleConnectorZipUri, MuleDomainZipUri] must have values

(Detailed) Steps to reproduce:

Run 15_Collibra_ESB_Server pipeline

(Optional) Fix recommendation:

Problem Description:

Application-generated backups stay on local disk. Need logic to sweeps backups to S3.

Expected Behavior:

Application-generated backups should be stored on S3 for long-term, off-host backup architecture

Actual Behavior:

Application-generated backups stay on local disk.

(Detailed) Steps to reproduce:

Configure application-backups. Check back after each run-window (files don't move)

Problem Description:

No longer need the Collibra Connect domain and connector application installed on the Mule ESB server. Remove parameters, and stop installation

Expected Behavior:

No need to provide parameter values

Actual Behavior:

Parameters still requested

(Detailed) Steps to reproduce:

Run 15_Collibra_ESB_Server pipeline

(Optional) Fix recommendation:

Problem Description:

EL 7 cannot use Collibra's boot-time startup scripts if ${APPLICATION_SOFTWARE_HOME} is not on /.

Collibra 5.4 currently attains EL 7 "compatibility" pretty much as a side-effect of EL 6 compatibility. That said, a number of things that are bad to do but still manage to work under EL 6 can fail rather horribly under EL7.

Expected Behavior:

When system reboots, the collibra-console or collibra-agent boot scripts ensure that the relevant DGC components start at boot.

Actual Behavior:

When system reboots, the collibra-console or collibra-agent boot scripts fail with:

systemd-sysv-generator[NNN]: stat() failed on /etc/rc.d/init.d/collibra-agent No such file or directory

Written to the boot-logs.

(Detailed) Steps to reproduce:

1. Install Collibra binaries onto a dedicated filesystem.
2. Reboot system
3. Find that software did not start on boot
4. Find that previously-noted error has been logged

(Optional) Fix recommendation:

See vendor's Solution document. Update automation to incorporate these fixes.

Problem Description:

Currently, logs for Collibra services are not pushed to CloudWatchLogs

Expected Behavior:

Logs for Collibra services are not pushed to CloudWatchLogs. The file-list should include:

• Agent Nodes: <INSTALL_ROOT>/data/agent/logs/agent_wrapper.log
• Monitor Nodes:
  • <INSTALL_ROOT>/data/monitoring/logs/prometheus.log
  • <INSTALL_ROOT>/data/agent/logs/agent_wrapper.log
• Console Nodes: <INSTALL_ROOT>/data/console/logs/console.log
• DGC Nodes:
  • <INSTALL_ROOT>/data/dgc/logs/dgc.log
  • <INSTALL_ROOT>/data/dgc/logs/dgc_recommender.log
  • <INSTALL_ROOT>/data/dgc/logs/dgc_jobs.log
• Jobserver Nodes: <INSTALL_ROOT>/data/spark-jobserver/logs/spark-job-server.log

Actual Behavior:

Not currently configured for export

Problem Description:

Seems a little odd that the "backup bucket" name would be optional. If so, what name is created/used?

Expected Behavior:

Name is a "required" value

Actual Behavior:

Name is tagged as "optional".

(Detailed) Steps to reproduce:

Run the 01_Create_S3-backup_bucket pipeline

(Optional) Fix recommendation:

Can we make mandatory?

Problem Description:

Some components (e.g. SMTP and some service-to-service comms) don't work when FIPS-mode is enabled.

Expected Behavior:

Service should be able to use SES for relaying

Actual Behavior:

First-noticed symtptom: service should be able to use SES for relaying

(Detailed) Steps to reproduce:

Use current automation to harden system and install Collibra

(Optional) Fix recommendation:

Add a direct, post-wam run of salt to disable FIPS-mode prior to reboot:

salt-call --local ash.fips_disable

Problem Description:

Vendor's Enterprise Linux 7 compatibility doesn't include systemd-compatibility. Need to author systemd unit files to (re)start services on boot

Expected Behavior:

Services (re)start on boot

Actual Behavior:

Services don't (re)start on boot.

(Detailed) Steps to reproduce:

Reboot the system

(Optional) Fix recommendation:

Problem Description:

Different service elements have different URLs to signify service-health. Update template to allow selection (or use mapping based on service)

Expected Behavior:

ELB marks instance healthy based on appropriate string

Actual Behavior:

ELB not marking instances healthy due to inappropriate check-string

(Optional) Fix recommendation:

Set appropriate check-string via parameter or mapping