Set up the annotations on the IMS API so that zero trust networking can be configured.

Comments: 04/Dec/19 12:46 PM;WAMOAR;Updated the BC and DC with app, environment, and role labels.
04/Dec/19 3:00 PM;WAMOAR;Done, rebuilt and manually deployed to dev and test.

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: WAMOAR

Comments:

Type: Story
Status: Backlog
Priority: Low
Resolution:
Assignee:

Python 3.8 was released on 2019-10-14. Due to the lack of pre-defined tooling for working with 3.8, the ppr-api proof of concept was developed using Python 3.7.4. We should upgrade to the latest version as soon as low-effort tooling is available, particularly docker images that we don't have to create ourselves.

Discuss with Thor/Kaine about getting credentials for 1Password. Store our secrets in 1Password and then find a way (sh script?) to pull the passwords in the CD and create the secrets object in OpenShift.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Determine the testing strategy for the PPR API.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Testing - please delete.

Comments:

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: WAMOAR

Overview

To speed deployment of the ppr-api (backend) a pre-existing container was used for the web server. When deployed to OpenShift, ppr-api consumes excessive resources, even when idle. We need to look more closely at this solution, as it is probably needlessly complex.

Details

The docker image from https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker was the first one that worked with FastAPI and it was good enough for the PoC. However, it wants to use two threads per core, so it tries to start with 32 threads. It takes a long time to start the pod, and once up they consume 500MB of memory and 50mcore of CPU when idle. It is currently throttled to four threads but this is possibly not the best long-tern solution.

Todo

• Learn uvicorn.
• Determine if we need gunicorn or not.
• If possible, deploy using a pure uvicorn image.

Repro Steps:

• Create a new registration (or use an existing known one) with an empty Year on the collateral
• Using the IMS-API
  -- Search for registration for the record using the serial number

Expected: The result is returned with a null year
Actual: The IMS-API returns a response with an http code of 500 (Internal Server Error)

This issue was captured in Sentry: https://sentry.io/organizations/bc-government-registries/issues/1391900774

Comments:

Type: Bug
Status: Under Review
Priority: Low
Resolution:
Assignee: JGUERTIN

Overview

A linter is a tool that helps keep our code clean by identifying poor formatting, style, or coding practices. We want a linter in the VS Code IDE so that developers don't accidentally commit poor-quality code to the repository.

Technical Details

Needs research. Leverage what's in the LEAR project's legal-api, although be skeptical of any custom settings: Use best practices in favor of non-standard personal preferences. Include a GitHub Action to enforce use.

To enable the CI and CD processes to exist outside of OpenShift, the build process must push its Docker images to an available registry. Currently the Github Actions are not able to push the images it creates, so we require a separate and manual process to rebuild those artifacts in OpenShift from source. This violates the CD principle of creating artifacts once for each integration.

Acceptance Criteria:
Given that a PR has merged into {master} in the PPR repo
When the "Build Continuous Integration Artifacts" job completes successfully
Then docker images for PPR-UI, PPR-API and IMS-API must have been pushed to Artifactory and tagged as {latest}

Comments: 16/Dec/19 12:09 PM;JGUERTIN;(flag) Flag added

Blocked until Artifactory is available

Type: Task
Status: Backlog
Priority: High
Resolution:
Assignee: JGUERTIN

When the pods start up for the ims-api, the health checks are failing. The pods take around 30 seconds to start up. Ideally we should not see error events in OpenShift for the pods. Set the delay to be 60s, to allow for the cluster being slow during patching, etc.

This is also happening with ppr-ui. Let's fix all the deploymentconfigs.

Comments: 05/Dec/19 1:03 PM;WAMOAR;Picking this up so that we prevent the error messages.
05/Dec/19 3:12 PM;WAMOAR;Done - deployed to dev and test.

Type: Task
Status: Done
Priority: High
Resolution: Done
Assignee: WAMOAR

James provided wireframes for the search screens. Mock these up. The mock ups probably will not make api calls.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Comments:

Type: Story
Status: Backlog
Priority: Low
Resolution:
Assignee:

We have decided to start with https://codecov.io to see if it meets our requirements for metrics around testing coverage of our code. Set up a proof of concept in the GitHub Actions to run a codecov report on PR submission. Keep it simple - just a PoC for now.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

As per the BC policy framework for working with GitHub (https://github.com/bcgov/BC-Policy-Framework-For-GitHub/blob/master/BC-Gov-Org-HowTo/Cheatsheet.md) we must:

Ensure the repo contains the minimum required content (LICENSE, README.md, CONTRIBUTING.md files)

When a new version of the front-end is deployed, it is not showing up in the browser. The browser is aggressively caching, neither F5 nor CTRL-R will load the new page. Using the browser in incognito mode will load the new page.

Comments:

Type: Task
Status: Backlog
Priority: High
Resolution:
Assignee:

We currently do not have any documentation describing how to do the GitHub workflow when using VS Code. Adapt the namex documentation (https://github.com/WalterMoar/namex/blob/master/docs/developer.md) for PPR.

In order to ensure we can wrap the existing functionality with an API and apply the strangler patter to move to a new implementation, we need to prove that we can interact with the existing system using a Java program.

Acceptance Criteria:
Given an IMS-API application is able to connect to the TEST IMS Host
When a call is made to the API with a single value search query
Then the API will successfully perform a search against the IMS Host and retrieve results

Comments: 04/Dec/19 1:19 PM;JGUERTIN;(flag) Flag added

While we are able to connect and send messages to the IMS Server, we have not yet been able to get a successful response back. Ali is working to see if we can determine where the problem occurs with our message and if we can provide correct details.

Until we are able to programmatically complete a single search, this issue will remain blocked.

10/Dec/19 11:14 AM;WAMOAR;PR67 deployed to dev.
12/Dec/19 2:07 PM;JGUERTIN;(flagoff) Flag removed

Flag removed, we were successfully able to execute a search

Type: Task
Status: Done
Priority: High
Resolution: Done
Assignee: JGUERTIN

Set up the zero trust networking for the front end.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Create the k8s readiness and liveness probes for the ppr-api.

Comments: 16/Dec/19 12:02 PM;JGUERTIN;Code has been reviewed and merged
16/Dec/19 3:13 PM;WAMOAR;Having problems starting up, it looks like the liveness probes are causing it to be killed. I'll sort out the problem in dev and report back.

Type: Task
Status: Under Review
Priority: Low
Resolution:
Assignee: JGUERTIN

Implement Launch Darkly in the client UI.

Comments: 11/Dec/19 10:20 AM;TCHAKANY;[~WAMOAR] says this is complete

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: BGILBERT

Background

We need to determine how to set up the Cross-Origin Resource Sharing (CORS - https://en.wikipedia.org/wiki/Cross-origin_resource_sharing) configuration for the API.

Details

We need to make this configurable at runtime, probably using the standard dotenv and os.getenv that we see in other BCRS apis. The configuration is currently hard-coded but we should probably go with:

• dev: http://localhost:8080 (desktop frontend) and https://ppr-dev.pathfinder.gov.bc.ca (dev frontend).
• test: https://ppr-test.pathfinder.gov.bc.ca (test frontend).
• prod: https://ppr.pathfinder.gov.bc.ca (prod frontend).

This will probably all change with apigee (or other gateway) but for now it is what we know.

• As a Registry staff member I need to be able to:
  • See which financing statements have been registered, extended, and discharged in the last month (may not be required going forward)
    *See what types of searches have been done in the last month e.g., search by serial # (may not be required going forward)
  • See who has been searching in the last month e.g., BC Online users, staff (may not be required going forward)

• As a taxation staff registering party I need to be able to
  • Draft, verify, register, change, and discharge a crown charge so that I can register the crown's interest in a piece(s) of collateral
  • Draft, verify, register, change, and discharge a miscellaneous Reg. Act charge register the crown's interest in a piece(s) of collateral
• As a Registry staff member I need to be able to:
  • Perform the same tasks with Crown and Misc. Reg Act Charges as I would with a financing statement

Set up a new endpoint in the ppr-api to call the ims-api and return whatever result is produced (as-is).

Comments:

Type: Task
Status: Cancelled
Priority: Low
Resolution:
Assignee:

Set up the annotations on the front end so that we can configure the zero trust networking.

Comments: 05/Dec/19 2:13 PM;WAMOAR;Done - deployed to dev and test.

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: WAMOAR

Set up annotations so that we can configure the zero trust networking.

Comments: 04/Dec/19 3:07 PM;WAMOAR;Sent to Bryan and Justin for review.
05/Dec/19 10:57 AM;WAMOAR;Done, deployed to dev and test.

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: WAMOAR

• As an applicant I need to be able to
• Apply and get appropriate access to financing statement, crown and misc Reg Act charge functionality

The relationships team is responsible for this work.

• Roles
• Permissions
• etc....

Once the health checks have been deployed, enable them in the OpenShift deploymentconfig files.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

The OpenShift cluster will be upgraded to version 4.2 sometime in 2020. We will be required to migrate to the new cluster, and will have hard deadlines to do so.

PEP257 is the guide for Python docstring conventions. Before we get going too far with the ppr-api we should get a good baseline of code documentation.

Comments: 11/Dec/19 2:12 PM;WAMOAR;Done - good thing, it's used by our api framework.

Type: Task
Status: Done
Priority: Medium
Resolution: Done
Assignee: WAMOAR

Comments:

Type: Story
Status: Backlog
Priority: Low
Resolution:
Assignee:

Overview

The OpenShift cluster has recently added self-service network security, allowing us to explicitly define the network connections that are allowed within our applications. This form of networking will eventually be mandatory within the cluster. We should implement it at the start of the project, so that it doesn't become a burden later - it will take more and more effort to implement the longer we leave it.

Technical Details

The namespaces are initially provided with three default networksecuritypolicy entries:

wamoar@ND080574:/mnt/c/Users/wamoar/VSCode/ppr$ oc get networksecuritypolicy
NAME                                      AGE
egress-internet-zwmtib-tools              22d
int-cluster-k8s-api-permit-zwmtib-tools   22d
intra-namespace-comms-zwmtib-tools        22d

These allow: internet to the namespace; namespace to the kubernetes APIs; and pod-to-pod networking within the namespace. These need to be tightened up, such as the first policy being restricted to certain pods, rather than the namespace as a whole.

Overview

In OpenShift, all of our application pods need to have Health Checks. These are ways that the cluster can check that a pod is healthy and ready to perform work. Currently there are no health checks in the API pods, so when there is a redeployment (or cluster patching) we will have an outage that affects customers. As this is something that will affect the users and is unwanted behaviour, it is considered a bug.

Technical Details

Borrow from https://github.com/bcgov/lear/blob/master/legal-api/src/legal_api/resources/ops.py although we should consider the repercussions of pinging the DB: we don't want a quick DB blip to cascade into a backend churn that causes an outage.

Level 1 monitoring is knowing whether or not our application / web services are up. The long-term plan is to use a cluster-external deploy of Prometheus, but in the meantime just use Uptime Robot to do the work.

Comments: 03/Dec/19 3:51 PM;WAMOAR;Setting up Uptime Robot for the UI and API in d/t/p.
03/Dec/19 3:52 PM;WAMOAR;Setup Complete.

Type: Task
Status: Done
Priority: Low
Resolution: Done
Assignee: WAMOAR

Once the ims-api is successfully calling the IMS Transactions for search, the ppr-api will need to be extended to call the ims-api. See https://github.com/WalterMoar/ppr/tree/ppr_pi_search for a very, very rough passthrough web service calling the ims-api.

Comments: 16/Dec/19 2:29 PM;WAMOAR;Was initially deployed but failing to start in OpenShift. Added more CPU to the pods, and it should start up now. Back in review.

Type: Task
Status: Under Review
Priority: Low
Resolution:
Assignee: WAMOAR

Any deployment configuration that require secrets or configmaps should also be "apply"ing those object when deployment is done. This allows removes the manual steps of creating / updating these files. It also allows us to create PR-specific deployments for running tests.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Set up the zero trust networking for the ims-api.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Background

The API comes with built-in and low-effort live demo documentation, courtesy of OpenAPI (aka Swagger) docs. Although it is low effort, some effort is indeed required to provide descriptive elements to the docs. We should also include completion of OpenAPI docs as part of our definition of done for API work.

Details

Check out the Swagger docs and figure out how to get better information into the docs.

Before we can put anything on the front end, we need a build config that works with Quasar.

The dream goal with Enterprise DB (EDB) is to offload the infrastructure operations to someone like Advanced Solutions (aka DXC, aka EDS, née HP). This would allow the BC Registry Services staff (DBAs) to focus on database administration at the application level, and not creating/verifying/restoring backups, dealing with corrupt database files, patching, disk space issues, etc, etc. I would also like to see the developers/SR team doing development work and not babysitting databases.

This idea comes from Thor, so he is probably the first person who should be in discussions. Maybe the lab in general would see it as a benefit, although I think we're the only users of EDB so far.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee: TCHAKANY

Comments:

Type: Story
Status: Backlog
Priority: Low
Resolution:
Assignee:

Right now anyone collaborator with write access can merge to master. We need to limit this, and the codeowners looks to be the way to do so. Will require investigation and testing.

Before I get going too far in the ppr-api, I need some time to re-read PEP8 (python style guide) and ensure that the existing code is pythonic enough.

Comments: 11/Dec/19 12:17 PM;WAMOAR;Read up on PEP8 to refresh pythonic habits.

Type: Task
Status: Done
Priority: Medium
Resolution: Done
Assignee: WAMOAR

We are using Jaeger for request tracing. The goal of this task is to integrate tracing with with the web application in order to give a holistic picture of requests through the system.

Specifically, we should use the OpenTelemetry client to push details to Jaeger.

Acceptance Criteria:
Given a user is browsing the PPR Web app
When an action is performed that invokes the PPR API
Then a trace log is sent to Jaeger with specifics of the request

Note that typically we will not log all interactions to Jaeger, but just a sample, such as 0.1%. For testing purposes we may capture all interactions, but we will scale this back at the completion of this task.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Integrate our app with the Auth components.

As a User
I want to use the PPR system as an Authorized User
because I want to protect my data.

Comments:

Type: Story
Status: Backlog
Priority: Low
Resolution:
Assignee:

Set up the zero trust networking for the IMS API

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee:

Set up a Hello World version of the backend using FastAPI and get it into production.

Before we get dependent on EDB, we should create a health check for it. This will give us an idea of how dependable it is.

In the ppr-api, create an endpoint that calls EDB and does a simple select. Add this endpoint to UptimeRobot for monitoring.

Comments:

Type: Task
Status: Backlog
Priority: Low
Resolution:
Assignee: