Git Product home page Git Product logo

cas-ggircs's Introduction

Welcome to GGIRCS

GGIRCS stands for Greenhouse Gas Industrial Reporting and Control System for the Climate Action Secretariat (CAS).

Description

GGIRCS is a tool designed to modernize the annual reporting and verification of greenhouse gas industrial emissions. This project is a fulfillment to meet the requirements of the BC government initiatives including Climate Strategy, Clean Growth and CleanBC, part of the Greenhouse Gas Industrial Reporting and Control Act [GGIRCA].

Status

  • Master Pipeline: CircleCI

  • Develop Pipeline:

  • Functional Test:

Usage

Release procedure

Clone repository

In terminal:

  • Run make verify to ensure minimum required versions of all dependencies are installed and postgres is online (see the dependencies section below)

  • Run make install to set up a development environment

  • Run make test to execute all pgTAP-based tests against a ggircs_test database

  • If needed, create a development database using $ createdb ggircs

  • Run sqitch deploy to deploy to a ggircs database

Using Sqitch


If you're new to Sqitch, the best place to start is with the tutorial and other docs.

Add Schema

  • sqitch add schema_[schema_name]

Add Table

  • sqitch add table*[table_name] --require schema*[schema_name] --set schema=[schema_name]

Dependencies


Installation

Version 10 or higher recommended. Usually available via your distribution's package system. Binaries and source are also available for download. The PostgreSQL wiki has a list of detailed installation guides where you can find some OS-specific instructions.

A role/user with the following options must be created (see this tutorial if you need information on how to create roles):

  • The role name is your current user name ($ whoami)

  • The role has the SUPERUSER option

1.7.0 or higher recommended. Usually available via your distribution's package system. Binaries and source are also available for download.

5.10.0 or higher. Included in most Unix distributions and on OS X. Windows users can install ActivePerl. The Perl website has OS-specific installation instructions.

Data

Excluding Test Data

The data-set that is parsed by the ETL process includes some test data that needs to be excluded from the actual data housed in the swrs schema. This is achieved in swrs/transform/table/ignore_organisation by adding the swrs_organisation_id of organisations that were created for test purposes to this table. A join on this table in the swrs/transform/view/final_report view then ignores these organisations when loading data into the final schema.

Data Architecture

(TBC)

Project Material Publication

The materials published include:

  • Code Repositories
  • /bcgov/cas-airflow-dags
  • /bcgov/cas-ciip-portal
  • /bcgov/cas-docker-metabase
  • /bcgov/cas-docker-sqitch
  • /bcgov/cas-ggircs
  • /bcgov/cas-ggircs-ciip-2018-extract
  • /bcgov/cas-ggircs-ciip-2018-schema
  • /bcgov/cas-ggircs-metabase
  • /bcgov/cas-ggircs-metabase-build
  • /bcgov/cas-ggircs-metabase-builder
  • /bcgov/cas-helm
  • /bcgov/cas-metascript
  • /bcgov/cas-pipeline
  • /bcgov/cas-postgres
  • /bcgov/cas-shelf
  • /bcgov/cas-shipit
  • /bcgov/cas-shipit-engine

Files in this repository


Documentation

Contributors

cas-ggircs's People

Contributors

bcerki avatar dependabot-preview[bot] avatar dependabot[bot] avatar dleard avatar hamzajaved avatar joshgamache avatar matthieu-foucault avatar mikevespi avatar pbastia avatar rajpersram avatar sepehr-sobhani avatar wenzowski avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cas-ggircs's Issues

CE Needs - Discovery Phase

CAS's Compliance and Enforcement team requires a simpler way to compare BCGHG_StatusReport Excel Sheets when submitted data seems awry. from submitted Two main priorities are easier file comparison and more effect search of compressed files.

With goals established, SD and UX will follow the research steps outlined in the GDX Service Design Playbook found here.

Internal Prep

  • Compile a list of questions and unknowns with the GGIRCS team to be clarified in discovery phase
  • Set up a Miro space to track research findings and stakeholders

The Context

  • Evaluate the Compliance SME's process document to see how this problem fits into the team's big-picture process
  • Create a stakeholder map to understand who is involved and/or affected

Role Shadowing / Conversations with C&E Team

  • Via the Compliance SME and users listed below, set up dates and times for unmoderated shadowing to understand business workflows
  • Record sessions and archive in C&E Research and Design folder in Teams.

(see Design phase ticket, 277)

Additional notes:
Use the Compliance Internal Users MS Teams channel for setup/discussions as needed. The list of internal users and contacts can be found in the above MS Teams channel wiki.
Here is the link to the business context info (meeting notes from June 30 meeting).

As a Compliance Officer, when a duplicate file has been uploaded, I would like the file to only be displayed once so that I don't have to spend my time guessing what is new.

Compliance Officers often receive updated versions of reports. This card will enable the user to navigate through versions of files to find new submissions and compare them against old ones.

Options:
Potentially a way that the Compliance Officer doesn't need to compare the MD5 hash themselves

Question: @johnnagle How accurate is the problem framed in this user story? How would the team like to view duplicate and updated files?

C&E Retrospective

Reflect on the work done by the GGIRCS team when the C&E team have used the file diff feature more.

  • Get confirmation from John on feature usage
  • Schedule retrospective with C&E and GGIRCS teams
  • Plan for retrospective

Create Metabase questions for the swrs_history schema tables

The swrs_history tables were created in #266. There need to be metabase questions that allow compliance to view the contents of these tables in a way that matches their current workflow.

AC:
Acceptance Criteria

  • a question in Metabase that would show status report spreadsheet info.
  • a second question in MB that shows the list of attachments for a given report version (all in the SWRS history schema)

Audit SWRS Schema for unused tables and views

Description

Through discussion with Qinghan, I've noticed there might be a potential issue with duplicate data in our view "Report With Org".

Not sure about the usage of this view/table.

Screenshot 2021-09-27 at 10.42.41 AM.png

This is probably an issue with multiple views or tables that are either unnecessary or already flagged as "deprecated"
Those tables/views can be hidden in the metabase configuration.
Auditing which tables/views are used by existing questions can be done in the test environment, by hiding tables and using the questions check script to detect if hiding a table/view triggers an error.

remove unused columns from ggircs_portal.facility/organisation tables

The following table columns are unused & should be removed.

facility...
report_id
swrs_report_id
swrs_organisation_id

organisation...
report_id
swrs_report_id
reporting_year
operator_trade_name
duns
operator_mailing_address
operator_city
operator_province
operator_postal_code
operator_country

import_swrs_organisation_facility needs to stop importing / inserting these columns.

comment from YT: create_application_mutation_chain() uses facility.report_id and will need refactoring

Issue public SWRS emissions to orgbook

Description

The MDT team wants scope 1 data (public emission data found in the GHG public reports) to be published on orgbook, for a set of organisations.

Script to issue to OrgBook: https://github.com/bcgov/mines-digital-trust/tree/develop/services/ghg-issuer-tools

This requires mapping the SWRS organisations to Orgbook entities (using the BC registry numbers)

We need:

  • credentials to send requests to orgbook
  • the list of orgs we want to publish the data for
  • the SWRS Org ID - BC reg number mapping

As a metabase user, I want to see the report comments of a SWRS report

We are currently not extracting the report comments from the xml file (only extracting the update comment). The report comment has important information related to the facility.

To Do:

  • Update the report materialized views and tables to add column for report_comment
  • Look at update_comment as an example (it is the same process except for it is a different xml tag)
  • Add unit tests

GGIRCS pods should wait for sqitch migrations before starting

With database migrations running during the helm release, as opposed to running as a pre-update hook, there is a risk that the application server starts before the database changes are applied. When database changes update the schema (i.e. a table, function or type signature is modified), this would result in posgraphile performing its introspection on the old schema, and any new change would not be propagated to the graphql API (e.g. a new column would not be available).

To solve this, the application pod should wait for the sqitch deployment to complete its migration, using an init container.

See bcgov/cas-ciip-portal#1998 for implementation details

Probability (1-5): 3

Effect (1-5): 5

As a Compliance officer, I can see differences between versions of SWRS reports

Description:

As a Compliance officer, I can quickly see differences between version of SWRS reports within two XML files displayed side by side.

image.png

Given a report ID, we can retrieve the original XML report via the foreign keys located in the swrs_history.report table.
The XML string needs to be formatted (e.g. with https://www.npmjs.com/package/xml-formatter), its syntax highlighted (e.g. with https://prismjs.com/) prior to the diff being rendered (e.g. with https://github.com/praneshr/react-diff-viewer)

Miro board reference: CE Discovery Space

Acceptance Criteria:

In the GGIRCS app,

  • two text boxes to input the report ID
  • differences in XML text can be seen in red and green
  • indicate where the file came from, date, name of file
  • diffs can be collapsed (summarize changes toggle)
  • When you type a report into the input box - if the number is a valid ID, there is a green checkmark displayed and the XML appears automatically below.
  • If you type an invalid ID, helper text tells you it's incorrect.
  • There can be an option to reverse the order of the two files if desired.

Parse non-timestamp values for uploaded_at in swrs_history.report_attachment

All date values we've seen in other tags have been in timestamptz format.
The date values being parsed in the swrs_transform.historical_report_attachment_data materialized view are mostly not in timestamptz format. The format I've observed has looked similar to this format: DD/MM/YYYY HH:MM:SS [AM/PM] [PST]

Todo:
The materialized view's uploaded_at column should be varchar instead of timestamptz.
Write a function that turns this non-timestamp format into a string that can be converted to timestamptz. Function must return NULL if it is not able to convert the string into a time stamp.
We may need to attempt the above several times to detect all the possible formats for the dates.
Use this function in the load_history_attachment() function to load the values as timestamptz into the swrs_history.report_attachment updated_at column

XML Diff: search for an existing report by swrs_report_id via relay

We are currently fetching all ids & doing an array.find() on the swrs_report_id
This could probably be done with a filter in the relay query.
I think this would entail changing the two ReportSelector components into a single fragment container with two allReports targets (left/right side) which use the swrs_report_id in the router query string to filter and return the proper report.

As a compliance officer, I can see which GHG reports have a Verification Statement requirement in Metabase

Description

According to GGERR, Division 2 (C&E team to confirm the source), reporting operations must submit a 3rd party verification statement if they meet certain emission thresholds in the current or past three years of reporting (also called the 3 Year Rule).

AC:

  • Allow users to see whether or not a GHG Report is required to have a verification statement

Implementation details

The following approach would be a possible way to implement this in metabase:

  1. create a question returning, for each report, the sum of CO2e, excluding the following records from the emission table:

    • gas_type = CO2bioC
    • unit_name = CoalMiningPitMines
    • activity_name = MobileCombustion
  2. If the total emissions of a report is >=25,000t, then there is a verification requirement for the 3 consecutive periods following, i.e.: Given a report from the 2019 reporting period, when the total emissions is >=25,000 tonnes of CO2e, then the vs_requirement_until column should have a value of 2022

  3. Aggregate the previous results per SWRS facility id, and return the maximum value for the vs_requirement_until column

  4. for each report where vs_requirement_until is >= 2020, create a question that returns the following:

    • company name
    • swrs facility id
    • facility name
    • facility type
    • primary naics
    • total emissions and report numbers for the 2017 to 2020 reporting periods (create a question per reporting period to easily join)
    • 2020 report has a verification statement
    • link to 2020 verification statement (will add once #288)

When validating the results against the compliance spreadsheet, the following needs to be checked (might need a meeting with John et al.):

Ensure that emissions in the report are not double counted when GHG report includes subprocess_name: Additional information for cement and lime production facilities only (not aggregated in totals); Additional information required when other activities selected are Activities in tablee 2 rows 2,4,5, or 6. Note, i am not sure if this is a quirk of Qinghan's extractions, or if this is something that MB would need to filter - this is just how we filter it using his extractions.

As a Compliance Officer, I want the history and list of attachments of SWRS reports available to be queried in metabase

This data would be extracted in a separate schema, swrs_history (feel free to suggest other names), which would have a report table identical to swrs.report but would contain all of the versions, not just the final reports, and including SaleClosePuchase reports which are filtered out by the final_report view.

The swrs_history schema would have a report_attachment table, listing the attachments for a given report.

TODO: When a new version of a SWRS report is extracted, we appear to receive the previous version of a report and its attachments in the new SWRS zip file. Check that this is indeed the case, or if something else is going on.

Acceptance Criteria

  • a swrs_history schema exists in the database
  • a report table exists in the swrs_history schema that contains all reports (not filtered like swrs.report)
  • an attachment table exists in the swrs_history schema that contains details on all the attachments relating to a report
  • data in the swrs_history schema is ready to be used in metabase to replicate the data in the status report file contained in each zip file

The status report spreadsheet includes some info that we are not currently parsing which can be done in the ETL for swrs_history.

Plan for capturing R&A process

The GGIRCS team will soon be working with R&A to document their process of receiving & processing data from SWRS.

Acceptance Criteria:

  • Create rough plan of attack for capturing process
  • Review plan and get feedback from CF and ES
  • Ideate potential types of maps (Service Blueprint vs. Business Process Map vs. System Model) using work done in the Mapping the System Miro board

Notes:

  • Keep CF/ES and team up-to-date on progress and outcomes

CE Needs - Opportunities in Process

CAS's Compliance and Enforcement team requires a simpler way to compare BCGHG_StatusReport Excel Sheets when submitted data seems awry. Two main priorities are easier file comparison and more effect search of compressed files.

Acceptance Criteria

  • Ask clarifying questions of team based on unknowns gathered above
  • Receive CE Team's feedback on process and answers to questions
  • Refine process outlined in CE Discovery Space Miro using CE Team's feedback
  • Get CE Team's green flag on process outlined in CE Discovery Space Miro
  • Hold session with GGIRCS & CE Team to identify opportunities
  • Generate User Stories
  • Track any other issues that came up in research that may be surfaced in future

Additional notes:
Use the Compliance Internal Users MS Teams channel for setup/discussions as needed. The list of internal users and contacts can be found in the above MS Teams channel wiki.
Here is the link to the business context info (meeting notes from June 30 meeting).

swrs.facility_details view should be refactored

This view may still be useful to metabase users.
If so, the way this view retrieives the naics code is highly ineffecient.

  • It should not join on swrs.naics_category_type (deprecated table)
  • The group by clause is probably unnecessary
  • Retrieving the proper naics code can probably be done by using the report_id and facility_id of a facility along with the naics_priority column in the naics table.

As a CAS user, I want to define the Carbon Tax Act fuels

Description:

CAS users should be able to add/edit the CTA fuels and any associated Normalized fuel types. This would be changing data in the ggircs_parameters.carbon_tax_act_fuel_type and ggircs_parameters.fuel_carbon_tax_detail tables

Acceptance Criteria:

Given I am in the CTA management UI
When I need to add a carbon tax act fuel type
Then I can add a cta fuel type

Given I am in the CTA management UI
When I need to edit a carbon tax act fuel type
Then I can edit a cta fuel type

Given I am in the CTA management UI
When I need to assign a normalized fuel type to a cta fuel type
Then I can assign the normalized fuel type to a cta fuel type

Given I am in the CTA management UI
When I need to un-assign a normalized fuel type from a cta fuel type
Then I can un-assign the normalized fuel type from a cta fuel type

Development Checklist:

  • carbon tax act fuel types can be added
  • carbon tax act fuel types can be edited
  • normalized fuel types can be assigned to a carbon tax act fuel type
  • normalized fuel types can be un-assigned from a carbon tax act fuel type

Explore reusable UI toolkit

AC
Not using react bootstrap
Potential constraints (developers to elaborate)
Reach out to BC Design System team about contributing (what's the process?)

  • the cas-ggircs repo uses the service-development-toolkit from button with a BCGOV theme
  • open issues in the service development toolkit for Popover ListGroup and react-bootstrap-typeahead functionality support
  • fix tests

Reference links
https://github.com/button-inc/service-development-toolkit

Compile BCCR Recommendations

Following the call with the BC Carbon Registry team leads on January 14th, this ticket documents the capturing of recommendations for service design & user research.

  • Create list of recommendations identified
  • Pass list to Lindsay for feedback/additions
  • Pass list on to BCCR team leads

Add combustible waste to the carbon_tax_act_fuel_type table

Combustible Waste has been added to the the list of taxed fuels in Schedule 2 of the CTA as of 2020-04-01.
Todo:

  • Add Combustible Waste to the carbon_tax_act_fuel_type table
  • Add rows to the fuel_charge table for Combustivle waste starting at the reporting period 2020-04-01
    Unnecessary, this was deployed with the fuel_charge UI & rates for Combustible Waste can be defined there.

Metabase alerts for fuel - emission_category assumptions

These alerts cannot be created until the latest cas-ggircs release has been deployed to production

The swrs carbon tax calculator makes some assumptions about the data ingested from swrs. These assumptions need to be validated via metabase alerts to let us know if there is a change.

Assumptions:
A fuel is only ever associated to one emission category
A fuel is never reported without an associated emission category

These assumptions are currently true for all data received beyond the 2011 reporting year.

Acceptance Criteria

  • add a metabase alert: Fuel has > 1 emission_category
  • add a metabase alert: Fuel has a fuel_amount and no emission_category (year > 2011)
    (these may end up being one alert)

As a CAS user, I want to manage the list of normalized SWRS fuel names

Description:

The list of fuels in the swrs.fuel_carbon_tax_details table is only editable by developers at the moment. This list contains the normalized fuel names, and connects them to the fuels defined in the carbon tax act. This feature would allow CAS staff to make updates without the development team's involvement.

This feature should provide a UI allowing CAS staff to manage the contents of the swrs.fuel_carbon_tax_details table

Acceptance Criteria:
Given I am logged into the ggircs app
When I am on the landing page
Then there is a option to manage the normalized fuel types

Given I am in the fuel types management UI
When I first load the page
Then any fuels that are not associated to a normalized fuel type are clearly shown
and I can easily set the normalized fuel type for that fuel

Given I am in the fuel types management UI
When I want to find a fuel type
Then I can search the normalized fuel types

Given I am in the fuel types management UI
When I need to edit a normalized fuel type
Then I can make a change to the normalized fuel type

Given I am in the fuel types management UI
When I need to remove a fuel's association with a normalized fuel type
Then I can remove that fuel from the normalized fuel type
and it is then shown at the top of the page as an unassociated fuel

Development Checklist

  • UI has a design
  • user can navigate to a page to manage the fuel types
  • fuel types without a normalized fuel type are shown at the top of the page
  • fuel types without a normalized fuel type at the top of the page have a dropdown to select a normalized fuel type
  • re-using the NAICS ui, clicking on a normalized fuel type shows the details and the associated fuels
  • normalized fuel types can be searched
  • normalized fuel types can be edited
  • fuels can be removed from a normalized fuel type, doing so adds them to the list of fuels without a normalized fuel type at the top of the page
  • Meets the DOD

Move xml-diff state variables into the router's query string

The swrsReportId's and some boolean state variables could possibly be moved into the router's query string so that the url of a diff can be saved/shared.
This likely involves passing the router through the QueryRenderer in _app.tsx like we do in the ciip_portal repo.

  • Passing the router as props like this has some side effects with the long-rendering / page-blocking Diff library component.
  • It appears UseRouter() creates a context at the highest level in the tree that it is called. Creating it in the adjacent ReportSelector components creates two different contexts which override each other, a parent component is likely needed to instantiate this hook
  • If we want to use the variables in the query string to refactor the relay request, some variable type parsing will need to be done in _app.tsx (example also in the ciip_portal repo)
  • The report objects in state need to stay in state, just the primitive variables should be moved into the query string.

Compile C&E Next Steps

Following the C&E Retrospective on January 14th, this ticket documents the capturing of next steps identified.

  • Create list of next steps identified
  • Pass list to Lindsay for feedback/additions
  • Schedule tickets for requested adjustments in ZenHub
  • Make a plan to schedule another retrospective when C&E team has used the tool more (captured in this ticket)

As a Compliance Officer, I want to directly access the documents listed in the file viewer so that I can look at the contents of the files

Description

Part of the compliance process involves opening the various attachments submitted with a SWRS report. Given the number of reports, attachments, and the fact that they are located across all archives, being able to download reports in a timely manner is paramount.

This user story's work is comprised of a metabase question (with a custom column using the concat function) that uses the attachment's and report's info to generate a link to ggircs application. This will allow the user to access links to documents directly from the file comparison viewer.

The file name in the URL should be URL-encoded (spaces replaced with %20, and other special characters, if any are present in the attachments file names) Modern browsers don't seem to mind spaces and slashes

The ggircs application already supports direct download of files through its API, so no work is needed on that end.

Scenarios

the swrs_history.report_attachment table has the following columns:

  • md5 hash
  • zip file name
  • file path

The swrs_transform.load_report_attachment(), needs to, for each record in the swrs_transform.historical_report_attachment_data materialized view, find the corresponding record in the swrs_extract.eccc_attachment table, based on the swrs_extract.eccc_attachment.swrs_report_id, swrs_extract.eccc_attachment.source_type_id and swrs_extract.eccc_attachment.uploaded_file_name columns.

Given that I am logged in metabase,
When viewing a question named attachments with download links
Then I can see a column named download_link, with a link to the ggircs application.

Given an attachment with file a path of Output_Prod%2FReport_1234_2020_SourceTypeId_42_some%20file.pdf and a zip_file_name of GHGBC_PROD_20200101.zip
When viewing the download_link column in the attachments with download links question
Then the link should be https://cas-ggircs.apps.silver.devops.gov.bc.ca/api/eccc/files/GHGBC_PROD_20200101.zip/download?filename=Output_Prod%2FReport_1234_2020_SourceTypeId_42_some%20file.pdf

CE Retros & Feedback Loops

While we wrap up our work with compliance, it will be helpful to set up feedback loops to have the GGIRCS team and CE team reflect together on the new solutions developed.

Acceptance Criteria:

  • Decide on check-in approach
  • Reach out to CE Team to set up conversation
  • Schedule session
  • Design session in the C&E Thinkspace Miro board

CE Needs: Design Phase

After determining outcomes from research phase from ticket 272, implement design steps to improve C&E's workflow for comparing Status Report Excel sheet to identify oddities in submitted data.

Feedback and testing with CE team will occur throughout steps.

Design Tasks

  • Idea generation on potential solutions
  • Lo-fi mockup
  • Hi-fi mockup with detailed interaction states
  • Guidance around using new solution

Acceptance Criteria

  • Provide mockups for potential solutions (via Figma or Miro)
  • Improved means for file comparison
  • More effective search methods of compressed files
  • Quickly identify dissimilar reports for a given Operator

Additional context

Background info and nice-to-haves listed in meeting notes, found here.

As a metabase user I can see the estimated carbon tax in the SWRS schema

Description

The reporting and analytics needs to estimate the carbon tax paid by reporting operation, in order to allow the province to define budgets, or to respond to information requests from other units.

The swrs schema has a carbon_tax_calculation view which was developed two years ago, but suffered from known performance issues. In addition to the performance issues, known issues related to the accuracy of the tax estimates exist in the calculator, and it should be re-written.

Features to implement (to be refined)

  • The Calculator should exclude LFO facilities from the results (display IF_a, IF_b and L_c instead)
  • An emission_category table, similar to the one used in CIIP, should exist, and indicate which emission categories are taxed (the waste should carbon-taxed, same as in CIIP)
  • A taxed_venting_emission_type table should exist, listing the venting emission types that are taxed
  • Missing indices (if any) should be added and the carbon_tax_calculation view should be rewritten to remain performant
  • Given that a reporting operation reported a flared CO2 gas amount of X tonnes of CO2e,
    when the carbon tax calculator is executed,
    then it should return a record with a Natural gas fuel amount of Y (as per WCI.20 EF for non-marketable gas, Table 20-3: 2.151 kgCO2/m3 non-marketable NG in BC - this reference must be in the view comment)
  • Given that a reporting operation reported a vented gas amount of X tonnes of CH4,
    when the carbon tax calculator is executed,
    then it should return a record with a Natural gas fuel amount of Y (as per WCI.353 Calculation of Greenhouse Gas Emissions, Equations 350-9 & 360-20 0.6785 kgCH4/m3- this reference must be in the view comment)

Notes

  • RT will be starting holidays on Dec 10 and may be away until May 2022. Scoping of work may want to be completed sooner.
  • Keep ES in the loop

Express server does not use updated SSL cert until it is restarted

In practice, this only happens if we don't push changes of that app to prod for a month, and if the pod wasn't affected by any maintenance.
When it does happen, the app becomes inaccessible to a non-technical user.

To resolve this, instead of having the express server taking care of the SSL termination, our applications should rely on an nginx reverse proxy deployed in a sidecar pod. See the ciip deployment for an example - using the chart from cas-template-app.

Probability (1-5): 2

Effect (1-5): 5

As a Compliance Officer, I want to flag files that are in progress

Description:

Related to #268, this was a feature shown in the mockup made by Raj. John clarified that this was a low priority at the moment but could be nice to have down the road.

This feature would allow COs to tag files that are being worked on as a way to mark what they were doing if they have to either step away or pass on to another colleague. The Flag icon would be a way to mark this state of reviewing files.

See this meeting recording for details from August 12, 2021.

Acceptance Criteria:

  • Option - use Ghirken:
    Given
    When
    Then
  • [Other AC]
  • [Other AC]
  • [Other AC]
  • Meets the DOD

Dynamically get columns used in join to finish this test

` -- Additional Data -> Activity
select results_eq(
$$
select substring((
select
indexdef
from
pg_indexes
where
schemaname = 'ggircs_swrs'
and
tablename = 'activity'
and
indexname = 'ggircs_activity_primary_key')
from '(?<=().+?(?=))')
$$,

-- columns used in join: ghgr_import_id, process_idx, sub_process_idx, activity_name
$$ select 'ghgr_import_id, process_idx, sub_process_idx, activity_name' $$,

'All columns in unique index ggircs_activity_primary_key are used in join when creating ggircs.additional_data -> ggircs.activity FK relation'

);`

As a Compliance Officer, I want to search the GGIRCS zip files

As a Compliance Officer, I want to search the GGIRCS zip files
So I can quickly mark them for comparison

This card is just a placeholder to see if we need to implement this functionality in the ggircs file browser instead of metabase. Iceboxed for now.

As a compliance officer, I want to see discrepancies between emissions of individual facilities and their parent facility, so that I can further review them

Description:

LFO facilities are comprised of individual facilities (IF_a, IF_b, L_c). Both the LFO and individual facilities are reported in SWRS, but discrepancies might exist.

If a discrepancy exists between the sum of the individual facilities and the LFO total, flag for further review - this may be an issue if one total is below the threshold and one is above the threshold.

When looking for discrepancies, the data can be aggregated at the following levels:

  • total facility emissions
  • process (Activity Name)
  • subprocess (Sub Activity Name)
  • emission type
  • gas type

AC

  • Given that I am loading the "LFO vs individual facilities emissions total" question, when an LFO has total CO2e emissions that do not match the sum of CO2e emission of its individual facilities, then the corresponding row is highlighted in red.
  • Given that I am loading the "LFO vs individual facilities emissions by Process" question, when an LFO's Activity has total CO2e emissions that do not match the sum of CO2e emission of its individual facilities for the same Process, then the corresponding row is highlighted in red.
  • Given that I am loading the "LFO vs individual facilities emissions by Sub Process" question, when an LFO's Sub Activity has total CO2e emissions that do not match the sum of CO2e emission of its individual facilities for the same Sub Process, then the corresponding row is highlighted in red.
  • Given that I am loading the "LFO vs individual facilities emissions by Emission Type" question, when an LFO's Emission Type has total CO2e emissions that do not match the sum of CO2e emission of its individual facilities for the same Emission Type, then the corresponding row is highlighted in red.
  • Given that I am loading the "LFO vs individual facilities emissions by Gas Type" question, when an LFO's Gas Type has total CO2e emissions that do not match the sum of CO2e emission of its individual facilities for the same Gas Type, then the corresponding row is highlighted in red.
  • Given that I am loading the "LFO vs individual facilities emissions" dashboard, when entering a reporting year, business name, and/or LFO facility name, the five questions above are filtered accordingly

`swrs_eccc` dag does not call the `ggircs_app_user` task

Description

Upon running the swrs_eccc dag, the ggircs app will lose permissions to access the schema, until the deploy_db dag is run again.
Instead of adding the task to the swrs_eccc dag, we should rely on the dag trigger mechanism, and update dags to do the following:

  • after extracting the eccc xml files and attachments, the cas_ggircs_swrs_eccc DAG should trigger a new cas_ggircs_load_db dag
  • the cas_ggircs_deploy_db should run the init db task and trigger the cas_ggircs_load_db DAG
  • the cas_ggircs_load_db dag should have the following task dependencies:
ggircs_etl >> ggircs_read_only_user
ggircs_etl >> ggircs_app_user
ggircs_app_schema >> ggircs_read_only_user
ggircs_app_schema >> ggircs_app_user
ggircs_etl >> trigger_ciip_deploy_db_dag

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.