Git Product home page Git Product logo

mes-data-poller-service's Introduction

mes-data-replication-service

Service that implements polling data from the replicated database (Aurora MySQL), writing to Dynamo DB tables.

source-db

The source database schema (Oracle) and DMS CDC grants.

destination-db

The destination database schema (Aurora MySQL). Contains the following two sub-folders:

sample-data-load

These scripts should be run sequentially on a non-integrated Aurora MySQL environment to create the tables and seed them with some sample data scenarios. Further details can be found at https://wiki.i-env.net/display/MES/TARS+Replica+-+Sample+Journal+Data

post-data-load

These should be run after the scripts in sample-data-load (for a non-integrated environment) or after DMS has set up and initial load completed (for a TARS integrated environment). Further details can be found at https://wiki.i-env.net/display/MES/TARS+Replica+-+Database+Objects

Integration tests

There are some integration tests in the solution. These spin up DynamoDB local + require MySQL in Docker to test against.

Integration pre-requisites

In order to run the integration tests, you need:

  • A JRE (tested with Java 8)
  • Docker (tested with Docker Desktop for Mac)

Running integration tests

To run the integration tests, open a cmd window and change to the e2e directory and run

docker-compose up --build

In a separate window run:

npm run test:int

If you get issues starting the integration tests due to ports already being bound, you can kill old instances of the test components by running:

npm run stack:down

remember to run

docker-compose down

to clean up the docker container.

E2E tests

There are some e2e tests for the journal and users pollers. These spin up DynamoDB local + MySQL in Docker to test against.

E2E pre-requisites

In order to run the E2E tests, you need:

  • A JRE (tested with Java 8)
  • Docker (tested with Docker Desktop for Mac)
  • A valid AWS credentials profile (either set as default or reference in the AWS_PROFILE envvar)

Running E2E tests

To run the E2E tests, open a cmd window and change to the e2e directory and run

docker-compose up --build

In a separate window run:

npm run e2e

If you get issues starting the E2E tests due to ports already being bound, you can kill old instances of the test components by running:

npm run stack:down

remember to run

docker-compose down

to clean up the docker container.

mes-data-poller-service's People

Contributors

chrisnappin avatar andrewsetterfield avatar ali-campbell avatar rossfauldsbjss avatar matthew2564 avatar richakl2 avatar dependabot[bot] avatar ammarhaiderbjss avatar cullima avatar jameshighfield avatar markpatt89 avatar marcbowen avatar snyk-bot avatar gittins7 avatar tombillingtonuk avatar calvin-j avatar hiddlaj avatar faggc avatar amobarak avatar appleabs avatar rstepnowski avatar mgoldson avatar sebolabs avatar

Watchers

Shaun Hare avatar James Cloos avatar Edward Wilson avatar Michał Piątek avatar Ricky Walker avatar Howard Rees avatar Szabi Keresztes avatar Przemek Sempruch avatar  avatar  avatar  avatar  avatar

Forkers

uk-gov-mirror

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.