Git Product home page Git Product logo

iudx-gis-interface's Introduction

Build Status Jenkins Coverage Unit Tests Security Tests Integration Tests

IUDX

iudx-gis-interface

The GIS interface is a resource server which defines the interactions between the consumers interested in GIS data and the upstream GIS Servers. The gis interface is IUDXs data discovery, data publication. It allows admins to publish their gis related info in accordance to the IUDX vocabulary annotated meta-data document. The consumers can access data from the gis interface using HTTPs.

Features

  • Provides data access from available resources using standard APIs.
  • Integration with authorization server (token introspection) to serve private data as per the access control policies set by the provider.
  • Allows admin to publish, edit or delete gis related info as per IUDX data descriptor.
  • Secure data access over TLS.
  • Scalable, service mesh architecture based implementation using open source components: Vert.X API framework, PostgreSQL for database and RabbitMQ for data broker.
  • Hazelcast and Zookeeper based cluster management and service discovery.

API Docs

The api docs can be found here.

Prerequisites

External Dependencies Installation

The Gis-interface Server connects with various external dependencies namely

  • PostgreSQL
  • RabbitMQ

Find the installations of the above along with the configurations to modify the database url, port and associated credentials in the appropriate sections here

Get Started

Docker based

  1. Install docker and docker-compose
  2. Clone this repo
  3. Build the images ./docker/build.sh
  4. Modify the docker-compose.yml file to map the config file you just created
  5. Start the server in production (prod) or development (dev) mode using docker-compose docker-compose up prod

Maven based

  1. Install java 13 and maven
  2. Use the maven exec plugin based starter to start the server mvn clean compile exec:java@gis-server

JAR based

  1. Install java 11 and maven
  2. Set Environment variables
export RS_URL=https://<rs-domain-name>
export LOG_LEVEL=INFO
  1. Use maven to package the application as a JAR mvn clean package -Dmaven.test.skip=true
  2. 2 JAR files would be generated in the target/ directory
    • iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar - clustered vert.x containing micrometer metrics
    • iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Running the clustered JAR

Note: The clustered JAR requires Zookeeper to be installed. Refer here to learn more about how to set up Zookeeper. Additionally, the zookeepers key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.

The JAR requires 3 runtime arguments when running:

  • --config/-c : path to the config file
  • --hostname/-i : the hostname for clustering
  • --modules/-m : comma separated list of module names to deploy

e.g. java -jar target/iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar --host $(hostname) -c configs/config.json -m iudx.gis.interface.database.archives.DatabaseVerticle,iudx.gis.interface.authenticator.AuthenticationVerticle ,iudx.gis.interface.metering.MeteringVerticle,iudx.gis.interface.database.postgres.PostgresVerticle

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx4096m"
$ java $RS_JAVA_OPTS -jar target/iudx.gis.interface-cluster-0.0.1-SNAPSHOT-fat.jar ...

Running the non-clustered JAR

The JAR requires 1 runtime argument when running:

  • --config/-c : path to the config file

e.g. java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export RS_JAVA_OPTS="-Xmx1024m"
$ java $RS_JAVA_OPTS -jar target/iudx.gis.interface-dev-0.0.1-SNAPSHOT-fat.jar ...

Testing

Unit tests

  1. Run the server through either docker, maven or redeployer
  2. Run the unit tests and generate a surefire report mvn clean test-compile surefire:test surefire-report:report
  3. Reports are stored in ./target/

Integration tests

Integration tests are through Postman/Newman whose script can be found from here.

  1. Install prerequisites
  2. Example Postman environment can be found here
  3. Run the server through either docker, maven or redeployer
  4. Run the integration tests and generate the newman report newman run <postman-collection-path> -e <postman-environment> --insecure -r htmlextra --reporter-htmlextra-export .
  5. Reports are stored in ./target/

Contributing

We follow Git Merge based workflow

  1. Fork this repo
  2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name, or refer to a milestone name as mentioned in Github -> Projects
  3. Commit to your fork and raise a Pull Request with upstream

License

View License

iudx-gis-interface's People

Contributors

abhi4578 avatar ananjaykumar2 avatar ankitmashu avatar code-akki avatar dependabot[bot] avatar divyasreemunagavalasa avatar gopal-mahajan avatar kailash avatar karun-singh avatar pranavrd avatar shreelakshmijoshi avatar sivanaikk avatar sushanthakumar avatar swaminathanvasanth avatar tharak-ram1 avatar

Watchers

 avatar  avatar  avatar  avatar

iudx-gis-interface's Issues

5.1. Integration with Auth 3.0 APIs

The GIS server should be integrated with Authentication Server 3.0 and JWT token flows. Both open and secure resources should be accessible only with tokens.

5.1.1. Identity-based access restriction
Similar to rs and fs only authenticated users should be able to access the resource.

5.3. Secure Resource Flow

5.3.1. Creation of Request router
Use a WebClient to create an HTTPS API request to the GIS Server. Get information about the GIS Server from the PostgreSQL database.

5.3.2. Connecting with external GIS server to get API key
Using the WebClient created with 5.3.1, call the GIS Server, parse the response, and get the API Key. Share the API Key, Server URL to the API Verticle.

5.6. API Response

Add URN-based formats for API Response. The URNs are defined in section 5.6 of the SRS document.

Use correct resource id while logging for audit

While creating the GIS entry, we are using the admin token and iid of the admin token as resource id, which is [rs.iudx.io], it is not available as catalog id.
due to this metering APIs (summary API) will eventually fails.

Resolution:
use resource id from request.

Remove duplicate code in database verticle

Remove code duplication in database verticle.

  • Merge PostgresVerticle and DatabseVerticle code, since both connect to the same Postgres DB.
  • Code changes across verticles for the above changes.
  • Update test cases accordingly

5.5. Metering/Audit information

Currently, the GIS does not store any audit information. We need to integrate with ImmuDB for Metering/Audit information similar to rs. The schema for the GIS audit DB shall be similar to rs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.