Git Product home page Git Product logo

stream-16-demo-app's Introduction

Demo Project for Online Stream #16,#17,#20, #21 - Microservice for Excel File Processing

Demo project for online stream #16 where a small microservice is created that processes Excel file uploaded through REST API, saves file to AWS S3, parses it to JSON and send that to another service.

On stream #17 the dockerization of the project is done together with additional properties and settings to prepare for running in production.

On stream #20 additional features are added: Sending of file to SFTP folder and then sending email. The integration with all external services is tested with running services in containers.

On stream #21 monitoring was added to service with Prometheus and Grafana setup to see application metrics

Diagram for Application

Access to Online Stream on YouTube

To get a link to online stream on YouTube please do the following:

  • 💰 Make any donation to support my volunteering initiative to help Ukrainian Armed Forces by means described on my website
  • 📧 Write me an email indicating donation amount and time
  • 📺 I will reply with the link to the stream on YouTube.

Thank you in advance for your support! Слава Україні! 🇺🇦

Technologies

  • Spring Boot
  • Apache POI
  • AWS Java SDK
  • SSHJ for SFTP
  • JavaMail
  • Prometheus for monitoring
  • TestContainers for testing
  • GreenMail for testing mail server

SFTP notes

To connect to SFTP server the SSH private key is used. The generated key is located in src/test/resources.

To generate new key you can use ssh-keygen tool on your system. Note, that key would be generated in OPENSSH format.

To convert to PEM format use the following command:

ssh-keygen -p -m pem -f /path/to/file

Building and Running the Application

To build the app, use gradle:

./gradlew build

To run application, use gradle:

./gradlew bootRun

After that you can access application on http://localhost:8080

Docker

To build docker image, use gradle:

./gradlew bootBuildImage

Then you run the docker container with exposing app port 8899:

docker run -p 8899:8080 -t stream-16-demo-app:0.0.1-SNAPSHOT

After that you can access application on http://localhost:8899

Running docker-compose with external dependencies

Project includes the docker-compose.yml file where external services are defined together with application.

Monitoring with Prometheus and Grafana

Prometheus and Grafana are started with docker-compose. Prometheus is set up to scape metrics from application that is also running in docker. The configuration for prometheus is defined in docker/prometheus.yml file.

Grafana is set up to read metrics from Prometheus. Grafana is started with default configuration defined in docker/grafana.ini. Default username and password to login to Grafana is admin:admin. You can import dashboards that were created during online stream from json files:

  • docker/grafana-dashboard-online-stream.json
  • docker/grafana-dashboard-spring-boot-stats.json

Documentation

Reference Documentation

For further reference, please consider the following sections:

Guides

The following guides illustrate how to use some features concretely:

Additional Links

These additional references should also help you:

stream-16-demo-app's People

Contributors

yuriytkach avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

stream-16-demo-app's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.