Git Product home page Git Product logo

refarch-kc's Introduction

Reefer Container Shipment solution - EDA reference implementation

The IBM Event Driven architecture reference implementation solution illustrates the deployment of real time analytics on event streams in the context of container shipment in an event driven architecture with event backbone, functions as service and event-driven microservices, and aims to illustrate the different event driven patterns like event sourcing, CQRS and Saga. There is a lot of content, so consider this to be a living book, for better reading experience go to the BOOK view.

TL;DR

If you want to just get the code, build and run we propose running locally with docker-compose.

To build and run the solution locally we are delivering some scripts which should help you to quickly get the solution cloned, built and deployed. Those bash scripts are under the scripts folder of this project.

git clone https://github.com/ibm-cloud-architecture/refarch-kc.git
cd refarch
./scripts/clone.sh

Running locally with docker-compose

  • Start Kafka, zookeeper and postgresql using: ./docker/startbackend.sh
  • Start the solution using the development settings: `docker-compose -f

Running in a local Kubernetes environment

The scripts provided in scripts/localk8s/ can be used to deploy to a local (vanilla) Kubernetes environment, such as a cluster provided by Docker Desktop.

Prereqs:

  • Helm (Helm v3 is recommended),
  • A Kubernetes cluster, and sufficient permissions to create namespaces, service accounts etc.

For more information, see the README for the scripts.

Building this booklet locally

The content of this repository is written with markdown files, built with Gatsby. For more information, see the README for the docs.

  1. Install NodeJS (https://nodejs.org/)
  2. git clone https://github.com/ibm-cloud-architecture/refarch-kc.git (or your forked repository if you plan to edit)
  3. cd refarch-kc/docs
  4. npm install
  5. npm run dev
  6. Go to http://127.0.0.1:8000/ in your browser.

Pushing the book to GitHub Pages

  1. Ensure that all your local changes to the master branch have been committed and pushed to the remote repository.
    1. git push origin master
  2. Ensure that you have the latest commits to the gh-pages branch, so you can get others' updates.
    git checkout gh-pages
    git pull origin gh-pages
    
    git checkout master
  3. Run mkdocs gh-deploy from the root refarch-kc directory.

Contribute

As this implementation solution is part of the Event Driven architecture reference architecture, the contribution policies apply the same way here.

refarch-kc's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

refarch-kc's Issues

Change images and references in docs to reflect changes made in refarch-kc-ui

We had an issue to remove code for the Customs and Docker services (refarch-kc-ui), however there are still some images in the docs that include these tiles. In the following locations:

https://ibm-cloud-architecture.github.io/refarch-kc/demo/readme/
https://ibm-cloud-architecture.github.io/refarch-kc/introduction/
https://ibm-cloud-architecture.github.io/refarch-kc/analysis/readme/

These may need to be updated to reflect the removal of these services in order to avoid confusion.

Site menu is broken

Steps to reproduce:

  1. Go to link https://ibm-cloud-architecture.github.io/refarch-kc/
  2. Click Business problem.. > Introduction -- generates transfer to page - https://ibm-cloud-architecture.github.io/refarch-kc/introduction - page displayed correctly
  3. Click Business problem.. > Introduction again -- generates 404 with the link https://ibm-cloud-architecture.github.io/refarch-kc/introduction/introduction - observe additional /introduction
  4. No other menu items are working then producing invalid links e.g.:
    https://ibm-cloud-architecture.github.io/refarch-kc/introduction/introduction/deployments/backing-services

Links should be fixed to be not relative to current context, but to site root.

Remove canned, hard-coded data for elements of the system and instead define default data via REST calls

Instead of having hard-coded data across multiple microservices that requires synchronization, remove all initial data / state from the microservices (fleets, ships, voyages, containers, cities), and instead define this state via a series of REST calls following deployment. This would allow for multiple pre-canned 'scenarios' with different fleets, voyages etc.

This can be done via a Job-based implementation for the deployment as a whole or as part of the individual microservices Deployment with an InitContainer.

Kafka connection and credentials flexibility

I noticed that the microservices hard-code the notion of using SASL PLAIN authentication, with a username of 'token', if an API key is provided (and optionally, the certificate to use to verify the server).

It appears this is sufficient for connecting to Event Streams. However, we might want to be more flexible if (for example) connecting to a Kafka created by Strimzi, which does not allow the use of PLAIN authentication (they support either mutual TLS, or SCRAM-SHA-512, which is effectively username and password).

This would require that the sasl.mechanism and username are configurable. I haven't looked at mutual TLS yet, but I imagine this would require injection of a client certificate instead of a username and password.

Several URLs open 404s

The following URLs open 404s:

As a result, several pages in the IBM Cloud Architecture Center now contain broken links, which our tests our flagging. Please let me know whether replacement URLs exist for those pages or if I should remove our references to them. Thanks.

Integrate existing Fleet Simulator & Reefer Simulator components into a perpetual simulation capability

Combine the existing Fleet Simulator capability that is exposed in the UI and the newer Reefer Simulator capability that is exposed through the Appsody Stack's inclusion of the Flassger/Swagger UI into a single simulator capability.

Ideally, the fleet simulation can be a perpetual motion simulator for the fleet itself (with location/position updates still available in the UI), while the UI interaction with the fleet would be to interject the co2sensor, o2sensor, & poweroff simulations that the reefer-simulator currently provides.

This could be done in an entirely new microservice or as an evolution of one of the two existing simulator microservices.

Add an overview of services deployed in the demonstration script

In the demonstration script, it would be good if there was an overview at the top with a list of all the services you need to have deployed in order to run through the demo guide.

It could also link out to the quick installation set up from @djones6 as a recommendation for getting set up quickly with the required pre-requisites.

Quickstart tutorial is not working

Hi Team,

I was trying todo the quickstart tutorial:
https://ibm-cloud-architecture.github.io/refarch-kc/quickstart-tutorial/deploy-on-openshift-playgrounds/

But the script in here kcontainer-quickstart-ocp.sh is cloning
https://github.com/ibm-cloud-architecture/refarch-kc-gitops .

Then it does a

86 cd refarch-kc-gitops/environments

and afterwards it tries to do

135 oc apply -k dev/infrastructure/

But there is not a dev directory in environments.
So that is broken.

Can you fix this or is there an updated guide ?

OpenLabs quick start using Lab6 instead

If you switch the lab to tell the user to Lab 6 "Bring Your Own Application" that's a better experience than using Lab 1
https://ibm-cloud-architecture.github.io/refarch-kc/business-scenario/quickstart-tutorial/

image

Using Lab 6 you can quickly find the OC console and the OC CLI login.
Also, the user would be confused with the other excersises in the left panel

This is what we recommend to users in Toolkit Workshop to use Lab 6
https://cloudnativetoolkit.dev/workshop/setup#1.-create-openshift-cluster

[quick start] Better guidance to indicate that there are 2 environment setup

While doing the quickstart lab https://ibm-cloud-architecture.github.io/refarch-kc/business-scenario/quickstart-tutorial/

Step 1 start with getting cluster on lean.openshift website, there is no obvious indication that there is another option to get a cluster on IBM Cloud using OpenLabs.

Maybe you make the 2 sections into carbon tab <Tabs>, or wrapped in a HTML section

The carbon tab would look like this
image

ANd perhaps favor the IBM Cloud since can be use with Free account and extended up to 8 hours, with learn.openshift I think the environment is lost after 60 minutes.

Migrate all topic names to be lower-case and hyphenated

This issue will span other repositories as the need for a more consistent naming approach occurs. Most simply, this should be able to be handled during the creation of the kafka-topics ConfigMap, but there may be alternative files and code paths that need to be adjusted to compensate for the change from camelCase to lower-case.

NOTE: The camelCase in the linked definition of the kafka-topics ConfigMap's keys is still correct. The left-hand values in the ConfigMap are the keys and are only used by Kubernetes to inject the values that are defined on the right-hand side of each line. The right-hand side values are what need to be hyphenated lower-case.

Linked repository issues:

Microservice helm charts use deprecated k8s API

The microservices will not install on newer levels of Kubernetes (1.16.x) as some deprecated APIs have been removed (eg. deployments that specify extensions/v1beta1 will no longer work, and should be updated to apps/v1).

Symptom:

Error: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"

Environment that this was observed:

$kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:08:14Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:18:29Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}

Reduce complexity of data model to only expose City to external users for shipping purposes

As City is the only relevant piece of information that we currently use for shipping, we should reduce the overall complexity of the exposed interfaces that require street address, state, country, zip code, etc.

Ideally, we could reduce this to city & country? (I know this would not give us absolute specificity for every real-world example, but it would be fine for demo purposes... The rest of the address elements would be a programming exercise.)

Depending on how far this change proliferates through the system - whether or not it's just the UI and we default the other fields to acceptable values OR we remove the deprecated elements from the data model altogether, this should proliferate from the beginning to the end of the data flow through the application.

Update Integration Tests to reflect new service names

Update https://github.com/ibm-cloud-architecture/refarch-kc/blob/master/itg-tests/es-it/ReeferItgTests.yaml once ibm-cloud-architecture/refarch-kc-gitops#17 is completed, as the new Appsody/OpenLiberty operators create services that are named differently than what we were explicitly creating in YAML previously (and it does not appear that we can control the service name at all through the AppsodyApplication or OpenLibertyApplication CRDs).

service/fleet-ms             => fleet-ms:9080
service/kc-ui                => kc-ui:3000
service/order-command-ms     => order-command-ms:9080
service/order-query-ms       => order-query-ms:9080
service/reefer-simulator     => reefer-simulator:8080
service/scoring-mp           => scoring-mp:9080
service/spring-container-ms  => spring-container-ms:8080 
service/voyages-ms           => voyages-ms:3000

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.