aaronkurz / hitl-ab-bpm Goto Github PK
View Code? Open in Web Editor NEWBusiness Process Improvement with Reinforcement Learning and Human-in-the-Loop.
License: MIT License
Business Process Improvement with Reinforcement Learning and Human-in-the-Loop.
License: MIT License
Gradle script etc
Add the pip install
for the Contextual Bandits package + requirements to the docker file.
Due to the nature of a prototype, the BPMN files are not uploaded by the user via the frontend due to the simulator plugin conventions to simulate a business process. Therefore, the BPMN files of the variants are stored in the backend. In order to start an instance, the BPMN files have to be pushed to the Camunda engine first via API calls. Therefore, a method has to be implemented to retrieve a BPMN file (names) from the backend (database).
More sophisticated reward function is needed - perhaps implementing the function from the phd paper?
Converting seconds to milliseconds in order to ensure faster execution of the processes / RL environment. This can be done in the BPMN files.
Get everything development-ready
Get everything development-ready
Linked to #32
The question arose in the meeting on 23/11/2021. While running multiple instances at the same time, do instances influence each other in terms of runtime or access to a shared resource. This is more of a research question if such a implementation if possible. If not, we just have to document the limits of our work and make the assumption that instances do not influence each others runtime.
Creates requests for new process definiton instances (not the decision which version).
Probably just a python script.
After successfully starting and terminating a batch of instances via API calls, the history service has to be called to retrieve valuable data from each instance run. If an instance has not been terminated and the history service is called, an error will be thrown. In order to avoid an error and ensure the proper execution, it has be ensured that every instance has terminated before the data is retrieved.
Take a look at https://github.com/david-cortes/contextualbandits. Omar is already taking care of installing the package and requirements in the docker file. Take a look at the example and implement a basic version of the RL env/agent. Most importantly understand how the RL env and agent is set up, which methods are important and how it works. Start with the examples and work your way up from there. The specific method does not matter right now. Define a VERY simple and basic reward function. The focus is on starting the execution of batches of instances from within the RL env and the orchestration of the different components of the prototype.
Linked to #39
alternatively => parsing bpmn files?
Create the first api endpoints for the frontend to fetch data/ BPMN files from the backend. For example:
The instance router houses the RL env, utility classes to interact with the Camunda engine via API calls, and utility to store and retrieve data from the backend database. When the action of the RL agent is initiated (an action is chosen), multiple utility methods have to be called in order to properly interact with the Camunda engine that simulates each instance. After the simulation of each instance has terminated, the data has to be retrieved, stored, and processed (batch: calculate the mean) before it is fed back to the RL env which then calculates the reward. For every action the RL agent, this loop has to be cleanly implemented and executed. Therefore some form of management is required to execute the methods accordingly and guarantee the correctness.
The goal is one simple method call within the RL env that handles the above mentioned requirements. Maybe multithreading is required to synchronize the environments.
In order to properly test our implementation we need decently complex business process variants. Start with the examples given in the phd thesis linked in the slack (pilot license). Model these variants and add extensions to each task according to the plugin GitHub: https://github.com/camunda-consulting/camunda-bpm-simulator. Feel free to decide what good time frames, ranges etc. are.
I mean... everyone will have to do it sometime.
It is still open how we store data that we retrieve from the simulations of a variant of the business process. We could simply use log files or tables as a database. It depends on the data we retrieve, how much data we retrieve, how we use the data in the RL environment, and what data we would like to store for later, e. g. historical data, additional key performance indicators (KPIs) to evaluate the decisions of the RL agent makes.
Three days each week according to the 12 ECTS calculation.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Create a conceptual mock up of the frontend which then can be used as a baseline for discussions with the stakeholders. Elements that should be included:
Find out how to assign cost to a task or a function (cost/min) to a task. Offers an additional dimension to compare the variants. This also helps in terms of finding useful parameters for the human expert. Does Camunda allow such definitions and how would you implement/model that.
In order to properly setup the prototype locally, the "normal" Camunda engine image must be changed to our own Java application where the simulator plugin is enabled. After finishing this task, let other group members as well as the stakeholders test the implementation.
See the official announcement on the course page.
"Gelber Zettel" Enrollment form:
https://www.static.tu.berlin/fileadmin/www/10002461/Pruefungsamt/Formulare_Bescheide/Pruefungsanmeldung.pdf
Dependency: #25
In order to properly implement the RL agent, especially the reward function, we need some client of evaluation and monitoring. In the https://github.com/david-cortes/contextualbandits/blob/master/example/policy_evaluation.ipynb, a evaluation example can be found. On top of that, it can be helpful to track the accumulated mean reward. Find out if there is a pre-implemented methods for that or how we would track the reward live (tensor board offers such functionality). This also could be implemented by us. Simply add a list in which the calculated reward is stored and plot it in the end (just an idea).
Implement the initial policy proposal which is used by default before the human expert configures it. As discussed in the meeting on 23/11/2021 we start if 50/50 and the batch size is determined by the human expert (e. g. n=10). Then 5 instances of variant A and 5 instances of variant B will be started. This require the utility implementation TODO.
Linked to #49
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.