redhat-italy / hacep Goto Github PK

When drools sessions are empty, hacep works fine with a predefined load (110 evts/sec) distributed on 800 fact key, consuming all facts in jms queue. After few million events, hacep seems to slow down its throughput (about half of initial throughput)and messages on queue begin to accumulate. Looking on hacep code, listener FactListenerPost is sync, so when it calls method saver.insert((Key) key, (Fact) value) , the thread which triggered the cache event may block until the listener callback completes.

Test stores with hacep

When an event, already inserted in the buffer of serialized session, is modified in the HAKieSession using modify construct in drools, this change is also reflected in the buffer. So, during the rebuild of the serialized session, the rules won't fire with the same behavior than the live session one, creating a different replicated session.

Add a camel optional reordering events for cases in which jmsxgroup can't be used

EAP friendly version

Since update(...) method is only used in TestContainerUpdate and UpgradeCommandTest we can look into refactoring it.

When a node is started while other nodes are inserting facts in grid, this exception arises.

hacep.txt

Useful for rules which fire multi-actions on externals systems.

If a node crashes in the middle of a multi-action (let's say it executed only 2 out of 5 external non-idempotent REST calls, you need to replay the rule but execute just the last 3 actions)

Possibly have a single component containing parameters for both EAP and plain JVM code

write unit tests

Grouping is not really needed, a simple Key can be used, leveraging Consistent Hashing.

• use the "grouping" string as a key for session
• use the "grouping" string as a key for session and add a cacheUpdated listener for Events cache

It's crucial to execute a snapshot right before a kieSession live update.

Test new Infinispan command based architecture instead of DeltaAware in HASession

If a node N has an error during a rule update while other nodes Nx did everything correctly, router can restart too early on Nx nodes

some test are failing while building with the community profile.

[INFO] Reactor Summary:
[INFO] 
[INFO] HACEP Parent ....................................... SUCCESS [ 20.220 s]
[INFO] HACEP Commons Test ................................. FAILURE [01:22 min]
[INFO] HACEP Core Model ................................... SKIPPED
[INFO] HACEP Core ......................................... SKIPPED
[INFO] HACEP Core Camel ................................... SKIPPED
[INFO] HACEP Examples ..................................... SKIPPED
[INFO] HACEP Example Model ................................ SKIPPED
[INFO] HACEP Example Rules ................................ SKIPPED
[INFO] HACEP Performance Client ........................... SKIPPED
[INFO] HACEP Base Playground .............................. SKIPPED
[INFO] HACEP JVM Playground ............................... SKIPPED
[INFO] HACEP EAP Playground ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:16 min
[INFO] Finished at: 2017-03-06T17:58:02+00:00
[INFO] Final Memory: 43M/326M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on project hacep-commons-test: Failed to clean project: Failed to delete /home/valdar/projects/hacep/hacep/hacep-commons-test/target/generated-sources/annotations -> [Help 1]```

Now that SessionKey is gone, the class Key could probably be simplified

Add javadocs for each class

Some users reported problems with a single node, see if we can support single node configurations too

On linux the build process will fail during the tests when there are interfaces plumbed up with IPv6.

Build will succeed if run with:

mvn -P community -Djava.net.preferIPv4Stack=true clean install

08:37:12,515 INFO [org.jboss.weld.deployer](MSC service thread 1-6) JBAS016005: Starting Services for CDI deployment: hacep-eap-playground-1.0-SNAPSHOT.war
08:37:12,519 INFO [org.jboss.weld.deployer](MSC service thread 1-8) JBAS016008: Starting weld service for deployment hacep-eap-playground-1.0-SNAPSHOT.war
08:37:12,779 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject](MSC service thread 1-8) Found kmodule: vfs:/C:/development/jbdevstudio/runtimes/jboss-eap/bin/content/hacep-eap-playground-1.0-SNAPSHOT.war/WEB-INF/lib/hacep-rules-1.0-SNAPSHOT.jar/META-INF/kmodule.xml
08:37:12,782 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject](MSC service thread 1-8) Virtual file physical path = C:\development\jbdevstudio\runtimes\jboss-eap\standalone\tmp\vfs\deployment\deploymentacd29d9b237a97d3\hacep-rules-1.0-SNAPSHOT.jar-15c380aad66e9340\hacep-rules-1.0-SNAPSHOT.jar
08:37:12,903 INFO [org.drools.compiler.kie.builder.impl.KieRepositoryImpl](MSC service thread 1-8) KieModule was added: ZipKieModule[releaseId=it.redhat.jdg.examples:hacep-rules:1.0-SNAPSHOT,file=C:\development\jbdevstudio\runtimes\jboss-eap\standalone\tmp\vfs\deployment\deploymentacd29d9b237a97d3\hacep-rules-1.0-SNAPSHOT.jar-15c380aad66e9340\hacep-rules-1.0-SNAPSHOT.jar]
08:37:12,981 ERROR [org.jboss.msc.service.fail](MSC service thread 1-8) MSC000001: Failed to start service jboss.deployment.unit."hacep-eap-playground-1.0-SNAPSHOT.war".WeldStartService: org.jboss.msc.service.StartException in service jboss.deployment.unit."hacep-eap-playground-1.0-SNAPSHOT.war".WeldStartService: Failed to start service
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1936) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_73]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_73]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_73]
Caused by: org.jboss.weld.exceptions.DeploymentException: WELD-001408 Unsatisfied dependencies for type [Cache<String, Object>] with qualifiers [@HACEPSessionCache] at injection point [[field] @Inject @HACEPSessionCache private it.redhat.hacep.cache.session.KieSessionSaver.sessionCache]
at org.jboss.weld.bootstrap.Validator.validateInjectionPoint(Validator.java:315)
at org.jboss.weld.bootstrap.Validator.validateInjectionPoint(Validator.java:284)
at org.jboss.weld.bootstrap.Validator.validateBean(Validator.java:147)
at org.jboss.weld.bootstrap.Validator.validateRIBean(Validator.java:167)
at org.jboss.weld.bootstrap.Validator.validateBeans(Validator.java:386)
at org.jboss.weld.bootstrap.Validator.validateDeployment(Validator.java:371)
at org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:379)
at org.jboss.as.weld.WeldStartService.start(WeldStartService.java:65)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
... 3 more

• there is a single fire() when events are applied back from the buffer in the current code
• doing a fire for each event could be time consuming (could be an option)
• define what rules can and cannot do in HACEP so users can decide

write some unit tests

To minimise event traffic, as by Galder suggestion:

https://docs.jboss.org/infinispan/8.2/apidocs/org/infinispan/filter/Converter.html

See PR #40

on session cache

currently only ERRORS rollback, WARNs don't

...to get rid of finalizers

See PR #40
Must be done after closing Issue #38

in UpdateVersionListener, verify:

• PRE doesn't carry correct new-value (is it a JDG bug?), so it's using POST at the moment
• PRE/POST huge duration (in same thread) can influence JGroups timeout?

Verify that suspending and restarting the camel route is enough to rebalance the JMS grouping

Message ordering is important, must be documented in the code and READMEs

At the moment HACEP supports only fireAllRules() approach.

Support fireUntilHalt() is needs some refactoring and closing of #15

Make AMQ grouping aware of JDG topology to minimise network hops