orbit / orbit Goto Github PK

View Code? Open in Web Editor NEW

1.7K 1.7K 173.0 6.54 MB

Orbit - Virtual actor framework for building distributed systems

License: BSD 3-Clause "New" or "Revised" License

Kotlin 98.24% Dockerfile 0.15% Shell 1.14% Starlark 0.13% Mustache 0.35%

actor-model actors distributed-actors distributed-systems grains java jvm kotlin orbit reactive virtual-actors

orbit's Introduction

Full Documentation

See the documentation website for full documentation, examples and other information.

Orbit 1

Looking for Orbit 1? Visit the orbit1 legacy project.

Developer & License

This project was developed by Electronic Arts and is licensed under the BSD 3-Clause License.

orbit's People

Contributors

Stargazers

Watchers

Forkers

johnou dgu123 leonhartley danielsperry daphantom yongchristophertang danielegasbarro claudiouzelac herman-rogers yanlinaung pakoito respu ifuyivara robertroeser sakesushibig based2 sirusdv afthill sukesh-alluri jleechan voronaam mfioccop duran-zhu corbindallas sarvex ohimors monolithic runc xinqianli biddyweb pazaan zhangf911 gabikliot crudbug tml renokun azogheb jthelin horzelski akydd fordream owenborstad kakamessi99 jedmond victorzhangl ligzy fiurgeist dominicgunn sayi21cn e1n wjtxt dechanaud oceanicwang turingworks maybemars theburgerfan suxinde2009 dejan-stankovic dhtuananh91 wgcells-sea alan-a-a amaranthlis samskivert jpennell vpothnis thaolt ramadam ckavi maniacs-sfa cgander shawntuatara oleksandr-baliev sangkyunyoon fish-potato thaylongs akkinenirajesh zuanoc crisperson insightsdev-dev guogongjun alainlompo ceckhardt mkram hfye l1b3rty vanjoeyg iuliandumitru alexj212 battegoal pologood lannerate myaniu armsnyder kaliwdsn gustavocoding openube mandarenmanman hyqgod oeohomos nixworks

orbit's Issues

Frozen class (cannot edit) when invoking actor from timer

2015-07-28 17:42:22,684 WARN ForkJoinPool-1-worker-22 Error calling timer
com.ea.orbit.exception.UncheckedException: java.lang.RuntimeException: SpaceActorFactory$SpaceActorReference: frozen class (cannot edit)
at com.ea.orbit.actors.runtime.ActorFactoryGenerator.getFactoryFor(ActorFactoryGenerator.java:111)
at com.ea.orbit.actors.runtime.ReferenceFactory.getFactory(ReferenceFactory.java:84)
at com.ea.orbit.actors.runtime.ReferenceFactory.getReference(ReferenceFactory.java:50)
at com.ea.orbit.actors.runtime.ReferenceFactory.ref(ReferenceFactory.java:98)
at com.ea.orbit.actors.Actor.getReference(Actor.java:74)
at SpaceActor.objectReapingTask(SpaceActor.java:103)
at SpaceActor$$Lambda$50/1533338994.call(Unknown Source)
at com.ea.orbit.actors.runtime.Execution$1.lambda$run$20(Execution.java:652)
at com.ea.orbit.actors.runtime.Execution$1$$Lambda$63/1095331087.get(Unknown Source)
at com.ea.orbit.actors.runtime.ExecutionSerializer$Runner.run(ExecutionSerializer.java:79)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.RuntimeException: SpaceActorFactory$SpaceActorReference: frozen class (cannot edit)
at com.ea.orbit.actors.core.shaded.javassist.ClassPool.checkNotFrozen(ClassPool.java:615)
at com.ea.orbit.actors.core.shaded.javassist.ClassPool.makeClass(ClassPool.java:809)
at com.ea.orbit.actors.core.shaded.javassist.ClassPool.makeClass(ClassPool.java:786)
at com.ea.orbit.actors.runtime.ActorFactoryGenerator.makeReferenceClass(ActorFactoryGenerator.java:135)
at com.ea.orbit.actors.runtime.ActorFactoryGenerator.getFactoryFor(ActorFactoryGenerator.java:99)
... 14 more

Stop Orbit clustering

Hi All,

Back with a question again. Thanks for the prompt answers on my other questions between.
If I want orbit to stop discovering other instances what should I do/

Lets say I want to create 3 instances of Orbit , but I dont want them to auto discover each other, what should I do?

Very Long Actor Name Issue

When using a very long actor name, the actor freezes in join().
When the actor is called alone, works ok, but when is called in conjunction with other actor methods, it freezes.

I have a example with the helloworld here:
https://github.com/quilombodigital/orbit/tree/longnameissue

Exception when using JpaStorageProvider

Hi,
I switched my little test project to MySQL and get now the following exception

com.ea.orbit.exception.UncheckedException: java.lang.IllegalArgumentException: An exception occurred while creating a query in EntityManager:
Exception Description: Problem compiling [select s from State s where s.stateId=:stateId].

coming from

c.ea.orbit.actors.runtime.Execution - Error reading actor state for: com.ea.orbit.actors.runtime.ReminderControllerFactory$ReminderControllerReference

because the ReminderController has a state as inner class (and no annotation), and that is not supported by the implementation of JpaStorageProvider according to your documentation.
Not sure if there are other Orbit internal actors with state, but that's the one I found.

Streams batch subscribing and unsubscribing.

Instead of iterating over hundreds of items and generating lots of messages / write states to the storage provider it would be nice to be able to batch subscribe and unsubscribe from streams.

Collaboration

Hi folks, your new project looks great and possibly like something I (at Netflix) could benefit from. We've been exploring Project Orleans and hadn't found something in Java and were considering our own implementation. We need an AP (as in CAP theorem) system that can provide stateful pub/sub with 100M+ topics/queues/actors and Orbit looks like it may fit the bill.

Can you provide information on your roadmap? If we have items we'd like to suggest changes for to enable adapting to our environment are you open to discussing them as issues?

If we find this fits our needs we are interested in different persistence providers (beyond Mongo) and discovery mechanisms (not JGroups) for example.

fast-serialization cloner

Might be worth benchmarking fast-serialization vs Kyro.

https://github.com/RuedigerMoeller/fast-serialization

nb.

RuedigerMoeller/fast-serialization#26 (comment)

Is there a way overcome chain blocking ?

Hi,
im calling actor1.m1 method which calls actor2.m2 method (example codes below). actor1 also being blocked until actor2 task is completed. Is there a way return actor2.m2 task and free actor1 from lock ?

interface Actor2 extends Actor {
    //long blocking task
     public Task<String> m2();
}

class Actor1Impl extends AbstracActor implements Actor1 {
     Actor2 actor2 = .... 
    // load actor2 etc  
    ....
    public Task<String> m1(){
      // doing my internal tasks
       ...
       // last task will be complated by actor2 and have no more internal job to do in this (act1) actor
       return Task.from(actor2.m2())
    }
    
}

also tried
return Task.supplyAsync(actor2.m2());
return Task.fromValue(actor2.m2()); // which causes not serializable exception
....

Thanks.

Code Formatter

Please make available the intellij and eclipse code formatter files.
Another option would be to use a maven formatter plugin to format automatically.
I tested the maven-java-formatter (https://github.com/mblanchette/maven-java-formatter-plugin), but I did not liked the formatter config file to be replicated accross all projects.

[]s, Ricardo

Observer method may be invoked from JGroups thread

Blocking IO method invoked from observer may disrupt JGroups..

2017-02-13T11:44:54,076 INFO  [Incoming-4,xxx,xxx] (xxx) gotcha []
java.lang.RuntimeException: null
	at xx
	at SimpleLocalEventDispatcher.publishEvent(SimpleLocalEventDispatcher.java:18) ~[classes/:?]
	at SimpleLocalEventDispatcher$ObjectInvoker.invoke(SimpleLocalEventDispatcher$ObjectInvoker.java) ~[classes/:?]
	at cloud.orbit.actors.runtime.ObjectInvoker.safeInvoke(ObjectInvoker.java:62) ~[orbit-core-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.InvocationHandler.invoke(InvocationHandler.java:155) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Execution.performInvocation(Execution.java:167) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Execution.lambda$null$0(Execution.java:90) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.ObserverEntry.run(ObserverEntry.java:65) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Execution.lambda$onInvocation$1(Execution.java:89) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.InternalUtils.safeInvoke(InternalUtils.java:192) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Execution.onInvocation(Execution.java:89) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Execution.onRead(Execution.java:74) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Hosting.onInvocation(Hosting.java:610) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Hosting.onRead(Hosting.java:595) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Messaging.onMessageReceived(Messaging.java:195) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Messaging.onRead(Messaging.java:150) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.extensions.PipelineExtension.onRead(PipelineExtension.java:93) ~[orbit-core-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.SerializationHandler.onRead(SerializationHandler.java:160) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.fireRead(DefaultHandlerContext.java:118) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.ClusterHandler.lambda$connect$0(ClusterHandler.java:70) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.cluster.JGroupsClusterPeer.doReceive(JGroupsClusterPeer.java:310) [orbit-infinispan-cluster-0.9.12.jar:?]
	at cloud.orbit.actors.cluster.JGroupsClusterPeer$1$1.receive(JGroupsClusterPeer.java:174) [orbit-infinispan-cluster-0.9.12.jar:?]
	at org.jgroups.JChannel.up(JChannel.java:768) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:157) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.stack.Protocol.up(Protocol.java:426) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.FORK.up(FORK.java:161) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.FRAG2.up(FRAG2.java:182) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.FlowControl.up(FlowControl.java:438) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.stack.Protocol.up(Protocol.java:426) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:294) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.UNICAST3.deliverBatch(UNICAST3.java:1100) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.UNICAST3.removeAndDeliver(UNICAST3.java:899) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:803) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:649) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.BARRIER.up(BARRIER.java:152) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:200) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:325) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.MERGE3.up(MERGE3.java:292) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.Discovery.up(Discovery.java:296) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.TP.passMessageUp(TP.java:1657) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1873) [jgroups-3.6.11.Final.jar:3.6.11.Final]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]

Default parallelism too high

By default Messaging and Execution will create their own ForkJoinPool with parallelism set to 1000, meaning that on a 64bit JRE (default thread stack size is 1 meg?) the JVM may use up to 2G of heap.

Orbit vs Akka

Hi,
I know that this is not the best place to ask such a questions but I didn't find a better one. Orbit seems to be a very interesting project. But I wonder why you created new library? I think Akka is already standard. So why Orbit?
What are pros (cons?) Orbit over Akka? Or maybe they address different problems?
I think such a information should be embedded into wiki/docs. I'm pretty sure that most of the people who first time see orbit already know something about Akka.

NPE sending message

Issue with shutdown sequence?

2015-05-05 10:17:46,085 ERROR ForkJoinPool-3-worker-299 Error locating server for X
java.util.concurrent.CompletionException: com.ea.orbit.exception.UncheckedException: java.lang.NullPointerException
at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1926)
at com.ea.orbit.actors.runtime.Hosting.selectNode(Hosting.java:331)
at com.ea.orbit.actors.runtime.Hosting.lambda$locateAndActivateActor$31(Hosting.java:264)
at com.ea.orbit.actors.runtime.Hosting$$Lambda$27/1732246598.get(Unknown Source)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1582)
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1574)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1689)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: com.ea.orbit.exception.UncheckedException: java.lang.NullPointerException
at com.ea.orbit.actors.cluster.ClusterPeer.sendMessage(ClusterPeer.java:282)
at com.ea.orbit.actors.runtime.Messaging.sendMessage(Messaging.java:377)
at com.ea.orbit.actors.runtime.Execution.sendMessage(Execution.java:1032)
at com.ea.orbit.actors.runtime.Execution.invoke(Execution.java:1040)
at com.ea.orbit.actors.runtime.ActorReference.invoke(ActorReference.java:101)
at com.ea.orbit.actors.runtime.HostingFactory$HostingReference.canActivate(HostingFactory$HostingReference.java)
... 9 more
Caused by: java.lang.NullPointerException
at com.ea.orbit.actors.cluster.ClusterPeer.sendMessage(ClusterPeer.java:278)
... 14 more

Orbit stage timer keeps JVM alive after failed startup

If you create a stage, then go off to start other services (like Jetty), but those fail, you want to cleanly shut down the JVM, but Orbit is sticking a fork in that process.

Stage declares a Timer and creates it in the implicit constructor:

    private Timer timer = new Timer("Orbit stage timer");

This causes a non-daemon thread to be started. So as soon as you "new Stage()" you are in a dangerous limbo state. You can't call Stage.shutdown() because the Stage isn't started yet, but you can't quit your JVM because there are non-daemon threads.

It would be ideal if Stage didn't create the timer until it was started. I'd submit this with a pull request, but I don't know how you want to handle that lazy initialization, so I'll leave this (hopefully simple) fix up to your discretion.

DefaultClassDictionary memory usage

In most scenarios cloud.orbit.actors.runtime.DefaultClassDictionary#getHashCodeToClassName is an OOM waiting to happen, let me know if you have any objections to removing it.

Ideally I would like to explore another class finder that simply scans for actor interfaces and implementations at build time instead of runtime @quilombodigital had some ideas on that.. :)

Reflection invoked every time invoke is called in Execution

As there is a large performance penalty with the use of reflection we should consider creating a singleton cache for annotation lookups.. electronicarts/orbit@0ccba49#diff-c8aa1f92453a884bb06ce94562693a7fR1212

Request Actor Deactivation

Support a new API that allows users to request early deactivation of actors.
This is useful for optimization purposes.

Timers violate actor thread safe guarantee

Actor task methods and actor timer methods run in parallel, is this a feature or bug?

Orbit Web

Thanks for building a great framework.
I was using the orbit -chat sample. Now the sample says that we can start only one server in one machine because of port limitations. Now if I start two servers in two machines will they auto discover each other? How does failover work? It will be nice if you can simply help me know how are things like HA configured.

Lastly can we have a better doc for Orbit-Web stating its functionalities. The orbit doc for other components is fantastic, but a bit more of web will be better.

Question : How to optimize Actor's read-write storage?

I'm currently working on a game.

I have the in-game item inventory use case which requires frequency read-write.

StorageExtension writes actor's state as one item/document on a database which consumes a lot of I/O if we consider the size of player's inventory.

So, my plan is writing custom read-write which put each in-game item as item/document on the separate table.

class PlayerActor extends AbstractActor<PlayerActor.State> implement Player
{
      public static class State
      {
            /** player's other information
            .
            .**/

           @JsonIgnore
           public PlayerInventory playerInventory;
      }

      @Override
      public Task<Void> writeState()
      {
            MyDBExtension mydbExtension = runtime.getFirstExtension(MyDBExtension.class);
            await(state().playerInventory.write(mydbExtension));
            return super.writeState();
      }

      @Override
      public Task<Boolean> readState()
      {
           State playerState = state();
           if(playerState.playerInventory == null)
           {
               playerState.playerInventory = new PlayerInventory("TableName", getIdentiy());
           }
           
           MyDBExtension mydbExtension = runtime.getFirstExtension(MyDB);
           await(playerState.playerInventory.read(mydbExtension));
           return super.readState();
      }
}

Here, My optimization questions.

Should I lock my custom read-write to make this thread-safe? I see comment in the source code that write/readState() is Asynchronous.
Is this the recommended pattern?

Here, My enhancement question.

If Orbit designed interface to handle custom read-write and conversion between old State, would this be a good idea?

What is best way to get Actor self reference ?

How can i self invoke Actor internal method ?
for example in Akka i can invoke ;
getSelf().tell(new MyPrivateMessage(),getSelf())

Question about using Servlets

Hi All,
I had a simple query. Couldn't locate the docs (Sorry if it is there and I skipped it).
Can we use Jetty Servelts with Orbit ? Much liek we can use Websockets.

Thanks

Query about Orbit rpc performance

Hello, I wanted to check the RPC call performance of Orbit. I saw that for things like a hello world, it takes 1 MS to make a request-reply. However other Actor frameworks does it in significantly less.

I want to know is it possible to configure Orbit to use things like Netty? Also Lets say I am in JVM 1 (in a 2 JVM cluster) and I am trying to find teh actor "hello", so if I make multiple calls to "hello", can I request somethings like Co-location. I think a RPC call in this scenario will be wasteful, may be local calls will suffice.

I am sure with time this optimizations will be done in Orbit, I myself was thinking of storing the orbit actor states in Infinsipan grid. It will be nice if you can clarify on these.

Ability to return delayed Task / CompletableFuture

Could you tell, is there a more simple way to create and return a delayed Task? Without such a construction:

CompletableFuture<String> future = new CompletableFuture<>();
registerTimer(() -> {
    future.complete("bingo!");
    return Task.done();
}, 1000, 0, TimeUnit.MILLISECONDS);
return Task.fromFuture(future);

Have I something missed?
Thanks!

Ability to restrict activations per stage / container?

Congrats on releasing!

I would like to be able to restrict activations of certain actors on a per stage / container basis without having to rely on multiple processes.

Eg, consider the scenario of a memory-usage heavy actor on a machine that in theory could only support 10 of.

I thought about doing this by creating multiple stages and restricting the sizes of the execution pools. However, actor auto discovery seems to get in the way here.

In the AsymmetricalStagesTest something similar is achieved through reflection by setting a custom Execution on the private execution field but that seems hack-y.

I can see that I could set the orbit.actors.autoDiscovery property but there is still no access the Execution instance to call the addActorClasses method.

Thoughts?

[Query] Implementation Model

Hello,

I went through the Orbit tutorials and the Orleans paper. I understand JGroups & Infinispan are used to manage distributed state. I have couple of questions:

How are virtual actors implemented on JVM ? using green thread like concept ? bytecode manipulation ?
Is there a concept of supervisor actor ? A single actor can have multiple children, for example, multiple requests on the same service URL will spawn multiple actors having different IDs.
How do you manage Actors ? Is there a management UI to control cluster - JMX ?

Thanks

NPE for actor in default package

If an actor is in the default package, NPE is thrown.

Task<T> vs Future<T>

Hi Joe and team. Congratulations on your OSS release of the Orbit project.

It is good to see more people understanding the importance of the actor paradigm for building scalable distributed systems 😄

I notice that Orbit Actor interfaces are defined using return type Task<T> to represent async promises.
Could you provide some more insight into why you found that the Future<T> / CompletableFuture<T> types from core Java was not directly suitable for that purpose?

There is a little info in your docs, but I was just interested to understand the deeper design rational from your experiences.
http://orbit.bioware.com/orbit-actor-concept-tasks.html

Best wishes for your project's success, from a C# near-neighbor 😉

Jorgen

ScheduledExecutorService in Task prevents clean JVM shutdown

If failAfter is ever used in Task non-daemon threads are created for schedulerExecutor which means the JVM will hang on shutdown. This can be reproduced by running cloud.orbit.actors.test.SingleNodeBenchmark.

JVM segfault when running tests

When running for example com.ea.orbit.actors.test.DeactivationTest from within IntelliJ 15 the JVM segfaults.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000727362728, pid=13152, tid=8736
#
# JRE version: Java(TM) SE Runtime Environment (8.0_25-b18) (build 1.8.0_25-b18)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.25-b02 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  0x0000000727362728
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x00000000292f6800):  JavaThread "ForkJoinPool-1-worker-16" daemon [_thread_in_Java, id=8736, stack(0x000000003b6e0000,0x000000003b7e0000)]

siginfo: ExceptionCode=0xc0000005, ExceptionInformation=0x0000000000000008 0x0000000727362728

Registers:
RAX=0xfffff568cf787ec0, RBX=0x0000000727362728, RCX=0x0000000000000000, RDX=0xfffff568cf787ebf
RSP=0x000000003b7dee20, RBP=0x000000003b7deee8, RSI=0x000000000390fefa, RDI=0x000000000ede5000
R8 =0x000000072815a250, R9 =0x000000000000005f, R10=0x00000000e4e6c4e5, R11=0xfe624e2852d72230
R12=0x0000000000000000, R13=0x000000003b7dede0, R14=0x000000003b7def40, R15=0x00000000292f6800
RIP=0x0000000727362728, EFLAGS=0x0000000000010206

Top of Stack: (sp=0x000000003b7dee20)
0x000000003b7dee20:   000000072815a230 0000000003034afc
0x000000003b7dee30:   ffffffffffffffff 000000002a4e25b0
0x000000003b7dee40:   0000000721fdf448 0000000721fdf538
0x000000003b7dee50:   0000000721fdf518 0000000721fdf500
0x000000003b7dee60:   000000003b7deee8 0000000002837958
0x000000003b7dee70:   000000003b7deee8 0000000002837958
0x000000003b7dee80:   000000072815a230 0000000727362728
0x000000003b7dee90:   0000000721fdf538 0000000000000001
0x000000003b7deea0:   0000000000000000 000000003b7dee98
0x000000003b7deeb0:   000000002a5003d4 000000003b7def40
0x000000003b7deec0:   000000002c7b0c28 000000002ed67bb0
0x000000003b7deed0:   000000002a5005b0 000000003b7dee80
0x000000003b7deee0:   000000003b7def30 000000003b7def98
0x000000003b7deef0:   00000000028377b0 0000000000000000
0x000000003b7def00:   0000000721e3c660 00000007170e9878
0x000000003b7def10:   0000000721df2278 0000015153e2b28d 

Instructions: (pc=0x0000000727362728)
0x0000000727362708:   01 00 00 00 e5 c4 e6 e4 00 00 00 00 c4 c4 e6 e4
0x0000000727362718:   00 00 00 00 00 00 00 00 be 28 e6 e4 00 00 00 00
0x0000000727362728:   21 dc 30 8b 29 00 00 00 da 02 00 f8 e8 c4 e6 e4
0x0000000727362738:   86 cc 68 57 00 00 00 00 21 00 00 00 00 00 00 00 


Register to memory mapping:

RAX=0xfffff568cf787ec0 is an unknown value
RBX=0x0000000727362728 is an oop
java.lang.String 
 - klass: 'java/lang/String'
RCX=0x0000000000000000 is an unknown value
RDX=0xfffff568cf787ebf is an unknown value
RSP=0x000000003b7dee20 is pointing into the stack for thread: 0x00000000292f6800
RBP=0x000000003b7deee8 is pointing into the stack for thread: 0x00000000292f6800
RSI=0x000000000390fefa is an unknown value
RDI=0x000000000ede5000 is an unknown value
R8 =0x000000072815a250 is an oop
java.lang.invoke.MemberName 
 - klass: 'java/lang/invoke/MemberName'
R9 =0x000000000000005f is an unknown value
R10=0x00000000e4e6c4e5 is an unknown value
R11=0xfe624e2852d72230 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x000000003b7dede0 is pointing into the stack for thread: 0x00000000292f6800
R14=0x000000003b7def40 is pointing into the stack for thread: 0x00000000292f6800
R15=0x00000000292f6800 is a thread


Stack: [0x000000003b6e0000,0x000000003b7e0000],  sp=0x000000003b7dee20,  free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x0000000727362728


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x000000002b175000 JavaThread "Orbit stage timer" [_thread_blocked, id=3660, stack(0x000000003de20000,0x000000003df20000)]
  0x000000002b178000 JavaThread "Orbit stage timer" [_thread_blocked, id=14060, stack(0x000000003df40000,0x000000003e040000)]
  0x000000002b177800 JavaThread "Orbit stage timer" [_thread_blocked, id=8340, stack(0x000000003dd20000,0x000000003de20000)]
  0x000000002b176000 JavaThread "Orbit stage timer" [_thread_blocked, id=11636, stack(0x000000003dae0000,0x000000003dbe0000)]
  0x000000002b176800 JavaThread "Orbit stage timer" [_thread_blocked, id=9520, stack(0x000000003d9e0000,0x000000003dae0000)]
  0x000000002b174800 JavaThread "Orbit stage timer" [_thread_blocked, id=15036, stack(0x000000003d8e0000,0x000000003d9e0000)]
  0x000000002b170800 JavaThread "Orbit stage timer" [_thread_blocked, id=8252, stack(0x000000003d7e0000,0x000000003d8e0000)]
  0x000000002b173800 JavaThread "Orbit stage timer" [_thread_blocked, id=3972, stack(0x000000003d6e0000,0x000000003d7e0000)]
  0x000000002b171800 JavaThread "Orbit stage timer" [_thread_blocked, id=14736, stack(0x000000003d5e0000,0x000000003d6e0000)]
  0x000000002b173000 JavaThread "Orbit stage timer" [_thread_blocked, id=8584, stack(0x000000003d4e0000,0x000000003d5e0000)]
  0x000000002b16f000 JavaThread "Orbit stage timer" [_thread_blocked, id=14188, stack(0x000000003d3e0000,0x000000003d4e0000)]
  0x000000002b16f800 JavaThread "Orbit stage timer" [_thread_blocked, id=13980, stack(0x000000003d2e0000,0x000000003d3e0000)]
  0x000000002b172000 JavaThread "Orbit stage timer" [_thread_blocked, id=11324, stack(0x000000003d1e0000,0x000000003d2e0000)]
  0x000000002b16b000 JavaThread "Orbit stage timer" [_thread_blocked, id=7332, stack(0x000000003d0e0000,0x000000003d1e0000)]
  0x000000002b16e000 JavaThread "Orbit stage timer" [_thread_blocked, id=7432, stack(0x000000003cfe0000,0x000000003d0e0000)]
  0x000000002b16d800 JavaThread "Orbit stage timer" [_thread_blocked, id=8284, stack(0x000000003cee0000,0x000000003cfe0000)]
  0x000000002b16c800 JavaThread "Orbit stage timer" [_thread_blocked, id=4856, stack(0x000000003cde0000,0x000000003cee0000)]
  0x000000002b169000 JavaThread "Orbit stage timer" [_thread_blocked, id=14588, stack(0x000000002d850000,0x000000002d950000)]
  0x000000002b16a800 JavaThread "Orbit stage timer" [_thread_blocked, id=2088, stack(0x000000002ebb0000,0x000000002ecb0000)]
  0x000000002b169800 JavaThread "Orbit stage timer" [_thread_blocked, id=12544, stack(0x000000003cce0000,0x000000003cde0000)]
  0x000000002b16c000 JavaThread "Orbit stage timer" [_thread_blocked, id=8344, stack(0x000000003cbe0000,0x000000003cce0000)]
  0x000000002b168000 JavaThread "Orbit stage timer" [_thread_blocked, id=5116, stack(0x000000003cae0000,0x000000003cbe0000)]
  0x000000002b164800 JavaThread "Orbit stage timer" [_thread_blocked, id=13984, stack(0x000000003c9e0000,0x000000003cae0000)]
  0x000000002b167800 JavaThread "Orbit stage timer" [_thread_blocked, id=9248, stack(0x000000003c8e0000,0x000000003c9e0000)]
  0x000000002b166800 JavaThread "Orbit stage timer" [_thread_blocked, id=14184, stack(0x000000003c7e0000,0x000000003c8e0000)]
  0x000000002b163800 JavaThread "Orbit stage timer" [_thread_blocked, id=7600, stack(0x000000003c6e0000,0x000000003c7e0000)]
  0x000000002b166000 JavaThread "Orbit stage timer" [_thread_blocked, id=9260, stack(0x000000003c5e0000,0x000000003c6e0000)]
  0x000000002b165000 JavaThread "Orbit stage timer" [_thread_blocked, id=1384, stack(0x000000003c4e0000,0x000000003c5e0000)]
  0x0000000029300000 JavaThread "ForkJoinPool-1-worker-136" daemon [_thread_blocked, id=7040, stack(0x000000003c3e0000,0x000000003c4e0000)]
  0x00000000292ff000 JavaThread "ForkJoinPool-1-worker-63" daemon [_thread_blocked, id=9896, stack(0x000000003c2e0000,0x000000003c3e0000)]
  0x00000000292fd800 JavaThread "ForkJoinPool-1-worker-177" daemon [_thread_blocked, id=6732, stack(0x000000003c1e0000,0x000000003c2e0000)]
  0x00000000292fe800 JavaThread "ForkJoinPool-1-worker-35" daemon [_thread_blocked, id=6636, stack(0x000000003c0e0000,0x000000003c1e0000)]
  0x00000000292fd000 JavaThread "ForkJoinPool-1-worker-78" daemon [_thread_blocked, id=10820, stack(0x000000003bfe0000,0x000000003c0e0000)]
  0x00000000292fc000 JavaThread "ForkJoinPool-1-worker-192" daemon [_thread_blocked, id=7508, stack(0x000000003bee0000,0x000000003bfe0000)]
  0x00000000292f8000 JavaThread "ForkJoinPool-1-worker-50" daemon [_thread_blocked, id=13084, stack(0x000000003bde0000,0x000000003bee0000)]
  0x00000000292fa800 JavaThread "ForkJoinPool-1-worker-164" daemon [_thread_blocked, id=12524, stack(0x000000003bce0000,0x000000003bde0000)]
  0x00000000292fb000 JavaThread "ForkJoinPool-1-worker-207" daemon [_thread_blocked, id=14092, stack(0x000000003bbe0000,0x000000003bce0000)]
  0x00000000292f9800 JavaThread "ForkJoinPool-1-worker-22" daemon [_thread_blocked, id=13896, stack(0x000000003bae0000,0x000000003bbe0000)]
  0x00000000292f9000 JavaThread "ForkJoinPool-1-worker-229" daemon [_thread_blocked, id=10392, stack(0x000000003b9e0000,0x000000003bae0000)]
  0x00000000292f6000 JavaThread "ForkJoinPool-1-worker-158" daemon [_thread_blocked, id=10672, stack(0x000000003b8e0000,0x000000003b9e0000)]
  0x00000000292f7800 JavaThread "ForkJoinPool-1-worker-93" daemon [_thread_blocked, id=12552, stack(0x000000003b7e0000,0x000000003b8e0000)]
=>0x00000000292f6800 JavaThread "ForkJoinPool-1-worker-16" daemon [_thread_in_Java, id=8736, stack(0x000000003b6e0000,0x000000003b7e0000)]
  0x00000000292f2000 JavaThread "ForkJoinPool-1-worker-201" daemon [_thread_blocked, id=8880, stack(0x000000003b5e0000,0x000000003b6e0000)]
  0x00000000292f5000 JavaThread "ForkJoinPool-1-worker-235" daemon [_thread_blocked, id=13600, stack(0x000000003b4e0000,0x000000003b5e0000)]
  0x00000000292f4800 JavaThread "ForkJoinPool-1-worker-121" daemon [_thread_blocked, id=13356, stack(0x000000003b3e0000,0x000000003b4e0000)]
  0x00000000292f3800 JavaThread "ForkJoinPool-1-worker-102" daemon [_thread_blocked, id=6164, stack(0x000000003b2e0000,0x000000003b3e0000)]
  0x00000000292f3000 JavaThread "ForkJoinPool-1-worker-31" daemon [_thread_blocked, id=12916, stack(0x000000003b1e0000,0x000000003b2e0000)]
  0x00000000292ee800 JavaThread "ForkJoinPool-1-worker-59" daemon [_thread_blocked, id=10180, stack(0x000000003b0e0000,0x000000003b1e0000)]
  0x00000000292f1800 JavaThread "ForkJoinPool-1-worker-244" daemon [_thread_blocked, id=11060, stack(0x000000003afe0000,0x000000003b0e0000)]
  0x00000000292ed800 JavaThread "ForkJoinPool-1-worker-7" daemon [_thread_blocked, id=6416, stack(0x000000003aee0000,0x000000003afe0000)]
  0x00000000292f0800 JavaThread "ForkJoinPool-1-worker-149" daemon [_thread_blocked, id=6200, stack(0x000000003ade0000,0x000000003aee0000)]
  0x00000000292f0000 JavaThread "ForkJoinPool-1-worker-220" daemon [_thread_blocked, id=8316, stack(0x000000003ace0000,0x000000003ade0000)]
  0x00000000292ef000 JavaThread "ForkJoinPool-1-worker-117" daemon [_thread_blocked, id=5920, stack(0x000000003abe0000,0x000000003ace0000)]
  0x00000000292ea000 JavaThread "ForkJoinPool-1-worker-106" daemon [_thread_blocked, id=13128, stack(0x000000003aae0000,0x000000003abe0000)]
  0x00000000292ed000 JavaThread "ForkJoinPool-1-worker-248" daemon [_thread_blocked, id=4260, stack(0x000000003a9e0000,0x000000003aae0000)]
  0x00000000292eb800 JavaThread "ForkJoinPool-1-worker-134" daemon [_thread_blocked, id=8016, stack(0x000000003a8e0000,0x000000003a9e0000)]
  0x00000000292ec000 JavaThread "ForkJoinPool-1-worker-205" daemon [_thread_blocked, id=6724, stack(0x000000003a7e0000,0x000000003a8e0000)]
  0x00000000292ea800 JavaThread "ForkJoinPool-1-worker-20" daemon [_thread_blocked, id=13372, stack(0x000000003a6e0000,0x000000003a7e0000)]
  0x00000000292e6000 JavaThread "ForkJoinPool-1-worker-91" daemon [_thread_blocked, id=14936, stack(0x000000003a5e0000,0x000000003a6e0000)]
  0x00000000292e8800 JavaThread "ForkJoinPool-1-worker-162" daemon [_thread_blocked, id=8412, stack(0x000000003a4e0000,0x000000003a5e0000)]
  0x00000000292e7800 JavaThread "ForkJoinPool-1-worker-160" daemon [_thread_blocked, id=7604, stack(0x000000003a3e0000,0x000000003a4e0000)]
  0x00000000292e9000 JavaThread "ForkJoinPool-1-worker-89" daemon [_thread_blocked, id=288, stack(0x000000003a2e0000,0x000000003a3e0000)]
  0x00000000292e4800 JavaThread "ForkJoinPool-1-worker-203" daemon [_thread_blocked, id=8772, stack(0x000000003a1e0000,0x000000003a2e0000)]
  0x00000000292e5800 JavaThread "ForkJoinPool-1-worker-175" daemon [_thread_blocked, id=1348, stack(0x000000003a0e0000,0x000000003a1e0000)]
  0x00000000292e7000 JavaThread "ForkJoinPool-1-worker-33" daemon [_thread_blocked, id=7188, stack(0x0000000039fe0000,0x000000003a0e0000)]
  0x00000000292e1000 JavaThread "ForkJoinPool-1-worker-76" daemon [_thread_blocked, id=12376, stack(0x0000000039ee0000,0x0000000039fe0000)]
  0x00000000292e4000 JavaThread "ForkJoinPool-1-worker-147" daemon [_thread_blocked, id=3616, stack(0x0000000039de0000,0x0000000039ee0000)]
  0x00000000292e2800 JavaThread "ForkJoinPool-1-worker-190" daemon [_thread_blocked, id=10772, stack(0x0000000039ce0000,0x0000000039de0000)]
  0x00000000292e1800 JavaThread "ForkJoinPool-1-worker-119" daemon [_thread_blocked, id=6744, stack(0x0000000039be0000,0x0000000039ce0000)]
  0x00000000292e3000 JavaThread "ForkJoinPool-1-worker-48" daemon [_thread_blocked, id=6680, stack(0x0000000039ae0000,0x0000000039be0000)]
  0x000000002969d800 JavaThread "ForkJoinPool-1-worker-233" daemon [_thread_blocked, id=7124, stack(0x00000000399e0000,0x0000000039ae0000)]
  0x0000000029699800 JavaThread "ForkJoinPool-1-worker-14" daemon [_thread_blocked, id=15192, stack(0x00000000398e0000,0x00000000399e0000)]
  0x000000002969c800 JavaThread "ForkJoinPool-1-worker-5" daemon [_thread_blocked, id=12160, stack(0x00000000397e0000,0x00000000398e0000)]
  0x000000002969c000 JavaThread "ForkJoinPool-1-worker-218" daemon [_thread_blocked, id=9080, stack(0x00000000396e0000,0x00000000397e0000)]
  0x000000002969a800 JavaThread "ForkJoinPool-1-worker-57" daemon [_thread_blocked, id=10648, stack(0x00000000395e0000,0x00000000396e0000)]
  0x000000002969b000 JavaThread "ForkJoinPool-1-worker-104" daemon [_thread_blocked, id=13680, stack(0x00000000394e0000,0x00000000395e0000)]
  0x0000000029696000 JavaThread "ForkJoinPool-1-worker-246" daemon [_thread_blocked, id=13088, stack(0x00000000393e0000,0x00000000394e0000)]
  0x0000000029699000 JavaThread "ForkJoinPool-1-worker-61" daemon [_thread_blocked, id=8492, stack(0x00000000392e0000,0x00000000393e0000)]
  0x0000000029698000 JavaThread "ForkJoinPool-1-worker-132" daemon [_thread_blocked, id=9768, stack(0x00000000391e0000,0x00000000392e0000)]
  0x0000000029697800 JavaThread "ForkJoinPool-1-worker-18" daemon [_thread_blocked, id=11724, stack(0x00000000390e0000,0x00000000391e0000)]
  0x0000000029696800 JavaThread "ForkJoinPool-1-worker-231" daemon [_thread_blocked, id=9388, stack(0x0000000038fe0000,0x00000000390e0000)]
  0x0000000029692000 JavaThread "ForkJoinPool-1-worker-46" daemon [_thread_blocked, id=5660, stack(0x0000000038ee0000,0x0000000038fe0000)]
  0x0000000029695000 JavaThread "ForkJoinPool-1-worker-171" daemon [_thread_blocked, id=12980, stack(0x0000000038de0000,0x0000000038ee0000)]
  0x0000000029694800 JavaThread "ForkJoinPool-1-worker-188" daemon [_thread_blocked, id=7596, stack(0x0000000038ce0000,0x0000000038de0000)]
  0x0000000029693800 JavaThread "ForkJoinPool-1-worker-3" daemon [_thread_blocked, id=6132, stack(0x0000000038be0000,0x0000000038ce0000)]
  0x0000000029693000 JavaThread "ForkJoinPool-1-worker-74" daemon [_thread_blocked, id=12408, stack(0x0000000038ae0000,0x0000000038be0000)]
  0x000000002968e800 JavaThread "ForkJoinPool-1-worker-145" daemon [_thread_blocked, id=10312, stack(0x00000000389e0000,0x0000000038ae0000)]
  0x0000000029691800 JavaThread "ForkJoinPool-1-worker-216" daemon [_thread_blocked, id=9988, stack(0x00000000388e0000,0x00000000389e0000)]
  0x0000000029690800 JavaThread "ForkJoinPool-1-worker-173" daemon [_thread_blocked, id=8140, stack(0x00000000387e0000,0x00000000388e0000)]
  0x000000002968f000 JavaThread "ForkJoinPool-1-worker-130" daemon [_thread_blocked, id=8360, stack(0x00000000386e0000,0x00000000387e0000)]
  0x0000000029690000 JavaThread "ForkJoinPool-1-worker-87" daemon [_thread_blocked, id=14040, stack(0x00000000385e0000,0x00000000386e0000)]
  0x000000002a849800 JavaThread "ForkJoinPool-1-worker-44" daemon [_thread_blocked, id=2492, stack(0x00000000384e0000,0x00000000385e0000)]
  0x000000002a84c800 JavaThread "ForkJoinPool-1-worker-72" daemon [_thread_blocked, id=11248, stack(0x00000000383e0000,0x00000000384e0000)]
  0x000000002a848800 JavaThread "ForkJoinPool-1-worker-115" daemon [_thread_blocked, id=7084, stack(0x00000000382e0000,0x00000000383e0000)]
  0x000000002a84b800 JavaThread "ForkJoinPool-1-worker-186" daemon [_thread_blocked, id=12420, stack(0x00000000381e0000,0x00000000382e0000)]
  0x000000002a848000 JavaThread "ForkJoinPool-1-worker-1" daemon [_thread_blocked, id=9312, stack(0x00000000380e0000,0x00000000381e0000)]
  0x000000002a84b000 JavaThread "ForkJoinPool-1-worker-143" daemon [_thread_blocked, id=14532, stack(0x0000000037fe0000,0x00000000380e0000)]
  0x000000002a84a000 JavaThread "ForkJoinPool-1-worker-98" daemon [_thread_blocked, id=3872, stack(0x0000000037ee0000,0x0000000037fe0000)]
  0x000000002a844000 JavaThread "ForkJoinPool-1-worker-214" daemon [_thread_blocked, id=13616, stack(0x0000000037de0000,0x0000000037ee0000)]
  0x000000002a846800 JavaThread "ForkJoinPool-1-worker-29" daemon [_thread_blocked, id=8568, stack(0x0000000037ce0000,0x0000000037de0000)]
  0x000000002a847000 JavaThread "ForkJoinPool-1-worker-100" daemon [_thread_blocked, id=15272, stack(0x0000000037be0000,0x0000000037ce0000)]
  0x000000002a842800 JavaThread "ForkJoinPool-1-worker-255" daemon [_thread_blocked, id=7832, stack(0x0000000037ae0000,0x0000000037be0000)]
  0x000000002a843800 JavaThread "ForkJoinPool-1-worker-184" daemon [_thread_blocked, id=9196, stack(0x00000000379e0000,0x0000000037ae0000)]
  0x000000002a845800 JavaThread "ForkJoinPool-1-worker-42" daemon [_thread_blocked, id=13740, stack(0x00000000378e0000,0x00000000379e0000)]
  0x000000002a845000 JavaThread "ForkJoinPool-1-worker-242" daemon [_thread_blocked, id=3664, stack(0x00000000377e0000,0x00000000378e0000)]
  0x000000002a83f000 JavaThread "ForkJoinPool-1-worker-156" daemon [_thread_blocked, id=11040, stack(0x00000000376e0000,0x00000000377e0000)]
  0x000000002a842000 JavaThread "ForkJoinPool-1-worker-227" daemon [_thread_blocked, id=14764, stack(0x00000000375e0000,0x00000000376e0000)]
  0x000000002a83e000 JavaThread "ForkJoinPool-1-worker-128" daemon [_thread_blocked, id=12940, stack(0x00000000374e0000,0x00000000375e0000)]
  0x000000002a841000 JavaThread "ForkJoinPool-1-worker-199" daemon [_thread_blocked, id=14524, stack(0x00000000373e0000,0x00000000374e0000)]
  0x000000002a840800 JavaThread "ForkJoinPool-1-worker-85" daemon [_thread_blocked, id=3712, stack(0x00000000372e0000,0x00000000373e0000)]
  0x000000002a83d800 JavaThread "ForkJoinPool-1-worker-55" daemon [_thread_blocked, id=4192, stack(0x00000000371e0000,0x00000000372e0000)]
  0x000000002a83f800 JavaThread "ForkJoinPool-1-worker-126" daemon [_thread_blocked, id=8872, stack(0x00000000370e0000,0x00000000371e0000)]
  0x0000000029f32800 JavaThread "ForkJoinPool-1-worker-12" daemon [_thread_blocked, id=12424, stack(0x0000000036fe0000,0x00000000370e0000)]
  0x0000000029f35800 JavaThread "ForkJoinPool-1-worker-197" daemon [_thread_blocked, id=10912, stack(0x0000000036ee0000,0x0000000036fe0000)]
  0x0000000029f32000 JavaThread "ForkJoinPool-1-worker-113" daemon [_thread_blocked, id=6072, stack(0x0000000036de0000,0x0000000036ee0000)]
  0x0000000029f35000 JavaThread "ForkJoinPool-1-worker-70" daemon [_thread_blocked, id=14996, stack(0x0000000036ce0000,0x0000000036de0000)]
  0x0000000029f34000 JavaThread "ForkJoinPool-1-worker-141" daemon [_thread_blocked, id=9188, stack(0x0000000036be0000,0x0000000036ce0000)]
  0x0000000029f30800 JavaThread "ForkJoinPool-1-worker-212" daemon [_thread_blocked, id=9780, stack(0x0000000036ae0000,0x0000000036be0000)]
  0x0000000029f31000 JavaThread "ForkJoinPool-1-worker-240" daemon [_thread_blocked, id=8048, stack(0x00000000369e0000,0x0000000036ae0000)]
  0x0000000029f33800 JavaThread "ForkJoinPool-1-worker-169" daemon [_thread_blocked, id=1392, stack(0x00000000368e0000,0x00000000369e0000)]
  0x0000000029f2f800 JavaThread "ForkJoinPool-1-worker-27" daemon [_thread_blocked, id=8904, stack(0x00000000367e0000,0x00000000368e0000)]
  0x0000000029f2c000 JavaThread "ForkJoinPool-1-worker-253" daemon [_thread_blocked, id=6068, stack(0x00000000366e0000,0x00000000367e0000)]
  0x0000000029f2f000 JavaThread "ForkJoinPool-1-worker-225" daemon [_thread_blocked, id=7456, stack(0x00000000365e0000,0x00000000366e0000)]
  0x0000000029f2b000 JavaThread "ForkJoinPool-1-worker-154" daemon [_thread_blocked, id=13220, stack(0x00000000364e0000,0x00000000365e0000)]
  0x0000000029f2d800 JavaThread "ForkJoinPool-1-worker-53" daemon [_thread_blocked, id=10112, stack(0x00000000363e0000,0x00000000364e0000)]
  0x0000000029f2e000 JavaThread "ForkJoinPool-1-worker-83" daemon [_thread_blocked, id=13848, stack(0x00000000362e0000,0x00000000363e0000)]
  0x0000000029f2c800 JavaThread "ForkJoinPool-1-worker-167" daemon [_thread_blocked, id=13432, stack(0x00000000361e0000,0x00000000362e0000)]
  0x0000000029f27800 JavaThread "ForkJoinPool-1-worker-40" daemon [_thread_blocked, id=14656, stack(0x00000000360e0000,0x00000000361e0000)]
  0x0000000029f2a800 JavaThread "ForkJoinPool-1-worker-111" daemon [_thread_blocked, id=11468, stack(0x0000000035fe0000,0x00000000360e0000)]
  0x0000000029f26800 JavaThread "ForkJoinPool-1-worker-182" daemon [_thread_blocked, id=10616, stack(0x0000000035ee0000,0x0000000035fe0000)]
  0x0000000029f29800 JavaThread "ForkJoinPool-1-worker-68" daemon [_thread_blocked, id=6584, stack(0x0000000035de0000,0x0000000035ee0000)]
  0x0000000029f28000 JavaThread "ForkJoinPool-1-worker-23" daemon [_thread_blocked, id=740, stack(0x0000000035ce0000,0x0000000035de0000)]
  0x0000000029f29000 JavaThread "ForkJoinPool-1-worker-208" daemon [_thread_blocked, id=11584, stack(0x0000000035be0000,0x0000000035ce0000)]
  0x0000000029c52800 JavaThread "ForkJoinPool-1-worker-139" daemon [_thread_blocked, id=10156, stack(0x0000000035ae0000,0x0000000035be0000)]
  0x0000000029c52000 JavaThread "ForkJoinPool-1-worker-94" daemon [_thread_blocked, id=12480, stack(0x00000000359e0000,0x0000000035ae0000)]
  0x0000000029c4e000 JavaThread "ForkJoinPool-1-worker-210" daemon [_thread_blocked, id=3316, stack(0x00000000358e0000,0x00000000359e0000)]
  0x0000000029c51000 JavaThread "ForkJoinPool-1-worker-180" daemon [_thread_blocked, id=5412, stack(0x00000000357e0000,0x00000000358e0000)]
  0x0000000029c50800 JavaThread "ForkJoinPool-1-worker-25" daemon [_thread_blocked, id=13724, stack(0x00000000356e0000,0x00000000357e0000)]
  0x0000000029c4f800 JavaThread "ForkJoinPool-1-worker-96" daemon [_thread_blocked, id=11308, stack(0x00000000355e0000,0x00000000356e0000)]
  0x0000000029c4f000 JavaThread "ForkJoinPool-1-worker-38" daemon [_thread_blocked, id=10012, stack(0x00000000354e0000,0x00000000355e0000)]
  0x0000000029c4a800 JavaThread "ForkJoinPool-1-worker-238" daemon [_thread_blocked, id=9104, stack(0x00000000353e0000,0x00000000354e0000)]
  0x0000000029c4d800 JavaThread "ForkJoinPool-1-worker-124" daemon [_thread_blocked, id=5004, stack(0x00000000352e0000,0x00000000353e0000)]
  0x0000000029c4c800 JavaThread "ForkJoinPool-1-worker-195" daemon [_thread_blocked, id=7060, stack(0x00000000351e0000,0x00000000352e0000)]
  0x0000000029c4c000 JavaThread "ForkJoinPool-1-worker-10" daemon [_thread_blocked, id=7500, stack(0x00000000350e0000,0x00000000351e0000)]
  0x0000000029c4b000 JavaThread "ForkJoinPool-1-worker-81" daemon [_thread_blocked, id=9976, stack(0x0000000034fe0000,0x00000000350e0000)]
  0x0000000029c46800 JavaThread "ForkJoinPool-1-worker-152" daemon [_thread_blocked, id=3336, stack(0x0000000034ee0000,0x0000000034fe0000)]
  0x0000000029c49800 JavaThread "ForkJoinPool-1-worker-223" daemon [_thread_blocked, id=3652, stack(0x0000000034de0000,0x0000000034ee0000)]
  0x0000000029c46000 JavaThread "ForkJoinPool-1-worker-109" daemon [_thread_blocked, id=14308, stack(0x0000000034ce0000,0x0000000034de0000)]
  0x0000000029c48000 JavaThread "ForkJoinPool-1-worker-251" daemon [_thread_blocked, id=15220, stack(0x0000000034be0000,0x0000000034ce0000)]
  0x0000000029c49000 JavaThread "ForkJoinPool-1-worker-66" daemon [_thread_blocked, id=5296, stack(0x0000000034ae0000,0x0000000034be0000)]
  0x0000000029c47800 JavaThread "ForkJoinPool-1-worker-137" daemon [_thread_blocked, id=1604, stack(0x00000000349e0000,0x0000000034ae0000)]
  0x0000000029c43800 JavaThread "ForkJoinPool-1-worker-165" daemon [_thread_blocked, id=9056, stack(0x00000000348e0000,0x00000000349e0000)]
  0x0000000029c45000 JavaThread "ForkJoinPool-1-worker-236" daemon [_thread_blocked, id=14912, stack(0x00000000347e0000,0x00000000348e0000)]
  0x0000000029c44800 JavaThread "ForkJoinPool-1-worker-51" daemon [_thread_blocked, id=11884, stack(0x00000000346e0000,0x00000000347e0000)]
  0x0000000029a9e800 JavaThread "ForkJoinPool-1-worker-122" daemon [_thread_blocked, id=12884, stack(0x00000000345e0000,0x00000000346e0000)]
  0x0000000029a9e000 JavaThread "ForkJoinPool-1-worker-193" daemon [_thread_blocked, id=8204, stack(0x00000000344e0000,0x00000000345e0000)]
  0x0000000029a9c800 JavaThread "ForkJoinPool-1-worker-8" daemon [_thread_blocked, id=15196, stack(0x00000000343e0000,0x00000000344e0000)]
  0x0000000029a9d000 JavaThread "ForkJoinPool-1-worker-79" daemon [_thread_blocked, id=11560, stack(0x00000000342e0000,0x00000000343e0000)]
  0x0000000029a99800 JavaThread "ForkJoinPool-1-worker-36" daemon [_thread_blocked, id=9552, stack(0x00000000341e0000,0x00000000342e0000)]
  0x0000000029a98800 JavaThread "ForkJoinPool-1-worker-221" daemon [_thread_blocked, id=8724, stack(0x00000000340e0000,0x00000000341e0000)]
  0x0000000029a9b800 JavaThread "ForkJoinPool-1-worker-150" daemon [_thread_blocked, id=10896, stack(0x0000000033fe0000,0x00000000340e0000)]
  0x0000000029a9a000 JavaThread "ForkJoinPool-1-worker-21" daemon [_thread_blocked, id=5780, stack(0x0000000033ee0000,0x0000000033fe0000)]
  0x0000000029a9b000 JavaThread "ForkJoinPool-1-worker-163" daemon [_thread_blocked, id=224, stack(0x0000000033de0000,0x0000000033ee0000)]
  0x0000000029a95000 JavaThread "ForkJoinPool-1-worker-107" daemon [_thread_blocked, id=6420, stack(0x0000000033ce0000,0x0000000033de0000)]
  0x0000000029a98000 JavaThread "ForkJoinPool-1-worker-249" daemon [_thread_blocked, id=5072, stack(0x0000000033be0000,0x0000000033ce0000)]
  0x0000000029a94000 JavaThread "ForkJoinPool-1-worker-178" daemon [_thread_blocked, id=13844, stack(0x0000000033ae0000,0x0000000033be0000)]
  0x0000000029a95800 JavaThread "ForkJoinPool-1-worker-120" daemon [_thread_blocked, id=13884, stack(0x00000000339e0000,0x0000000033ae0000)]
  0x0000000029a96800 JavaThread "ForkJoinPool-1-worker-64" daemon [_thread_blocked, id=7252, stack(0x00000000338e0000,0x00000000339e0000)]
  0x0000000029a97000 JavaThread "ForkJoinPool-1-worker-135" daemon [_thread_blocked, id=8280, stack(0x00000000337e0000,0x00000000338e0000)]
  0x0000000029a90800 JavaThread "ForkJoinPool-1-worker-206" daemon [_thread_blocked, id=11920, stack(0x00000000336e0000,0x00000000337e0000)]
  0x0000000029a93800 JavaThread "ForkJoinPool-1-worker-92" daemon [_thread_blocked, id=4952, stack(0x00000000335e0000,0x00000000336e0000)]
  0x0000000029a92000 JavaThread "ForkJoinPool-1-worker-77" daemon [_thread_blocked, id=1652, stack(0x00000000334e0000,0x00000000335e0000)]
  0x0000000029a91000 JavaThread "ForkJoinPool-1-worker-234" daemon [_thread_blocked, id=10204, stack(0x00000000333e0000,0x00000000334e0000)]
  0x0000000029a8f800 JavaThread "ForkJoinPool-1-worker-49" daemon [_thread_blocked, id=7748, stack(0x00000000332e0000,0x00000000333e0000)]
  0x0000000029a92800 JavaThread "ForkJoinPool-1-worker-191" daemon [_thread_blocked, id=13632, stack(0x00000000331e0000,0x00000000332e0000)]
  0x000000002b775800 JavaThread "ForkJoinPool-1-worker-6" daemon [_thread_blocked, id=8528, stack(0x00000000330e0000,0x00000000331e0000)]
  0x000000002b775000 JavaThread "ForkJoinPool-1-worker-148" daemon [_thread_blocked, id=10516, stack(0x0000000032fe0000,0x00000000330e0000)]
  0x000000002b774000 JavaThread "ForkJoinPool-1-worker-219" daemon [_thread_blocked, id=14508, stack(0x0000000032ee0000,0x0000000032fe0000)]
  0x000000002b773800 JavaThread "ForkJoinPool-1-worker-34" daemon [_thread_blocked, id=7020, stack(0x0000000032de0000,0x0000000032ee0000)]
  0x000000002b76f800 JavaThread "ForkJoinPool-1-worker-105" daemon [_thread_blocked, id=2436, stack(0x0000000032ce0000,0x0000000032de0000)]
  0x000000002b772800 JavaThread "ForkJoinPool-1-worker-176" daemon [_thread_blocked, id=8364, stack(0x0000000032be0000,0x0000000032ce0000)]
  0x000000002b76f000 JavaThread "ForkJoinPool-1-worker-247" daemon [_thread_blocked, id=10564, stack(0x0000000032ae0000,0x0000000032be0000)]
  0x000000002b772000 JavaThread "ForkJoinPool-1-worker-62" daemon [_thread_blocked, id=14516, stack(0x00000000329e0000,0x0000000032ae0000)]
  0x000000002b76e000 JavaThread "ForkJoinPool-1-worker-133" daemon [_thread_blocked, id=8260, stack(0x00000000328e0000,0x00000000329e0000)]
  0x000000002b771000 JavaThread "ForkJoinPool-1-worker-19" daemon [_thread_blocked, id=14076, stack(0x00000000327e0000,0x00000000328e0000)]
  0x000000002b770800 JavaThread "ForkJoinPool-1-worker-204" daemon [_thread_blocked, id=6092, stack(0x00000000326e0000,0x00000000327e0000)]
  0x000000002b76d800 JavaThread "ForkJoinPool-1-worker-90" daemon [_thread_blocked, id=7088, stack(0x00000000325e0000,0x00000000326e0000)]
  0x000000002b76c800 JavaThread "ForkJoinPool-1-worker-161" daemon [_thread_blocked, id=4048, stack(0x00000000324e0000,0x00000000325e0000)]
  0x000000002b769000 JavaThread "ForkJoinPool-1-worker-232" daemon [_thread_blocked, id=4112, stack(0x00000000323e0000,0x00000000324e0000)]
  0x000000002b76c000 JavaThread "ForkJoinPool-1-worker-47" daemon [_thread_blocked, id=7964, stack(0x00000000322e0000,0x00000000323e0000)]
  0x000000002b76b000 JavaThread "ForkJoinPool-1-worker-118" daemon [_thread_blocked, id=11132, stack(0x00000000321e0000,0x00000000322e0000)]
  0x000000002b76a800 JavaThread "ForkJoinPool-1-worker-189" daemon [_thread_blocked, id=2456, stack(0x00000000320e0000,0x00000000321e0000)]
  0x000000002b766800 JavaThread "ForkJoinPool-1-worker-4" daemon [_thread_blocked, id=12840, stack(0x0000000031fe0000,0x00000000320e0000)]
  0x000000002b767800 JavaThread "ForkJoinPool-1-worker-75" daemon [_thread_blocked, id=12556, stack(0x0000000031de0000,0x0000000031ee0000)]
  0x000000002b769800 JavaThread "ForkJoinPool-1-worker-146" daemon [_thread_blocked, id=6432, stack(0x0000000031be0000,0x0000000031ce0000)]
  0x000000002b768000 JavaThread "ForkJoinPool-1-worker-217" daemon [_thread_blocked, id=13876, stack(0x0000000031ae0000,0x0000000031be0000)]
  0x000000002b19b000 JavaThread "ForkJoinPool-1-worker-32" daemon [_thread_blocked, id=12872, stack(0x00000000319e0000,0x0000000031ae0000)]
  0x000000002b197000 JavaThread "ForkJoinPool-1-worker-103" daemon [_thread_blocked, id=8944, stack(0x00000000318e0000,0x00000000319e0000)]
  0x000000002b19a000 JavaThread "ForkJoinPool-1-worker-174" daemon [_thread_blocked, id=11400, stack(0x00000000317e0000,0x00000000318e0000)]
  0x000000002b199800 JavaThread "ForkJoinPool-1-worker-245" daemon [_thread_blocked, id=8468, stack(0x00000000316e0000,0x00000000317e0000)]
  0x000000002b198800 JavaThread "ForkJoinPool-1-worker-60" daemon [_thread_blocked, id=10732, stack(0x00000000315e0000,0x00000000316e0000)]
  0x000000002b195800 JavaThread "ForkJoinPool-1-worker-131" daemon [_thread_blocked, id=12632, stack(0x00000000314e0000,0x00000000315e0000)]
  0x000000002b196800 JavaThread "ForkJoinPool-1-worker-202" daemon [_thread_blocked, id=7852, stack(0x00000000313e0000,0x00000000314e0000)]
  0x000000002b198000 JavaThread "ForkJoinPool-3-worker-20" daemon [_thread_blocked, id=13520, stack(0x00000000312e0000,0x00000000313e0000)]
  0x000000002b193800 JavaThread "ForkJoinPool-1-worker-17" daemon [_thread_blocked, id=7816, stack(0x00000000311e0000,0x00000000312e0000)]
  0x000000002b194000 JavaThread "ForkJoinPool-3-worker-27" daemon [_thread_blocked, id=3516, stack(0x00000000310e0000,0x00000000311e0000)]
  0x000000002b195000 JavaThread "ForkJoinPool-1-worker-88" daemon [_thread_blocked, id=9200, stack(0x0000000030fe0000,0x00000000310e0000)]
  0x000000002b192800 JavaThread "ForkJoinPool-1-worker-159" daemon [_thread_blocked, id=9612, stack(0x0000000030ee0000,0x0000000030fe0000)]
  0x000000002b192000 JavaThread "ForkJoinPool-3-worker-2" daemon [_thread_blocked, id=5196, stack(0x0000000030de0000,0x0000000030ee0000)]
  0x000000002b191000 JavaThread "ForkJoinPool-1-worker-230" daemon [_thread_blocked, id=9772, stack(0x0000000030ce0000,0x0000000030de0000)]
  0x000000002b18d000 JavaThread "ForkJoinPool-1-worker-45" daemon [_thread_blocked, id=11928, stack(0x0000000030be0000,0x0000000030ce0000)]
  0x000000002b190000 JavaThread "ForkJoinPool-1-worker-116" daemon [_thread_blocked, id=13492, stack(0x0000000030ae0000,0x0000000030be0000)]
  0x000000002b18c800 JavaThread "ForkJoinPool-3-worker-9" daemon [_thread_blocked, id=15296, stack(0x00000000309e0000,0x0000000030ae0000)]
  0x000000002b18f800 JavaThread "ForkJoinPool-3-worker-16" daemon [_thread_blocked, id=13016, stack(0x00000000308e0000,0x00000000309e0000)]
  0x000000002b18b800 JavaThread "ForkJoinPool-1-worker-187" daemon [_thread_blocked, id=14824, stack(0x00000000307e0000,0x00000000308e0000)]
  0x000000002b18e000 JavaThread "ForkJoinPool-3-worker-23" daemon [_thread_blocked, id=6328, stack(0x00000000306e0000,0x00000000307e0000)]
  0x000000002b18e800 JavaThread "ForkJoinPool-1-worker-2" daemon [_thread_blocked, id=7096, stack(0x00000000305e0000,0x00000000306e0000)]
  0x00000000295fa800 JavaThread "ForkJoinPool-1-worker-73" daemon [_thread_blocked, id=9684, stack(0x00000000304e0000,0x00000000305e0000)]
  0x00000000295f9800 JavaThread "ForkJoinPool-3-worker-30" daemon [_thread_blocked, id=14492, stack(0x00000000303e0000,0x00000000304e0000)]
  0x00000000295fb000 JavaThread "ForkJoinPool-1-worker-144" daemon [_thread_blocked, id=9796, stack(0x00000000302e0000,0x00000000303e0000)]
  0x00000000295f9000 JavaThread "ForkJoinPool-3-worker-5" daemon [_thread_blocked, id=11140, stack(0x00000000301e0000,0x00000000302e0000)]
  0x00000000295f5000 JavaThread "ForkJoinPool-3-worker-12" daemon [_thread_blocked, id=9372, stack(0x00000000300e0000,0x00000000301e0000)]
  0x00000000295f8000 JavaThread "ForkJoinPool-3-worker-19" daemon [_thread_blocked, id=2604, stack(0x000000002ffe0000,0x00000000300e0000)]
  0x00000000295f7800 JavaThread "ForkJoinPool-3-worker-26" daemon [_thread_blocked, id=11876, stack(0x000000002fee0000,0x000000002ffe0000)]
  0x00000000295f3800 JavaThread "ForkJoinPool-1-worker-215" daemon [_thread_blocked, id=12716, stack(0x000000002fde0000,0x000000002fee0000)]
  0x00000000295f6800 JavaThread "ForkJoinPool-3-worker-1" daemon [_thread_blocked, id=6884, stack(0x000000002fce0000,0x000000002fde0000)]
  0x00000000295f4800 JavaThread "ForkJoinPool-3-worker-8" daemon [_thread_blocked, id=4316, stack(0x000000002fbe0000,0x000000002fce0000)]
  0x00000000295f6000 JavaThread "ForkJoinPool-1-worker-30" daemon [_thread_blocked, id=1528, stack(0x000000002fae0000,0x000000002fbe0000)]
  0x00000000295f2000 JavaThread "ForkJoinPool-3-worker-15" daemon [_thread_blocked, id=8680, stack(0x000000002f9e0000,0x000000002fae0000)]
  0x00000000295f3000 JavaThread "ForkJoinPool-1-worker-101" daemon [_thread_blocked, id=12200, stack(0x000000002f8e0000,0x000000002f9e0000)]
  0x00000000295f1800 JavaThread "ForkJoinPool-1-worker-172" daemon [_thread_blocked, id=14748, stack(0x000000002f7e0000,0x000000002f8e0000)]
  0x00000000295ee800 JavaThread "ForkJoinPool-1-worker-243" daemon [_thread_blocked, id=13540, stack(0x000000002f6e0000,0x000000002f7e0000)]
  0x00000000295f0800 JavaThread "ForkJoinPool-1-worker-58" daemon [_thread_blocked, id=3220, stack(0x000000002f5e0000,0x000000002f6e0000)]
  0x00000000295ed000 JavaThread "ForkJoinPool-3-worker-22" daemon [_thread_blocked, id=3624, stack(0x000000002f4e0000,0x000000002f5e0000)]
  0x00000000295f0000 JavaThread "ForkJoinPool-1-worker-129" daemon [_thread_blocked, id=14388, stack(0x000000002f3e0000,0x000000002f4e0000)]
  0x00000000295ec000 JavaThread "ForkJoinPool-1-worker-200" daemon [_thread_blocked, id=13796, stack(0x000000002f2e0000,0x000000002f3e0000)]
  0x00000000295ef000 JavaThread "ForkJoinPool-1-worker-15" daemon [_thread_blocked, id=8188, stack(0x000000002f1e0000,0x000000002f2e0000)]
  0x00000000295ed800 JavaThread "ForkJoinPool-1-worker-86" daemon [_thread_blocked, id=14980, stack(0x000000002f0e0000,0x000000002f1e0000)]
  0x0000000028960800 JavaThread "ForkJoinPool-1-worker-157" daemon [_thread_blocked, id=14880, stack(0x000000002efe0000,0x000000002f0e0000)]
  0x0000000028960000 JavaThread "Orbit stage timer" [_thread_blocked, id=12624, stack(0x000000002eee0000,0x000000002efe0000)]
  0x000000002895f000 JavaThread "Orbit stage timer" [_thread_blocked, id=9996, stack(0x000000002eab0000,0x000000002ebb0000)]
  0x0000000028961800 JavaThread "Orbit stage timer" [_thread_blocked, id=12832, stack(0x000000002d9e0000,0x000000002dae0000)]
  0x000000002895b800 JavaThread "ForkJoinPool-3-worker-29" daemon [_thread_blocked, id=7120, stack(0x000000002d750000,0x000000002d850000)]
  0x000000002895d000 JavaThread "ForkJoinPool-3-worker-4" daemon [_thread_blocked, id=14340, stack(0x000000002d650000,0x000000002d750000)]
  0x000000002895e800 JavaThread "Orbit stage timer" [_thread_blocked, id=14296, stack(0x000000002d550000,0x000000002d650000)]
  0x000000002b112000 JavaThread "ForkJoinPool.commonPool-worker-1" daemon [_thread_blocked, id=13936, stack(0x000000002d450000,0x000000002d550000)]
  0x0000000029ad5000 JavaThread "ForkJoinPool-1-worker-228" daemon [_thread_blocked, id=10216, stack(0x000000002d350000,0x000000002d450000)]
  0x000000002b5a9800 JavaThread "ForkJoinPool-1-worker-43" daemon [_thread_blocked, id=8484, stack(0x000000002d250000,0x000000002d350000)]
  0x0000000029b50800 JavaThread "ForkJoinPool-1-worker-114" daemon [_thread_blocked, id=8160, stack(0x000000002d050000,0x000000002d150000)]
  0x0000000029e37800 JavaThread "ForkJoinPool-1-worker-185" daemon [_thread_blocked, id=7428, stack(0x000000002cf50000,0x000000002d050000)]
  0x000000002b3ff800 JavaThread "ForkJoinPool-3-worker-11" daemon [_thread_blocked, id=4364, stack(0x000000002cc50000,0x000000002cd50000)]
  0x000000002aeed800 JavaThread "ForkJoinPool-3-worker-18" daemon [_thread_blocked, id=9304, stack(0x000000002cb50000,0x000000002cc50000)]
  0x000000002a0b7800 JavaThread "ForkJoinPool-3-worker-25" daemon [_thread_blocked, id=10620, stack(0x000000002ca50000,0x000000002cb50000)]
  0x0000000029a55000 JavaThread "Messaging timer" [_thread_blocked, id=14948, stack(0x000000002c950000,0x000000002ca50000)]
  0x000000002b19c000 JavaThread "Orbit stage timer" [_thread_blocked, id=7320, stack(0x000000002c650000,0x000000002c750000)]
  0x00000000278f7800 JavaThread "Monitor Ctrl-Break" daemon [_thread_in_native, id=13640, stack(0x00000000283e0000,0x00000000284e0000)]
  0x0000000027861000 JavaThread "Service Thread" daemon [_thread_blocked, id=14856, stack(0x00000000281a0000,0x00000000282a0000)]
  0x00000000277d8800 JavaThread "C1 CompilerThread3" daemon [_thread_blocked, id=6136, stack(0x00000000280a0000,0x00000000281a0000)]
  0x00000000277cb000 JavaThread "C2 CompilerThread2" daemon [_thread_in_native, id=10284, stack(0x0000000027fa0000,0x00000000280a0000)]
  0x00000000277c7000 JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=11472, stack(0x0000000027ea0000,0x0000000027fa0000)]
  0x00000000277d5000 JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=11964, stack(0x0000000027da0000,0x0000000027ea0000)]
  0x00000000277d4000 JavaThread "Attach Listener" daemon [_thread_blocked, id=9008, stack(0x0000000027ca0000,0x0000000027da0000)]
  0x00000000277d2800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=11752, stack(0x0000000027ba0000,0x0000000027ca0000)]
  0x000000000282a000 JavaThread "Finalizer" daemon [_thread_blocked, id=9956, stack(0x00000000276a0000,0x00000000277a0000)]
  0x0000000002823000 JavaThread "Reference Handler" daemon [_thread_blocked, id=12140, stack(0x00000000275a0000,0x00000000276a0000)]
  0x000000000272e800 JavaThread "main" [_thread_blocked, id=13944, stack(0x0000000002580000,0x0000000002680000)]

Other Threads:
  0x00000000257d8800 VMThread [stack: 0x00000000274a0000,0x00000000275a0000] [id=8636]
  0x0000000027863000 WatcherThread [stack: 0x00000000282a0000,0x00000000283a0000] [id=4068]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap:
 PSYoungGen      total 284672K, used 221107K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 263168K, 75% used [0x0000000715d00000,0x0000000721ff3108,0x0000000725e00000)
  from space 21504K, 99% used [0x0000000727300000,0x00000007287f9bd0,0x0000000728800000)
  to   space 21504K, 0% used [0x0000000725e00000,0x0000000725e00000,0x0000000727300000)
 ParOldGen       total 349696K, used 23344K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 6% used [0x00000005c1600000,0x00000005c2ccc108,0x00000005d6b80000)
 Metaspace       used 18801K, capacity 19496K, committed 19840K, reserved 1067008K
  class space    used 2200K, capacity 2332K, committed 2432K, reserved 1048576K

Card table byte_map: [0x0000000011bf0000,0x0000000012bf0000] byte_map_base: 0x000000000ede5000

Marking Bits: (ParMarkBitMap*) 0x00000000713fd490
 Begin Bits: [0x0000000013ea0000, 0x000000001be48000)
 End Bits:   [0x000000001be48000, 0x0000000023df0000)

Polling page: 0x0000000000d60000

CodeCache: size=245760Kb used=7749Kb max_used=9916Kb free=238010Kb
 bounds [0x0000000002830000, 0x00000000031f0000, 0x0000000011830000]
 total_blobs=3114 nmethods=2711 adapters=315
 compilation: enabled

Compilation events (10 events):
Event: 3.176 Thread 0x00000000277d8800 nmethod 3733 0x0000000002a355d0 code [0x0000000002a35720, 0x0000000002a35890]
Event: 3.176 Thread 0x00000000277d8800 3730       2       com.ea.orbit.actors.runtime.Message::withFromNode (7 bytes)
Event: 3.176 Thread 0x00000000277c7000 3741       4       java.lang.invoke.LambdaForm$DMH/359023572::invokeStatic_L_L (14 bytes)
Event: 3.176 Thread 0x00000000277d8800 nmethod 3730 0x0000000002a37a10 code [0x0000000002a37b60, 0x0000000002a37cd0]
Event: 3.176 Thread 0x00000000277d8800 3731       2       com.ea.orbit.actors.runtime.Message::withHeaders (7 bytes)
Event: 3.176 Thread 0x00000000277d8800 nmethod 3731 0x0000000002a376d0 code [0x0000000002a37820, 0x0000000002a37990]
Event: 3.176 Thread 0x00000000277d8800 3732       2       com.ea.orbit.actors.runtime.Message::withInterfaceId (7 bytes)
Event: 3.176 Thread 0x00000000277d8800 nmethod 3732 0x0000000002a37390 code [0x0000000002a374e0, 0x0000000002a37630]
Event: 3.177 Thread 0x00000000277c7000 nmethod 3741 0x000000000296be50 code [0x000000000296bfa0, 0x000000000296c048]
Event: 3.177 Thread 0x00000000277c7000 3540       4       java.lang.Long::toUnsignedString0 (49 bytes)

GC Heap History (8 events):
Event: 1.664 GC heap before
{Heap before GC invocations=1 (full 0):
 PSYoungGen      total 153088K, used 131584K [0x0000000715d00000, 0x0000000720780000, 0x00000007c0000000)
  eden space 131584K, 100% used [0x0000000715d00000,0x000000071dd80000,0x000000071dd80000)
  from space 21504K, 0% used [0x000000071f280000,0x000000071f280000,0x0000000720780000)
  to   space 21504K, 0% used [0x000000071dd80000,0x000000071dd80000,0x000000071f280000)
 ParOldGen       total 349696K, used 0K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 0% used [0x00000005c1600000,0x00000005c1600000,0x00000005d6b80000)
 Metaspace       used 14582K, capacity 14726K, committed 14848K, reserved 1062912K
  class space    used 1716K, capacity 1783K, committed 1792K, reserved 1048576K
Event: 1.675 GC heap after
Heap after GC invocations=1 (full 0):
 PSYoungGen      total 153088K, used 15820K [0x0000000715d00000, 0x0000000720780000, 0x00000007c0000000)
  eden space 131584K, 0% used [0x0000000715d00000,0x0000000715d00000,0x000000071dd80000)
  from space 21504K, 73% used [0x000000071dd80000,0x000000071ecf3268,0x000000071f280000)
  to   space 21504K, 0% used [0x000000071f280000,0x000000071f280000,0x0000000720780000)
 ParOldGen       total 349696K, used 16K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 0% used [0x00000005c1600000,0x00000005c1604000,0x00000005d6b80000)
 Metaspace       used 14582K, capacity 14726K, committed 14848K, reserved 1062912K
  class space    used 1716K, capacity 1783K, committed 1792K, reserved 1048576K
}
Event: 2.705 GC heap before
{Heap before GC invocations=2 (full 0):
 PSYoungGen      total 153088K, used 147404K [0x0000000715d00000, 0x0000000720780000, 0x00000007c0000000)
  eden space 131584K, 100% used [0x0000000715d00000,0x000000071dd80000,0x000000071dd80000)
  from space 21504K, 73% used [0x000000071dd80000,0x000000071ecf3268,0x000000071f280000)
  to   space 21504K, 0% used [0x000000071f280000,0x000000071f280000,0x0000000720780000)
 ParOldGen       total 349696K, used 16K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 0% used [0x00000005c1600000,0x00000005c1604000,0x00000005d6b80000)
 Metaspace       used 18147K, capacity 18736K, committed 18944K, reserved 1067008K
  class space    used 2179K, capacity 2304K, committed 2304K, reserved 1048576K
Event: 2.724 GC heap after
Heap after GC invocations=2 (full 0):
 PSYoungGen      total 153088K, used 21494K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 131584K, 0% used [0x0000000715d00000,0x0000000715d00000,0x000000071dd80000)
  from space 21504K, 99% used [0x000000071f280000,0x000000072077dab0,0x0000000720780000)
  to   space 21504K, 0% used [0x000000071dd80000,0x000000071dd80000,0x000000071f280000)
 ParOldGen       total 349696K, used 12720K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 3% used [0x00000005c1600000,0x00000005c226c0b8,0x00000005d6b80000)
 Metaspace       used 18147K, capacity 18736K, committed 18944K, reserved 1067008K
  class space    used 2179K, capacity 2304K, committed 2304K, reserved 1048576K
}
Event: 2.887 GC heap before
{Heap before GC invocations=3 (full 0):
 PSYoungGen      total 153088K, used 153078K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 131584K, 100% used [0x0000000715d00000,0x000000071dd80000,0x000000071dd80000)
  from space 21504K, 99% used [0x000000071f280000,0x000000072077dab0,0x0000000720780000)
  to   space 21504K, 0% used [0x000000071dd80000,0x000000071dd80000,0x000000071f280000)
 ParOldGen       total 349696K, used 12720K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 3% used [0x00000005c1600000,0x00000005c226c0b8,0x00000005d6b80000)
 Metaspace       used 18524K, capacity 19186K, committed 19328K, reserved 1067008K
  class space    used 2189K, capacity 2321K, committed 2432K, reserved 1048576K
Event: 2.900 GC heap after
Heap after GC invocations=3 (full 0):
 PSYoungGen      total 153088K, used 21494K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 131584K, 0% used [0x0000000715d00000,0x0000000715d00000,0x000000071dd80000)
  from space 21504K, 99% used [0x000000071dd80000,0x000000071f27dbf0,0x000000071f280000)
  to   space 21504K, 0% used [0x0000000727300000,0x0000000727300000,0x0000000728800000)
 ParOldGen       total 349696K, used 22656K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 6% used [0x00000005c1600000,0x00000005c2c200f8,0x00000005d6b80000)
 Metaspace       used 18524K, capacity 19186K, committed 19328K, reserved 1067008K
  class space    used 2189K, capacity 2321K, committed 2432K, reserved 1048576K
}
Event: 2.993 GC heap before
{Heap before GC invocations=4 (full 0):
 PSYoungGen      total 153088K, used 153078K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 131584K, 100% used [0x0000000715d00000,0x000000071dd80000,0x000000071dd80000)
  from space 21504K, 99% used [0x000000071dd80000,0x000000071f27dbf0,0x000000071f280000)
  to   space 21504K, 0% used [0x0000000727300000,0x0000000727300000,0x0000000728800000)
 ParOldGen       total 349696K, used 22656K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 6% used [0x00000005c1600000,0x00000005c2c200f8,0x00000005d6b80000)
 Metaspace       used 18573K, capacity 19250K, committed 19584K, reserved 1067008K
  class space    used 2189K, capacity 2321K, committed 2432K, reserved 1048576K
Event: 3.004 GC heap after
Heap after GC invocations=4 (full 0):
 PSYoungGen      total 284672K, used 21478K [0x0000000715d00000, 0x0000000728800000, 0x00000007c0000000)
  eden space 263168K, 0% used [0x0000000715d00000,0x0000000715d00000,0x0000000725e00000)
  from space 21504K, 99% used [0x0000000727300000,0x00000007287f9bd0,0x0000000728800000)
  to   space 21504K, 0% used [0x0000000725e00000,0x0000000725e00000,0x0000000727300000)
 ParOldGen       total 349696K, used 23344K [0x00000005c1600000, 0x00000005d6b80000, 0x0000000715d00000)
  object space 349696K, 6% used [0x00000005c1600000,0x00000005c2ccc108,0x00000005d6b80000)
 Metaspace       used 18573K, capacity 19250K, committed 19584K, reserved 1067008K
  class space    used 2189K, capacity 2321K, committed 2432K, reserved 1048576K
}

Deoptimization events (10 events):
Event: 2.538 Thread 0x000000000272e800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x0000000002e15a38 method=java.lang.ThreadLocal.get()Ljava/lang/Object; @ 6
Event: 2.542 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002f5d9bc method=java.util.regex.Matcher.match(II)Z @ 86
Event: 2.543 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002f5d9bc method=java.util.regex.Matcher.match(II)Z @ 86
Event: 2.548 Thread 0x000000002aeed800 Uncommon trap: reason=unreached action=reinterpret pc=0x0000000002da7d54 method=java.util.Date.normalize()Lsun/util/calendar/BaseCalendar$Date; @ 4
Event: 2.553 Thread 0x000000002aeed800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x0000000002e15a38 method=java.lang.ThreadLocal.get()Ljava/lang/Object; @ 6
Event: 2.665 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002ffac9c method=java.util.TimSort.countRunAndMakeAscending([Ljava/lang/Object;IILjava/util/Comparator;)I @ 43
Event: 2.697 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002ffac9c method=java.util.TimSort.countRunAndMakeAscending([Ljava/lang/Object;IILjava/util/Comparator;)I @ 43
Event: 2.702 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002ffac9c method=java.util.TimSort.countRunAndMakeAscending([Ljava/lang/Object;IILjava/util/Comparator;)I @ 43
Event: 2.779 Thread 0x000000000272e800 Uncommon trap: reason=bimorphic action=maybe_recompile pc=0x0000000002e5e9c4 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 203
Event: 2.780 Thread 0x000000000272e800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000002ffac9c method=java.util.TimSort.countRunAndMakeAscending([Ljava/lang/Object;IILjava/util/Comparator;)I @ 43

Internal exceptions (10 events):
Event: 2.701 Thread 0x000000000272e800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071d629308) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 2.701 Thread 0x000000000272e800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071d62f4a8) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 2.702 Thread 0x000000000272e800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071d639af0) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 2.703 Thread 0x000000000272e800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071d643de8) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.050 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad5b6e0) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.050 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad61d00) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.051 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad66898) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.051 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad6b518) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.051 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad70198) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]
Event: 3.066 Thread 0x00000000292f4800 Exception <a 'java/security/PrivilegedActionException'> (0x000000071ad74e18) thrown at [D:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u25\1677\hotspot\src\share\vm\prims\jvm.cpp, line 1312]

Events (10 events):
Event: 3.065 loading class com/google/common/cache/RemovalCause$5 done
Event: 3.068 Thread 0x000000002b177800 Thread added: 0x000000002b177800
Event: 3.086 Thread 0x000000002b178000 Thread added: 0x000000002b178000
Event: 3.120 Thread 0x00000000292fd000 DEOPT PACKING pc=0x0000000002bc385d sp=0x000000003c0df070
Event: 3.121 Thread 0x00000000292fd000 DEOPT UNPACKING pc=0x00000000028775bf sp=0x000000003c0dedf8 mode 0
Event: 3.170 Thread 0x000000002b175000 Thread added: 0x000000002b175000
Event: 3.171 loading class com/ea/orbit/actors/test/actors/SomeActor
Event: 3.171 loading class com/ea/orbit/actors/test/actors/SomeActor done
Event: 3.171 loading class com/ea/orbit/actors/test/actors/SomeActor
Event: 3.171 loading class com/ea/orbit/actors/test/actors/SomeActor done

---------------  S Y S T E M  ---------------

OS: Windows 8.1 , 64 bit Build 9600 

CPU:total 8 (4 cores per cpu, 2 threads per core) family 6 model 60 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2

Memory: 4k page, physical 33459324k(25704220k free), swap 67013756k(56401168k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.25-b02) for windows-amd64 JRE (1.8.0_25-b18), built on Oct  7 2014 14:25:37 by "java_re" with MS VC++ 10.0 (VS2010)

time: Sun Nov 29 16:35:44 2015
elapsed time: 3 seconds (0d 0h 0m 3s)

Documentation points to wrong location for JGroups config

Thanks for this interesting project! I'm just playing around so far to get acquainted with the virtual actor model.

This page says that you should modify classpath:/conf/jgroups.xml but the default seems to be classpath:/conf/udp-jgroups.xml (in JGroupsClusterPeer).

That it's called udp-jgroups.xml makes me wonder if it's supposed to work over TCP. I just tried the Hello world example from the wiki with the configuration below and that worked however (originally found at https://github.com/belaban/JGroups/blob/master/conf/tcp.xml).

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
        <TCP bind_port="7800"
             recv_buf_size="${tcp.recv_buf_size:130k}"
             send_buf_size="${tcp.send_buf_size:130k}"
             max_bundle_size="64K"
             sock_conn_timeout="300"

             thread_pool.min_threads="0"
             thread_pool.max_threads="20"
             thread_pool.keep_alive_time="30000"/>

        <TCPPING async_discovery="true"
                 initial_hosts="${jgroups.tcpping.initial_hosts:localhost[7800],localhost[7801]}"
                 port_range="2"/>
        <MERGE3  min_interval="10000"
                 max_interval="30000"/>
        <FD_SOCK/>
        <FD timeout="3000" max_tries="3" />
        <VERIFY_SUSPECT timeout="1500"  />
        <BARRIER />
        <pbcast.NAKACK2 use_mcast_xmit="false"
                        discard_delivered_msgs="true"/>
        <UNICAST3 />
        <pbcast.STABLE desired_avg_gossip="50000"
                       max_bytes="4M"/>
        <pbcast.GMS print_local_addr="true" join_timeout="2000"/>
        <MFC max_credits="2M"
             min_threshold="0.4"/>
        <FRAG2 frag_size="60K"  />
        <!--RSVP resend_interval="2000" timeout="10000"/-->
        <pbcast.STATE_TRANSFER/>
</config>

ConcurrentModificationException in JavaMessageSerializer.serializeMessage

I get the exception below when sending a lot of messages in a for loop.
I don't have a repo I can easily share but it was something like sending ~20000 msg in ~4 seconds.

Basically the problem is that the serialization code of the standard collections (like HashMap, ArrayList, etc.) is not thread safe and must be handled in the application (either cloning or synchronising)

15:18:46.772 [ForkJoinPool-1-worker-228] ERROR c.e.o.a.runtime.SerializationHandler - Error serializing message
java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) ~[na:1.8.0_65]
    at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_65]
    at java.util.HashSet.writeObject(HashSet.java:286) ~[na:1.8.0_65]
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[na:1.8.0_65]
    at com.ea.orbit.actors.runtime.JavaMessageSerializer.serializeMessage(JavaMessageSerializer.java:62) ~[orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.SerializationHandler.write(SerializationHandler.java:69) ~[orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.MessageLoopback.write(MessageLoopback.java:85) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.writeMessage(Messaging.java:398) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.writeInvocation(Messaging.java:360) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.write(Messaging.java:320) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Hosting.async$writeInvocation(Hosting.java:724) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.concurrent.TaskContext.lambda$wrap$21(TaskContext.java:215) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_65]
    at com.ea.orbit.concurrent.Task.internalComplete(Task.java:103) ~[orbit-commons-0.7.1.jar:na]
    at com.ea.orbit.concurrent.Task.lambda$from$3(Task.java:154) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_65]
    at com.ea.orbit.concurrent.Task.internalComplete(Task.java:103) ~[orbit-commons-0.7.1.jar:na]
    at com.ea.orbit.concurrent.Task.lambda$from$3(Task.java:154) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[na:1.8.0_65]
[...]
15:18:46.776 [ForkJoinPool-1-worker-228] ERROR c.e.o.a.net.DefaultHandlerContext - Error writing: 
com.ea.orbit.exception.UncheckedException: Error serializing message
    at com.ea.orbit.actors.runtime.SerializationHandler.write(SerializationHandler.java:79) ~[orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.MessageLoopback.write(MessageLoopback.java:85) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.writeMessage(Messaging.java:398) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.writeInvocation(Messaging.java:360) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Messaging.write(Messaging.java:320) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:158) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.Hosting.async$writeInvocation(Hosting.java:724) [orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.concurrent.TaskContext.lambda$wrap$21(TaskContext.java:215) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_65]
    at com.ea.orbit.concurrent.Task.internalComplete(Task.java:103) ~[orbit-commons-0.7.1.jar:na]
    at com.ea.orbit.concurrent.Task.lambda$from$3(Task.java:154) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[na:1.8.0_65]
    at com.ea.orbit.concurrent.Task.internalComplete(Task.java:103) ~[orbit-commons-0.7.1.jar:na]
    at com.ea.orbit.concurrent.Task.lambda$from$3(Task.java:154) ~[orbit-commons-0.7.1.jar:na]
    at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595) ~[na:1.8.0_65]
    at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[na:1.8.0_65]
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[na:1.8.0_65]
Caused by: java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) ~[na:1.8.0_65]
    at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_65]
    at java.util.HashSet.writeObject(HashSet.java:286) ~[na:1.8.0_65]
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
    at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174) ~[na:1.8.0_65]
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[na:1.8.0_65]
    at com.ea.orbit.actors.runtime.JavaMessageSerializer.serializeMessage(JavaMessageSerializer.java:62) ~[orbit-actors-stage-0.7.1.jar:na]
    at com.ea.orbit.actors.runtime.SerializationHandler.write(SerializationHandler.java:69) ~[orbit-actors-stage-0.7.1.jar:na]
    ... 30 common frames omitted

Are WebSockets supported?

Are WebSockets supported as a Frontend solution?

I don't see any way for customers to interact with Java as Frontend in most web applications today. Besides that I would never build a web application which uses Java as Frontend since you will have an additional dependency which might not be available to the user.

Any roadmap for orbit

I think orbit is workable but still far from complete or good enough.Is there any roadmao for orbit or WIP ?

WaitFreeExecutionSerializer memory footprint

WaitFreeExecutionSerializer is created for every actor instance, AtomicBoolean and AtomicInteger can be replaced with volatile boolean / int and static AtomicReferenceFieldUpdaters combo to save memory.

Unrecognized Windows Sockets error

Since the author of issue #257 deleted his account and I'm with the same issue. I created other issue and the previous issue should be closed.

Note that all code is the same of this wiki

Setup:

PS C:\Users\petit> gradle -v
------------------------------------------------------------
Gradle 4.0
------------------------------------------------------------

Build time:   2017-06-14 15:11:08 UTC
Revision:     316546a5fcb4e2dfe1d6aa0b73a4e09e8cecb5a5

Groovy:       2.4.11
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_131 (Oracle Corporation 25.131-b11)
OS:           Windows 10 10.0 amd64

build.gradle:

apply plugin: 'java'
apply plugin: 'application'

repositories {
    jcenter()
}

dependencies {
    compile "cloud.orbit:orbit-runtime:1.1.0"
}

mainClassName = "Main"

Output:

PS C:\Users\petit\Documents\Code\orbit-test> ./gradlew run

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Jul 05, 2017 7:43:12 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport start
INFO: ISPN000078: Starting JGroups channel orbit-helloworld-cluster
Jul 05, 2017 7:43:12 PM org.jgroups.stack.DiagnosticsHandler bindToInterfaces
**WARNING: failed to join /224.0.75.75:7500 on net0: java.net.SocketException: Unrecognized Windows Sockets error: 0: no I
net4Address associated with interface**

-------------------------------------------------------------------
GMS: address=orbit-helloworld-cluster, cluster=orbit-helloworld-cluster, physical address=192.168.2.107:57133
-------------------------------------------------------------------
Jul 05, 2017 7:43:14 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport viewAccepted
INFO: ISPN000094: Received new cluster view for channel orbit-helloworld-cluster: [orbit-helloworld-cluster|0] (1) [orbi
t-helloworld-cluster]
Jul 05, 2017 7:43:14 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport startJGroupsChannelIfNeeded
INFO: ISPN000079: Channel orbit-helloworld-cluster local address is orbit-helloworld-cluster, physical addresses are [19
2.168.2.107:57133]
Jul 05, 2017 7:43:14 PM org.infinispan.factories.GlobalComponentRegistry start
INFO: ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.4.Final
Here: Welcome to orbit
You said: 'Welcome to orbit', I say: Hello from 1705436675 !
Jul 05, 2017 7:43:15 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport stop
INFO: ISPN000080: Disconnecting JGroups channel orbit-helloworld-cluster
Jul 05, 2017 7:43:15 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport stop
INFO: ISPN000082: Stopping the RpcDispatcher for channel orbit-helloworld-cluster


BUILD SUCCESSFUL in 5s
2 actionable tasks: 1 executed, 1 up-to-date

Hosting busy loop

Hosting can go into a crazy busy loop (most noticeable when the node is shutdown), there was a recent change that added logging to this method which completely kills the node. During one test it logged about 3k lines.. logging.txt

Storage extensions shut down before execution serializer is shutdown

Storage extensions are shut down before the execution serializer which means storage extensions that override cloud.orbit.lifecycle.Startable#stop and shut down connection pools etc. may cause problems for Actors which invoke writeState() inside deactivateAsync.

task.cancel() has no effect on taks waiting in the queue.

Hi,
i ve two actors Actor1 and Actor2. Actor2 m2 method is a long running method. Im calling Actor2.m2() multiple times from actor1 and adding response task to a list. Sometimes i need to cancel waiting tasks (actor2.m2()) and i call task.cancel() for all tasks i already added list. But actor2 executes tasks i ve cancelled before.
Does it mean , an actor method once called there is nothing i can do to stop it?

interface Actor1 extends Actor {
  public Task<Void> m1();
  public Task<Void> cancel();
}

interface Actor2 extends Actor{
// this is long running task
 public Task<Void> m2(String prm);
}

class Actor1Impl extends AbstractActor implements Actor1 {
   private List<Task> taskCache = new ArrayList();   
   private Actor2 act2;
   public Task<Void> activateAsync(){
        act2 = Actor.getReference(Actor2.class);
   }

   public Task<Void>  m1(){
      taskCache.clear();
      for(int i=0;i<100;i++)
         taskCache.add(act1.m2());
      return Task.done();
   }
   public void cancel() {
       for(Task t:taskCache) {
        t.cancel();
       }
       taskCache.clear();
   }

}

Actor1 methods is being called by JavaFX thread (a start and a cancel button).

Thanks.

Conflicting documentation with respect to timers

The wiki page for timers states:
"Timers do not keep an actor activated and will disappear on deactivation."

However, the method comment for AbstractActor.registerTimer reads:
"The timer is automatically disposed on deactivation. The timer calls will keep the actor active."

Do timers keep an actor active or not?

Registered timer with 0 period value doesn't work properly

As an example, please try something like:

registerTimer(() -> tick(), 1000, 0, TimeUnit.MILLISECONDS);

Unfortunately, there will be no Exception (good old java Threads). A main discrepancy here is that Runtime interface doc said:

...
* @param period       the period of subsequent ticks (if ZERO then will only tick once)
...

But we are unable to register Timer with 0 period value due to this precondition check:

if (period <= 0)
            throw new IllegalArgumentException("Non-positive period.");

We have to use Timer.schedule(TimerTask task, long delay); method instead. I think, scheduling some task just once is very useful. Please, could you provide this ability? I would say, that overloaded registerTimer() will be the best.

"Orbit stage timer" keeps JVM alive after failed startup

If you create a stage, then go off to start other services (like Jetty), but those fail, you want to cleanly shut down the JVM, but Orbit is sticking a fork in that process.

Stage declares a Timer and creates it in the implicit constructor:

    private Timer timer = new Timer("Orbit stage timer");

Error serializing message Object already installed

Exception popped up in production.

2017-01-24 19:53:14,066 ERROR [ForkJoinPool-1-worker-23] (cloud.orbit.actors.runtime.SerializationHandler) Error serializing message
java.lang.IllegalArgumentException: Object already installed: com.sulake.h4k.common.actors.observers.SessionObserver:jUP6WdiZRUFC4FoXdlVWpC
	at cloud.orbit.actors.runtime.LocalObjects.registerLocalObject(LocalObjects.java:235) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.LocalObjects.registerLocalObject(LocalObjects.java:216) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.LocalObjects.getOrAddLocalObjectReference(LocalObjects.java:188) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.Stage.registerObserver(Stage.java:1126) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.BasicRuntime.registerObserver(BasicRuntime.java:82) ~[orbit-core-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.JavaMessageSerializer$1.replaceObject(JavaMessageSerializer.java:125) ~[orbit-runtime-0.9.12.jar:?]
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1144) ~[?:1.8.0_102]
	at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378) ~[?:1.8.0_102]
	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174) ~[?:1.8.0_102]
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_102]
	at cloud.orbit.actors.runtime.JavaMessageSerializer.serializeMessage(JavaMessageSerializer.java:91) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.SerializationHandler.write(SerializationHandler.java:69) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:171) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.MessageLoopback.write(MessageLoopback.java:94) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:171) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Messaging.writeMessage(Messaging.java:396) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Messaging.writeInvocation(Messaging.java:358) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Messaging.write(Messaging.java:318) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.net.DefaultHandlerContext.write(DefaultHandlerContext.java:171) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.actors.runtime.Hosting.async$writeInvocation(Hosting.java:741) ~[orbit-runtime-0.9.12.jar:?]
	at cloud.orbit.concurrent.TaskContext.lambda$wrap$3(TaskContext.java:256) ~[orbit-commons-0.9.12.jar:?]
	at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_102]
	at cloud.orbit.concurrent.Task.internalComplete(Task.java:122) ~[orbit-commons-0.9.12.jar:?]
	at cloud.orbit.concurrent.Task.lambda$from$1(Task.java:173) ~[orbit-commons-0.9.12.jar:?]
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_102]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_102]
	at cloud.orbit.concurrent.Task.internalComplete(Task.java:122) ~[orbit-commons-0.9.12.jar:?]
	at cloud.orbit.concurrent.Task.lambda$from$1(Task.java:173) ~[orbit-commons-0.9.12.jar:?]
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) [?:1.8.0_102]
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) [?:1.8.0_102]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [?:1.8.0_102]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595) [?:1.8.0_102]
	at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) [?:1.8.0_102]
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_102]
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_102]
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_102]
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_102]

Avoid using Infinispan Uberjars

Since this causes several classpath issues like https://developer.jboss.org/thread/275777.

Instead of depending on infinispan-embedded, please use infinispan-core

StatelessWorker Exception

As discussed in Gitter:
see https://github.com/Fiurgeist/OrbitIssue

There you can find a unit test which doesn't make a lot of sense as it is, because I streamlined it to focus on the issue.
It's basically just a for loop which calls a StatelessWorker a 100 times and in that actor another StatelessWorker is called.
And every 10th or so run of the test I get this exception:

Jul 22, 2015 5:06:12 PM com.ea.orbit.concurrent.ExecutorUtils lambda$newScalingThreadPool$0
Uncaught Exception
java.lang.Error: Error loading class: null
    at com.ea.orbit.actors.runtime.Execution.classForName(Execution.java:776)
    at com.ea.orbit.actors.runtime.Execution.classForName(Execution.java:762)
    at com.ea.orbit.actors.runtime.Execution.access$300(Execution.java:93)
    at com.ea.orbit.actors.runtime.Execution$Activation.getOrCreateInstance(Execution.java:432)
    at com.ea.orbit.actors.runtime.Execution.executeMessage(Execution.java:989)
    at com.ea.orbit.actors.runtime.Execution.lambda$handleOnMessageReceived$24(Execution.java:912)
    at com.ea.orbit.actors.runtime.Execution$$Lambda$55/2071701248.get(Unknown Source)
    at com.ea.orbit.actors.runtime.ExecutionSerializer.lambda$offerJob$32(ExecutionSerializer.java:116)
    at com.ea.orbit.actors.runtime.ExecutionSerializer$$Lambda$56/619859981.run(Unknown Source)
    at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(Unknown Source)
    at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
    at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
    at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at com.ea.orbit.actors.runtime.Execution.classForName(Execution.java:770)
    ... 13 more

Issues working with the IDE.

I dont know if something is messed up in my setup, but something is strange.
cloned now, first time build with intellij.
"mvn clean install" with several exceptions. (showing success)
hello test did not worked from IDE, including .bat
chat did not worked from IDE, including .bat
"mvn clean install" more a couple of times.
chat worked using .bat, hello worked using .bat
hello main and hello test still dont work using run at the IDE.

Hum... I guess classpath search is showing the nonsense exceptions:
java.lang.ClassNotFoundException: sun/plugin2/util/ParameterNames
java.lang.ClassNotFoundException: com/sun/deploy/ui/DialogTemplate

The hello test error is:
ERROR com.ea.orbit.actors.runtime.Hosting - Error locating server for com.ea.orbit.samples.hello.IHello

[]s, Ricardo Mello (gandhi)

ps.: by the way the statelessworkertest sometimes fails at my machine with 5 and not 4.

Some errors

Hi folks! I'm using the HelloWorld example (in java) in Windows with gradle. But I'm no successfully, the gradle wrapper is returning these erros

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Jun 22, 2017 8:36:09 PM org.infinispan.remoting.transport.jgroups.JGroupsTransport start INFO: ISPN000078: Starting JGroups channel orbit-helloworld-cluster Jun 22, 2017 8:36:09 PM org.jgroups.stack.DiagnosticsHandler bindToInterfaces WARNING: failed to join /224.0.75.75:7500 on net0: java.net.SocketException: Unrecognized Windows Sockets error: 0: no I net4Address associated with interface=

State Machine in Container: implementation glitch ?

This code:

    catch (Throwable ex)
    {
        throw new UncheckedException(ex);
    }

in method Container.start() leaves the Container state in STARTING. Is that the intention ? Wouldn't we rather want to set the state back to CREATED before throwing the exception ?

Upgrading a cluster (question)

Thanks for the cool framework!

How can I make zero down time upgrades work? Say I have a cluster running happily and I want to introduce a new version of an actor? Ideally I'd add nodes with the new version of the Actor, while removing the old nodes and eventually all activations would be running the new version and no-one will have experienced any downtime.

I can make the actor's state be forward and backward compatible without too much drama, so that should deal with activation and deactivation. But what about changes to the Actor interface or implementation classes?

How do you see this process working?

Thanks again!

Extending ActorProfiler to provide automatic thread dumps

To debug "Missing counterpart (pending message) for message.." it might be desirable to extend the ActorProfile extension allowing for a configurable threshold in which if the execution time is exceeded the service automatically takes a thread dump to show what's blocking.. wdyt?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.