dos-group / aura Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 8.0 3.01 MB

Distributed Execution Engine

License: Apache License 2.0

Java 92.82% Shell 7.18%

aura's People

Contributors

Stargazers

Watchers

Forkers

aalexandrov teots akunft lauritzthamsen kfleischmann tobiasherb elbehery

aura's Issues

Repository conventions: develop branch + WIP pull requests

yesterday @TobiasHerb and I discussed how integrate changes more often and how to make ongoing work more visible for all on github.

here two ideas:

first idea: use a develop branch and roughly follow the gitflow workflow. this way, we have a dedicated branch to exchange changes that should eventually land in master but haven't been tested on the cluster before, while the master still only points to versions that have been tested on the cluster.

second idea: use WIP pull requests to describe ongoing work in feature branches. this allows to discuss ongoing work (on the level of code and commits) and makes ongoing efforts more visible.

i created a wiki page for this proposal: Repository Conventions

Connection to ZooKeeper lost on OS X

running the example clients currently fails on OS X with the following output:

2014-07-04 13:47:15,339 |  INFO [main] (LocalClusterSimulator.java:98) - CREATE TMP DIRECTORY: '/var/folders/5g/lk8wz6sd62b63m831_rh3h1w0000gn/T/zookeeper'
2014-07-04 13:47:15,820 |  INFO [nioEventLoopGroup-2-1] (DataReader.java:96) - network server bound to address /141.23.83.200:55283
2014-07-04 13:47:15,824 |  INFO [localEventLoopGroup-4-1] (DataReader.java:96) - network server bound to address local:5d47babf-0251-48ce-8bac-415aa2980314
2014-07-04 13:47:15,827 |  INFO [nioEventLoopGroup-6-1] (IOManager.java:312) - network server bound to address /141.23.83.200:55283
2014-07-04 13:47:21,044 | ERROR [main] (NIOServerCnxnFactory.java:44) - Thread Thread[main,5,main] died
java.lang.IllegalStateException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /aura
    at de.tuberlin.aura.workloadmanager.InfrastructureManager.<init>(InfrastructureManager.java:116)
    at de.tuberlin.aura.workloadmanager.InfrastructureManager.getInstance(InfrastructureManager.java:133)
    at de.tuberlin.aura.workloadmanager.WorkloadManager.<init>(WorkloadManager.java:77)
    at de.tuberlin.aura.workloadmanager.WorkloadManager.<init>(WorkloadManager.java:57)
    at de.tuberlin.aura.client.executors.LocalClusterSimulator.<init>(LocalClusterSimulator.java:128)
    at de.tuberlin.aura.client.executors.LocalClusterSimulator.<init>(LocalClusterSimulator.java:63)
    at de.tuberlin.aura.demo.examples.IntegrationTests.main(IntegrationTests.java:480)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

this is the case for both the state on master (e.g. SimpleClient at 90147c3) and develop (e.g. IntegrationTests at 87451d6).

stepping through these clients sometimes leads to successful runs, which might suggest a timing issue and not a general problem with OS X.

IllegalState during event handling of state machine

Sometimes threads die unexpectedly during the execution of jobs. In particular, running all three test cases of the IntegrationTest class currently fails sometimes with the following output. The test case does not fail with this problem on every run and not yet once when executing a single test case. It didn't fail with this problem on the CI yet, mostly on my machine.

Seems to be a problem related to the event system and state machine transitions.

2014-07-10 14:31:23,452 |  INFO [Thread-19] (LocalIntegrationTest.java:423) - Submit: Job: 3 layered - point2point + point2point connection - run 1/1
2014-07-10 14:31:23,454 |  INFO [Thread-20] (LocalIntegrationTest.java:423) - Submit: Job: 2 layered - point2point connection - run 1/1
2014-07-10 14:31:23,461 |  INFO [pool-1-thread-1] (WorkloadManager.java:141) - TOPOLOGY 'Job: 3 layered - point2point + point2point connection' SUBMITTED
2014-07-10 14:31:23,461 |  INFO [pool-1-thread-1] (TopologyController.java:96) - ASSEMBLE TOPOLOGY 'Job: 3 layered - point2point + point2point connection'
2014-07-10 14:31:23,463 |  INFO [Thread-21] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TOPOLOGY_STATE_PARALLELIZED
2014-07-10 14:31:23,463 |  INFO [Thread-21] (TopologyController.java:304) - CHANGE STATE OF TOPOLOGY 'Job: 3 layered - point2point + point2point connection' [2bcc0f31-df39-480d-9458-66667f68f6a5] FROM TOPOLOGY_STATE_CREATED TO TOPOLOGY_STATE_PARALLELIZED  [TOPOLOGY_TRANSITION_PARALLELIZE]
2014-07-10 14:31:23,464 |  INFO [Thread-21] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TOPOLOGY_STATE_SCHEDULED
2014-07-10 14:31:23,464 |  INFO [Thread-21] (TopologyController.java:304) - CHANGE STATE OF TOPOLOGY 'Job: 3 layered - point2point + point2point connection' [2bcc0f31-df39-480d-9458-66667f68f6a5] FROM TOPOLOGY_STATE_PARALLELIZED TO TOPOLOGY_STATE_SCHEDULED  [TOPOLOGY_TRANSITION_SCHEDULE]
2014-07-10 14:31:23,470 |  INFO [pool-4-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Sink-0 [d96a6791-d1b5-47c3-8e5f-7b7c59106756] ON EXECUTION UNIT (0) ON MACHINE [dab02f2f-6c5e-47de-b5fd-d7ca93f2589b]
2014-07-10 14:31:23,470 |  INFO [FSM-Sink-0-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,471 |  INFO [FSM-Sink-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [d96a6791-d1b5-47c3-8e5f-7b7c59106756] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,472 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Sink]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = d96a6791-d1b5-47c3-8e5f-7b7c59106756,  name = Sink },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 30e6c492-401c-4ec7-9097-cbfe57163e7c,  name = Middle }]],  outputGates = [] } }
2014-07-10 14:31:23,472 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,481 |  INFO [pool-7-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Sink-1 [0b3b1d7d-edce-448a-9e3d-1f6daaa0deac] ON EXECUTION UNIT (0) ON MACHINE [d511f745-09c1-477c-a15e-9dbc4fe33819]
2014-07-10 14:31:23,481 |  INFO [FSM-Sink-1-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,482 |  INFO [FSM-Sink-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [0b3b1d7d-edce-448a-9e3d-1f6daaa0deac] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,483 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Sink]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = 0b3b1d7d-edce-448a-9e3d-1f6daaa0deac,  name = Sink },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = e4fdc2c8-dda9-4102-9da6-b5f9ae83efde,  name = Middle }]],  outputGates = [] } }
2014-07-10 14:31:23,484 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,491 |  INFO [pool-4-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Middle-1 [30e6c492-401c-4ec7-9097-cbfe57163e7c] ON EXECUTION UNIT (1) ON MACHINE [dab02f2f-6c5e-47de-b5fd-d7ca93f2589b]
2014-07-10 14:31:23,493 |  INFO [TaskDriver-Middle-1-EventDispatcher] (TaskDataProducer.java:393) - OUTPUT CONNECTION FROM Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] TO TASK Sink [d96a6791-d1b5-47c3-8e5f-7b7c59106756] IS ESTABLISHED
2014-07-10 14:31:23,493 |  INFO [FSM-Middle-1-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,494 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Middle]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 30e6c492-401c-4ec7-9097-cbfe57163e7c,  name = Middle },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 69526009-8518-438c-86e6-cf00d751e18a,  name = Source }]],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = d96a6791-d1b5-47c3-8e5f-7b7c59106756,  name = Sink }]] } }
2014-07-10 14:31:23,494 |  INFO [TaskDriver-Sink-0-EventDispatcher] (TaskDataConsumer.java:551) - INPUT CONNECTION FROM Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] TO TASK Sink [d96a6791-d1b5-47c3-8e5f-7b7c59106756] IS ESTABLISHED
2014-07-10 14:31:23,494 |  INFO [FSM-Middle-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,497 |  INFO [FSM-Sink-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [d96a6791-d1b5-47c3-8e5f-7b7c59106756] FROM TASK_STATE_OUTPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,499 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,500 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,504 |  INFO [pool-7-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Middle-0 [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] ON EXECUTION UNIT (1) ON MACHINE [d511f745-09c1-477c-a15e-9dbc4fe33819]
2014-07-10 14:31:23,506 |  INFO [TaskDriver-Middle-0-EventDispatcher] (TaskDataProducer.java:393) - OUTPUT CONNECTION FROM Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] TO TASK Sink [0b3b1d7d-edce-448a-9e3d-1f6daaa0deac] IS ESTABLISHED
2014-07-10 14:31:23,506 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Middle]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = e4fdc2c8-dda9-4102-9da6-b5f9ae83efde,  name = Middle },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = 6ac103d9-0483-4a17-8fe2-14859a9834df,  name = Source }]],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = 0b3b1d7d-edce-448a-9e3d-1f6daaa0deac,  name = Sink }]] } }
2014-07-10 14:31:23,506 |  INFO [TaskDriver-Sink-1-EventDispatcher] (TaskDataConsumer.java:551) - INPUT CONNECTION FROM Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] TO TASK Sink [0b3b1d7d-edce-448a-9e3d-1f6daaa0deac] IS ESTABLISHED
2014-07-10 14:31:23,506 |  INFO [FSM-Middle-0-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,509 |  INFO [FSM-Middle-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,510 |  INFO [FSM-Sink-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [0b3b1d7d-edce-448a-9e3d-1f6daaa0deac] FROM TASK_STATE_OUTPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,511 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,512 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,516 |  INFO [pool-4-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Source-1 [69526009-8518-438c-86e6-cf00d751e18a] ON EXECUTION UNIT (2) ON MACHINE [dab02f2f-6c5e-47de-b5fd-d7ca93f2589b]
2014-07-10 14:31:23,516 |  INFO [FSM-Source-1-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_INPUTS_CONNECTED
2014-07-10 14:31:23,517 |  INFO [FSM-Source-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Source [69526009-8518-438c-86e6-cf00d751e18a] FROM TASK_STATE_CREATED TO TASK_STATE_INPUTS_CONNECTED  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,518 |  INFO [TaskDriver-Source-1-EventDispatcher] (TaskDataProducer.java:393) - OUTPUT CONNECTION FROM Source [69526009-8518-438c-86e6-cf00d751e18a] TO TASK Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] IS ESTABLISHED
2014-07-10 14:31:23,518 |  INFO [TaskDriver-Middle-1-EventDispatcher] (TaskDataConsumer.java:551) - INPUT CONNECTION FROM Source [69526009-8518-438c-86e6-cf00d751e18a] TO TASK Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] IS ESTABLISHED
2014-07-10 14:31:23,519 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Source]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 69526009-8518-438c-86e6-cf00d751e18a,  name = Source },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 30e6c492-401c-4ec7-9097-cbfe57163e7c,  name = Middle }]] } }
2014-07-10 14:31:23,520 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,523 |  INFO [FSM-Middle-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Middle [30e6c492-401c-4ec7-9097-cbfe57163e7c] FROM TASK_STATE_OUTPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,524 |  INFO [FSM-Source-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Source [69526009-8518-438c-86e6-cf00d751e18a] FROM TASK_STATE_INPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,526 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,526 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,533 |  INFO [pool-7-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Source-0 [6ac103d9-0483-4a17-8fe2-14859a9834df] ON EXECUTION UNIT (2) ON MACHINE [d511f745-09c1-477c-a15e-9dbc4fe33819]
2014-07-10 14:31:23,534 |  INFO [FSM-Source-0-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_INPUTS_CONNECTED
2014-07-10 14:31:23,534 |  INFO [FSM-Source-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Source [6ac103d9-0483-4a17-8fe2-14859a9834df] FROM TASK_STATE_CREATED TO TASK_STATE_INPUTS_CONNECTED  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,535 |  INFO [TaskDriver-Source-0-EventDispatcher] (TaskDataProducer.java:393) - OUTPUT CONNECTION FROM Source [6ac103d9-0483-4a17-8fe2-14859a9834df] TO TASK Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] IS ESTABLISHED
2014-07-10 14:31:23,535 |  INFO [TaskDriver-Middle-0-EventDispatcher] (TaskDataConsumer.java:551) - INPUT CONNECTION FROM Source [6ac103d9-0483-4a17-8fe2-14859a9834df] TO TASK Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] IS ESTABLISHED
2014-07-10 14:31:23,536 |  INFO [pool-1-thread-1] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Source]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = 6ac103d9-0483-4a17-8fe2-14859a9834df,  name = Source },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = e4fdc2c8-dda9-4102-9da6-b5f9ae83efde,  name = Middle }]] } }
2014-07-10 14:31:23,537 |  INFO [FSM-Middle-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Middle [e4fdc2c8-dda9-4102-9da6-b5f9ae83efde] FROM TASK_STATE_OUTPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_INPUTS_CONNECTED]
2014-07-10 14:31:23,537 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,537 |  INFO [FSM-Source-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Source [6ac103d9-0483-4a17-8fe2-14859a9834df] FROM TASK_STATE_INPUTS_CONNECTED TO TASK_STATE_READY  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,539 |  INFO [pool-1-thread-2] (WorkloadManager.java:141) - TOPOLOGY 'Job: 2 layered - point2point connection' SUBMITTED
2014-07-10 14:31:23,539 | ERROR [Thread-21] (EventDispatcher.java:326) - 
java.lang.IllegalStateException
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:383)
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:379)
    at de.tuberlin.aura.core.common.statemachine.StateMachine$FiniteStateMachine$3.handleEvent(StateMachine.java:551)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.dispatch(EventDispatcher.java:324)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.access$300(EventDispatcher.java:23)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher$1.run(EventDispatcher.java:91)
    at java.lang.Thread.run(Thread.java:745)
2014-07-10 14:31:23,541 |  INFO [pool-1-thread-2] (TopologyController.java:96) - ASSEMBLE TOPOLOGY 'Job: 2 layered - point2point connection'
2014-07-10 14:31:23,543 | ERROR [Thread-21] (NIOServerCnxnFactory.java:44) - Thread Thread[Thread-21,5,main] died
java.lang.IllegalStateException
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:383)
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:379)
    at de.tuberlin.aura.core.common.statemachine.StateMachine$FiniteStateMachine$3.handleEvent(StateMachine.java:551)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.dispatch(EventDispatcher.java:324)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.access$300(EventDispatcher.java:23)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher$1.run(EventDispatcher.java:91)
    at java.lang.Thread.run(Thread.java:745)
2014-07-10 14:31:23,543 | ERROR [Thread-21] (NIOServerCnxnFactory.java:44) - Thread Thread[Thread-21,5,main] died
java.lang.IllegalStateException
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:383)
    at de.tuberlin.aura.workloadmanager.TopologyController$10.stateAction(TopologyController.java:379)
    at de.tuberlin.aura.core.common.statemachine.StateMachine$FiniteStateMachine$3.handleEvent(StateMachine.java:551)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.dispatch(EventDispatcher.java:324)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.access$300(EventDispatcher.java:23)
    at de.tuberlin.aura.core.common.eventsystem.EventDispatcher$1.run(EventDispatcher.java:91)
    at java.lang.Thread.run(Thread.java:745)
2014-07-10 14:31:23,545 |  INFO [Thread-28] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TOPOLOGY_STATE_PARALLELIZED
2014-07-10 14:31:23,547 |  INFO [Thread-28] (TopologyController.java:304) - CHANGE STATE OF TOPOLOGY 'Job: 2 layered - point2point connection' [ab5f70fc-6046-4dae-a84f-5956e7173f84] FROM TOPOLOGY_STATE_CREATED TO TOPOLOGY_STATE_PARALLELIZED  [TOPOLOGY_TRANSITION_PARALLELIZE]
2014-07-10 14:31:23,548 |  INFO [Thread-28] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TOPOLOGY_STATE_SCHEDULED
2014-07-10 14:31:23,548 |  INFO [Thread-28] (TopologyController.java:304) - CHANGE STATE OF TOPOLOGY 'Job: 2 layered - point2point connection' [ab5f70fc-6046-4dae-a84f-5956e7173f84] FROM TOPOLOGY_STATE_PARALLELIZED TO TOPOLOGY_STATE_SCHEDULED  [TOPOLOGY_TRANSITION_SCHEDULE]
2014-07-10 14:31:23,553 |  INFO [pool-4-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Sink-1 [0cc56dd6-bde4-46a6-b681-71677f0f350b] ON EXECUTION UNIT (3) ON MACHINE [dab02f2f-6c5e-47de-b5fd-d7ca93f2589b]
2014-07-10 14:31:23,553 |  INFO [FSM-Sink-1-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,553 |  INFO [FSM-Sink-1-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [0cc56dd6-bde4-46a6-b681-71677f0f350b] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,554 |  INFO [pool-1-thread-2] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Sink]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 0cc56dd6-bde4-46a6-b681-71677f0f350b,  name = Sink },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = f294dab8-b39f-4f02-8dce-a0b761b195a8,  name = Source }]],  outputGates = [] } }
2014-07-10 14:31:23,555 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,562 |  INFO [pool-7-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Sink-0 [db77ca68-ae7a-4c99-9968-7d586071c6a7] ON EXECUTION UNIT (3) ON MACHINE [d511f745-09c1-477c-a15e-9dbc4fe33819]
2014-07-10 14:31:23,562 |  INFO [FSM-Sink-0-EventDispatcher] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TASK_STATE_OUTPUTS_CONNECTED
2014-07-10 14:31:23,563 |  INFO [FSM-Sink-0-EventDispatcher] (TaskDriver.java:391) - CHANGE STATE OF TASK Sink [db77ca68-ae7a-4c99-9968-7d586071c6a7] FROM TASK_STATE_CREATED TO TASK_STATE_OUTPUTS_CONNECTED  [TASK_TRANSITION_OUTPUTS_CONNECTED]
2014-07-10 14:31:23,564 |  INFO [pool-1-thread-2] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Sink]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = db77ca68-ae7a-4c99-9968-7d586071c6a7,  name = Sink },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = fc5e321e-114a-4613-9d02-e4a78034e8c3,  name = Source }]],  outputGates = [] } }
2014-07-10 14:31:23,564 |  INFO [TopologyControllerEventDispatcher] (EventDispatcher.java:331) - no listener registered for event CONTROL_EVENT_REMOTE_TASK_STATE_UPDATE
2014-07-10 14:31:23,571 |  INFO [pool-4-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Source-1 [f294dab8-b39f-4f02-8dce-a0b761b195a8] ON EXECUTION UNIT (0) ON MACHINE [dab02f2f-6c5e-47de-b5fd-d7ca93f2589b]
2014-07-10 14:31:23,573 |  INFO [pool-1-thread-2] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Source]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = f294dab8-b39f-4f02-8dce-a0b761b195a8,  name = Source },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = dab02f2f-6c5e-47de-b5fd-d7ca93f2589b,  netAddress = /141.23.109.200:55425,  controlAddress = /141.23.109.200:55426, HardwareDescriptor = { uid = 6ea8d352-2350-4bd1-8950-b9c533e6e6e7,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = 69f4ede3-d04a-4471-9e62-525989e6bdd2,  sizeOfHDD = 250140434432 } } },  uid = 0cc56dd6-bde4-46a6-b681-71677f0f350b,  name = Sink }]] } }
2014-07-10 14:31:23,581 |  INFO [pool-7-thread-1] (TaskExecutionManager.java:106) - EXECUTE TASK Source-0 [fc5e321e-114a-4613-9d02-e4a78034e8c3] ON EXECUTION UNIT (0) ON MACHINE [d511f745-09c1-477c-a15e-9dbc4fe33819]
2014-07-10 14:31:23,582 |  INFO [pool-1-thread-2] (TopologyDeployer.java:86) - TASK DEPLOYMENT DESCRIPTOR [Source]: DeploymentDescriptor = { nodeDescriptor = AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = fc5e321e-114a-4613-9d02-e4a78034e8c3,  name = Source },  nodeBindingDescriptor = NodeBindingDescriptor = { inputGates = [],  outputGates = [[AbstractNodeDescriptor = { machine = MachineDescriptor = { uid = d511f745-09c1-477c-a15e-9dbc4fe33819,  netAddress = /141.23.109.200:55430,  controlAddress = /141.23.109.200:55431, HardwareDescriptor = { uid = 8e0159a6-fb1b-492c-9005-2113a7b8432a,  cpuCores = 4,  sizeOfRAM = 1908932608, HDDDescriptor = { uid = f182db85-a1e9-4182-9796-d2655f14eb1d,  sizeOfHDD = 250140434432 } } },  uid = db77ca68-ae7a-4c99-9968-7d586071c6a7,  name = Sink }]] } }
2014-07-10 14:31:23,584 |  INFO [Thread-28] (EventDispatcher.java:331) - no listener registered for event FSM_STATE_EVENT_TOPOLOGY_STATE_DEPLOYED
2014-07-10 14:31:23,584 |  INFO [Thread-28] (TopologyController.java:304) - CHANGE STATE OF TOPOLOGY 'Job: 2 layered - point2point connection' [ab5f70fc-6046-4dae-a84f-5956e7173f84] FROM TOPOLOGY_STATE_SCHEDULED TO TOPOLOGY_STATE_DEPLOYED  [TOPOLOGY_TRANSITION_DEPLOY]

Add configuration (manager)

Add a configuration manager and config files for aura modules so that one can specify e.g. the buffer size, io threads, etc... in a configuration file instead of using hard coded constants.

The typesafe configuration library seems suitable, as it provides hierarchical configs, enhanced json syntax, ... see https://github.com/typesafehub/config

CI

It would be nice to build and test AURA automatically. To do so, I would set up Jenkins and everybody needs to write some tests ;)

Coding conventions

Since I expect the number of contributors to the codebase to grow in the next weeks, I would like to discuss the question of coding conventions for the project.

I am really quite flexible with respect to whatever your preferences are, as long as we keep the code style consistent in the whole project. However, based on my experience I would like to bring out three important points.

1. Stick to A Standard

I prefer to stick to a well-known coding standard with comes with formatters for common IDE's like Eclipse and IntelliJ.

2. Stick to Tab Indention

On the not-so-off chance that you have a fragment of complex code that requires line alignment, sticking to spaces (as opposed to tabs) allows people to see your code as you intended regardless of the viewer.

{
    int x = if (a.very.complex.condition) ? // 4 spaces
          ? true                            // 10 spaces
          : false;                          // 10 spaces
}

The alternative would be to mix tabs and spaces for special alignment:

{
    int x = if (a.very.complex.condition) ? // 1 tab
          ? true                            // 1 tab + 6 spaces
          : false;                          // 1 tab + 6 spaces
}

3. Enable On/Off Tags In Code Formatters

Even the best code formatter is not equipped for all cases, and I don't see nothing wrong in deviating from the standard for a line or two if it substantially increases the readability of your code.

The most important case where I would love to keep the code formatter from breaking my nicely aligned code is in I use fluent syntax.

@formatter:off
// I really don't want the formatter to screw up this shit!
Plan plan = new PlanBuilder(
    new NodeBuilder()
        .setAttrX()
        .setAttrY()
        .setLeftInput(new NodeBuilder()
            .setAttrX(...)
            .setAttrY(...)
            .build())
        .setRightInput(new NodeBuilder()
            .setAttrX(...)
            .setAttrY(...)
            .build())
        .build());
@formatter:on

I searched for something that that meets the above points and found Maven Code Style And Code Conventions.

What do you think?

Rename frontend classes

In cloud setups relying on automated scripts and jps, having the same main class name as a similar system can cause issues.

I suggest the following renames in order to avoid confusion:

TaskManager => AuraTaskManager
WorkloadManager => AuraWorkloadManager

Buffer management issues (BUFFER_LOAD_FACTOR)

ok, i only have a rough understanding of our buffer management, but on certain machine configurations AURA fails for buffer management related reasons.

how much of the main memory is reserved for buffers is currently defined relatively to the available memory.

when the BUFFER_LOAD_FACTOR is chosen very small (e.g. 1%), there might not be (m)any buffers on small machines, see https://travis-ci.org/lauritzthamsen/AURA/builds/29074045

may be it'd be wise to mix relative memory usage with specific limits such as a lower limit of buffers to reserve memory for (MIN_BUFFER_COUNT). does that make sense?

when the BUFFER_LOAD_FACTOR is however set to be a bigger chuck (e.g. 40%), the system currently can run out of memory, see https://travis-ci.org/lauritzthamsen/AURA/builds/29075554

that shouldn't happen in any case, right? at least the system should tell the user on startup that there isn't enough memory for AURA / the query (yet, not with a fixed 1 GB requirement i'd say).

how to assign input parameters to nodes/operators

Hey. I am wondering how to assign input parameters to nodes or operators. The topology is built by using native class types instead of objects.

E.g. reading a file source at a specific position, what would be a nice way to assign parameters to the node.

final Topology.AuraTopologyBuilder atb1 = ac.createTopologyBuilder();

atb1.addNode(new Topology.ComputationNode(UUID.randomUUID(), "Source", 1, 1), Arrays.asList(FileSource.class, Record1.class))
                .connectTo("Sink", Topology.Edge.TransferType.ALL_TO_ALL)
                .addNode(new Topology.ComputationNode(UUID.randomUUID(), "Sink", 1, 1), Sink.class);

Buffers not freed at TASK_STATE_FINISHED

Running this OperatorTest fails even though all tuples are being processed correctly because buffers are currently still being freed when the TASK_STATE_FINISHED event is received:

2014-09-22 14:51:41,794 | INFO FSM-Source2-0-EventDispatcher - CHANGE STATE OF TASK Source2 [e8de05ca-9efa-4284-9df5-2a8f4e7f5954] FROM TASK_STATE_RUNNING TO TASK_STATE_FINISHED [TASK_TRANSITION_FINISH]
2014-09-22 14:51:44,288 | ERROR FSM-Join1-0-EventDispatcher - 1 buffers are not freed.
java.lang.IllegalStateException: 1 buffers are not freed.
at de.tuberlin.aura.core.memory.BufferAllocator.checkForMemoryLeaks(BufferAllocator.java:163)
at de.tuberlin.aura.core.memory.BufferAllocatorGroup.checkForMemoryLeaks(BufferAllocatorGroup.java:157)
at de.tuberlin.aura.taskmanager.TaskDriver.teardownDriver(TaskDriver.java:209)
at de.tuberlin.aura.taskmanager.TaskExecutionUnit$ExecutionUnitRunner$2.stateAction(TaskExecutionUnit.java:190)
at de.tuberlin.aura.taskmanager.TaskExecutionUnit$ExecutionUnitRunner$2.stateAction(TaskExecutionUnit.java:181)
at de.tuberlin.aura.core.common.statemachine.StateMachine$FiniteStateMachine$3.handleEvent(StateMachine.java:424)
at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.dispatch(EventDispatcher.java:324)
at de.tuberlin.aura.core.common.eventsystem.EventDispatcher.access$300(EventDispatcher.java:23)
at de.tuberlin.aura.core.common.eventsystem.EventDispatcher$1.run(EventDispatcher.java:91)
at java.lang.Thread.run(Thread.java:745)

NullPointerException during input gate connection phase

In the TaskDataConsumer, the queues assigned to an input gate are bound to an absorber before all channels are connected, which causes a nullpointer.

The error only occurs on the cluster and can easily be reproduced with the three layer all2all join point2point test.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.