nodelets vs thread pool about rclcpp HOT 7 CLOSED

I think the use case for the multi-threaded executor is to take advantage of multiple core systems to execute as many jobs in parallel as possible. Though I also don't recall seeing it mentioned in the design documents. If you take a look at multi_threaded_executor.hpp, it uses std::thread::hardware_concurrency to determine the number of threads to spawn:

https://github.com/ros2/rclcpp/blob/master/rclcpp/include/rclcpp/executors/multi_threaded_executor.hpp#L47

http://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency

This call is supposed to return the number of cores of the CPU executing the code, though it might not be implemented on all platforms/architectures.

It's up to the user to avoid data races in the MultiThreadedExecutor. It provides a mutex to ensure that only one thread accesses the get_next_executable function, but there isn't more synchronization than that.

from rclcpp.

Yes, the multi-threaded executor that is currently implemented will not tie work coming from a particular node to a particular thread.

from rclcpp.

The executor could be more sophisticated and have some notion of affinity. We don't currently have this, but it would be interesting to try it out to see if our abstractions provide enough information to achieve it.

from rclcpp.

Agreed. I'm not sure what @BobDean had in mind when he opened this question, but one use case I see for thread affinity is prioritizing work from a certain node by scheduling it in a high-priority/realtime thread.

It should be fairly easy to implement a "ThreadAffinityExecutor" (or whatever we want to call it), since the AnyExecutable struct has a shared pointer to the parent node of the executable.

from rclcpp.

I am wondering about its applicability to tightly timed nodes, and the ability to debug multi-threaded behavior when the worker is moving between threads between executions. what is the use case for the multithreaded executor? did i miss it on the design docs?

from rclcpp.

but there isn't more synchronization than that.

Well, there is also the callback groups, which can prevent certain user provided code from being run in parallel or in a reentrant fashion. This might let the user avoid having their own synchronization patterns and can allow the system to avoid things like locking the worker threads. See my roscon talk:

http://roscon.ros.org/2014/wp-content/uploads/2014/07/ROSCON-2014-Why-you-want-to-use-ROS-2.pdf (see page 17)

from rclcpp.

@BobDean I'm going to close this issue too. I think my most recent comment addresses some of the "why multithreaded executor?". Another thing I'll add is that the multi-threaded executor allows you to utilize as much of the system as possible when your CPU bound (basically a thread-pool as you said). The difference is that allows uncoordinated and uncoupled pieces of code (separate nodes, a.k.a. nodelets) to shared the thread pool, which is a bit above and beyond what a normal thread-pool does for you.

Feel free to continue the discussion or move it to the mailing list.

from rclcpp.