I don't know the full scope of doing this, but I know for sure that in order to get persistent durability, we'll need to add a new member to the durability_policy_t here: https://github.com/ros2/rmw/blob/master/rmw/include/rmw/types.h#L191 . I'm sure there is other work to get it down to the RMW implementation properly.

Feature request

Feature description

Right now, when requesting endpoint info from rmw implementations, e.g. rmw_get_publishers_info_by_topic, we receive inconsistent results for publishers created with the same QoS settings. This makes it hard to create subscriptions adaptively when using queried QoS profiles. It also means that the reporting is noisy/lossy, we do not know what settings were really given to the endpoints.

• Define a set of constants in the RMW API to define "infinite duration"
• State the contract on the RMW APIs rmw_get_publishers_info_by_topic and rmw_get_subscriptions_info_by_topic that these infinity values will be reported by RMW implementations
• State the contract on subscription/publisher creation APIs that an input of Zero duration for QoS policies will be interpreted and reported as Infinity.

The DDS Spec's OMG IDL PSM (Platform Specific Model) defines the following (page 139)

struct Duration_t {
long sec;
unsigned long nanosec;
};
const long DURATION_INFINITE_SEC = 0x7fffffff;
const unsigned long DURATION_INFINITE_NSEC = 0x7fffffff;

const long DURATION_ZERO_SEC = 0;
const unsigned long DURATION_ZERO_NSEC = 0;

We should do something similar. However, since we use a uint64_t for both fields of rmw_time_t, I'd suggest the infinity value being "uint64_max" for both seconds and nanoseconds.

Work Log

• #301
• ros2/rmw_cyclonedds#288 Change rmw_cyclonedds to conform to API
• Change rmw_fastrtps to conform to API
• Change rmw_connext to conform to API

Hello !

I am developing a custom implementation of RMW, and I can't figure what rmw_take and rmw_take_with_info should do.
Especially the interaction with rclcpp callbacks.

This comment could apply to other undocumented functions.

This will make debugging much easier.

From discussion on ros2/rmw_connext#43

typedef struct CustomParticipantInfo
{
eprosima::fastrtps::Participant * participant;
ReaderInfo * secondarySubListener;
WriterInfo * secondaryPubListener;
rmw_guard_condition_t * graph_guard_condition;
} CustomParticipantInfo;

What is the purpose of the secondarySubListener in the above strucutre, which is implemented in custom_participant_info.hpp ?

See discussion: #311 (comment)

The corresponding parameters and variables upstream/downstream of rmw should also be renamed to be consistent. API docs will also need to be updated.

We have a lot of different formats for the names of QoS policy settings at the moment. The inconsistency makes it inconvenient for users to change the options without looking them up.

If we unify the names, what should the format be?

1. RMW_QOS_POLICY_{SETTING}_{CATEGORY} e.g. how we have RMW_QOS_POLICY_KEEP_LAST_HISTORY (closest to DDS naming e.g. RELIABLE_RELIABILITY_QOS)
2. RMW_QOS_POLICY_{CATEGORY}_{SETTING} e.g. how we have RMW_QOS_POLICY_HISTORY_SYSTEM_DEFAULT
3. RMW_QOS_POLICY_{SETTING} e.g. how we have RMW_QOS_POLICY_RELIABLE
4. RMW_QOS_{CATEGORY}_POLICY_{SETTING} e.g. how we have rmw_qos_history_policy_t

or perhaps a different (shorter) option

Objectives

We make some investigations to find out the dependencies of transport latencies between different nodes. Therefore we made an set up with one sender-receiver-computer (IOM) and one repeater-computer (CPM). The IOM publishes defined data-packages on a constant frequency f with the sending timestamp t1 to the network. The CPM subscribes this, takes the receiving time t2, copies the data and publishes the data on time t3. The IOM subscribes at t4 and logs the data.

Afterwards the transport times between the sender-receiver-computer and the repeater-computer and vice versa shall be calculated. We assume them to be equal (Dt = t2-t1 = t4-t3) and use the formula

Dt = ( (t4-t1) - (t3-t2) ) / 2

which means, that the clocks of both computers don't have to be synchronized.

Problem

We assume the transport time Dt to be independent from the sending frequency f, as long as the network is not overloaded. But our results show an linear increasing transport time Dt on linear decreasing frequencies f (as you can see in the following section). In the same time, our timestamps t1 and t2 show that the sending frequency seems to be correct.

Results

Below you will see the results of our investigation. Information to our set up you will find in the next section. We do some tests with LAN and WLAN. In both cases there are some strange behaviours. Regarding the latency-issue it's on both the same. The ordinate axis is limited to 20 ms for a better readability. Latencies bigger than 20 ms are counted as packages out of bounds. So lets take a look to the WLAN test results:

Configuration with a D-Link DIR 510L (5G Hz WLAN)

Configuration with a D-Link DIR 510L (2G4 Hz WLAN)

Configuration with a TP-Link M7350 (5G Hz WLAN)

• 50 Hz
  
• 100 Hz
  
• 200 Hz
  

Configuration with a TP-Link M7350 (2G4 Hz WLAN)

• 50 Hz
  
• 100 Hz
  
• 200 Hz
  

For us the results are not as expected. Can you help us to find out the reason for it? Is there something like a fixed buffer or something else we don't see at the moment???

Set up

The computers are Raspberry Pi 2 with Raspbian installed on. We did our investigation with openSplice and FastRTPS. But there are no significant differences related to this behaviour. The ROS2 build is alpha7.

Below you will find the code for this two nodes:

sender-receiver-computer (IOM)

• iom.cpp

#include "rclcpp/rclcpp.hpp"
#include <string>

#include "test_lat_msg/msg/test_lat_msg.hpp"

// reference time stamp
static std::chrono::steady_clock::time_point t_start;
static FILE *fd_log;

void cb_subscriber(const test_lat_msg::msg::LatMsgCpm::SharedPtr msg)
{
	// calc receive time
	uint64_t t4 = (std::chrono::steady_clock::now()-t_start).count();
	
	// store received data to logfile
	fwrite (&msg->id, sizeof(msg->id), 1, fd_log);
	fwrite (&msg->t1, sizeof(msg->t1), 1, fd_log);
	fwrite (&msg->t2, sizeof(msg->t2), 1, fd_log);
	fwrite (&msg->t3, sizeof(msg->t3), 1, fd_log);
	fwrite (&t4, sizeof(t4), 1, fd_log);

	fwrite (&(msg->payload[0]), sizeof(uint8_t), 32, fd_log);

	fflush(fd_log);
}

int main(int argc, char * argv[])
{
	// aircraft number with default value - 0 should not use as valid value
	int flzNo = 0;
	// frequency of data output [Hz]
	int freq  = 10;
	
	// replace default values if some arguments given
	// first argument: aircraft number
	if( argc > 1)
	{
		flzNo = std::stoi(argv[1]);
	}
	// second argument: frequency of data output
	if( argc > 2)
	{
		freq = std::stoi(argv[2]);
	}

	// create a log-file with a unique name
	...

	// create names for node and topics
	...

	// initialize ROS2
	rclcpp::init(argc, argv);
	// initialize a node
	auto node = rclcpp::node::Node::make_shared(flzNodeName);
	// initialize a puplisher for IOM-messages
	auto publisher = node->create_publisher<test_lat_msg::msg::LatMsgIom>
	  (iomTopicName, rmw_qos_profile_sensor_data);

	auto subscriber = node->create_subscription<test_lat_msg::msg::LatMsgCpm>
	  (cpmTopicName, cb_subscriber, rmw_qos_profile_sensor_data);

	// take a reference time stamp
	t_start = std::chrono::steady_clock::now();

	// declare a publisher message
	test_lat_msg::msg::LatMsgIom pupmsg;
	// add aircraft number as IOM-ID to the puplisher message
	pupmsg.iomid = flzNo;
	// create a payload
	for(uint64_t i=0; i<32; i++)
	{
		pupmsg.payload[i] = i;
	}

	// set loop rate for publishing messages
	rclcpp::WallRate loop_rate(freq);
	// a counter for the message id
	uint64_t cnt = 0;
	while ( rclcpp::ok() )
	{
		pupmsg.id = cnt++;
		// put sending time to the message
		pupmsg.t1 = (std::chrono::steady_clock::now()-t_start).count();
		publisher->publish(pupmsg);

		// iterate callbacks		
		rclcpp::spin_some(node);
		// time control of the loop
		loop_rate.sleep();
	}

	return 0;
}

epeater-computer (CPM)

• cpm.cpp

#include "rclcpp/rclcpp.hpp"
#include <string>
#include "CpmRelay.h"

int main(int argc, char * argv[]){
	// count of aircrafts or rather count of relays provided by this CPM
	// the default value is 5
	uint8_t flzCnt = 5;
	
	// replace default value if some arguments given
	// first argument: aircraft count
	if( argc > 1){
		flzCnt = std::stoi(argv[1]);
	}

	// initialize ROS2
	rclcpp::init(argc, argv);
	// initialize a node
	auto node = rclcpp::node::Node::make_shared("test_cpm_test");

	// create an instance for each relay
	std::vector<test::CpmRelay*> cpmRelay;
	for(int i=1; i<=flzCnt; i++){
		cpmRelay.push_back(new test::CpmRelay(node, i));
	}

	std::cout << "spin" << std::endl;
	rclcpp::spin(node);

	return 0;
}

• CpmRelay.h

#ifndef CPMRELAY_H_INCLUDED
#define CPMRELAY_H_INCLUDED

#include "rclcpp/rclcpp.hpp"
#include <string>

#include "test_lat_msg/msg/test_lat_msg.hpp"

namespace test
{

class CpmRelay
{

private:
	uint8_t flzNo;
	std::string cpmTopicName;
	std::string iomTopicName;

	rclcpp::node::Node::SharedPtr node;
	
	rclcpp::publisher::Publisher<test_lat_msg::msg::LatMsgCpm>::SharedPtr
		publisher;
	
	rclcpp::subscription::Subscription<test_lat_msg::msg::LatMsgIom>::SharedPtr
		subscriber;

	test_lat_msg::msg::LatMsgCpm pupmsg;
	
	/// offset-time / start time
	std::chrono::steady_clock::time_point t_start;

public:
	CpmRelay( rclcpp::node::Node::SharedPtr node, uint8_t flzNo );

private:
	void cb_subscriber(const test_lat_msg::msg::LatMsgIom::SharedPtr msg);
};

}

#endif // CPMRELAY_H_INCLUDED

• CpmRelay.cpp

#include "rclcpp/rclcpp.hpp"
#include <string>

#include "test_lat_msg/msg/test_lat_msg.hpp"

#include "CpmRelay.h"

test::CpmRelay::CpmRelay( rclcpp::node::Node::SharedPtr pnode, 
                          uint8_t pflzNo)
{ 
	this->flzNo = pflzNo;
	this->node = pnode;

	// get start time
	this->t_start = std::chrono::steady_clock::now();

	cpmTopicName  = "t_flz" + std::to_string(flzNo) + "_cpm";
	iomTopicName  = "t_flz" + std::to_string(flzNo) + "_iom";
	
	std::cout << "Relay no. " << std::to_string(flzNo) << " listen to " 
	          << iomTopicName << " and writes to " << cpmTopicName << std::endl;

	publisher = node->create_publisher<test_lat_msg::msg::LatMsgCpm>
	  (cpmTopicName, rmw_qos_profile_sensor_data);

	using namespace std::placeholders; // for `_1`
	subscriber = node->create_subscription<test_lat_msg::msg::LatMsgIom>
	  (iomTopicName, 
	   std::bind(&CpmRelay::cb_subscriber, this,_1),
	   rmw_qos_profile_sensor_data);
}

void
test::CpmRelay::cb_subscriber(const test_lat_msg::msg::LatMsgIom::SharedPtr msg)
{
	// get relative receiving time
	pupmsg.t2 = (std::chrono::steady_clock::now()-t_start).count();

	// copy data
	pupmsg.iomid = msg->iomid;
	pupmsg.id = msg->id;
	pupmsg.t1 = msg->t1;
	pupmsg.payload = msg->payload;

	// get relative sending time
	pupmsg.t3 = (std::chrono::steady_clock::now()-t_start).count();
	publisher->publish(pupmsg);
}

Feel free to ask for further information.

Kind regards,
Alex

Feature request

Feature description

In order to reduce dynamic memory allocations during publish and receipt of messages during the operation of a Node, this API would ideally allow for users of the RMW API to request the middleware to preallocate memory for predefined message types and sizes.

Implementation considerations

Implement a struct that hides the implementation of the buffer allocation that future calls to publish will take as an argument. If a publisher is called with the argument, then it will use the previously-allocated buffer rather than allocating a new one as necessary.

typedef struct RMW_PUBLIC_TYPE rmw_publisher_allocation_t
{
  const char * implementation_identifier;
  void * data;
} rmw_publisher_allocation_t;

typedef struct RMW_PUBLIC_TYPE rmw_subscription_allocation_t
{
  const char * implementation_identifier;
  void * data;
} rmw_subscription_allocation_t;

These can then be used by the rmw_publish and rmw_subscription methods to provide a reserved space for serializing/deserializing incoming/outgoing messages.

Before any messages were publish or take, these would need to be initialized via an API as follows:

rmw_init_publisher_allocation_t(rmw_publisher_allocation_t*, const rosidl_message_typesupport_t*, const rosidl_message_bounds_t*);
rmw_init_subscription_allocation_t(rmw_publisher_allocation_t*, const rosidl_message_typesupport_t*, const rosidl_message_bounds_t*)
rmw_fini_publisher_allocation_t(rmw_publisher_allocation_t*);
rmw_fini_subscription_allocation_t(rmw_subscription_allocation_t*)

This will also required additional information from the typesupport to get the serialized message sizes, also taking into account the potential size of unbounded fields. To account for unbounded fields, I propose adding a second structure rosidl_message_bounds_t that will give the user a way of specifying known fixed bounds for unbounded messages.

For fixed and bounded sized messages, the API should take a const rosidl_message_typesupport_t* which will then be allocated by the RMW implementation.

The rosidl_message_bounds_t structure would be on a per message basis and describe any of the unbounded fields. Besides the nominal case, there are three main cases to handle.

# Message with unbounded primitive
uint8[] data

struct Msg__bounds {
  size_t data__length = 0;
};

# Message with string array
string[] data

# special structure for delimiting string bounds
struct string__bounds {
  size_t bounds = 0;
};

struct Msg__bounds {
  size_t data__length = 0;
  string__bounds data__bounds;
};

# Message with unbounded array of complex structures
std_msgs/Header header[]

# special structure for delimiting string bounds
struct string__bounds {
  size_t bounds = 0;
};

struct std_msgs__msg__Header__bounds {
  string__bounds frame_id__bounds;
};

struct Msg__bounds {
  size_t header__length = 0;
  std_msgs__msg__Header__bounds header__bounds;
};

Serialized length can be retrieved via:

rmw_get_serialized_length(const rosidl_message_typesupport_t*, const rosidl_message_bounds_t*);

These structures can then be used in conjunction with rmw_publisher:

rmw_publisher_allocation_t alloc;
rmw_init_publisher_allocation(&alloc, ts, bounds);
# TODO: What happens when ros_message violates bounds given?
rmw_publish(publisher, ros_message, alloc);
# Also rmw_publish_serialized_message()

Or with rmw_subscriber:

rmw_subscription_allocation_t alloc;
rmw_init_subscription_allocation(&alloc, ts, bounds);
rmw_create_subscription(node, ts, topic, qos, ignore_local, alloc)

# TODO: What happens when incoming message violates bounds?
rmw_take()

Open Questions:

• What do we do when a message violates bounds?
• Drop message
• Error and allow a retry
• Realloc and retry

Feature request

Feature description

Since a subscriber can receive a topic message only if its QOS profile is compatible with the QOS profile of the publisher a method to retrieve this information from the available topics should be very useful.

Implementation considerations

I think that the command
$ ros2 topic info /topic
can be a good place to introduce this important feature.

Ref: https://answers.ros.org/question/304946/ros2-retrieving-qos-settings-for-a-topic/

Feature request

Feature description

Developer can use this interface to confirm reader receive all sent messages. For some use cases, this confirmation is very important.

wait_for_acknowledgments is defined in DDS spec.

The description in DDS spec

2.2.2.4.1.12 wait_for_acknowledgments
This operation blocks the calling thread until either all data written by the reliable DataWriter entities is acknowledged by all
matched reliable DataReader entities, or else the duration specified by the max_wait parameter elapses, whichever happens
first. A return value of OK indicates that all the samples written have been acknowledged by all reliable matched data readers;
a return value of TIMEOUT indicates that max_wait elapsed before all the data was acknowledged.

Implementation considerations

If DDS exposes this interface, we can directly use (Such as FastDDS). If not, return unsupported.

Acceptance Criteria:

• Add service structures and functions to C-API

For reusability purposes, the rmw_serialized_message_t type should be excluded from this rmw package. The data type shall be placed in rcutils as it is basically only a vector of characters.

Feature request

Feature description

Expose resource limit QoS. If not, KEEP_ALL history policy is not configurable at all.
See comment ros2/rclcpp#727 (comment).

Also, it will be needed if we want to implement KEEP_ALL policy with intraprocess communication (see ros2/rclcpp#727).

Feature request

Reconsidering 1-to-1 mapping of ROS nodes to DDS participants

Feature description

As a result of the research published here:
https://github.com/nobleo/ros2_performance
A ROS discourse discussion was started here:
https://discourse.ros.org/t/reconsidering-1-to-1-mapping-of-ros-nodes-to-dds-participants/10062
at this discussion it was requested that this feature be posted as an issue.

From the discourse discussion:
dirk-thomas:
"Instead of introducing an option the current idea is to associate the DDS participant with the context created during rmw_init. That would imply that common applications using a single init / context will only use a single DDS participant - even if they are composed of multiple ROS nodes."

Argumentation for change: Making every Node in rclcpp a DDS participant creates significant CPU overhead. Use cases exists where all Nodes run on a single piece of hardware. The CPU overhead can be reduced by creating 1 participant per process space.

Implementation considerations

From the discourse discussion:
dirk-thomas:
"... Any help is appreciated. It will likely start with a design article to discuss the side affects of the intended change. E.g. the ROS node name is currently being used for the DDS participant name. When that mapping goes away there needs to be a replacement mechanism to communicate the node name."

kyrofa:
"Not trying to debate the performance gains of this approach, but I think it’s worth pointing out that SROS 2 (and indeed DDS-Security in general) currently only supports security at the domain participant level. Using the same participant for multiple nodes will make security very difficult, as all nodes will have effectively the same identity and thus the same access control."

See discussion in:
ros2/rcl#30 (comment)

I believe it's a good idea to shift the error handling from rmw to c_utilities to avoid duplication and typedefs everywhere and have clean dependencies.

Feature request

Feature description

The auto-populated seq field was dropped from Header in transition from ROS1 to ROS2 (ros2/common_interfaces#2). This data was useful in measuring lost messages, and I imagine will be even more helpful with ROS2's focus on unreliable communication channels.

Implementation considerations

Some implementation hints were provided in https://discourse.ros.org/t/foxy-message-api-review/13105/17

rmw_time_t is defined as:

typedef struct RMW_PUBLIC_TYPE rmw_time_t
{
  uint64_t sec;
  uint64_t nsec;
} rmw_time_t;

This differs from rcutils_duration_value_t which admits a value as just a int64_t holding nanoseconds. It probably makes sense to replace rmw_time_t with a type alias for rcutils_duration_value_t, and to make clear that it is a duration, not a time point.

Feature request

Feature description

Currently, the API does not provide (or documents) a generic mechanism to wait for publishers (subscriptions) to match subscriptions (publishers) -- only the current match count is available for query. Thus, code that monitors these counts has to sleep and retry. That's inefficient and potentially brittle. It'd be nice to enable such waits.

Implementation considerations

It seems to me that a new pub/sub event would be the simplest solution. All Tier 1 RMW implementations have a form of pub/sub matching listener that can be leveraged:

• eProsima Fast-DDS has it, and both rmw_fastrtps_cpp and rmw_fastrtps_dynamic_cpp already use it (see here and here).
• RTI Connext has it, and rmw_connext_cpp already uses it (see here and here).
• ADLINK CycloneDDS has it, but rmw_cyclonedds_cpp doesn't use it yet (see here).

It's worth noting that some rcl tests appear to rely on pub/sub matching triggering the graph guard condition. Regardless of that not being true for any of the current Tier 1 RMW implementations, it seems to me that would create too many unwarranted awakes. Also, pub/sub matching does not imply a graph change.

To avoid the current compiler warnings: http://ci.ros2.org/job/ros2_batch_ci_linux/70/warnings17Result/

https://github.com/ros2/rmw/blob/master/rmw/include/rmw/serialized_message.h is an extremely thin wrapper around rcutils_uint8_array_*. The problem with this is that users expect that these functions return RMW_* error codes, when in reality they return RCUTILS_* error codes. Even the documentation is confused about this; it is documented as returning RMW_* error codes, which is just incorrect.

My suggestion is that we make these static inline functions instead, with a conversion between the RCUTILS error codes that are returned from the rcutils_uint8_array functions and the RMW error codes that they are documented to return.

Feature request

Feature description

Design document here.
The idea is to implement everything in the rmw layer, and avoid changes in upstream layers as much as possible.

Tasks already done:

• Port rmw_fastrtps.
• Create common package rmw_dds_common, to allow easiest porting of remaining rmw implementations.
• Disable failing cross vendor tests (@ivanpauno)
• (security) Update sros2 and test_security.

To do:

• Migrate rmw_connext (not needed, we changed to rmw_connextdds which is one-participant-per-context)
• Migrate rmw_cyclonedds

Feature request

Feature description

Draft overview and background is ros2/design#282,

Introduce dds contentfiltertopic to ROS, dds specification is DDS v1.4 2.2.2.3.3 ContentFilteredTopic Class. ContentFilteredTopic can be created on parent topic and DataReader only receives the messages that matches with filtering expression and expression parameters. If implementation does not support this feature, it returns not support.

• rmw implementation support.

• expected user for this feature.

  • action feedback / status topics.
  • parameter_events topic.
  • user defined topics.

Implementation considerations

• rmw and rcl implementation
  • ContentFilteredTopic requires parent topic, returning capability for ContentFilteredTopic with rmw_create_subscriotion rmw_subscription_t. So that rclcpp/rcl can tell if the implementation supports it or not.
  • Based on parent topic and subscription, it can create ContentFilteredTopic with subtopic name, filtering expression and expression parameters. Then implementation creates ContentFilteredTopic and replace the DataReader.
  • Expression parameter can be changed via setter dynamically, also getter should be provided. but filtering expression cannot be changed unless re-creating ContentFilteringTopic.
  • Test must be added to make sure ContentFIlteredTopic supported and functions.
• rcl and rclcpp
  • Action feedback and status to use ContentFilteredTopic.
  • Introduce new class for parameter_events filtering which calls ContentFilteredTopic.
  • Filtering in rcl based when rmw does not support ContentFilteredTopic.

@wjwwood and I were talking yesterday and realised that the default QoS profile for services might lead to undesired behaviour.

The default durability is RMW_QOS_POLICY_TRANSIENT_LOCAL_DURABILITY. @wjwwood pointed out that this was set before wait_for_service was implemented, as a way to compensate for race conditions in server/client bringup.

With this setting, in the case of a server that starts, dies, and restarts, it will receive the service request from the client twice. While the client is protected from receiving multiple service responses, the server is not protected from side effects of receiving multiple service calls.

If a service does anything like write to file, increment a counter, toggle a value, etc, the transient_local setting is probably not a good idea.

A better default might be volatile, and we should be careful about exposing the transient_local setting to users.

Feature request

Feature description

Similar to previously added events like ros2/ros2#822 or on liveliness lost, it would be good to add support to on sample lost events (which might be really helpful when debugging communication issues).

For ROS, on message lost might be a more appropriate name.

For DDS based rmw implementations, implementing this should be pretty straightforward:

• Connext: sample lost status, data reader listener callbacks.
• FastRTPS: AFAIS, it's not supported yet: https://github.com/eProsima/Fast-RTPS/blob/8842bdd35bac66b731fe3f2f5e79fe7417355c4b/include/fastdds/rtps/reader/ReaderListener.h#L41.
• CycloneDDS: https://github.com/eclipse-cyclonedds/cyclonedds/blob/623b35d83b9d061187c816f37f4a92798f3b35d1/src/core/ddsc/include/dds/ddsc/dds_public_listener.h#L37.

Detailed list of tasks: #226 (comment)

Implementation considerations

For rmw implementations that don't support the feature, they can return RMW_RET_UNSUPPORTED, as for example fastrtps already does when the creation of an ON_*_INCOMPATIBLE_QOS event is requested.

DDS also support a status callback called on sample rejected. The difference with the previous one, is that this one reflects a reader not having resources to receive the sample and not a communication problem. I think it would be interesting to add support to that one too, maybe both can be wrapped in the same event?

As of now checks for both the service names and topic names are same, but there should be separate length checks for service names as the service names are suffixed with a Request or a Reply in rmw_dds implementations which limits the actual length of the service names that can be used. Note: special consideration for rmw_connext implementation which suffixes the GUID of the participant to the service_name along with Reply. See: ros2/rmw_connext#285

Closed in favor of ros2/design#259 (comment)

Bug report

Required Info:

• Operating System:
  • Ubuntu 18.04
• Version or commit hash:
  • Crystal patch3 and master
• DDS implementation:
  • Fast-RTPS and OpenSplice
• Client library (if applicable):
  • rclcpp

Steps to reproduce issue

Run in different terminals

ros2 run examples_rclcpp_minimal_subscriber subscriber_lambda

ps aux | grep subscriber_lambda

ros2 run examples_rclcpp_minimal_client client_main

ps aux | grep subscriber_lambda

Expected behavior

The memory used by the subscriber_lambda process does not change while creating an additional node that does not share any pub/sub/client/service with it.

Actual behavior

The memory used by the subscriber_lambda process increases.

Before:
RSS: 21192 VRT: 433292

After:
RSS: 21452 VRT: 498828

The increase in memory is proportional to the number of pub/sub/client/service created in the new node.
For example, slightly changing the code for examples_rclcpp_minimal_client client_main in order to create 12 clients instead of only 1 causes the following:

After:
RSS: 21720 VRT: 498828

This behavior is observed with both Fast-RTPS 1.7.0, 1.7.2 and OpenSplice (version shipped with Crystal patch3).
Note that with OpenSplice, this should not happen if nodes are run in the same process.

Moreover, creating a FastRTPS example and simulating this behavior, there is only an increase in VRT.

Feature request

It looks like the RMW is allocating some memory whenever a new end-point is discovered.
However, this aggressive approach does not scale well (especially if every node contains a ParameterServer and a ParameterClient).

For example, in Fast-RTPS v1.7.0, considering a single process application with multiple nodes.
If we have 1 node and we add a second one, the memory overhead is approximately 10Mb.
If we have 20 nodes and we add a 21st one, the memory overhead is 23Mb.

I guess that the reason for this behavior is the following scenario:
Node A publishes to topic 1.
Node B is created and discovers Node A and its publisher.
Later, Node B creates a subscriber to topic 1 and it can immediately receive messages as everything is already set-up.

Feature request

Feature description

The DDS Security standard includes a Logging plugin that handles logging of security events to a special topic. This is implemented and available in RTI Connext, but currently must be enabled via source code, and the corresponding configuration is not exposed in any way by RMW. This should be changed to allow users to enable either DDS Security logging to the topic or a specified file through startup configuration.

Implementation considerations

One reasonable way to present this to the user would be to make this configurable via environment variables which, if set, can configure whether logging to the secure topic, to a file, or both are to be enabled. For example, we could have:

RMW_SECURITY_LOG_TO_TOPIC=true/false
RMW_SECURITY_LOG_TO_FILE= <path to file>

The default setting in the absence of an override should be to log to the topic, but not to a file.

Given that namespaces are supported by now, we should refactor the rmw interface to not only give one accumulated list of all topics, but rather distinguish them between publishers, subscribers, services and clients.

Meta-ticket to track the progress in exposing more QoS settings.
Please update the tasklist below when a setting has been decided to be exposed.

• [ ]
• [ ]

here the event-related function is declared in a isolated event.h header file, but it is not included in the rmw.h.
when we implement a rmw-xxx, we have to include another rmw/event.h, why the event is so special.

... currently only ensures that the history depth of the data reader and writer is at least the specified size.

While this allow the user to use vendor specific configuration to override the QoS parameters it prevents the user from setting a not-to-be-changed queue size.

We might want to consider offering both options to the developer and decide which behavior should be the default.

This issue tracks the progression of rmw to Quality Level 1 and a 1.0 version level. It follows the outline described in REP 2004.

• Version Policy
  • Follows ROS Core Quality Declaration
• Version >= 1.0.0
• Change Control Process
  • Follows ROS Core
• PRs automatically tested on Jenkins

Documentation

• Per-feature documentation.
  • Features still needing documentation on README.md or equivalent (endian.hpp, filesystem_helper.hpp, find_and_replace.hpp, find_library.hpp, join.hpp, split.hpp, pointer_traits.hpp, visibility_control.hpp)
• Per-item documentation in public API.
  • Items need doc blocks (endian.hpp, filesystem_helper.hpp, thread_safety_annotations.hpp, visibility_control.hpp)
  • Items needing doc improvements (join.hpp)
• Declared set of licenses
• Copyright statement in each source file
• Quality Declaration document (#205)

Testing:

• System tests, which cover "features" (#203)
• Tests covering public API (#203)
• Code coverage (with #203)
• Performance tests
• Performance test policy: ROS Core
• Package code style: ROS Core
• Uses ament_lint_common

Dependencies:

• Runtime "ROS" dependencies are level 1
  • rcutils
• non-ROS dependencies are equivalent level 1
  • c++ standard library

Platform Support

• Supports all tier 1 platforms

Feature request

Feature description

In the presence of multiple publishers, with potentially different QoS settings, it is useful to automatically setup a subscription with a QoS that is compatible with the majority (if not all) of the publishers. Presumably, what it means for two QoS profiles to be compatible depends on the RMW implementation. Therefore, it makes sense to offer an API that, given one or more endpoints, returns a QoS profile that is compatible with the most endpoints.

Immediately useful in the following tools:

• ros2topic for choosing QoS based on existing publishers/subscriptions (ros2/ros2cli#594)
• rosbag2 for choosing QoS for recording and playback
• ros1_bridge (for example, ros2/ros1_bridge#304)
• domain_bridge for bridging multiple publishers on the same topic

Implementation considerations

Given the non-symmetric nature of publisher and subscription QoS policies, it may make sense to add two API's: one for getting a publisher QoS and one for getting a subscription QoS.

There already exists an implementation for DDS RMWs in rosbag2, see related issue about moving logic to a common place: ros2/rosbag2#601

I'm proposing we actually move the logic to the RMW layer, since it may vary depending on the RMW implementation.
Perhaps moving this logic to RMW is overkill, as the existing QoS policies are fairly specific to DDS. Alternatively, we could hard-code the compatibility rules in rcl.

Feature request

Feature description

A couple of destruction methods have signatures that make it impossible to obtain an allocator from an active context, which means that any of the members those methods are meant to deallocate cannot use a user defined allocator to do so.

An example is the rmw_destroy_guard_condition

I suspect there are a couple more, such as rmw_destroy_wait_set, etc.

This could be done by either sticking the context or node pointer in the relevant structs, or passing it into the method like how rmw_destroy_publisher does it. Though this might be API breaking for all existing rmw implementations.

Implementation considerations

On the other hand, I'm not sure if the recommended way to allocate or deallocate dynamic memory is via rmw_allocate and rmw_free. It looks like those just boil down to a malloc and free/delete call.

I presume that using allocators (that is, rcutils_allocator_t) would be preferable?

Currently, the macro RMW_CHECK_TYPE_IDENTIFIERS_MATCH has an error message of the format:

" implementation '%s'(%p) does not match rmw implementation '%s'(%p)"

However, in the rmw implementations this macro is used for checking the types of typesupport identifiers, not only rmw implementation identifiers, leading to confusing and incorrect error messages such as:

could not create publisher: type support handle implementation 'connext_static'(0x2b54f95e9671) does not match rmw implementation 'introspection'(0x2b54f97eb691)

This macro should be reworded to be more general, or a different macro should be made for matching typesupport identifiers.

Nodes are creating with a name as well as a namespace. The function rmw_create_node only returns the name though. It should include the namespace as well - either as separate return values or as a combined value.

Follow up of #95.

This is not an issue but rather a question about ROS2 & FastRTPS.

https://answers.ros.org/question/358361/fastrtps-communication-over-other-network/

Bug report

Seeing a heap over-read reported by ASAN when running cartographer or navigation on MacOS (compiled with ASAN enabled)

Required Info:

• Operating System:
  • MacOS 10.16.4
• Installation type:
  • compiled from source
• Version or commit hash:
  • latest dashing .repos file
• DDS implementation:
  • Fast-RTPS
• Client library (if applicable):
  • reproduces in cartographer and navigation modules, multiple nodes

Steps to reproduce issue

1. enable ASAN by adding below to cartographer and navigation CMakeLists.txt files

   set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer -fsanitize=address")
   set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_LINKER_FLAGS_DEBUG} -fno-omit-frame-pointer -fsanitize=address")

2. recompile cartographer, navigation (didn't try recompiling all of ROS with ASAN, not sure if that's a mistake)

   colcon build --cmake-args -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Debug -DFORCE_DEBUG_BUILD=True --symlink-install

3. run cartographer, navigation as normal

4. observe ASAN flagging 8-byte heap over-read

Expected behavior

No traps from ASAN

Actual behavior

ASAN flags over-read and dies

Additional information

I've reviewed a few different nodes, and they all have some form of this stack trace, where a node (multiple of them are killed by ASAN) is doing something that invokes rmw::impl::cpp:parse_key_value()

[occupancy_grid_node-2] =================================================================
[occupancy_grid_node-2] ==37032==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000042d5 at pc 0x0001034d46df bp 0x70000b8f0c30 sp 0x70000b8f03e0
[occupancy_grid_node-2] READ of size 8 at 0x6020000042d5 thread T3
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] 0 0x1034d46de in __asan_memcpy (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x596de)
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] 1 0x100906a8c in std::__1::enable_if<((std::__1::integral_constant<bool, true>::value) || (!(__has_construct<std::__1::allocator, bool*, bool>::value))) && (is_trivially_move_constructible::value), void>::type std::__1::allocator_traits<std::__1::allocator >::__construct_backward(std::__1::allocator&, bool*, bool*, bool*&) memory:1699
[occupancy_grid_node-2] 2 0x10090550a in std::__1::vector<unsigned char, std::__1::allocator >::__swap_out_circular_buffer(std::__1::__split_buffer<unsigned char, std::__1::allocator&>&) vector:931
[occupancy_grid_node-2] 3 0x101aee75a in void std::__1::vector<unsigned char, std::__1::allocator >::__push_back_slow_path<unsigned char const&>(unsigned char const&&&) (librclcpp.dylib:x86_64+0xdc75a)
[occupancy_grid_node-2] 4 0x111460d93 in std::__1::vector<unsigned char, std::__1::allocator >::push_back(unsigned char const&) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x21d93)
[occupancy_grid_node-2] 5 0x11146005a in rmw::impl::cpp::parse_key_value(std::__1::vector<unsigned char, std::__1::allocator >) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x2105a)
[occupancy_grid_node-2] 6 0x11145d855 in ParticipantListener::onParticipantDiscovery(eprosima::fastrtps::Participant*, eprosima::fastrtps::rtps::ParticipantDiscoveryInfo&&) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x1e855)
[occupancy_grid_node-2] 7 0x103057aa6 in eprosima::fastrtps::rtps::PDPSimpleListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) (libfastrtps.1.dylib:x86_64+0x1c8aa6)
[occupancy_grid_node-2] 8 0x102ee419b in eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) (libfastrtps.1.dylib:x86_64+0x5519b)
[occupancy_grid_node-2] 9 0x102ee4372 in eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) (libfastrtps.1.dylib:x86_64+0x55372)
[occupancy_grid_node-2] 10 0x102ef3a8c in eprosima::fastrtps::rtps::MessageReceiver::proc_Submsg_Data(eprosima::fastrtps::rtps::CDRMessage_t*, eprosima::fastrtps::rtps::SubmessageHeader_t*) (libfastrtps.1.dylib:x86_64+0x64a8c)
[occupancy_grid_node-2] 11 0x102ef1869 in eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) (libfastrtps.1.dylib:x86_64+0x62869)
[occupancy_grid_node-2] 12 0x102efaaac in eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) (libfastrtps.1.dylib:x86_64+0x6baac)
[occupancy_grid_node-2] 13 0x102f57a91 in eprosima::fastrtps::rtps::UDPTransportInterface::perform_listen_operation(eprosima::fastrtps::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t) (libfastrtps.1.dylib:x86_64+0xc8a91)
[occupancy_grid_node-2] 14 0x102f5b402 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, void (eprosima::fastrtps::rtps::UDPTransportInterface::)(eprosima::fastrtps::rtps::UDPChannelResource, eprosima::fastrtps::rtps::Locator_t), eprosima::fastrtps::rtps::UDPTransportInterface*, eprosima::fastrtps::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >(void*) (libfastrtps.1.dylib:x86_64+0xcc402)
[occupancy_grid_node-2] 15 0x7fff5c05a2ea in _pthread_body (libsystem_pthread.dylib:x86_64+0x32ea)
[occupancy_grid_node-2] 16 0x7fff5c05d248 in _pthread_start (libsystem_pthread.dylib:x86_64+0x6248)
[occupancy_grid_node-2] 17 0x7fff5c05940c in thread_start (libsystem_pthread.dylib:x86_64+0x240c)
[occupancy_grid_node-2]
[occupancy_grid_node-2] 0x6020000042d8 is located 0 bytes to the right of 8-byte region [0x6020000042d0,0x6020000042d8)
[occupancy_grid_node-2] allocated by thread T3 here:
[occupancy_grid_node-2] 0 0x1034e3b92 in wrap__Znwm (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x68b92)
[occupancy_grid_node-2] 1 0x101ac4e88 in std::__1::__libcpp_allocate(unsigned long, unsigned long) (librclcpp.dylib:x86_64+0xb2e88)
[occupancy_grid_node-2] 2 0x101aca084 in std::__1::allocator::allocate(unsigned long, void const*) (librclcpp.dylib:x86_64+0xb8084)
[occupancy_grid_node-2] 3 0x101ac9ec0 in std::__1::allocator_traits<std::__1::allocator >::allocate(std::__1::allocator&, unsigned long) (librclcpp.dylib:x86_64+0xb7ec0)
[occupancy_grid_node-2] 4 0x101aedf89 in std::__1::__split_buffer<unsigned char, std::__1::allocator&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator&) (librclcpp.dylib:x86_64+0xdbf89)
[occupancy_grid_node-2] 5 0x101aede0c in std::__1::__split_buffer<unsigned char, std::__1::allocator&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator&) (librclcpp.dylib:x86_64+0xdbe0c)
[occupancy_grid_node-2] 6 0x101aee70c in void std::__1::vector<unsigned char, std::__1::allocator >::__push_back_slow_path<unsigned char const&>(unsigned char const&&&) (librclcpp.dylib:x86_64+0xdc70c)
[occupancy_grid_node-2] 7 0x111460d93 in std::__1::vector<unsigned char, std::__1::allocator >::push_back(unsigned char const&) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x21d93)
[occupancy_grid_node-2] 8 0x11146005a in rmw::impl::cpp::parse_key_value(std::__1::vector<unsigned char, std::__1::allocator >) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x2105a)
[occupancy_grid_node-2] 9 0x11145d855 in ParticipantListener::onParticipantDiscovery(eprosima::fastrtps::Participant*, eprosima::fastrtps::rtps::ParticipantDiscoveryInfo&&) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x1e855)
[occupancy_grid_node-2] [WARN] [occupancy_grid_node]: submap_slices and last_frame_id is empty
[occupancy_grid_node-2] 10 0x103057aa6 in eprosima::fastrtps::rtps::PDPSimpleListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) (libfastrtps.1.dylib:x86_64+0x1c8aa6)
[occupancy_grid_node-2] 11 0x102ee419b in eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) (libfastrtps.1.dylib:x86_64+0x5519b)
[occupancy_grid_node-2] 12 0x102ee4372 in eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) (libfastrtps.1.dylib:x86_64+0x55372)
[occupancy_grid_node-2] 13 0x102ef3a8c in eprosima::fastrtps::rtps::MessageReceiver::proc_Submsg_Data(eprosima::fastrtps::rtps::CDRMessage_t*, eprosima::fastrtps::rtps::SubmessageHeader_t*) (libfastrtps.1.dylib:x86_64+0x64a8c)
[occupancy_grid_node-2] 14 0x102ef1869 in eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) (libfastrtps.1.dylib:x86_64+0x62869)
[occupancy_grid_node-2] 15 0x102efaaac in eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) (libfastrtps.1.dylib:x86_64+0x6baac)
[occupancy_grid_node-2] 16 0x102f57a91 in eprosima::fastrtps::rtps::UDPTransportInterface::perform_listen_operation(eprosima::fastrtps::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t) (libfastrtps.1.dylib:x86_64+0xc8a91)
[occupancy_grid_node-2] 17 0x102f5b402 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_deletestd::__1::__thread_struct >, void (eprosima::fastrtps::rtps::UDPTransportInterface::)(eprosima::fastrtps::rtps::UDPChannelResource, eprosima::fastrtps::rtps::Locator_t), eprosima::fastrtps::rtps::UDPTransportInterface*, eprosima::fastrtps::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >(void*) (libfastrtps.1.dylib:x86_64+0xcc402)
[occupancy_grid_node-2] 18 0x7fff5c05a2ea in _pthread_body (libsystem_pthread.dylib:x86_64+0x32ea)
[occupancy_grid_node-2] 19 0x7fff5c05d248 in _pthread_start (libsystem_pthread.dylib:x86_64+0x6248)
[occupancy_grid_node-2] 20 0x7fff5c05940c in thread_start (libsystem_pthread.dylib:x86_64+0x240c)
[occupancy_grid_node-2]
[occupancy_grid_node-2] Thread T3 created by T0 here:
[occupancy_grid_node-2] 0 0x1034cf02d in wrap_pthread_create (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x5402d)
[occupancy_grid_node-2] 1 0x102f5b30d in std::__1::thread::thread<void (eprosima::fastrtps::rtps::UDPTransportInterface::)(eprosima::fastrtps::rtps::UDPChannelResource, eprosima::fastrtps::rtps::Locator_t), eprosima::fastrtps::rtps::UDPTransportInterface*, eprosima::fastrtps::rtps::UDPChannelResource*&, eprosima::fastrtps::rtps::Locator_t const&, void>(void (eprosima::fastrtps::rtps::UDPTransportInterface::&&)(eprosima::fastrtps::rtps::UDPChannelResource, eprosima::fastrtps::rtps::Locator_t), eprosima::fastrtps::rtps::UDPTransportInterface*&&, eprosima::fastrtps::rtps::UDPChannelResource*&&&, eprosima::fastrtps::rtps::Locator_t const&&&) (libfastrtps.1.dylib:x86_64+0xcc30d)
[occupancy_grid_node-2] 2 0x102f578e6 in eprosima::fastrtps::rtps::UDPTransportInterface::CreateInputChannelResource(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, eprosima::fastrtps::rtps::Locator_t const&, bool, unsigned int, eprosima::fastrtps::rtps::TransportReceiverInterface*) (libfastrtps.1.dylib:x86_64+0xc88e6)
[occupancy_grid_node-2] 3 0x102f5748a in eprosima::fastrtps::rtps::UDPTransportInterface::OpenAndBindInputSockets(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::TransportReceiverInterface*, bool, unsigned int) (libfastrtps.1.dylib:x86_64+0xc848a)
[occupancy_grid_node-2] 4 0x102f40671 in eprosima::fastrtps::rtps::UDPv4Transport::OpenInputChannel(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::TransportReceiverInterface*, unsigned int) (libfastrtps.1.dylib:x86_64+0xb1671)
[occupancy_grid_node-2] 5 0x102efa214 in eprosima::fastrtps::rtps::ReceiverResource::ReceiverResource(eprosima::fastrtps::rtps::TransportInterface&, eprosima::fastrtps::rtps::Locator_t const&, unsigned int) (libfastrtps.1.dylib:x86_64+0x6b214)
[occupancy_grid_node-2] 6 0x102ef85f9 in eprosima::fastrtps::rtps::NetworkFactory::BuildReceiverResources(eprosima::fastrtps::rtps::Locator_t&, unsigned int, std::__1::vector<std::__1::shared_ptreprosima::fastrtps::rtps::ReceiverResource, std::__1::allocator<std::__1::shared_ptreprosima::fastrtps::rtps::ReceiverResource > >&) (libfastrtps.1.dylib:x86_64+0x695f9)
[occupancy_grid_node-2] 7 0x102efde05 in eprosima::fastrtps::rtps::RTPSParticipantImpl::createReceiverResources(eprosima::fastrtps::rtps::LocatorList_t&, bool) (libfastrtps.1.dylib:x86_64+0x6ee05)
[occupancy_grid_node-2] 8 0x102efd0e7 in eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) (libfastrtps.1.dylib:x86_64+0x6e0e7)
[occupancy_grid_node-2] 9 0x102f05a2e in eprosima::fastrtps::rtps::RTPSDomain::createParticipant(eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) (libfastrtps.1.dylib:x86_64+0x76a2e)
[occupancy_grid_node-2] 10 0x102f08882 in eprosima::fastrtps::Domain::createParticipant(eprosima::fastrtps::ParticipantAttributes const&, eprosima::fastrtps::ParticipantListener*) (libfastrtps.1.dylib:x86_64+0x79882)
[occupancy_grid_node-2] 11 0x11145c087 in rmw_fastrtps_shared_cpp::create_node(char const*, char const*, char const*, eprosima::fastrtps::ParticipantAttributes) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x1d087)
[occupancy_grid_node-2] 12 0x11145cf07 in rmw_fastrtps_shared_cpp::__rmw_create_node(char const*, char const*, char const*, unsigned long, rmw_node_security_options_t const*) (librmw_fastrtps_shared_cpp.dylib:x86_64+0x1df07)
[occupancy_grid_node-2] 13 0x1113f2de3 in rmw_create_node (librmw_fastrtps_cpp.dylib:x86_64+0x1cde3)
[occupancy_grid_node-2] 14 0x1021f37fc in rmw_create_node (librmw_implementation.dylib:x86_64+0x47fc)
[occupancy_grid_node-2] 15 0x102077874 in rcl_node_init (librcl.dylib:x86_64+0x19874)
[occupancy_grid_node-2] 16 0x101aa2c75 in rclcpp::node_interfaces::NodeBase::NodeBase(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::shared_ptrrclcpp::Context, rcl_node_options_t const&, bool) (librclcpp.dylib:x86_64+0x90c75)
[occupancy_grid_node-2] 17 0x101aa37fd in rclcpp::node_interfaces::NodeBase::NodeBase(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::shared_ptrrclcpp::Context, rcl_node_options_t const&, bool) (librclcpp.dylib:x86_64+0x917fd)
[occupancy_grid_node-2] 18 0x101a87440 in rclcpp::Node::Node(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, rclcpp::NodeOptions const&) (librclcpp.dylib:x86_64+0x75440)
[occupancy_grid_node-2] 19 0x101a872b2 in rclcpp::Node::Node(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, rclcpp::NodeOptions const&) (librclcpp.dylib:x86_64+0x752b2)
[occupancy_grid_node-2] 20 0x10083da58 in cartographer_ros::(anonymous namespace)::OccupancyGridNode::OccupancyGridNode(double, double) occupancy_grid_node_main.cc:56
[occupancy_grid_node-2] 21 0x10083d458 in cartographer_ros::(anonymous namespace)::OccupancyGridNode::OccupancyGridNode(double, double) occupancy_grid_node_main.cc:58
[occupancy_grid_node-2] 22 0x10083d311 in std::__1::__compressed_pair_elem<cartographer_ros::(anonymous namespace)::OccupancyGridNode, 1, false>::__compressed_pair_elem<double&, double&, 0ul, 1ul>(std::__1::piecewise_construct_t, std::__1::tuple<double&, double&>, std::__1::__tuple_indices<0ul, 1ul>) memory:2153
[occupancy_grid_node-2] 23 0x10083cec3 in std::__1::__compressed_pair<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>, cartographer_ros::(anonymous namespace)::OccupancyGridNode>::__compressed_pair<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>&, double&, double&>(std::__1::piecewise_construct_t, std::__1::tuple<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>&>, std::__1::tuple<double&, double&>) memory:2255
[occupancy_grid_node-2] 24 0x10083c4f3 in std::__1::__compressed_pair<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>, cartographer_ros::(anonymous namespace)::OccupancyGridNode>::__compressed_pair<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>&, double&, double&>(std::__1::piecewise_construct_t, std::__1::tuple<std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>&>, std::__1::tuple<double&, double&>) memory:2256
[occupancy_grid_node-2] 25 0x10083bd35 in std::__1::__shared_ptr_emplace<cartographer_ros::(anonymous namespace)::OccupancyGridNode, std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode> >::__shared_ptr_emplace<double&, double&>(std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>, double&&&, double&&&) memory:3668
[occupancy_grid_node-2] 26 0x10083b244 in std::__1::__shared_ptr_emplace<cartographer_ros::(anonymous namespace)::OccupancyGridNode, std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode> >::__shared_ptr_emplace<double&, double&>(std::__1::allocator<cartographer_ros::(anonymous namespace)::OccupancyGridNode>, double&&&, double&&&) memory:3669
[occupancy_grid_node-2] 27 0x10083adb6 in std::__1::shared_ptr<cartographer_ros::(anonymous namespace)::OccupancyGridNode> std::__1::shared_ptr<cartographer_ros::(anonymous namespace)::OccupancyGridNode>::make_shared<double&, double&>(double&&&, double&&&) memory:4327
[occupancy_grid_node-2] 28 0x10082b9a3 in std::__1::enable_if<!(is_array<cartographer_ros::(anonymous namespace)::OccupancyGridNode>::value), std::__1::shared_ptr<cartographer_ros::(anonymous namespace)::OccupancyGridNode> >::type std::__1::make_shared<cartographer_ros::(anonymous namespace)::OccupancyGridNode, double&, double&>(double&&&, double&&&) memory:4706
[occupancy_grid_node-2] 29 0x10082b659 in main occupancy_grid_node_main.cc:193
[occupancy_grid_node-2] 30 0x7fff5be663d4 in start (libdyld.dylib:x86_64+0x163d4)
[occupancy_grid_node-2]
[occupancy_grid_node-2] SUMMARY: AddressSanitizer: heap-buffer-overflow (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x596de) in __asan_memcpy
[occupancy_grid_node-2] Shadow bytes around the buggy address:
[occupancy_grid_node-2] 0x1c0400000800: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 fa
[occupancy_grid_node-2] 0x1c0400000810: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa 00 00
[occupancy_grid_node-2] 0x1c0400000820: fa fa 00 00 fa fa 00 fa fa fa 00 00 fa fa 00 00
[occupancy_grid_node-2] 0x1c0400000830: fa fa 00 fa fa fa 00 00 fa fa 00 00 fa fa 00 fa
[occupancy_grid_node-2] 0x1c0400000840: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa fd fa
[occupancy_grid_node-2] =>0x1c0400000850: fa fa fd fa fa fa fd fa fa fa[05]fa fa fa 00 00
[occupancy_grid_node-2] 0x1c0400000860: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[occupancy_grid_node-2] 0x1c0400000870: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[occupancy_grid_node-2] 0x1c0400000880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[occupancy_grid_node-2] 0x1c0400000890: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[occupancy_grid_node-2] 0x1c04000008a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[occupancy_grid_node-2] Shadow byte legend (one shadow byte represents 8 application bytes):
[occupancy_grid_node-2] Addressable: 00
[occupancy_grid_node-2] Partially addressable: 01 02 03 04 05 06 07
[occupancy_grid_node-2] Heap left redzone: fa
[occupancy_grid_node-2] Freed heap region: fd
[occupancy_grid_node-2] Stack left redzone: f1
[occupancy_grid_node-2] Stack mid redzone: f2
[occupancy_grid_node-2] Stack right redzone: f3
[occupancy_grid_node-2] Stack after return: f5
[occupancy_grid_node-2] Stack use after scope: f8
[occupancy_grid_node-2] Global redzone: f9
[occupancy_grid_node-2] Global init order: f6
[occupancy_grid_node-2] Poisoned by user: f7
[occupancy_grid_node-2] Container overflow: fc
[occupancy_grid_node-2] Array cookie: ac
[occupancy_grid_node-2] Intra object redzone: bb
[occupancy_grid_node-2] ASan internal: fe
[occupancy_grid_node-2] Left alloca redzone: ca
[occupancy_grid_node-2] Right alloca redzone: cb
[occupancy_grid_node-2] Shadow gap: cc
[occupancy_grid_node-2] ==37032==ABORTING

Bug report

Required Info:

• Operating System:
  • All platforms
• Installation type:
  • Source (ci.ros2.org)
• Version or commit hash:
  • vcs export --exact
• DDS implementation:
  • N/A
• Client library (if applicable):
  • N/A

Steps to reproduce issue

colcon build --packages-up-to rmw
colcon test --packages-up-to rmw

Expected behavior

Tests on test/test_names_and_types.cpp:131 and test/test_names_and_types.cpp:137 pass.

Actual behavior

13: [ RUN      ] rmw_names_and_types.rmw_names_and_types_fini
13: /home/jenkins-agent/workspace/nightly_linux-aarch64_debug/ws/src/ros2/rmw/rmw/test/test_names_and_types.cpp:131: Failure
13: Expected equality of these values:
13:   rmw_names_and_types_fini(&names_and_types)
13:     Which is: 0
13:   11
13: /home/jenkins-agent/workspace/nightly_linux-aarch64_debug/ws/src/ros2/rmw/rmw/test/test_names_and_types.cpp:137: Failure
13: Expected equality of these values:
13:   rmw_names_and_types_fini(&names_and_types)
13:     Which is: 0
13:   11
13: [  FAILED  ] rmw_names_and_types.rmw_names_and_types_fini (0 ms)

Additional information

I was looking at vcs exact output between today and yesterday and I have no inkling as to why these tests which were passing yesterday are failing today. These should be shot for shot tests of code here and here.

I'm trying to determine roughly what the diff is between QoS functionality available via rmw interfaces vs dds itself. From a read through rmw/{types, qos_profiles, rmw}.h it looks like there are some diffs:

• reliability: seems to just specify reliable/best effort, could not find mention of things like ordering, max_blocking_time
• ownership: with respect to ability of DataWriters to write to a topic/object, could not find mention of this
• liveliness: determine writer is present independent of rxed data

i think there are others as well, but wanted to check to see what is intended/planned.

Feature request

Feature description

Every now and then there is a change to the RMW interface that necessitates updating all the RMW implementations — for example #187, which gives the application a way to control options offered by the underlying implementation that are not otherwise exposed in the regular RCL interfaces. This then obviously results in the updated RMW implementations no longer being compatible with older versions of ROS2.

Having a version number for the RMW interface allows an implementation to be compatible with a larger range of ROS2 versions using a single source version.

One could also do the same in reverse: let RMW implementations advertise the version they implement, then provide compatibility with older RMW implementations where possible with a small effort. (#187 would have been pretty trivial.) The more different implementations of the RMW interface exist outside the ROS2 sources, the more valuable this becomes.

Implementation considerations

In my view the implementation could be as simple as defining an RMW_VERSION macro in rmw.h, expanding to an integer that gets incremented whenever there is an interface change. That would have a minimal impact on managing the RMW interface but would give RMW implementers options.

While letting RMW implementations advertise the interface version they implement and providing compatibility with older RMW implementations would definitely be very nice for the RMW implementers, it would also require some real effort in maintaining the RMW interface layer. That might be a bit too much to ask for 😄 .

Bug report

rmw / fastrtps appears to throw an assert when another node becomes unreachable. This makes for some hard to notice "hmmm why did it stop working?" scenarios

Required Info:

• Operating System:
  • Ubuntu 18.04 server (arm64) running on Raspberry Pi
• Installation type:
  • binaries from ros-dashing-ros-base
• Version or commit hash:
  • dashing
• DDS implementation:
  • Fast-RTPS
• Client library (if applicable):
  • rclcpp???

Steps to reproduce issue

Start TB3 latest package which is split between three CPU's - wheel control on OpenCR, robot state publisher on Raspberry Pi, and rviz / main workload processing on Mac or Ubuntu PC

Debugging some issue that I saw when running rviz on Ubuntu, wanted to switch over to Mac and see if maybe that issue stopped happening. Shut down Ubuntu, and the instance of rmw/fast-RTPS throws an exception while the Ubuntu side is shutting down.

Unfortunately, I don't notice the exception on Raspberry Pi side until several minutes later when I'm scratching my head trying to figure out why I can't run basic teleop on Mac anymore... oh yeah, rmw threw an error and my bot is basically dead.

Expected behavior

if one node goes down, the other nodes shouldn't crash and burn

Actual behavior

crashing and burning

Additional information

[robot_state_publisher-1] terminate called after throwing an instance of 'rclcpp::exceptions::RCLError'
[robot_state_publisher-1] what(): failed to publish message: cannot publish data, at /tmp/binarydeb/ros-dashing-rmw-fastrtps-shared-cpp-0.7.5/src/rmw_publish.cpp:52, at /tmp/binarydeb/ros-dashing-rcl-0.7.6/src/rcl/publisher.c:257

Bug report

Required Info:

• Operating System:
  • Debian and Ubuntu
• Installation type:
  • From source
• Version or commit hash:
  • Eloquent and Foxy HEAD
• DDS implementation:
  • Fast-RTPS and CycloneDDS
• Client library (if applicable):
  • Tested with rclpy, problem is based in RMW

Steps to reproduce issue

The fast (1kHz) publisher and slow (10Hz) subscriber
https://gist.github.com/cleitner/93decfa79a99a8a3b59a795df02b99e7
https://gist.github.com/cleitner/e5eb35f6bcf6639425c06c03dd13fbff
expose problems with buffer bloat and, more relevant here, issues with the rmw_publish interface.

As described in #176, the RMW implementations can cause rmw_publish to block when history is set to KEEP_ALL

Calling the two scripts with

$ ./pub_back_pressure.py
$ ./sub_back_pressure.py

will sometimes exhibit blocking times > 20ms in rmw_publish and more often RMW_RET_ERROR when the internal RESOURCE_LIMITS setting is hit.

...
Failed to publish at 10032 with Failed to publish: cannot publish data, at .../src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_publish.cpp:53, at .../src/ros2/rcl/rcl/src/rcl/publisher.c:290
Failed to publish at 10032 with Failed to publish: cannot publish data, at .../src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_publish.cpp:53, at .../src/ros2/rcl/rcl/src/rcl/publisher.c:290
Slowed down at 10032 for 41.456 ms
Slowed down at 10035 for 101.639 ms
Failed to publish at 10039 with Failed to publish: cannot publish data, at .../src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_publish.cpp:53, at .../src/ros2/rcl/rcl/src/rcl/publisher.c:290
Failed to publish at 10032 with Failed to publish: cannot publish data, at .../src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_publish.cpp:53, at .../src/ros2/rcl/rcl/src/rcl/publisher.c:290
...

With

$ RMW_IMPLEMENTATION=rmw_cyclonedds_cpp ./pub_back_pressure.py
$ RMW_IMPLEMENTATION=rmw_cyclonedds_cpp ./sub_back_pressure.py

the publisher blocks with (seemingly) random timeouts, but no errors:

...
Slowed down at 3205 for 21.131 ms
Slowed down at 3207 for 21.206 ms
Slowed down at 3209 for 209.976 ms
Slowed down at 3210 for 275.831 ms
Slowed down at 3211 for 173.350 ms
Slowed down at 3212 for 97.646 ms
Slowed down at 3213 for 43.757 ms
...

The subscriber crashes with OoM because CycloneDDS doesn't seem to limit the incoming buffer.

Expected behavior

At least consistent behavior regarding returning errors or blocking.

The IMHO correct result would be an indication of a need for blocking, akin to the taken parameter to rmw_take to allow pure polling operation of rmw_publish, i.e. in cyclic realtime context.

No crashing in the subscriber with default settings.

Actual behavior

rmw_publish blocks when some unknown internal (and external) limit is hit. Fast-RTPS also returns RMW_RET_ERROR, without having experiences a real error.

CycloneDDS subscriber can be crashed with OoM if the publisher exceeds the subscribers processing bandwidth.

Additional information

The write function of the DDS DataWriter interface has a OUT_OF_RESOURCES return code and specifies a max_block_time parameter for the RELIABILITY QoS, which would preferrably be set to 0.

The interface of rmw_publish could be made symmetric to rmw_take:

rmw_ret_t
rmw_publish(
  const rmw_publisher_t * publisher,
  const void * ros_message,
  bool * published,
  rmw_publisher_allocation_t * allocation);

Feature request

Initial context: ros2/rosbag2#773

Feature description

Services can't be recorded and replayed. This prevents actions from being replayed which is an important feature for offline debugging/testing.

CC: @emersonknapp, @Karsten1987

Some of the include guards are still using ROS_MIDDLEWARE_INTERFACE rather than RMW.

Feature request

• Allow users to define a static topology during rmw_init, such that the system will allocate only as many resources as specified by the user.

Motivation

• Currently, ROS2 does not support the use of a static topology: the user gets to specificy exactly how many publishers, subscribers, topics, etc. a node needs, and thus the system allocates for exactly that many -- no more, no less. While Fast-RTPS and RTI Connext Pro do not require this feature, since they allow for dynamic allocation, I foresee that with the introduction of DDS-XCRE and the use of ROS in embedded devices, we will eventually have the need to statically allocate resources during initialization.
• While it is possible to over-allocate resources, this solution is obviously not ideal (especially if you are working in a low-resource device).

Feature description

• We see two ways to implement this: 1) creating rmw_pre_init() and rmw_post_init() functions or 2) Passing a parameter to rmw_init that would be used to specify the topology.
  e.g,:

• Option 1

rmw_ret_t rmw_pre_init(rmw_init_options_t * context_ptr) //not necessarily void *
{
  //pre-init stuff here
  return RMW_RET_OK;
}

rmw_ret_t rmw_post_init(rmw_init_options_t * context_ptr) //not necessarily void *
{
  // post-init stuff here
  return RMW_RET_OK;
}

int main(void)
{
  rmw_init_options_t * context_ptr = ...; 
  rmw_ret_t retval = rmw_pre_init(context_ptr);
  // Check for and handle errors
  retval = rmw_init();
  if (ret == RMW_RET_OK) {
    retval = rmw_post_init(context_ptr);
    // Check for and handle errors
  } else {
    // error handling...
  }

  return 0;
}

• Option 2

rmw_ret_t entry_callback(void * caller_context_ptr)
{
  // do pre-init stuff
  return RMW_RET_OK;
}

rmw_ret_t exit_callback(void * caller_context_ptr, rmw_ret_t current_rmw_init_status)
{
  // do post-init stuff
  return RMW_RET_OK;
}

int main(void)
{
  struct rmw_init_callbacks_t init_callbacks {
    .m_caller_context_ptr = NULL,
    .m_callback_at_rmw_init_entry = &entry_callback,
    .m_callback_at_rmw_init_exit = &exit_callback
  };

  rmw_ret_t ret = rmw_init(init_callbacks); // new rmw_init
  // error handling...

  return 0;
}

rmw_ret_t rmw_init(const rmw_init_callbacks_t *  callback_ptr) // new rmw_init
{
  rmw_ret_t retval = RMW_RET_ERROR;
  retval = callback_ptr->m_callback_at_rmw_init_entry(callback_ptr->m_caller_context_ptr);
  // check for and handle errors
  retval = old_rmw_init(); //old rmw_init
  // check for and handle errors 
  retval = callback_ptr->m_callback_at_rmw_init_exit(callback_ptr->m_caller_context_ptr, retval);
  // check for and handle errors
  return retval
}

** Credit to @wjwwood @serge-nikulin for the original examples (which I may have bastardized when I elaborated on them)

Implementation considerations

• Option 1

  • Pros;
    1. Backwards-compatible (if you haven't needed to do any pre-init or post-init steps, your code won't have to change).
  • Cons:
    1. User code written initially for a dynamic topology will require some changes before it is compatible with the static DDS implementation.

• Option 2

  • Pros:
    1. When switching from a dynamic to a static topology, the user code will not have to change since the rmw implementation will handle the static allocation of resources. If the users have an idea of what resources are needed, they will be able to specify this in a configuration file, and if they don't, then there can be a default over-allocating configuration that can later be tuned for the specific use-case.
  • Cons:
    1. Not backwards-compatible.

• At a high-level for the static topology, you would need to specify a graph of how your nodes are connected and what kind of resources they need to communicate (on an edge-by-edge basis). e.g., :

  • Number of nodes
  • Per node:
    • Number of publishers
    • Number of subscribers
    • Topics
  • Per publisher/subscriber:
    • Number of Writers/Readers
    • Number of samples
  • Routes between nodes
  • QoS settings?

@dejanpan @dirk-thomas @wjwwood @serge-nikulin FYI

The current API only passes a topic name which can only return the number of publishers / subscribers independently if they are already matched (due to timing) or will never match (due to incompatible QoS settings).

The new API needs to be coupled to a publisher / subscriber handle since it needs that context to decide if another entity is actually matching this entities QoS.