Fallback content filtering

[fujitatomoya]

Feature request

Background

Content Filtering Subscription is new feature introduced in Humble, see Content-Filtering-Subscription
Currently Content Filtering Subscription depends on RMW implementation, so that if underlying RMW implementation does not support Content Filtering Subscription interfaces, user application cannot use this feature in ROS 2.
This could be problem for the application cz its behavior could not be consistent once RMW implementation is changed.
The application can still manage this situation in user space, but this would be burden for user application code.
Besides, subscriptions are the endpoints that we use mostly in application.

This has been discussed on ros2/design#282 (comment)

Feature description

We will provide fallback filtering function in rcl or else where that DOES NOT depend on rmw implementation.
Fallback filtering will conceal the rmw dependency from user perspective, so that user application can get rid of if rmw does not support code from the user space.

Implementation considerations

• draft design is introduced ContentFilteredTopic Design Proposal. design#282 (comment) and ContentFilteredTopic Design Proposal. design#282 (comment)
• implement concrete functions into rmw_dds_common or other c++ package, and export C interface to rcl. So that typesupport_cpp problem can be resolved. And to address SQL parser and tokenizer, we can depend on PEGTL. (external API changes may be required.)

Performance

Fallback filtering will be only Reader-Filtering.I
It always runs executor to take out the message from rmw, and then de-serialize the message to see what's in the message for filtering.
This said that there will be no performance improvement compared to filtering in user application.
Reader-Filtering in rmw is different from above, since there is no need to take out the data using executor, besides it does not need to de-serialize the message before filtering. This is achieved by TypeObject in Fast-DDS and TypeCode in RTI Connext DDS.

Tasks

• Basic Design (already proposed ContentFilteredTopic Design Proposal. design#282 (comment))
• ROS 2 filtering expressions and parameters (could be subset of DDS spec, but needs to be discussed.)
• Implementation (rcl and other repos. such as tokenizer, type support c/c++, client libraries)
• Documentation (must be aligned and explained with fallback filtering.)

[fujitatomoya]

Some consideration as below, comments and feedback are welcome.

• RCL filtering is opt-out. and if RMW support CFT, RCL filtering is disabled internally.
• We will provide tokenizer and parsers for RCL filtering expression and parameters.
  • This will be checked at the 1st place regardless of underlying filtering.
  • Subset of DDS spec.
• RMW filtering and RCL fallback filtering are exclusive. (but ROS 2 filtering expression and parameters are applied for both case)
  • with RCL filtering, if the expression and parameters cannot be applied, it will return error.
  • with RMW filtering, if the expression and parameters cannot be applied, it will print the warning but passed to RMW filtering. and RMW returns error if cannot be applied.
• Environment value to use RCL fallback only even if RMW filtering is supported.
  • For instance, RMW supports CFT but some problems observed in RMW implementation. So we can use this flag to fallback RCL filtering in force w/o code change.

[fujitatomoya]

Development Status Update, the followings are ready for review.

• A fallback to support content filters in rcl rcl_content_filter_fallback#1
• call rcl content filter fallback #996
• rcl content filter fallback supported in the rcl demos#579

CC: @wjwwood @ivanpauno @MiguelCompany @asorbini @clalancette

[fujitatomoya]

see ros2/demos#579 (comment)