RTT GDC Talk

Robot Operating System (ROS) on RT-Thread

Introduction to ROS1/ROS2
RT-Thread Packages (rosserial and micro_ros)
Future Plan

Ph.D. Student at the University of Exeter, in the U.K.

Research: Is Deep Learning secure for Robots?

Hi, I'm a PhD student at the University of Exeter, in the U.K. Here's my github username. I would like to introduce Robot Operating System, or ROS on RT-Thread, because my current research focuses on deep learning security for robots, that tries to seek the answer for the question: Is Deep Learning secure for Robots? In recent years, we see quite a lot of deep learning applications in robotics, such as object detection for autonomous driving. But here's the problem, what if deep learning models are vulnerable to adversarial attacks, which means neural networks can make wrong or even rediculous predictions once under attack, and I'll give a demonstration in the next slide. The demo project is implemented using Robot Operating System or ROS. So first, I'll give you a brief introduction to ROS ... How to communicate with ROS on RT-Thread. At last.

Why do we need ROS?

ROS is not an operating system, it is a set of software libraries and tools.

This is a project that demonstrates how to attack deep learning models, and it is implemented using ROS. I'll illustrate each component one by one. At the bottom right corner, we have a Gazebo Simulator for ROS. Actually, the simulator runs on a remote server, so the network connection makes it a little bit slow here. Anyway, we have a simulated robot with a camera, and the image captured by the simulated camera is sent to the browser. We can see the input image and the output of object detection. We can see a bounding box for the traffic sign 60. The last component is the control panel where we can control the simulated robot using keyboard. The object detection model seems to be pretty stable, right?. But actually, it is not. As I said, deep learning models are not secure. So we can generate a traffic sign out of no where. As you can see here, we have a lot of traffic signs now. For humans, we know that it is not a traffic sign, it's looks lile random noises. But deep learning models will recognize it as traffic signs. Imagine, if you put this image on your t-shirt, and walk on the street, the aunomous driving car will see a person wearing traffic signs. That's crazy, right? So now, do you still think deep learning models are secure? Perhaps not. Just a short demo, let's move back to take look at the overall architecture of this project.

Adversarial ROS Detection

ROS decouples software from hardware.

The same code works for both simulated robots and real robots.

In this project, eveything is published as ROS topics. If we would like to send out information, we can publish a topic; On the other hand, if we would like to receive information, we subscribe to a topic. For example, if we would like to send out the image from the simulator to the browser, we can publish a topic named /image. While if we would like to receive the control command from the keyboard, we can subscribe to a topic named /cmd_vel. So what if the benefit of doing this? Well, it decouple each component. For example, from the browsers perspective, it receives images from the topic /image.It has no idea where the image comes from. If we replace the simulator with a real robot. We don't really need to change the code for the browser, because we still have the same topic /image. Similarly, we can replace the keyboard with a controller, and eveything remains the same as long as we have the same topic.

ROS for Autonomous Driving

Carla Simulator - ROS Bridge

Why do we need RT-Thread?

RT-Thread provides support for real-time applications. ROS runs on Linux, MacOS, Windows.

rosserial	micro_ros
ROS1	ROS2
C++	C/C++
serial & TCP	serial & UDP
8bit - 32bit MCUs	32bit MCUs
amd64 arm64 armhf	amd64 arm64

ROS1 on RT-Thread (rosserial)

Choose the distribution (ROS1)

ROS1	Ubuntu	Debian
Noetic	Ubuntu 20.04 (Focal)	Debian 10 (Buster)
Melodic	Ubuntu 18.04 (Bionic)	Debian 9 (Stretch)
Kinetic	Ubuntu 16.04 (Xenial)	Debian 8 (Jessie)

Install ROS1 (Noetic)

                            
                                $ sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
                                $ sudo apt install curl # if you haven't already installed curl
                                $ curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
                                $ sudo apt update

                                $ sudo apt install ros-noetic-ros-base
                                $ echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
                                $ source ~/.bashrc

                                $ sudo apt install python3-rosdep
                                $ sudo rosdep init
                                $ rosdep update

Install rosserial

                            
                                $ sudo apt install ros-noetic-rosserial-arduino

Ubuntu

                            
                                $ roscore
                                $ rosrun rosserial_python serial_node.py /dev/ttyUSB0

RT-Thread

                            
                                msh > rosserial_hello_world_serial_example

Echo the topic

                            
                                $ rostopic echo /chatter

rosserial port on RT-Thread (RTTHardware.h)

                            
                                #ifndef ROSSERIAL_UART_NAME
                                    #define ROSSERIAL_UART_NAME "uart2"
                                #endif

                                class RTTHardware {
                                    public:
                                        RTTHardware();

                                        void setBaud(long baud);                                    
                                        int  getBaud();

                                        void init();

                                        int  read();
                                        void write(uint8_t* data, int length);

                                        unsigned long time();
                                };

rosserial port on RT-Thread (RTTTcpHardware.h)

                            
                                #ifndef ROSSERIAL_TCP_RECV_TIMEOUT
                                    #define ROSSERIAL_TCP_RECV_TIMEOUT 200000
                                #endif

                                #ifndef ROSSERIAL_TCP_SEND_TIMEOUT
                                    #define ROSSERIAL_TCP_SEND_TIMEOUT 200000
                                #endif

                                class RTTTcpHardware {
                                public:
                                    RTTTcpHardware();

                                    void setConnection(const char* url, int port = 11411);

                                    void init();

                                    int  read();
                                    void write(const uint8_t* data, int length);

                                    unsigned long time();
                                };

ROS2 on RT-Thread (micro_ros)

Choose the distribution (ROS2)

ROS2	Ubuntu	Debian	RHEL
Galactic	Ubuntu 20.04 (Focal)	Debian 10 (Buster)	RHEL 8 64-bit
Foxy	Ubuntu 20.04 (Focal)	Debian 10 (Buster)	-
Dashing	Ubuntu 18.04 (Bionic)	Debian 9 (Stretch)	-

Install ROS2 (Galactic)

                            
                                $ sudo apt install software-properties-common
                                $ sudo add-apt-repository universe
                                $ sudo apt update && sudo apt install curl gnupg lsb-release
                                $ sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key  -o /usr/share/keyrings/ros-archive-keyring.gpg
                                $ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
                                $ sudo apt update

                                $ sudo apt install ros-galactic-ros-base
                                $ echo "source /opt/ros/galactic/setup.bash" >> ~/.bashrc
                                $ source ~/.bashrc

                                $ sudo apt install python3-rosdep
                                $ sudo rosdep init
                                $ rosdep update

Ubuntu

                            
                                $ docker run -it -p 9999:9999/udp --privileged microros/micro-ros-agent:galactic udp4 -p 9999

RT-Thread

                            
                                msh > microros_pub_int32

Echo the topic

                            
                                $ docker exec -it YOUR_DOCKER_CONTAINER_NAME /bin/bash
                                $ ros2 topic echo /micro_ros_rtt_node_publisher

micro_ros port on RT-Thread (micro_ros_rtt.h)

                            
                                #if defined MICRO_ROS_USE_SERIAL
                                    bool   rtt_transport_open(struct uxrCustomTransport * transport);
                                    bool   rtt_transport_close(struct uxrCustomTransport * transport);
                                    size_t rtt_transport_write(struct uxrCustomTransport* transport, const uint8_t * buf, size_t len, uint8_t * err);
                                    size_t rtt_transport_read(struct uxrCustomTransport* transport, uint8_t* buf, size_t len, int timeout, uint8_t* err);
                                    int clock_gettime(clockid_t unused, struct timespec *tp);
                                #endif  // MICRO_ROS_USE_SERIAL

                                #if defined MICRO_ROS_USE_UDP
                                    bool   rtt_udp_transport_open(struct uxrCustomTransport * transport);
                                    bool   rtt_udp_transport_close(struct uxrCustomTransport * transport);
                                    size_t rtt_udp_transport_write(struct uxrCustomTransport* transport, const uint8_t * buf, size_t len, uint8_t * err);
                                    size_t rtt_udp_transport_read(struct uxrCustomTransport* transport, uint8_t* buf, size_t len, int timeout, uint8_t* err);
                                    int clock_gettime(clockid_t unused, struct timespec *tp);
                                #endif  // MICRO_ROS_USE_UDP

Future Plan

Test more sensors (Lidar)

Upstream the port

Thanks

https://ros.wuhanstudio.uk