CPM Remote: A Remote Access to the CPM Lab

Laboratories with model vehicles offer a middle ground between field tests and simulations. Thereby, the lab-oratories benefit from the advantages of more realistic setups and reduce acquisition and maintenance costs. However, the efficient use of a fixed laboratory presents further organizational hurdles. In addition, special hardware requirements, complex installation processes, and the cost and length of travel to the laboratories discourage users from getting involved. In this paper, we present the web app CPM Remote, which addresses these hurdles. The Cyber-Physical Mobility (CPM) Lab is an open source platform for networked and autonomous vehicles. Our online framework www.cpm-remote.de. provides a simulation environment and remote access to our CPM Lab (based in Aachen, Germany), making it accessible from anywhere in the world. The simulation environment is outsourced to our servers, reducing the hardware and software requirements of the users. CPM Remote aims to offer a user-friendly platform for making solutions to current research questions more transparent since results can be reproduced and extended quickly with little effort.


A. Motivation
Algorithms for networked and autonomous driving can be developed using both field tests and computer simulations.The two approaches are usually not used exclusively but complement each other.However, operating a single test vehicle or a fleet of vehicles is time-consuming and maintenance-intensive.Both the operation and conversion of the vehicles require special permits and trained personnel.In contrast, simulations can be used much more flexibly but do not fully reflect reality.However, simulations can depict certain aspects, such as realistic vehicle physics.But the higher the degree of realism, the higher the demands on the user's hardware.
Laboratories that use model vehicles provide a middle ground between these two approaches.The use of model vehicles retains the advantages of field testing, such as realistic driving physics, in combination with lower maintenance and acquisition costs.However, the setup is more complex than in a simulation, as the lab operator has to establish and maintain communication in the laboratory.
Furthermore, other non-technical hurdles limit the operation of a laboratory as well.The Cyber-Physical Mobility (CPM) Lab for example is located in the city of Aachen, and users potentially have to travel to use the lab.Depending on the distance, this journey is associated with effort and costs.When a user travels to the lab, we need to reserve a time *www.cpm-remote.com 1 RWTH Aachen University -Chair for Embedded Software, Ahornstrasse 55, 52074 Aachen, Germany email: {lastname}@embedded.rwth-aachen.deslot in the lab.This time slot is usually longer than needed, compensating for multiple runs and possible readjustments.Allocation of time slots represents a second hurdle.A free slot, access to the lab and the building must be guaranteed; this results in inaccessible laboratories during the night or on holidays.Efficient use of the lab is not easy to achieve because the extra time in each slot causes idle time where no experiment is conducted.Furthermore, the lab users must be trained or accompanied in their use, which is not always possible, especially in a pandemic situation.Most of these obstacles are surmountable when the number of users is low but become more problematic as the number of users increases.

B. Contribution of This Paper
This paper presents an approach to overcome these limitations based on the CPM Lab.CPM Remote is a web app that allows access to both the computer simulation and the lab itself from anywhere in the world.Due to its loweffort setup, CPM Remote reduces the setup time and allows inexperienced users to access the lab immediately.An essential component of CPM Remote is the outsourcing of the simulation to our servers.The servers run several simulations in parallel and thus open the simulation to user's hardware which can not sustain a resource-intensive simulation.The available computing power is limited and identical for each user.Submitted experiments can be executed in the lab one after the other without idle times between experiments.In addition, we developed a real-time remote access, which allows to use the lab during working hours in other time zones without staff being present in the lab.
In addition to the lower hurdles, CPM Remote offers a platform with defined hardware and software, increasing the comparability of algorithms.Complex and modern problems can firstly be simulated and then tested in the CPM Lab.Hence, CPM Remote makes research and teaching more transparent, as results and algorithms can be recreated and extended with less effort.

C. Related Work
Recently, the interest in self-driving laboratories has increased.All around the world, such laboratories are actively used for cutting-edge research and education.Examples are the Robotarium [1], Duckietown [2], Scaled Smart City [3] and F1tenth (F1/10) [4].These laboratories differ in their area of application as well as their architecture, purpose of use, and public availability.In the following, we describe two laboratories that are conceptually closest to our lab platform.
The Robotarium [1] is a laboratory in the area of robotic interaction and control strategies.While the Robotarium and the CPM Lab have less in common in terms of their application area, the architecture and usage concept are comparable.Just as the CPM Lab, the Robotarium has a series of robots (vehicles for CPM Lab) with wireless communication hardware located on a stationary test-bed using an external indoor positioning system.In order to make their lab accessible to users around the globe, the Robotarium offers a web-based access.However, it is necessary to set up the Robotarium's simulation software by installing dependencies and the simulation environment itself on a supported platform.Furthermore, the simulation environment consumes hardware resources that have to be provided by the user.Our remote access is a full-stack online app, that eliminates the installation effort and minimizes the requirements for the processing capabilities of the user's device.
The second laboratory that we discuss is Duckietown [2].In contrast to the Robotarium, Duckietown provides no remote access but overlaps with the CPM Lab in their area of application.Both labs operate in the area of self-driving vehicles although Duckietown focuses more on perception rather than networked vehicles.A Duckietown lab in particular consists of two main components: the Duckiebots and the Duckietown.While the Duckiebot is a self-driving physical vehicle with attached sensors, the Duckietown is a model city consisting of streets, intersections, and traffic signs.The Duckiebot's onboard sensors are used to detect the streets and traffic signs and locate the Duckiebot within Duckietown.As the Duckietown concept focuses on environment perception and not on vehicle control, the Duckiebot's driving dynamics are based on a very simplified model and not on the Ackermann steering commonly applied in vehicles.Duckietown is designed to test the user implemented algorithms mainly on-site whereby the Duckiebot hardware must be purchased by the user and is usually not provided or shared by the founders.Although a simulation environment exists, it is currently in the beta stage and not supported on all major platforms [5].Nevertheless, Duckietown is characterized by wide acceptance and a large number of active users worldwide.The Duckietown Foundation regularly hosts driving Olympics where competitors from all around the world participate to solve new self-driving challenges [5].This proves that there is considerable interest in self-driving laboratories.
The CPM Lab is such a self-driving laboratory, but unique in the area of networked decision making, trajectory planning, and control.In this paper, we give a short overview on our CPM Lab in Section II.The methodology and implementation of our remote access is described in Section III and IV.Section V specifies various fields of application for CPM Remote.

II. THE CPM LAB
Since CPM Remote provides a remote access to the CPM Lab, we briefly introduce the lab and explain its architecture.More detailed information can be found in [6] and [7].CPM Lab Architecture: Our lab consists of 20 µCars with corresponding external computing units, a camera-based indoor positioning system connected to the main computing unit and a local network to which vehicles and computing units are connected.CPM Remote provides a remote interface for the lab to make it available a scalable number of users.Figure adapted from [6].
Our CPM Lab is a self-driving laboratory focusing on networked decision-making, trajectory planning, and control.As shown in Figure 1, the lab itself offers a model-scale testing platform consisting of a scalable number of lab vehicles called µCars, which can be seamlessly extended by simulated vehicles [8].Currently, up to 20 µCars in a scale of 1 : 18 are available driving on the underlying map with a size of 16 m 2 .While low and mid-level computations, e.g., hardware abstraction and sensor fusion, is performed onboard the vehicle, every vehicle has its own external computing unit for high-level computations.The vehicles and corresponding external computing units are connected to the lab's local area network and communicate using the Data Distribution Service (DDS) based middleware [9].
Our lab setup contains our open-source software, called Lab Control Center (LCC) which can be used for simulation and lab experiments.This software can conduct hybrid experiments where some vehicles are real and some vehicles are simulated.
By now, we use the lab actively in research and education as we integrated it into several teaching courses.Currently, our lab is only physically accessible.CPM Remote extends the lab by providing a remote interface.Using CPM Remote, researchers and educational facilities worldwide can implement, simulate and remotely test their own decision, planning, and control algorithms in real physical experiments in the CPM Lab.

III. CPM REMOTE
A remote interface to the CPM Lab opens our existing infrastructure to people out of reach of Aachen, Germany, and offers further challenges and opportunities.This chapter discusses our approach to provide a web app with on-server rather than on-client computation to not only remotely access our lab but also reduce the inhibition threshold to use the simulation capabilities offered by CPM Lab.To introduce the features of CPM Remote, Figure 2 depicts the development cycle a user will go through when conducting simulations and experiments in our lab.It consists of six steps from which steps 1-3 on the left belong to the user-operated web app, while steps 4-6 on the right are automatically executed in the CPM Lab.We describe these steps in III-A and introduce a special use-case which offers experienced users a real-time remote access to the lab for faster prototyping in III-B.

A. Development Cycle
Users who want to develop algorithms for networked decision-making, trajectory planning, and control for autonomous driving from home can start using CPM Remote by signing up on www.cpm-remote.de.A personal account will be created and a guided tour will introduce users to the web interface and its core functionalities.We provide readyto-run examples ranging from basic to advanced experiments and additional documentation.New and existing projects can be created, uploaded, or discarded using our project manager and a web editor enables users to edit them on our website, representing the first step of the development cycle.

Web Editor and Toolchain
CPM Remote experiments are developed using the C++ programming language and therefore have to be written in a code editor and compiled in order to be executed.This is why initial engagement with a new development platform often involves a setup procedure like installing toolchains, libraries, and accompanying software like an IDE.Especially inexperienced users might get overwhelmed and unmotivated to get started.In order to reduce the inhibition threshold to use our lab, CPM Remote offers a code editor including an easy-to-use compiler built into our web app.Providing a web editor with syntax highlighting, code map, and file manager not only reduces the amount of third-party software required to use our lab but also promotes community teamwork as every user will be then familiar with the same interface.Compiling code into an executable binary might be considered difficult among beginners because pitfalls like mismatching toolchain versions may occur.To counteract, we have set up the compilation environment on our server, available for users of CPM Remote with a single press of a button.Moreover this setup lowers the requirements on user hardware.We use the same approach of on-server rather than on-client computation to run the compiled experiment binaries in a simulation, which comes next in the development cycle.

Simulation
Before running newly developed algorithms directly with our µCars in the lab, it is recommended to test them for correct behavior in a simulation environment.Because CPM Lab and its simulation use the same interface to control lab vehicles, we can run experiment binaries in both environments interchangeably.The simulated lab helps to save time and resources while providing safety for equipment and operators.Similar to compiling experiments, users can start the simulation with a button that triggers our server to run the experiment binary compiled in the previous step.While the server is computing the simulation, it constantly sends experiment data back to the web app, which visualizes them in various forms as Figure 3 shows.If desired, users can download all experiment data for further analysis with external tools after the simulation.Although the computation runs on a single server, multiple users can use CPM Remote concurrently because resources are managed such that every user experience is equally performant.Using onserver computation, it is possible to use our web app for simulations even on low-end hardware, independent of the operating system and CPU architecture because all is needed is a recent web browser.The submission process discussed next describes how we also reduce the effort to transfer experiments from the simulation into the real lab.

Submit for lab experiment
CPM Remote provides online submission for experiments to be executed in the CPM Lab without the need for physical presence in our lab.Therefore, we replace the effort to travel, the schedule of time slots, and the process of briefing guests on how to operate the lab with a single submission page.Users can select the amount of µCars, their starting positions, and further experiment properties like the duration.Once submitted the validation process described next takes over and upon success, experiments will be executed in the lab as soon as possible.

Validation
Whenever code matures from the simulation into the real world, special care should be applied to ensure the experiment is safe for staff and equipment.Hence CPM Remote will automatically validate every submitted experiment to check for exceeded physical limits, potential vehicle crashes, or leaving our lab floor.If the validation succeeds without errors, the submission will be queued for execution.In case of error, we notify the user about the exact cause.Consequently, automated validation provides safety in our lab and additional feedback for users who might have overlooked unintended vehicle behavior during simulation.

Run experiment
After successful validation, experiments will append to a queue waiting for execution in the CPM Lab.In order to minimize the lab hours required to run a submitted experiment and subsequently maximize the number of experiments able to run per day, we automated many setup steps.Few manual procedures for a lab operator are to make sure the µCars are charged and to place as many of them on the map, as the experiment with the most vehicles requires.Starting execution of the queue on the main computer will then sequentially run all experiments.Because every submitted experiment might have different initial conditions, our lab automation drives µCars to their starting positions and parks vehicles which are not required anymore, before the next experiment in the queue starts.A ceiling-mounted camera attached to the main computer records the driving area while storing all sent DDS messages per experiment during execution.Once all experiments are completed, clearing the lab takes an operator an equally short amount of time as the setup procedure took.During the experiments, no physical presence is required.Additionally, all processes to provide users with their recorded experiment data are also automated, which represents the sixth and final step of our development cycle.

Send Results
After the execution queue has finished, we automatically upload accumulated experiment data to the server and notify users via e-mail that the results are ready for review.CPM Remote returns a synchronized view of recorded vehicle data and a video feed to emulate a lab-like experience.This includes a SQLite database with all actual DDS messages sent during the experiment, which the user can analyse either online or download and inspect in detail.
This section discussed six steps of the CPM-Remote development cycle, which offer rapid prototyping for networked decision-making, trajectory planning, and control for autonomous driving with model-scale vehicles to a worldwide audience.In the following, we describe a reduced cycle with faster prototyping capabilities for advanced users.

B. Use Case: Live Session
Experienced users who are familiar with the CPM Lab, like our staff, our students, or any other group using our lab extensively, can request a live session in order to reduce the time between submitting and receiving experiments results to an absolute minimum.This live session allows users to skip steps 2, 3, and 4 namely simulation, submission, and validation to directly conduct lab experiments from home.Granted with direct control of the lab, they can start and stop experiments through the web app, which streams a live video synchronized with incoming vehicle data similar to the replay functionality introduced in step 6 of the development cycle.This kind of remote control to a lab requires low latency data transmission protocols which is why we developed CPM Remote using WebSockets [10].Further implementation decisions are discussed in the following Chapter IV in more detail.

IV. IMPLEMENTATION
This chapter provides an insight into the software architecture of the CPM Remote platform and describes the used technologies.
The CPM Remote platform is implemented based on the state-of-the-art free and open-source MongoDB, Express, Angular, and NodeJS (MEAN) Software Stack.The MEAN Stack is a highly scalable high-performance framework [11] for multi-user web applications such as CPM Remote.Furthermore, it allows for rapid development of modular applications.As MEAN-based applications are written in Javascript or Typescript, seamless communication between Front-and Backend is ensured where code can also be reused.Another advantage is that NodeJS comes with various extensions that are effortlessly integrated into the application almost.
Figure 4 shows, the MEAN Stack is applied in the following way: The Frontend, written in Angular, consists of publicly available and private user-specific sites.The Backend is written in Express and accesses the MongoDB database to retrieve user information and provide access to the user's project data.Furthermore, the Backend starts, stops, and controls the compilation and simulation sessions that are running on our servers.
In order to provide an environment for compilation and simulation, we chose the concept of containerization.Containerization has many benefits in executing multiple applications on the same host system: high scalability, efficient resource management, and isolated code execution.Furthermore, modern engines instantiate containers almost instantly and operate with nearly no overhead [12].Therefore, a containerization engine is well-suited for the CPM Remote project as multiple containers can run in parallel without too much additional effort.We chose the containerization virtualization engine Docker.The Docker containers include the CPM Library as well as widely used libraries to compile the user code and simulate experiments.Thereby, Docker is configurated such that every user obtains a part of the computing resources of our servers, independently of the number of parallel users.Furthermore, Docker isolates the local network of the container such that DDS-communications of simulation vehicles in experiments from different users do not collide.While running simulated experiments, the Docker container opens a web socket to directly connect to the Frontend, resulting in lower communication latencies.
In addition to the introduced server architecture, the CPM Remote platform comes with a separated Real-Time Lab Access server.This server is responsible for conducting realtime lab experiments controllable by a remote user via the web Frontend.

V. FIELDS OF APPLICATION
CPM Remote has two main application areas.First, it provides a unified research platform for networked and autonomous driving algorithms and can be used to compare different algorithms and approaches.Second, educational institutions can use CPM Remote in their teaching courses.Due to the comparatively low hardware, software, and setup requirements, students can gain practical experience without the need for powerful hardware.
When creating additional material for CPM Remote, we categorize possible users into three target groups: The first group consists of users with no to little programming experience.This group includes, among others, students at secondary schools.We want to accompany this target group in gaining experience by designing materials so that the gamification effect contributes to a playful engagement with the topic.The second group consists of users with programming experience, such as university students.Here we pose more complex problems that we partly transferred from the real world.The combination of practical work and tasks with a high degree of realism can increase student engagement.We focus the material on the consolidation and expansion of paradigms and algorithms.The last group consists of scientists or professionals with experience in programming and autonomous driving.We provide real-world problems to challenge the algorithms and an objective comparison method between algorithms.
We have already implemented applications for two of the three groups.For Group 3, we have developed realworld scenarios and integrated them into CPM Remote.This project, named CPM Olympics [13], makes different solution algorithms comparable.We have supplemented the defined software and hardware of CPM Remote with defined tasks and evaluation methods.Thus, the stack is completed by creating an identical basis for all users, apart from the actual planning algorithm.The identical basis for all users allows us to compare the different algorithms objectively.We plan to hold a competition for the CPM Olympics in 2023.
The second application engages students [14].They learn about the application and problems of autonomous driving in a practical course, combining practical work in the laboratory with a fast and straightforward setup process.The students first develop their code in CPM Remote and then send it to the lab, which replaces the need to use a virtual machine or a Linux installation.In addition, students can work entirely from home; in the current pandemic situation, working oncampus would be difficult or even temporarily impossible.CPM Remote will be extended and continuously improved through user feedback.Potential applications will be planned and integrated into CPM Remote in an iterative development process.

VI. CONCLUSION
The Cyber-Physical Mobility (CPM) Lab in Aachen, Germany, provides a middle ground between real experiments and computer simulations.In this paper, we introduced an approach to provide remote access to the CPM Lab to make it accessible to the public.CPM Remote allows users to use the CPM Lab without traveling by submitting experiments to the web platform for later execution in the lab.In addition, CPM Remote provides a full web-based simulation of the CPM Lab, significantly reducing installation time.The elimination of the setup stage has two effects.First, research conducted on CPM Remote becomes much more transparent, as results can be easily reproduced.Second, it makes it easier for inexperienced users to get started with research in the first place.CPM Remote aims to be a platform for teaching and research.For teaching, it focuses on easy and fast deployment, even on low-power hardware.In research, it focuses on the comparability and reproducibility of research results.
Fig. 1.CPM Lab Architecture: Our lab consists of 20 µCars with corresponding external computing units, a camera-based indoor positioning system connected to the main computing unit and a local network to which vehicles and computing units are connected.CPM Remote provides a remote interface for the lab to make it available a scalable number of users.Figure adapted from[6].

Fig. 3 .
Fig. 3. Simulation View: On the left are eight simulated µCars in blue on a dark road network with white lane markings.For each vehicle, the planned trajectory is depicted in blue and the driven trajectory in red.On the right, the longitudinal accelerations of µCar 1, 5, and 7 are plotted over time.The console below displays log messages.

Fig. 4 .
Fig. 4. Overview of the Software Architecture of the CPM Remote.CPM remote applies several state-of-the-art technologies such as the MEAN Stack and the Container Virtualization Engine Docker.Stacked components visualize if a component is based on another.Furthermore, the arrows illustrate the communication flow (control and data).