Distributed Search and Rescue with Robot and Sensor Teams

. We develop a network of distributed mobile sensor systems as a solution to the emergency response problem. The mobile sensors are inside a building and they form a connected ad›hoc network. We discuss cooperative localization algorithms for these nodes. The sensors collect temperature data and run a distributed algorithm to assemble a temperature gradient. The mobile nodes are controlled to navigate using this temperature gradient. We also discuss how such networks can assist human users to (cid:2)nd an exit. We have conducted an experiment at a facility used to train (cid:2)re(cid:2)ghters in order to understand the environment and to test component technology. Results from experiments at this facility as well as simulations are presented here.


Motivation
We consider search and rescue applications in which heterogeneous groups of agents (humans, robots, static and mobile sensors) enter an unknown building and disperse while following gradients of temperature and concentration of toxins, and looking for immobile humans. The agents deploy the static sensors and maintain line of sight visibility and communication connectivity whenever possible. Since different agents have different sensors and therefore different pieces of information, communication is necessary for tasking the network, sharing information, and for control.
An ad-hoc network is formed by a group of mobile hosts upon a wireless local network interface. It is a temporary network formed without the aid of any established infrastructure or centralized administration. A sensor network consists of a collection of sensors and distributed over some area that form an ad-hoc network. Our heterogeneous teams of agents (sensors, robots, and humans) constitute distributed adaptive sensor networks and are well-suited for tasks in extreme environments, especially when the environmental model and the task specifications are uncertain and the system has to adapt to it. Applications of this work cover search and rescue for first responders, monitoring and surveillance, and infrastructure protection.

Sensors Robots
We combine networking, sensing, and control to control the flow of information in search and rescue in unknown environments. Specifically, this research examines (1) localization in an environment with no infrastructure such as a burning building (for both sensors and robots) (2) information flow across a sensor network that can localize on the fly for delivering the most relevant and current information to its consumer, maintaining current maps, and automating localization; (3) using feedback from the sensor network to control the autonomous robots for placing sensors, collecting data from sensors, and locating targets; and (4) delivering the information gathered from the sensor network (integrated as a global picture) to human users. The paper will detail our technical results in these 4 areas and describe an integrated experiment for navigation in burning buildings.

Localization
Localization in dynamic environments such as posed by search and rescue operations is difficult because no infrastructure can be presumed and because simple assumptions such as line of sight to known features cannot be guaranteed. We have been investigating the use of low cost radio beacons that can be placed in the environment by rescue personnel or carried by robots (see Figure 2). In this paradigm, as the robot moves, it periodically sends out a query, and any tags within range respond by sending a reply. The robot can then estimate the distance to each responding tag by determining the time elapsed between sending the query and receiving the response. The advantage of such a method is that it does not require line of sight between tags and the mobile robot, making it useful in many environmental conditions where optical methods fail. Note that, since each tag transmits a unique ID number, distance readings are automatically associated with the appropriate tags, so the data Fig. 2. A radio tag, approximately 12 × 9 cm in size, with which a robot can communicate to obtain range data. Such tags can be scattered into a burning building as firefighters move about or even be deployed by robots themselves. Since these tags are placed without careful survey, their position must be calculated along with the position of the mobile agents themselves.
association problem, a difficult issue in environments that can be visually obscured, is solved trivially.
Since the position of the tags is unknown to start and can potentially change during operation, it is necessary to localize both the receiver and the beacons simultaneously. This problem is often known as Simultaneous Localization and Mapping (SLAM). Although generally it is assumed that a receiver is able to measure both range and bearing to "features", we can assume only that range to tags is known and that this measurement may be very noisy. We have adapted the well-known estimation techniques of Kalman filtering, Markov methods, and Monte Carlo localization to solve the problem of robot localization from range-only measurements [1] [5]. All three of these methods estimate robot position as a distribution of probabilities over the space of possible robot positions. In the same work we presented an algorithm capable of solving SLAM in cases where approximate a priori estimates of robot and landmark locations exist. The primary difficulty stems from the annular distribution of potential relative locations that results from a range only measurement. Since the distribution is highly non-Gaussian, SLAM solutions based on Kalman filtering falter. In theory, Markov methods (probability grids) and Monte Carlo methods (particle filtering) have the flexibility to handle annular distributions. Unfortunately, the scaling properties of these methods severely limit the number of landmarks that can be mapped. In truth, Markov and Monte Carlo methods have much more flexibility than we need; they can represent arbitrary distributions while we need only to deal with very well structured annular distributions. What is needed is a compact way to represent annular distributions together with a computationally efficient way of combining annular distributions with each other and with Gaussian distributions. In most cases, we expect the results of these combinations to be well approximated by mixtures of Gaussians so that standard techniques such as Kalman filtering or multiple hypothesis tracking could be applied to solve the remaining estimation problem.
We have also extended these results to deal with the case when the tag locations are unknown using a geometrically inspired batch processing method. The basic idea is to store the robot locations and measured ranges the first few times the landmark is encountered and then obtain an estimate of landmark position by intersecting circles on the plane. Once an estimate of a new landmark is produced, the landmark is added to the Kalman filter where its estimate is then improved along with the estimates of the other (previously seen) landmarks. Because it takes advantage of the special structure of the problem, the resulting approach is less computationally cumbersome and avoids the local maxima problems associated with standard batch optimization techniques.
To collect data for this experiment, we used an instrumented autonomous robot that has highly accurate (2 cm) positioning for groundtruth using RTK GPS receivers as well as a fiber optic gyro and wheel encoders. Position is updated at 100 Hz. We equipped this robot with a RF ranging system (Pinpoint from RF Technologies) that has four antennae pointing in four directions and a computer to control the tag queries and process responses. For each tag response, the system produces a time-stamped distance estimate to the responding tag, along with the unique ID number for that tag. The distance estimate is simply an integer estimate of the distance between the robot and the tag.
The localization experiment was conducted on a flat, area about 30 meters by 40 meters in size. We distributed 13 RF tags throughout the area and then programmed the robot to drive in a repeating path among the tags. With this setup, we collected three kinds of data: the ground truth path of the robot from GPS and inertial sensors, the dead reckoning estimated path of the robot from inertial sensors only, and the range measurements to the RF tags. Results from our experiments are shown in Figures 3 and 4. Greater details can be found in [2].

Information Flow
Sensors detect information about the area they cover. They can store this information locally or forward it to a base station for further analysis and use. Sensors can also use We have developed distributed protocols for navigation tasks in which a distributed sensor field guides a user across the field [3]. We use the localization techniques presented above to compute environmental maps and sensor maps, such as temperature gradients. These maps are then used for human and robot navigation to a target, while avoiding danger (hot areas). Figure 1(Left) shows a picture of a room in which a fire was started. We have collected a temperature gradient map during the fire burning experiment as shown in Figure 1. The Mote sensors 1 were deployed by hand at the locations marked in the figure. The sensors computed multi-hop communication paths to a base station placed at the door. Data was sent to the base station over a period of 30 minutes.

Directional Guidance
We used the structure of the data we collected during the fire burning exercise to develop a navigation guidance algorithm designed to guide a user to the door, in  The Flashlight prototype we designed and built is shown in Figure 5 (Left). This device can be carried by a human user or placed on a mobile robot (or flying robot) to interact with a sensor field. The beam of the Flashlight is sensor-to-sensor, multi-hop routed RF messages which send or return information.
The Flashlight consists of an analog compass, alert LED, pager vibrator, a 3 position mode switch, a power switch, a range potentiometer, some power conditioning circuitry, and a microcontroller based CPU/RF transceiver. The processing and RF communication components of the Flashlight and the sensor network are Berkeley Motes, shown in Figure 5 (Center, Right). A switch selects the sensor type (light, sound, temperature, etc.). When the user points the Flashlight in a direction, if sensor reports of the selected type are received from any sensors in that direction, a silent vibrating alarm activates. The vibration amplitude can be used to encode how far (in number of hops) was the sensor that triggered. The potentiometer is used to set the detection range (calibrated in number of network hops from sensor to sensor.) The electronic compass supplies heading data indicating the pointed direction of the device.
We have deployed 12 Mote sensors along corridors in our building and used the Flashlight and the communication infrastructure presented here to guide a human user out of the building. Figure 6 shows the map. The Flashlight interacted with sensors to compute the next direction of movement towards the exit. For each interaction, the user did a rotation scan until the Flashlight was pointed in the direction computed from the sensor data. The user then walked in that direction to the next sensor. Each time we recorded the correct direction and the direction detected by the Flashlight. The directional error was 8% (or 30 degrees) on average. However, because the corridors and office doorways are wide, and the sensors sufficiently dense, the exit was identified successfully. The user was never directed towards a blocked or wrong configuration. An interesting question is how dense should the sensors be, given the feedback accuracy.

Control of a Network of Robots
Robots augment the surveillance capabilities of a sensor network by using mobility. Each robot must use partial state information derived from its sensors and from the communication network to control in cooperation with other robots the distribution of robots and the motion of the team. We treat this as a problem of formation control where the motion of the team is modeled as an element of a Lie group, while the shape of the formation is a point in shape space. We seek abstractions and control laws that allow partial state information to be used effectively and in a scalable manner.
Our platforms are car-like robots equipped with omnidirectional cameras as their primary sensors. The communication among the robots relies on IEEE 802.11b networking. By using information from its camera system each robot is only able to estimate its distance and bearing from their teammates. However, if two robots exchange their bearing to each other, they are also able to estimate their relative orientations [6]. We use this idea to combine the information of a group of two or more robots in order to improve the knowledge of the group about their relative position.
We have developed control protocols for using such a team of robots in connection with a sensor network to explore a known building. We assume that a network of Mote sensors previously deployed in the environment guide the robots towards the source of heat. The robots can modify their trajectories and still find the building exit. The robots can also switch between the potential fields (or temperature gradients) computed and stored in the sensor network (see Figure 7). The first switch occurs automatically when the first robot encounters a Mote sensor at a given location. The robots move toward the fire and stop at a safer distance (given by the temperature gradient). They stay there until they are asked to evacuate the building, at which point they use the original potential field to find the exit.

User Feedback
When robots or people interact with the sensor network, it becomes an extension of their capabilities, basically extending their sensory systems and ability to act over a much large range. We have developed software that allows an intuitive, immersive display of environments. Using, panoramic imaging sensors that can be carried by small robots into the heart of a damaged structure, the display can be coupled to head mounted, head tracking sensors that enable a remote operator to look around in the environment without the delay associated with mechanical pan and tilt mechanisms (see Figure 8).
The data collected from imaging systems such as visible cameras and IR cameras are displayed on a wearable computer to give the responder the most accurate and current information. Distributed protocols collect data from the geographically dispersed sensor network and integrate this data into a global map such as a temperature gradient that can also be displayed on a wearable computer to the user.

Discussion
The three groups met on August 23, 2002, at the Allegheny County firefighting training facility to conduct preliminary experiments involving a search and rescue exercise in a burning building (see Figure 1). A Mote sensor network was deployed manually in the building to collect temperature data and deliver the data to an outside point. A network of robots navigated the space. A network of cameras took panoramic images and IR images that were subsequently used to localize the robots. A network of radio tags was also used for localization. Although these modules were not integrated, the data collected during this exercise was used off-site to test the algorithms described in this paper. The firefighters who assisted us expressed great eagerness for having the kinds of support our vision provides.   8. 360 degree panorama of a room in the burning building taken by a catadioptric system. The image can be viewed using a combination of a head tracker/head mounted display to enable a digital pan and tilt with very low latency.