When a drone flies into a burning building, its cameras can fail in seconds: feeds dissolve into gray, and laser scanners scatter unpredictably off airborne ash. The solution, demonstrated by Carnegie Mellon University's Robotics Institute in its Super Odometry system, is sensor fusion—strategically combining external perception sensors like cameras and lidar with internal motion sensors so the robot keeps track of where it is even when its "eyes" stop working.

The Challenge of Robotic Autonomy in Extreme Environments

Techniques such as odometry and simultaneous localization and mapping (SLAM) use cameras and lidar to estimate motion and construct maps, letting machines understand where they are and how they are moving. While effective in controlled or clear conditions, as noted by Carnegie Mellon's researchers, this camera and lidar-based perception can fail in challenging environments such as dark, smoky, dusty, or rapidly changing settings, impairing a robot's ability to orient and operate safely.

The problem is structural, not incidental. A survey of advanced sensing and control technologies for autonomous robots observes that integrating and analyzing data from multiple sensor sources is imperative to generate comprehensive, accurate and reliable information, precisely because no single sensor works everywhere. A robot navigating smoke, darkness, stairs or rough terrain cannot bet its position estimate on one fragile input.

Sensor Fusion: A Strategy for Resilient Navigation

Sensor fusion is the process of combining multiple sensor data to reduce uncertainty or improve accuracy in the state estimation of a dynamic system. As the engineering firm eInfochips explains, the point is to compensate for the limitations of single sensors: a wheel encoder, for instance, reports erroneous motion when wheels slip, and such errors are exacerbated by sensor noise and the physical environment.

The payoff is redundancy. Texas Instruments demonstrates the same principle in human-robot collaboration, where fusing radar and vision data overcomes the limitations of standalone vision, radar or lidar data. In navigation, the same logic applies: if one sensor degrades, others carry the estimate forward. This article covers the concepts and strategies, not the underlying mathematics of filters like the Extended Kalman Filter that implement them.

Categories of Sensors for Robotic Perception

Robotic sensors fall into two broad families. Exteroceptive sensors—cameras, lidar, radar—gather information about the robot's surroundings. Proprioceptive sensors, such as inertial measurement units (IMUs) and kinematic sensors, measure the robot's own motion and state.

Each family fails differently. Exteroceptive sensors are accurate in clear conditions but vulnerable to smoke, dust, darkness and fast movements. Proprioceptive sensors keep working when visibility vanishes, but they measure only the robot's own motion, not the world. A legged-robotics research paper describes a system whose core estimator requires only inertial and kinematic measurements to achieve low drift, with stereo vision and lidar incorporated in a complementary manner—redundancy that also compensates for mechanical compliance and terrain deformation such as mud or loose stones.

How Super Odometry Dynamically Adapts to Extreme Environments

The clearest illustration of the fusion strategy is Super Odometry, developed by a Carnegie Mellon research team advised by Robotics Institute faculty members Sebastian Scherer and Wenshan Wang. The table below models how its reliance shifts as conditions degrade.

Dynamic sensor weighting in a resilient navigation system
Environmental conditionExternal sensors (camera/lidar)Internal sensors (IMU/kinematics)Fusion system output
Clear, controlled conditionsPrimary source: cameras and lidar estimate motion and build mapsSupporting roleAccurate localization and mapping
Degrading conditions (low light, dust)Partially degraded; still contributingIncreasingly weightedContinuous localization with reduced accuracy margin
Fully degraded (smoke, darkness)Effectively blind: feeds dissolve, laser scans scatterDominant source: IMU estimates movement without reliable visual inputPosition and motion tracking maintained

The logic follows a human analogy the researchers drew explicitly: in dense fog or complete darkness, people rely more heavily on internal cues like balance and motion signals from the body. Super Odometry strengthens the robot's internal sense of motion the same way, blending traditional camera and lidar methods with a module that uses IMU data—acceleration and rotation—to estimate movement when visual input is unreliable.

Super Odometry Movie S3: Evaluation of 13 types of degradation in a single runTo visually demonstrate the 'Super Odometry' system's performance and resilience in various degraded conditions.Watch on YouTube

Maintaining Localization and Perception in Degraded Conditions

The design principle is captured by researcher Zhao's framing of the trade-off: "External perception makes systems more accurate, but internal sensing helps systems survive." External sensors deliver precision; internal sensors deliver continuity. A system that leans on whichever is currently trustworthy gets both.

In real-world tests reported by the Robotics Institute, Super Odometry kept robots moving safely through tough conditions including stairs, low light and smoke—situations that would blind a camera-only system. The adaptive strategy ensures the robot maintains its sense of position and motion even when its primary "eyes" are compromised.

The approach is not exhaustive: Super Odometry is one system, and the examples here illustrate principles rather than survey every fusion technology. But the pattern generalizes. Any robot destined for extreme environments should be designed so that no single sensor's failure ends the mission.

Ensuring Continuous Autonomy Through Adaptive Sensor Fusion

Robotic systems operating in extreme environments must implement sensor fusion strategies that dynamically combine diverse sensor modalities, prioritizing internal motion sensors when external perception is degraded. The measurable indicator of success is concrete: the robot maintains accurate localization and continues its mission despite environmental challenges that would typically cause primary external sensors to fail. For teams evaluating such systems, the question to ask is not which sensor the robot uses, but what happens when that sensor stops working—and whether the fusion architecture keeps the mission alive when it does.

Sources