ETH AI Digest: #22

Simulators slash real-world experiments, lightning-fast robot vision, and breakthrough neural operators master diverse physics equations

In this week's digest:

1. Simulation-Informed Deep Learning — SimPEL uses low-fidelity simulators as Bayesian priors, reducing physical experiments by half while bridging sim-to-real gaps with Gaussian processes

2. Smart Robot Camera Control — ActLoc predicts localization accuracy across viewing angles in one forward pass, achieving state-of-the-art navigation while being 75× faster than previous methods

3. Universal PDE Neural Operators — HyPINO combines hypernetworks with synthetic data to solve diverse partial differential equations, achieving 100× accuracy improvements through iterative refinement

---

Selected Papers of the Week

---

1. Simulation Priors for Data-Efficient Deep Learning

Bridging sim-to-real gaps with low-fidelity simulators as Bayesian priors.

SimPEL framework: (a) estimates prior score by combining simulator and sim-to-real Gaussian Proccess (GP) priors, (b) models data likelihood with Gaussian distribution for closed-form score expression, and (c) uses SVGD updates in function space projected to parameter space for iterative prior-to-posterior transitions.

✍️ Authors: Lenart Treven, Bhavya Sukhija, Jonas Rothfuss, Stelian Coros, Florian Dörfler, Andreas Krause

🏛️ Lab: Learning & Adaptive Systems Group

⚡ Summary

SIMPEL addresses the challenge of learning accurate models of complex real-world systems when data is scarce.

The method combines first-principles simulators with data-driven learning by using low-fidelity simulators as priors in Bayesian neural networks, while modeling the sim-to-real gap with Gaussian processes.

This approach benefits from simulator knowledge in low-data regimes and leverages deep learning's flexibility as more data becomes available.

Experiments across robotics, biology, and agriculture demonstrate significant improvements in sample efficiency, reducing the number of physical experiments needed for complex tasks by a factor of two.

👉 Read the full paper

---

2. ActLoc: Learning to Localize on the Move via Active Viewpoint Selection

Teaching robots to look in the right direction for better navigation through intelligent camera orientation selection.

ActLoc framework for active viewpoint selection in robot navigation. The model learns to predict localization accuracy distributions over viewpoint angles at 3D locations (left) and integrates with path planning to select robust viewpoints during navigation (right).

✍️ Authors: Jiajie Li, Boyang Sun, Luca Di Giammarino, Hermann Blum, Marc Pollefeys

🏛️ Lab: Computer Vision and Geometry Group

⚡ Summary

Visual localization for robots suffers from cameras' narrow field of view, making viewpoint selection critical for navigation reliability.

ActLoc addresses this challenge with an attention-based model that predicts localization accuracy across multiple viewing angles in a single efficient forward pass.

The system integrates these predictions into path planning, enabling robots to select optimal camera orientations that balance localization robustness with motion constraints.

Compared to previous approaches, ActLoc achieves state-of-the-art accuracy while being significantly faster (110ms vs 8230ms per waypoint), making it suitable for real-time robot navigation in various environments.

👉 Read the full paper

🌐 Project page

---

3. HyPINO: Multi-Physics Neural Operators via HyperPINNs and the Method of Manufactured Solutions

Hypernetwork-based approach solves diverse PDEs using synthetic data and iterative refinement.

Figure 1. HyPINO pipeline overview: (a-b) processes supervised MMS and unsupervised physics-informed training data to generate task-specific PINN weights, (c-d) maps coordinates to solutions using combined physics and supervised losses, and (e-f) enables zero-shot inference with optional downstream refinement for unseen PDEs.

✍️ Authors: Rafael Bischof, Michal Piovarči, Michael A. Kraus, Siddhartha Mishra, Bernd Bickel

🏛️ Lab: Computational Design Laboratory, Seminar for Applied Mathematics

⚡ Summary

HyPINO addresses the challenge of developing neural operators that can generalize across diverse partial differential equations without task-specific fine-tuning.

The approach combines a Swin Transformer-based hypernetwork with synthetic data from the Method of Manufactured Solutions and physics-informed objectives.

Experimental results show HyPINO outperforms existing methods on benchmark problems, with iterative refinement achieving up to 100x improvement in solution accuracy.

The framework provides effective initialization for fine-tuning, converging faster and to lower errors than randomly initialized or meta-learned alternatives.

👉 Read the full paper

---