ETH AI Digest: #21

Multi-camera 3D tracking, psychology-driven AI agents, and auto-differentiation achieves 92× JAX speedup

In this week's digest:

1. Multi-Camera 3D Point Tracking — MVTracker fuses multi-view features into unified 3D point clouds using transformer refinement, tracking arbitrary points across cameras without per-sequence optimization

2. Psychology-Enhanced AI Agents — MBTI-in-Thoughts framework conditions LLM agents with personality traits, showing emotional types excel at narratives while analytical types dominate strategic tasks

3. High-Performance Auto-Differentiation — DaCe AD bridges ML and scientific computing with memory-efficient gradient computation, achieving 92× speedups over JAX without code modifications

---

Selected Papers of the Week

---

1. Multi-View 3D Point Tracking

Tracking 3D points across multiple camera views with fewer cameras and no per-sequence optimization.

MVTracker. The first data-driven multi-view 3D point tracker that fuses multi-view features into unified 3D point clouds and uses transformer-based refinement to track arbitrary 3D points across multiple cameras without per-sequence optimization.

✍️ Authors: Frano Rajič, Haofei Xu, Marko Mihajlovic, Siyuan Li, Irem Demir, Emircan Gündoğdu, Lei Ke, Sergey Prokudin, Marc Pollefeys, Siyu Tang

🏛️ Lab: Computer Vision and Geometry Group, Computer Vision and Learning Group

⚡ Summary

Madupite is a high-performance solver that tackles large-scale Markov decision processes by combining inexact policy iteration methods with distributed computing.

The solver can efficiently handle problems exceeding single-machine memory constraints and achieves fast convergence even with high discount factors through customizable solution algorithms.

Implemented in C++ with a Python API, madupite demonstrates superior performance on applications ranging from epidemiology to control systems, solving problems with millions of states.

This represents a significant advancement over existing solvers, enabling exact solutions to previously intractable decision-making problems.

👉 Read the full paper

🌐 Project page

💻 Github repo

---

2. Psychologically Enhanced AI Agents

Enhancing AI agents with psychological traits for improved task performance.

✍️ Authors: Maciej Besta, Shriram Chandran, Robert Gerstenberger, Mathis Lindner, Marcin Chrapek, Sebastian Hermann Martschat, Taraneh Ghandi, Patrick Iff, Hubert Niewiadomski, Piotr Nyczyk, Jürgen Müller, Torsten Hoefler

🏛️ Lab: Scalable Parallel Computing Lab

⚡ Summary

This paper introduces MBTI-in-Thoughts, a framework that enhances LLM agents by conditioning them with personality traits based on psychological profiles.

By priming agents with different MBTI personality types through prompt engineering, the researchers demonstrate that emotional types excel at narrative tasks while analytical types perform better in strategic scenarios.

The framework enables structured multi-agent communication and verifies personality persistence through standardized testing, all without requiring fine-tuning.

This approach bridges psychological theory and AI design, offering a lightweight method to align agent traits with specific task demands.

👉 Read the full paper

💻 Github repo

---

3. DaCe AD: Unifying High-Performance Automatic Differentiation for Machine Learning and Scientific Computing

Bridging machine learning and scientific computing with a memory-efficient, high-performance automatic differentiation engine.

Overview of existing solution for automatic differentiation.

✍️ Authors: Afif Boudaoud, Alexandru Calotoiu, Marcin Copik, Torsten Hoefler

🏛️ Lab: Scalable Parallel Computing Lab

⚡ Summary

DaCe AD addresses key limitations in automatic differentiation frameworks by unifying support for both machine learning and scientific computing.

The system uses a novel ILP-based algorithm to optimize the trade-off between storing and recomputing intermediate values, maximizing performance within memory constraints.

Without requiring code modifications, DaCe AD outperforms JAX by 92× on average across NPBench benchmarks, with some cases showing speedups of over 2,700×.

This breakthrough enables scientists to efficiently compute gradients for complex programs, facilitating the integration of AI techniques into scientific simulations.

👉 Read the full paper

---

Other noteworthy articles

• Measuring Scalar Constructs in Social Science with LLMs: Comparing pointwise scoring, pairwise ranking, and finetuning approaches for measuring continuous language constructs (like emotional intensity or negativity).

---