ETH AI Digest: #20
Python tools solve causal inference challenges, unlabeled data boosts multi-task learning, and breakthrough solver tackles massive decision problems
In this week's digest:
Robust Causal Inference Toolkit — New Python package tackles weak instrument bias in econometrics, providing statisticians with validated methods and real-world demonstrations using education-wage data
Smart Multi-Objective Learning — Pseudo-labeling with Bregman losses harnesses unlabeled data to balance competing model objectives, reducing labeled sample requirements through statistical cost shifting
Million-State Decision Problem Solver — Madupite combines distributed computing with inexact policy iteration to solve previously intractable Markov decision processes with superior performance
Selected Papers of the Week
1. A statistician's guide to weak-instrument-robust inference in instrumental variables regression with illustrations in Python
A comprehensive Python-based toolkit for valid causal inference when instruments are weak.
✍️ Authors: Malte Londschien
⚡ Summary
This paper addresses the challenge of performing valid statistical inference in instrumental variables regression when instruments are weak, which can lead to biased estimates and incorrect inference.
The authors provide a comprehensive overview of weak-instrument-robust methods, including Anderson-Rubin, conditional likelihood-ratio, and Lagrange multiplier tests, with detailed mathematical formulations and proofs.
All methods are implemented in the ivmodels Python package, making them accessible to researchers and practitioners.
The paper illustrates these methods using Card's (1995) dataset on estimating the causal effect of education on wages, demonstrating how different approaches yield varying confidence intervals.
2. On the sample complexity of semi-supervised multi-objective learning
How Bregman losses and pseudo-labeling enable efficient trade-offs between competing objectives using unlabeled data.
✍️ Authors: Tobias Wegel, Geelon So, Junhyung Park, Fanny Yang
🏛️ Lab: Statistical Machine Learning Group
⚡ Summary
This paper investigates when unlabeled data can reduce the labeled sample complexity in multi-objective learning, where models must balance multiple competing objectives.
For general losses, the authors prove a hardness result showing that labeled sample complexity depends on the complexity of the larger function class, even with unlimited unlabeled data.
However, for Bregman divergence losses (like square and cross-entropy), they demonstrate that a simple pseudo-labeling algorithm can significantly reduce labeled data requirements by shifting statistical cost to unlabeled data.
Unlike in single-objective learning, unlabeled data helps by determining the relative importance of inputs to different tasks, enabling better trade-offs without requiring the marginals to carry information about labels.
3. Inside madupite: Technical Design and Performance
Solving million-state Markov decision processes through distributed computing and customizable algorithms.
✍️ Authors: Matilde Gargiani, Robin Sieber, Philip Pawlowsky, John Lygeros
🏛️ Lab: Automatic Control Laboratory
⚡ Summary
Madupite is a high-performance solver that tackles large-scale Markov decision processes by combining inexact policy iteration methods with distributed computing.
The solver can efficiently handle problems exceeding single-machine memory constraints and achieves fast convergence even with high discount factors through customizable solution algorithms.
Implemented in C++ with a Python API, madupite demonstrates superior performance on applications ranging from epidemiology to control systems, solving problems with millions of states.
This represents a significant advancement over existing solvers, enabling exact solutions to previously intractable decision-making problems.
Other noteworthy articles
COMET-poly: Machine Translation Metric Grounded in Other Candidates: Enhancing translation metrics by mimicking human evaluators who compare multiple alternatives