The Conference for Machine Learning Innovation

Underspecified pipelines: Why good models underperform in production

Session
Join the ML Revolution!
Register until October 20:
✓ Save up to $233
✓ Team discount
✓ Extra Specials for Freelancers
Register Now
Join the ML Revolution!
Register until October 20:
✓ Save up to $233
✓ Team discount
✓ Extra Specials for Freelancers
Register Now
Join the ML Revolution!
Register until November 03:
✓ Save up to €494
✓ 10% Team Discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Register until November 03:
✓ Save up to €494
✓ 10% Team Discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Until the Conference starts:
✓ Group discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Until the Conference starts:
✓ Group discount
✓ Special discount for freelancers
Register Now

Machine Learning models often exhibit poor performance when applied to real-world problems. Surprisingly, this effect is observed even when rigorous validation procedures have been followed, and the model generalization measured under laboratory conditions seems acceptable. Recent research identifies underspecified pipelines as the main culprit for models that score well on hold-out sets but perform poorly once operationalized. 

In this advanced talk, we will go over the research results, explain the theoretical foundations of the underspecification effect, discuss several case studies using ensemble and deep learning models, and provide suggestions for training models with credible inductive biases. As part of the talk, we will also do a Python-based demo using some toy examples to get a better idea of how underspecification manifests itself.

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Singapore Singapore .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Berlin Berlin .

Take me to the full program of Zum vollständigen Programm von Munich Munich .

This Session Diese Session belongs to the gehört zum Programm von MunichMunich program. Take me to the current program of . Hier geht es zum aktuellen Programm von Singapore Singapore , Berlin Berlin or oder Munich Munich .

Behind the Tracks