The Conference for Machine Learning Innovation

Evaluating machine learning solutions for subjective problems

Session
Join the ML Revolution!
Register until the conference starts:
✓ 50% off on all prices
✓ 10% team discount
Register Now
Join the ML Revolution!
Register until the conference starts:
✓ 50% off on all prices
✓ 10% team discount
Register Now
Join the ML Revolution!
Register until November 4:
✓ 2 in 1 conference special
✓ Save up to €220
✓ 10 % Team Discount
Register Now
Join the ML Revolution!
Register until November 4:
✓ 2 in 1 conference special
✓ Save up to €220
✓ 10 % Team Discount
Register Now
Join the ML Revolution!
Register until the conference starts:
✓ 2-in-1 conference special
✓ 10 % Team Discount
Register Now
Join the ML Revolution!
Register until the conference starts:
✓ 2-in-1 conference special
✓ 10 % Team Discount
Register Now
Infos
Tuesday, December 7 2021
15:45 - 16:30

Machine Learning research uses massive curated datasets and mathematical metrics to evaluate their methods, which allows models to be compared up to decimal points of percentages. While these metrics are not always perfect, they mostly provide a good indication of how well the model performs. When deploying machine learning systems in the real world, we often encounter problems where it’s harder to evaluate the quality of the results. 

This is especially the case when dealing with solutions that can only be evaluated subjectively, where the model decision influences the outcome, or where the possible result set is so large that it’s impossible to create a golden evaluation set. Prime examples of this are search engines and recommender systems, where all three of these characteristics come into play together, but there’s a wide range of machine learning solutions that are hard to evaluate due to at least one of these characteristics. 

The only way to really evaluate these systems is by running them in production and analyzing the results, which can be risky, costly, and too late. In this talk, we’ll examine some alternative and intermediate evaluation methods, which can help you build and evaluate these systems with more confidence and lower risk.

This Session belongs to the Diese Session gehört zum Programm vom BerlinBerlin program. Take me to the program of . Hier geht es zum Programm von Singapore Singapore .

Take me to the full program of Zum vollständigen Programm von Berlin Berlin .

This Session belongs to the Diese Session gehört zum Programm vom BerlinBerlin program. Take me to the program of . Hier geht es zum Programm von Munich Munich .

This Session Diese Session belongs to the gehört zum Programm von BerlinBerlin program. Take me to the current program of . Hier geht es zum aktuellen Programm von Singapore Singapore , Berlin Berlin or oder Munich Munich .

Behind the Tracks