The Conference for Machine Learning Innovation

Efficient Transformers

Session
Join the ML Revolution!
Register until October 20:
✓ Save up to $233
✓ Team discount
✓ Extra Specials for Freelancers
Register Now
Join the ML Revolution!
Register until October 20:
✓ Save up to $233
✓ Team discount
✓ Extra Specials for Freelancers
Register Now
Join the ML Revolution!
Register until November 03:
✓ Save up to €494
✓ 10% Team Discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Register until November 03:
✓ Save up to €494
✓ 10% Team Discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Until the Conference starts:
✓ Group discount
✓ Special discount for freelancers
Register Now
Join the ML Revolution!
Until the Conference starts:
✓ Group discount
✓ Special discount for freelancers
Register Now
Infos

Transformers are the new go-to technology for Natural Language Processing (NLP) and are also starting to gain traction in the computer vision community. However, despite all their successes and widespread adoption, they have one major drawback: Their computation and memory requirements grow quadratically with the input size. Hence training transformer models from scratch is a very resource-intensive task.

In this session we want to take a look at the current state of the research into efficient transformer layers, i.e. reformulations of the vanilla transformers that have computation and/or memory requirements of O(n*log(n)) or even O(n). If your knowledge about transformers or complexity theory is a bit rusty, do not worry: The session will start with a short refresher on both topics so you can make the most of it.

This Session originates from the archive of Diese Session stammt aus dem Archiv von BerlinBerlin and  und MunichMunich . Take me to the program of . Hier geht es zum aktuellen Programm von Singapore Singapore .

This Session originates from the archive of Diese Session stammt aus dem Archiv von BerlinBerlin and  und MunichMunich . Take me to the program of . Hier geht es zum aktuellen Programm von Berlin Berlin .

This Session originates from the archive of Diese Session stammt aus dem Archiv von BerlinBerlin and  und MunichMunich . Take me to the program of . Hier geht es zum aktuellen Programm von Munich Munich .

This Session Diese Session originates from the archive of stammt aus dem Archiv von BerlinBerlin and  und MunichMunich . Take me to the current program of . Hier geht es zum aktuellen Programm von Singapore Singapore , Berlin Berlin or oder Munich Munich .

Behind the Tracks