The Conference for Machine Learning Innovation

Efficient Transformers

Session
Join the ML Revolution!
Register until April15:
✓Save up to 310 €
✓ 2-in-1 conference special
✓10 % Team Discount
Register Now
Join the ML Revolution!
Register until April15:
✓Save up to 310 €
✓ 2-in-1 conference special
✓10 % Team Discount
Register Now
Join the ML Revolution!
Register until December 12:
✓ML Intro Day for free
✓Raspberry Pi or C64 Mini for free
✓Save up to $580
Register Now
Join the ML Revolution!
Register until December 12:
✓ML Intro Day for free
✓Raspberry Pi or C64 Mini for free
✓Save up to $580
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now

Transformers are the new go-to technology for Natural Language Processing and are also starting to gain traction in the computer vision community. However, despite all their successes and widespread adaption, they have one major drawback: Their computation and memory requirements grow quadratically with the input size. Hence training transformer models from scratch is a very resource-intensive task.

In this session we want to take a look at the current state of the research into efficient transformer layers, i.e. reformulations of the vanilla transformers that have computation and/or memory requirements of O(n*log(n)) or even O(n). If your knowledge about transformers or complexity theory is a bit rusty, do not worry: The session will start with a short refresher on both topics so you can make the most of it.

Take me to the full program of Zum vollständigen Programm von Munich Munich .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Singapore Singapore .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Berlin Berlin .

This Session Diese Session belongs to the gehört zum Programm von MunichMunich program. Take me to the current program of . Hier geht es zum aktuellen Programm von Munich Munich , Singapore Singapore or oder Berlin Berlin .

Behind the Tracks