The Conference for Machine Learning Innovation

Efficient Transformers

Session
Join the ML Revolution!
Register until the conference starts:
✓ 2-in-1 conference special
✓ 10 % Team Discount
Register Now
Join the ML Revolution!
Register until the conference starts:
✓ 2-in-1 conference special
✓ 10 % Team Discount
Register Now
Join the ML Revolution!
Register until August 12:
✓ML Intro Day for free
✓Save up to $380
Register Now
Join the ML Revolution!
Register until August 12:
✓ML Intro Day for free
✓Save up to $380
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now
Infos
Wednesday, June 23 2021
10:15 - 11:00

Transformers are the new go-to technology for Natural Language Processing and are also starting to gain traction in the computer vision community. However, despite all their successes and widespread adaption, they have one major drawback: Their computation and memory requirements grow quadratically with the input size. Hence training transformer models from scratch is a very resource-intensive task.

In this session we want to take a look at the current state of the research into efficient transformer layers, i.e. reformulations of the vanilla transformers that have computation and/or memory requirements of O(n*log(n)) or even O(n). If your knowledge about transformers or complexity theory is a bit rusty, do not worry: The session will start with a short refresher on both topics so you can make the most of it.

Take me to the full program of Zum vollständigen Programm von Munich Munich .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Singapore Singapore .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Berlin Berlin .

This Session Diese Session belongs to the gehört zum Programm von MunichMunich program. Take me to the current program of . Hier geht es zum aktuellen Programm von Munich Munich , Singapore Singapore or oder Berlin Berlin .

Behind the Tracks