The Conference for Machine Learning Innovation

Efficient Transformers

Session
Join the ML Revolution!
Register until March 11:
✓Save more than 500 € and get ML Intro Day for free
✓ Workshop day for free
✓10 % Team Discount
Register Now
Join the ML Revolution!
Register until March 11:
✓Save more than 500 € and get ML Intro Day for free
✓ Workshop day for free
✓10 % Team Discount
Register Now
Join the ML Revolution!
Register until December 12:
✓ML Intro Day for free
✓Raspberry Pi or C64 Mini for free
✓Save up to $580
Register Now
Join the ML Revolution!
Register until December 12:
✓ML Intro Day for free
✓Raspberry Pi or C64 Mini for free
✓Save up to $580
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now
Join the ML Revolution!
Register until November 7th:
✓Save up to € 210
✓10% Team Discount
Register Now

Transformers are the new go-to technology for Natural Language Processing and are also starting to gain traction in the computer vision community. However, despite all their successes and widespread adaption, they have one major drawback: Their computation and memory requirements grow quadratically with the input size. Hence training transformer models from scratch is a very resource-intensive task.

In this session we want to take a look at the current state of the research into efficient transformer layers, i.e. reformulations of the vanilla transformers that have computation and/or memory requirements of O(n*log(n)) or even O(n). If your knowledge about transformers or complexity theory is a bit rusty, do not worry: The session will start with a short refresher on both topics so you can make the most of it.

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Online Edition Online Edition .

Take me to the full program of Zum vollständigen Programm von Munich Munich .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Singapore Singapore .

This Session belongs to the Diese Session gehört zum Programm vom MunichMunich program. Take me to the program of . Hier geht es zum Programm von Berlin Berlin .

This Session Diese Session belongs to the gehört zum Programm von MunichMunich program. Take me to the current program of . Hier geht es zum aktuellen Programm von Online Edition Online Edition , Munich Munich , Singapore Singapore or oder Berlin Berlin .

Behind the Tracks