Skip to content
Episode from the podcast Argmax
2: data2vec artwork

Episode

2: data2vec

Released Monday, March 7th 2022 53 min

Show notes

About this episode

Todays paper: data2vec (https://arxiv.org/abs/2202.03555) Summary of the paper A multimodal SSL algorithm that predicts latent representation of different types of input. Highlights of discussion What are the motivations of SSL and multimodalHow does the student teacher learning work?What are similarities and differences between ViT, BYOL, and Reinforcement Learning algorithms.

More from Argmax

All episodes

Mixture of Experts

54 min

In this episode we talk about the paper "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean.

LoRA

1 hr 2 min

We talk about Low Rank Approximation for fine tuning Transformers. We are also on YouTube now! Check out the video here: https://youtu.be/lLzHr0VFi3Y

15: InstructGPT

57 min

In this episode we discuss the paper "Training language models to follow instructions with human feedback" by Ouyang et al (2022). We discuss the RLHF paradigm and how important RL is to tuning GPT.

14: Whisper

49 min

This week we talk about Whisper. It is a weakly supervised speech recognition model.

13: AlphaTensor

49 min

We talk about AlphaTensor, and how researchers were able to find a new algorithm for matrix multiplication.

This podcast, its content and its artwork are not owned by, affiliated with, or endorsed by Castquire.

Up next

  1. Nothing queued. Use + Queue on any episode to line it up.

0:00 0:00