Sentence and Contextual Word Representations (2/12/2019)
Content:
- Sentence Representations
- Contextual Word Representations
Reading Material
- Highly Recommended Reading: Illustrated BERT (Alammar 2019)
- Reference: Learning Sentence Representations (Hill et al. 2016)
- Reference: Skip-thought Vectors (Kiros et al. 2015)
- Reference: Semi-supervised Sequence Learning (Dai and Le 2015)
- Reference: Paraphrase Detection (Dolan et al. 2005)
- Reference: Semantic Relatedness (Marelli et al. 2014)
- Reference: Recognizing Textual Entailment (Dagan et al. 2006)
- Reference: Paraphrastic Sentence Embeddings (Wieting et al. 2015)
- Reference: Paraphrase Database
- Reference: ParaNMT-50M (Wieting et al. 2018)
- Reference: context2vec (Melamud et al. 2016)
- Reference: Learned in Translation: Contextualized Word Vectors (McMann et al. 2017)
- Reference: Deep Contextualized Word Representations (Peters et al. 2018)
- Reference: Bidirectional Transformers (Devlin et al. 2018)
- Reference: Inference -> Generalization (Conneau et al. 2017)
- Reference: Paraphrase ID -> Generalization (Wieting and Gimple 2018)
- Reference: Comparison of Training Objectives (Zhang and Bowman 2018)
Slides: Sentence Representation Slides
Sample Code: Sentence Representation Code Examples