language-model

This talk discusses the evolving field of transfer learning, from LSTMs to large language models, and shows new direction on the transferability in large language model.

Adding new language support to the BLOOM multilingual language model for zero-shot prompting.

We introduce xCodeEval, the largest executable multilingual multitask benchmark to date consisting of 25M document-level coding examples from about 7.5 K unique problems covering up to 17 programming languages with execution-level parallelism.

Shows multitask multilingual generalization in language model.

A semi-parametric prompt tuning method improving multitask generalization for parameter-efficient fine-tuning with cross-task zero-shot generalization.

Investigating scaling laws and practical guidance for training language models under constrained compute budgets.

The crystallization of modeling methods around the Transformer architecture has been a boon for practitioners. Simple, well-motivated architectural variations that transfer across tasks and scale, increasing the impact of modeling research. However, …

Over 2,000 prompts for roughly 170 datasets are available through PromptSource framework.

T0 shows zero-shot task generalization on English natural language prompts, outperforming GPT-3 on many tasks, while being 16x smaller!

We propose a trasductive approach for few shot cross-lingual classification.