Institutsseminar/2021-07-09

Termin (Alle Termine)
	Datum	Fr 9. Juli 2021, 14:00 Uhr
	Dauer	30 min
	Raum	https://sdqweb.ipd.kit.edu/wiki/Institutsseminar/Microsoft_Teams
	Vorheriger Termin	Fr 2. Juli 2021
	Nächster Termin	Fr 16. Juli 2021

Vorträge

Vortragende(r)	Dennis Grötzinger
Titel	Exploring The Robustness Of The Natural Language Inference Capabilties Of T5
Vortragstyp	Bachelorarbeit
Betreuer(in)	Jan Keim
Kurzfassung	Large language models like T5 perform excellently on various NLI benchmarks. However, it has been shown that even small changes in the structure of these tasks can significantly reduce accuracy. I build upon this insight and explore how robust the NLI skills of T5 are in three scenarios. First, I show that T5 is robust to some variations in the MNLI pattern, while others degenerate performance significantly. Second, I observe that some other patterns that T5 was trained on can be substituted for the MNLI pattern and still achieve good results. Third, I demonstrate that the MNLI pattern translate well to other NLI datasets, even improving accuracy by 13% in the case of RTE. All things considered, I conclude that the robustness of the NLI skills of T5 really depend on which alterations are applied.