File(s) not publicly available
SemEval-2007 Task 10: English Lexical Substitution Task
presentationposted on 2023-06-07, 20:32 authored by Diana McCarthy, Roberto Navigli
In this paper we describe the English Lexical Substitution task for SemEval. In the task, annotators and systems find an alternative substitute word or phrase for a target word in context. The task involves both finding the synonyms and disambiguating the context. Participating systems are free to use any lexical resource. There is a subtask which requires identifying cases where the word is functioning as part of a multiword in the sentence and detecting what that multiword is.
PublisherThe Association for Computational Linguistics
Event nameProceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007)
Event locationPrague, Czech Republic
Department affiliated with
- Informatics Publications
NotesOriginality: This paper describes a new approach for evaluating word sense disambiguation algorithms and the lexical resources that these algorithms rely on by evaluating the ability of a system to find a good substitute for a word in context. In contrast to previous work, the method does not require a man-made inventory. Rigour: We collected annotations from 5 annotators for over 2000 sentences and evaluated systems using our new methodology in an international competition (SemEval 2007). Significance: This paper makes a major contribution because the task makes no requirement on the inventory of word meaning and the task reflects capabilities that will benefit NLP applications such as question answering. Impact: The competition was run in 2007 and we had 8 different international teams participating, with a total of 10 systems. This is a high number of systems for a brand new task. We anticipate that this research will spawn new directions in the evaluation of lexical sense inventories and lexical disambiguation. Outlet: SemEval is the fourth of a successful series of international evaluation workshops, initially called SENSEVAL. This workshop was held at ACL 2007. Full papers were reviewed.
Full text available