University of Sussex
Browse

File(s) not publicly available

SemEval-2007 Task 10: English Lexical Substitution Task

presentation
posted on 2023-06-07, 20:32 authored by Diana McCarthy, Roberto Navigli
In this paper we describe the English Lexical Substitution task for SemEval. In the task, annotators and systems find an alternative substitute word or phrase for a target word in context. The task involves both finding the synonyms and disambiguating the context. Participating systems are free to use any lexical resource. There is a subtask which requires identifying cases where the word is functioning as part of a multiword in the sentence and detecting what that multiword is.

History

Publication status

  • Published

Publisher

The Association for Computational Linguistics

Publisher URL

Pages

6.0

Presentation Type

  • paper

Event name

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007)

Event location

Prague, Czech Republic

Event type

conference

Department affiliated with

  • Informatics Publications

Notes

Originality: This paper describes a new approach for evaluating word sense disambiguation algorithms and the lexical resources that these algorithms rely on by evaluating the ability of a system to find a good substitute for a word in context. In contrast to previous work, the method does not require a man-made inventory. Rigour: We collected annotations from 5 annotators for over 2000 sentences and evaluated systems using our new methodology in an international competition (SemEval 2007). Significance: This paper makes a major contribution because the task makes no requirement on the inventory of word meaning and the task reflects capabilities that will benefit NLP applications such as question answering. Impact: The competition was run in 2007 and we had 8 different international teams participating, with a total of 10 systems. This is a high number of systems for a brand new task. We anticipate that this research will spawn new directions in the evaluation of lexical sense inventories and lexical disambiguation. Outlet: SemEval is the fourth of a successful series of international evaluation workshops, initially called SENSEVAL. This workshop was held at ACL 2007. Full papers were reviewed.

Full text available

  • No

Peer reviewed?

  • Yes

Legacy Posted Date

2012-02-06

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Keywords

    Uncategorised value

    Licence

    Copyright not evaluated

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC