University of Sussex

File(s) not publicly available

Relational evaluation schemes

posted on 2023-06-07, 22:49 authored by Ted Briscoe, John Carroll, Jonathan Graham, Ann Copestake
We describe extensions to a scheme for evaluating parse selection accuracy based on named grammatical relations between lemmatised lexical heads. The scheme is intended to directly reflect the task of recovering grammatical and logical relations, rather than more arbitrary details of tree topology. There is a manually annotated test suite of 500 sentences which has been used by several groups to perform evaluations. We are developing software to create larger test suites automatically from existing treebanks. We are considering alternative relational annotations which draw a clearer distinction between grammatical and logical relations in order to overcome limitations of the current proposal.


Publication status

  • Published

Presentation Type

  • paper

Event name

Beyond PARSEVAL Workshop at the 3rd International Conference on Language Resources and Evaluation

Event location

Las Palmas, Gran Canaria

Event type


Department affiliated with

  • Informatics Publications

Full text available

  • No

Peer reviewed?

  • Yes

Legacy Posted Date


Usage metrics

    University of Sussex (Publications)


    No categories selected