Overview of the CLEF ehealth evaluation lab 2018

Hanna Suominen, Liadh Kelly, Lorraine Goeuriot, Aurélie Névéol, Lionel Ramadier, Aude Robert, Evangelos Kanoulas, Rene Spijker, Leif Azzopardi, Dan Li, Jimmy, João Palotti, Guido Zuccon

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

41 Citations (Scopus)
8 Downloads (Pure)


In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offered three tasks: Task 1 on multilingual information extraction to extend from last year’s task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year’s pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and multilingual settings that builds on the 2013–17 Information Retrieval tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). Herein, we describe the resources created for these tasks, outline our evaluation methodology adopted and provide a brief summary of participants of this year’s challenges and results obtained. As in previous years, the organizers have made data and tools associated with the lab tasks available for future research and development.

Original languageEnglish
Title of host publicationExperimental IR Meets Multilinguality, Multimodality, and Interaction - 9th International Conference of the CLEF Association, CLEF 2018, Proceedings
EditorsEric SanJuan, Fionn Murtagh, Jian Yun Nie, Laure Soulier, Linda Cappellato, Patrice Bellot, Josiane Mothe, Chiraz Trabelsi, Nicola Ferro
Place of PublicationCham
Number of pages16
ISBN (Print)9783319989310
Publication statusPublished - 14 Sep 2018
Event9th International Conference of the CLEF Association, CLEF 2018 - Avignon, France
Duration: 10 Sep 201814 Sep 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11018 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference9th International Conference of the CLEF Association, CLEF 2018


  • entity linking
  • evaluation
  • health records
  • information extraction
  • information retrieval
  • medical informatics
  • self-diagnosis
  • systematic reviews
  • test-set generation
  • text classification
  • text segmentation
  • total recall

Cite this