Micro Blog Contextualization CLEF & Inex tracks data and tools

Cultural Microblog Contextualization CLEF 2016 WorkShop

Corpus

• English
• XML retrieval system powered by Indri
• Perl API for the XML retrieval system using standard LWP
• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)
• Spanish
• XML retrieval system powered by Indri
• Perl API for the XML retrieval system using standard LWP
• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)
• French
• XML retrieval system powered by Indri
• Perl API for the XML retrieval system using standard LWP
• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)
• Portuguese
• Sentence retrieval system powered by Indri
• Perl API for the XML retrieval system using standard LWP
• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)

Tweet Contextualization INEX 2013 & 2014 track

Tools

• Baseline system for 2013 & 2014
• Perl API for the baseline using standard LWP
• Bash script to convert the API output using the baseline format into a Tweet Contextualization run

Corpus

• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)
• The same corpus split into ten archives of 100 folders for slow or restricted internet connexions:
  1. Part 1 (tgz archive)
  2. Part 2 (tgz archive)
  3. Part 3 (tgz archive)
  4. Part 4 (tgz archive)
  5. Part 5 (tgz archive)
  6. Part 6 (tgz archive)
  7. Part 7 (tgz archive)
  8. Part 8 (tgz archive)
  9. Part 9 (tgz archive)
  10. Part 10 (tgz archive)
• List of wikipedia page titles in the corpus with (gz archive)

Topics

• 2013
  • 2013 Topics in xml format
  • 2013 Topics in txt format
• 2014
  • 2014 Topics in xml format
  • 2014 Topics in txt format

Baselines

• 2013
  • Baseline run: sentence retrieval based on Indri
  • Reference run generated using the perl API
  • Parameter file to generate the reference with the perl API
  • Reference document extraction (XML output)
  • Parameter file for the reference document retrieval in XML format using the perl API
  • Reference document extraction (PoS tagged)
  • Parameter file for the reference document retrieval with PoS annotations using the perl API
• 2014
  • Reference document extraction (XML output)
  • Parameter file for the reference document retrieval in XML format using the perl API
  • Reference document extraction (textual content)
  • Parameter file for the reference document retrieval (textual content) using the perl API

Evaluation 2013

• Evaluation program for informativeness (perl script)
• Prior reference
• Pool reference
• All references
• Results based on prior reference
• Results based on pool reference
• Results based on all references

Pilot tasks

• French subtrack
• French corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)
• Spanish subtrack
• Spanish corpus in a single xml file (gzip compression).
• Corpus split into 1000 folders, one file per page (tgz archive).
• Spanish topics in text format.

Tweet Contextualization INEX 2012 track

Tools

• Baseline system for 2012
• Perl API for the baseline using standard LWP
• Bash script to convert the API output using the baseline format into a Tweet Contextualization run -->

Corpus

• Corpus in a single xml file (gzip compression)
• Corpus splited into 1000 folders, one file per page (tgz archive)
• List of wikipedia page titles in the corpus with (gz archive)

Topics

• Topics in json format
• Topics in txt format (two columns)
• Example of file that can be used with the APIs (generates as many files as topics with PoS tagged results)

Evaluation

• Official Informativeness evaluation by organizers
• Readme
• Average scores
• Detailed scores
• Reference (q-rels)
• Official Readability evaluation by participants
• Readme
• Estimated Average scores
• Failed estimations
• Source code of the readability evaluation interface (without data)
• Unofficial Entity evaluation (Update 2012/12/10)
  • Readme
  • Average scores
  • Detailed scores
  • Entity Reference (q-rels)
  • Evaluation Perl script
  • Document Paragraph source for Reference
  • Submitted runs converted into entity runs

QA@INEX 2011

Tools

• Baseline system
• Perl API for the baseline using libcurl
• Perl API for the baseline using standard LWP

Corpus

• Corpus in a single xml file (gzip compression)
• Corpus split into 1000 folders, one file per page (tgz archive)

Topics

• Topics in xml format
• Topics in csv format

Evaluation

• Evaluation ToolKit
• Readability evaluation
• QA 2011 Track Overview preprint

Other tools

• Restad: a YeSQL XML indexer
• Tool to split the corpus XML file into folders and distinct files
• Tools to generate a corpus in TC XML format from a dump of the wikipedia

Contact: eric.sanjuan at univ-avignon.fr