TQE: Transcription Quality Evaluation

1    Project Title & Acronym and Abstract

Title: Transcription Quality Evaluation
Acronym: TQE
The current proposal is about a completely automatic Transcription Quality Evaluation (TQE) tool. Input is a corpus with audio files and phone transcriptions (PTs). Audio and PTs are aligned, phone boundaries are derived, and for each segment-phone combination it is determined how well they fit together, i.e. for each phone a TQE measure (a confidence measure) is determined, e.g. ranging from 0-100%, indicating how well the fit is, what the quality of the phone transcription is. The output of the TQE tool will consist of a TQE measure and the segment boundaries for each phone in the corpus. The tool will be useful for validating, obtaining, and selecting phone transcriptions, for detecting phone strings (e.g.  words) with deviating pronunciation, and, in general, it can be usefully applied in all research – in various (sub-)fields of humanities and language and speech technology (L&ST) – in which audio and PTs are involved.

2    Coordinator

Name: Dr. H. Strik
Organization: Centre for Language & Speech Technology (CLST) [6], Radboud University
Address: Erasmusplein 1, 6525 HT  Nijmegen, The Netherlands
E-mail: H.Strik [at] let.ru.nl
Tel: +31 – 24 – 3616104
Fax: +31 – 24 – 3612907
Role(s): User and Technology Provider

3    Composition of the Project Team

Name: Drs. R. van Veenendaal
Organization: Institute for Dutch Lexicology (INL – Instituut voor Nederlandse Lexicologie), HLT Agency (TST-Centrale – Centrale voor Taal- en Spraaktechnologie) [7]
Address: Witte Singel/Doelencomplex, Matthias de Vrieshof 2-3, 2311 BZ  Leiden, Netherlands
E-mail: remco.vanveenendaal [at] inl.nl
Tel: +31 – 71 – 5272495 / +32 32654601
Fax: +31 – 71 – 5272115
Role(s): Data Provider

Name: Drs. D. Broeder
Organization: Max Planck Institute for Psycholinguistics (MPI) [8]
Address: Wundtlaan 1, 6525 XD  Nijmegen, The Netherlands
E-mail: Daan.Broeder [at] mpi.nl
Tel: +31 – 24 – 3521103
Fax: +31 – 24 – 3521213
Role(s): Technology Provider

4    CLARIN centre

Organization: Max Planck Institute for Psycholinguistics (MPI) [8]
Name: Drs. D. Broeder
Address: Wundtlaan 1, 6525 XD  Nijmegen, The Netherlands
E-mail: Daan.Broeder [at] mpi.nl
Tel: +31 – 24 – 3521103
Fax: +31 – 24 – 3521213


User survey: A user survey has been carried out, you can find the results here: [PDF] - [DOC]

Application: For the application, go to http://vps8639.xlshosting.net/TQE/

Literature (mentioned in the proposal)


  • [1] C. Cucchiarini, A. Neri, H. Strik (2009) “Oral proficiency training in Dutch L2: The contribution of ASR-based corrective feedback”. Speech Communication 51(10) 853-863.
  • [2] H. Strik, K. Truong, F. de Wet, C. Cucchiarini (2009) “Comparing different approaches for automatic pronunciation error detection”. Speech Communication. 51(10) 845-852.
  • [3] J. van Doremalen, Catia Cucchiarini, Helmer Strik (submitted) Automatic Detection of Vowel Pronunciation Errors Using Multiple Information Sources. Submitted to the IEEE ASRU2009 Workshop (http://lands.let.ru.nl/~doremalen/publications/Doremalen-EtAl-ASRU2009-PED.pdf)
  • [4] M. Gubian, B. Schuppler, J. van Doremalen, E. Sanders, L. Boves (2009) Novelty Detection as a Tool for Automatic Detection of Orthographic Transcription Errors. Proc. of 13-th International Conference on Speech and Computer (SPECOM’2009).
  • [5] C. Van Bael, L. Boves, H. van den Heuvel, H. Strik (2007) Automatic phonetic transcription of large speech corpora. Computer Speech & Language, Volume 21, Issue 4, pp. 652-668.