Implications of Potentially Erroneous Feedback in CALL Systems

(In Dutch: “Implicaties van potentieel foutieve feedback voor CALL-systemen”)

The ultimate objective of this project is to advance our understanding of the effects on learning of the frequency and the seriousness of feedback errors in advanced Computer Assisted Language Learning (CALL) systems. The domain in which we work is pronunciation training in the acquisition of Dutch as a second language. Pronunciation training is an outstanding example of a situation in which the need for interactive multimedia environments is obvious. At the same time it is obvious that state-of-the-art speech and language technology will make occasional assessment errors when it comes to evaluating the correctness of responses in interactive student-system dialogs, both with respect to contents and pronunciation quality or fluency. Empirical investigation of the impact of feedback errors in interactive multimedia courses requires that such courses are available.

For this purpose, we have developed Computer-Assisted Pronunciation Training (CAPT) program based on Automatic Speech Recognition for learners of Dutch with different mother tongues (Dutch-CAPT). The program offers realistic language input in the form of short movies with Dutch actors and audio recordings of several native speakers. It contains a wide range of pronunciation exercises including minimal pairs and dialogues and it provides automatic, easy-to-understand, and reliable feedback on the student’s pronunciation, by focussing on eleven problematic Dutch sounds. The program, whose design has been based on a thorough analysis of typical segmental errors made by non-native speakers of Dutch, has been tested in a series of experiments. The subjects of these experiments were beginner learners from various countries across the world, who were following a Dutch course. One group of subjects used this program for a few hours over a period of four weeks; another group used a program that did not provide any feedback on pronunciation for approximately the same amount of time; a third group who did not train with any program was used as control. The pronunciation of these subjects before and after training was scored along the dimension ‘segmental quality’ by a number of experts. The results of these experiments, which are currently being examined, will reveal whether this program, which has the advantage of providing immediate and automatic feedback, does indeed lead to significant improvements in the learner’s pronunciation, and whether it is conducive to higher learning gains than more traditional forms of training without feedback.

This project has been carried out within the framework of the NWO ‘Programma voor onderwijsonderzoek’ (PROO).

Publications

• A. Neri, C. Cucchiarini, H. Strik
  The effectiveness of computer-based corrective feedback for improving segmental quality in L2-Dutch
  To appear in ReCALL. [PDF]
• C. Cucchiarini, A. Neri, F. de Wet, H. Strik
  ASR-based pronunciation training: Scoring accuracy and pedagogical effectiveness of a system for Dutch L2 learners
  Proceedings of Interspeech-2007, Antwerp, Belgium, pp. 2181-­2184. [PDF]
• A. Neri, C. Cucchiarini & H. Strik
  Pronunciation training in Dutch as a second language on the basis of automatic speech recognition.
  Stem-, Spraak- en Taalpathologie, Vol. 15, No. 2, pp. 157-167. [PDF]
• A. Neri, C. Cucchiarini, H. Strik
  Selecting segmental errors in L2 Dutch for optimal pronunciation training.
  IRAL – International Review of Applied Linguistics, 44, pp. 357–404, 2006. [PDF]
• A. Neri, C. Cucchiarini, H. Strik
  Improving segmental quality in L2 Dutch by means of Computer Assisted Pronunciation Training with Automatic Speech Recognition.
  Proceedings of CALL 2006, Antwerp, Belgium, pp. 144-151. [PDF]
• A. Neri, C. Cucchiarini, H. Strik
  ASR-based corrective feedback on pronunciation: Does it really work?
  Proceedings of Interspeech-2006, Pittsburg, USA, pp. 1982-1985. [PDF]
• K. Truong, A. Neri, F. de Wet, C. Cucchiarini, H. Strik
  Automatic detection of frequent pronunciation errors made by L2-learners.
  Proc. of InterSpeech 2005 (IS2005), Lisbon, 4-8 Sept. 2005, pp. 1345-1348. [PDF]
• K. Truong, A. Neri, F. de Wet, C. Cucchiarini, H. Strik
  Automatic pronunciation error detection: an acoustic-phonetic approach.
  Acoustic Society of America Conference, workshop on Second Language Speech Learning, Vancouver 14-15 May 2005.
• K. Truong, A. Neri, C. Cucchiarini, H. Strik (2004)
  Automatic pronunciation error detection: an acoustic-phonetic approach.
  Proceedings of the InSTIL/ICALL Symposium, 17-19 June, Venice, Italy, pp. 135-138. [PDF]
• A. Neri, C. Cucchiarini, H. Strik (2004)
  Segmental errors in Dutch as a second language: How to establish priorities for CAPT.
  Proceedings of the InSTIL/ICALL Symposium, 17-19 June, Venice, Italy, pp. 13-16. [PDF]
• A. Neri, C. Cucchiarini & H. Strik (2003)
  Automatic Speech Recognition for second language learning: How and why it actually works.
  Proceedings of 15th ICPhS, Barcelona, Spain, pp. 1157-1160. [PDF]
• A. Neri, C. Cucchiarini, H. Strik, L. Boves (2002)
  The pedagogy-technology interface in Computer Assisted Pronunciation Training.
  Computer Assisted Language Learning, 15:5, pp. 441-467. [PDF]
• A. Neri, C. Cucchiarini, H. Strik (2002)
  Feedback in Computer Assisted Pronunciation Training: when technology meets pedagogy.
  Proc. of the 10th Int. CALL Conference on “CALL professionals and the future of CALL research”, University of Antwerp, pp. 179-188. [PDF]
• A. Neri, C. Cucchiarini, H. Strik (2002)
  Feedback in computer assisted pronunciation training: Technology push or demand pull?
  Proc. of ICSLP-2002, Denver, USA, pp. 1209-1212. [PDF]
• A. Neri, C. Cucchiarini & H. Strik (2001)
  Effective feedback on L2 pronunciation in ASR-based CALL.
  Proc. of the workshop on Computer Assisted Language Learning, Artificial Intelligence in Education Conference, AIED-2001, San Antonio, Texas, pp. 40-48. [PDF]

Presentations