↑

MPC: My Pronunciation Coach – STW Valoristion Grant phase 1

Title: ‘My Pronunciation Coach’ (MPC)
Exploring the potential of speech technology in language learning

Abstract:
As a result of increasing internationalization there is a growing demand from the business community for people who speak foreign languages well.  The number of people that want or need to learn foreign languages is steadily increasing. However, there are not enough teachers for all of them, especially for practicing pronunciation, which is virtually impossible in a standard classroom setting. We plan to develop a computer program, called ‘My Pronunciation Coach’ (MPC), that students can use to practice pronunciation anytime and anywhere they want. The first version of this system will be intended for Dutch students that want to learn English (fixed language pair). However, we will also look at the commercial and technological aspects of porting the program to other languages and adapting the content to the interests of different user groups.
With MPC students can practice their pronunciation in English: they produce utterances, MPC assesses their pronunciation, checks whether any pronunciation errors were made, provides feedback on the errors detected, and suggests appropriate exercises for improvement. Dedicated technology will be developed to make it possible to perform these tasks automatically within a computer program. Since the system will have to cope with English spoken with a whole range of Dutch accents, this is a challenging task requiring innovative technology, developed and optimized for this specific task. MPC will be web based, providing the students the opportunity to use it anytime and anywhere they want.
Our final goal is to develop a high-end product that educational institutions teaching English can use to support teachers in providing feedback to their Dutch students on their pronunciation of the English language.

Partners:

Goal:
We intend to develop and market MPC, a product for practicing English pronunciation by Dutch students. The market survey that has already been conducted shows that there is currently a strong tendency towards internationalization. By a quick introduction on the market, we would be able to use the commercial potential of the MPC product as much as possible. The knowledge required for developing the product is present and can be fully utilized. The STW Valorization Grant will be used to conduct both a commercial and technological feasibility study, aimed at developing a prototype product.
The goals encompass the following aspects:

  1. Drawing up an overview of the products currently available on the market, including the prices and quality. This involves all products irrespective of the language pairs they use.
  2. Making an inventory of user demands. Which factors are important to users with regard to the Computer Assisted Pronunciation Training (CAPT) application to be developed.
  3. Drawing up a preliminary financial plan
  4. Making an inventory of frequent errors made by Dutch people learning English.
  5. Making an inventory of existing technology for handling non-native speech (for speech recognition, assessment, error detection, etc.).
  6. Making an inventory of the feasibility of porting technology between languages.
  7. Carry out pilot experiment 1; porting the technology developed for detecting errors in the pronunciation of sounds for Dutch to English.
  8. Carry out pilot experiment 2; porting the technology developed for detecting errors in the word stress for Dutch to English.

Our final goal is to develop a high-end product that educational institutions teaching English can use to support teachers in providing feedback to their Dutch students on their pronunciation of the English language.

References (mentioned in the proposal):

  • Abercrombie, D. (1991[1956]) Teaching pronunciation, in A. Brown (ed.), Teaching English Pronunciation: A Book of readings, Routledge: London, 87-95
  • Benzeghiba, M., R. D. Mori, O. Deroo, S. Dupont, T. Erbes, D. Jouvet, L. Fissore, P. Laface, A. Mertins, C. Ris, R. Rose, V. Tyagi, and C. Wellekens, Automatic speech recognition and speech variability: a review, Speech Communication, vol. 49, no. 10-11, pp. 763-786, 2007.
  • Bohn, O. and J. Flege, The production of new and similar vowels by adult German learners of English studies in second language acquisition, vol. 14, no. 2, pp. 131-158, 1992
  • Celce-Murcia, M., Brinton, D.M., Goodwin, J.M., (1996) Teaching Pronunciation, Cambridge: CUP.
  • Collins, B., and Mees, I. (1981). The Sounds of English and Dutch. The Hague: Leiden University Press.
  • Collins, B., and Mees, I. (2003). The Phonetics of English and Dutch. (5th ed.). Leiden: Brill.
  • Compernolle, D. van (2001) Recognizing speech of goats, wolves, sheep and..non-natives, Speech Communication, vol. 35, no. 1-2, pp. 81-79, 2001
  • Cucchiarini, C., Neri, A., and Strik, H., (2009) Oral proficiency training in Dutch L2: The contribution of ASR-based corrective feedback, Speech Communication, pp. 853-863.
  • Doeleman, R. (1998). Native reactions to nonnative speech. Tilburg: Tilburg University Press.
  • Doremalen, J. van, Strik, H., Cucchiarini, C. (2009) Optimizing non-native speech recognition for CALL applications. Proceedings of Interspeech-2009, Brighton, UK, pp. 592-595.
  • El Tatawi, M. (2002) Corrective feedback in second language acquisition, Working papers in TESOL and Applied Linguistics, Vol. 2, 1-19.
  • Eskenazi, M.. 1996, Detection of foreign speakers’ pronunciation errors for second language training – preliminary results, Proceedings of the International Conference on Spoken Language Processing (ICSLP), Philadelphia.
  • Flege, J. (1995) Second language speech learning: theory, findings and problems. In W. Strange (Ed.) Speech perception and linguistic experience, Baltimore: York Press, 233-272.
  • Franco, H., Abrash, V., Precoda, K., Bratt, H., Rao, R., Butzberger, J., Rossier, R., Cesari, F., 2000, The SRI Eduspeak System: recognition and pronunciation scoring for language learning, Proceedings ESCA ETRW INSTiL2000, Dundee Scotland, p. 123- 128.
  • Gussenhoven, C., and Broeders, A. (1976). The pronunciation of English. A course for Dutch learners. Groningen: Wolters-Noordhoff-Longman.
  • Gussenhoven, C., & Broeders, A. (1997). English pronunciation for student teachers. (2nd ed.). Groningen: Wolters-Noordhoff.
  • Kim, Y., Franco, H., Neumeyer, L, 1997, Automatic pronunciation scoring of specific phone segments for language instruction, Proceedings Eurospeech 97, Rhodes Greece, p. 645-648.
  • Logan, J. Lively, S. and Pisoni, D. (1991) Training Japanese listeners to identify English /r/ and /l/: a first report, Journal of the Acoustical Society of America, 89, 874-886.
  • Lyster, R. and Ranta, L. (1997) Corrective feedback and learner uptake, Studies in Second Language Acquisition, 19, 37-66.
  • Koet, T. (2007). Polder English in Dutch ears. PhD Thesis, Utrecht/Amsterdam.
  • Mackey, A. (2006) Feedback, noticing and Instructed Second Language Learning, Applied Linguistics, 27, 405-430.
  • Mak, B.S., Ng, M., Tam, Y., Chan, Y., Chan, K., Leung, K., 2003, PLASER; Pronunciation learning via automatic speech recognition, Proc. HLT-NAACL, p. 23-29.
  • Menzel, W., Herron, D., Bonaventura, P., Morton, R. (2000) Automatic detection and correction of non-native English pronunciations, Proceedings of InSTILL, Dundee, Scotland, 49-56.
  • Munro, M.J., Derwing, T.M. (1995) Foreign accent, comprehensibility and intelligibility in the speech of second language learners, Language Learning 45, 73-97.
  • Neumeyer, L., Franco, H., Weintraub, M., and Price, P., 1996, Automatic Text-Independent Pronunciation Scoring of Foreign Language Student Speech. Proceedings International Conference on Spoken Language Processing, Philadelphia, PA.
  • Precoda, K., Halverson, C., Franco, H., 2000, Effect of speech recognition-based pronunciation feedback on second language pronunciation ability, Proc.ISCA ITRW INSTiL2000, Dundee Scotland, p.102-105.
  • Strik, H., Truong, K., de Wet, F. and Cucchiarini C.,(2009) Comparing different approaches for automatic pronunciation error detection”, Speech Communication, pp. 845-852.
  • Van der Haagen, M. (1998). Caught between norms. The English pronunciation of Dutch Learners. The Hague: Holland academic graphics.
  • Van de Voort, M. (1999) Gluren naar de buren, Nijmegen: UTN.
  • Wang, H., & Van Heuven, V. (2003). Mutual intelligibility of Chinese, Dutch and American speakers of English. Linguistics in the Netherlands 2003, pp. 213-224.
  • Werker, J.F. and Tees, R.C. (1984) Cross-language speech perception: Evidence for perceptual reorganization during the first year of life, Infant Behavior and Development, 7, 49-63.
  • Witt, S. and Young, S.J., 2000, Phone-level pronunciation scoring and assessment for interactive language learning, Speech Communication, 30 (2-3). pp. 95-108. ISSN 0167-6393.