Version	Draft
Date	24/9/99

1 Report Summary *

2 Introduction *

3 The ISLE demonstrator *

2. Report Summary

The purpose of this report is to describe the functionalities of a training environment for spoken language learning system. In this report we will describe the set of speech-driven exercises that have been designed and how these will be integrated in a complete courseware aiming at developing the students’ capabilities to autonomously communicate in the four fundamental linguistic skills: listening, reading, writing and talking.

 

3. Introduction

The main purpose of this document is to describe the features that a training environment for spoken language learning should have in order to train or improve students’ spoken skills.

It is our opinion that the learning process should aim to the development of global communication capabilities. A multimedia foreign language course should lead gradually the student to be autonomously able to communicate in the four fundamental linguistic skills: listening, reading, writing and talking. Therefore, in order to be effective, a training environment willing to develop oral abilities will have to provide a comprehensive learning environment dealing also with the other fundamental linguistic skills. In addition, it should also offer some understanding on the culture and civilisation of the country concerned.

Up to now, existing foreign language courseware, has been rather ineffective as far as the oral capabilities development are concerned. This is mainly due to two different aspects: from one side, the limited diffusion of computer with enough computational power and with satisfactory multimedia capabilities, and from the other, the unsatisfactory performance of the speech recognition systems especially as far as foreign speakers are concerned.

During the development of the first prototype, the work of the consortium has been focused on two major points: a) to demonstrate that the available speech recognition technology has the potential to set up a learning environment with good teaching performances and b) solve the technical issues concerning the integration of the speech-driven exercises within existing foreign language learning courses.

In this document we will therefore describe all the features that a multimedia foreign language course should have in order to successfully integrate oral training activities.

In the second chapter we will describe the features that traditional multimedia foreign language courseware should have, while in the third chapter we will give the description of the voice driven exercises we are developing to perform oral activities. Particular attention will be given to the description of the feedback provided to the users and on the "recovery" actions that the system will take based on the measured users’ performance.

In the last chapter a description of the final version of the ISLE prototype will be proposed including the structure of the course and the graphical layout.

1. Techniques and Educational Strategies for a Multimedia language learning courseware

A multimedia foreign language course should lead gradually the student to be autonomously able to communicate in the four fundamental linguistic skills: listening, reading, talking and writing. It should also offer some understanding on the culture and civilisation of the country concerned.

Given these objectives, we think it may be useful to point out the techniques we are going to use to show their most distinctive aspects without any classificatory pretence.

Direct presentation

By Direct presentation we mean an information’s communication technique that reproduces and transfers on software the typical slide show technique: short test, drawings, graphics. Its main function is to support the tutor in his/her explanation putting at his/her disposal some visual material to comment. There is neither educational dialogue nor exercise nor learning test. Menus, buttons and active areas allow the student to personalise the learning path.

Motion graphics

The main function, and the added value that is given by motion in comparison with still images is the time dimension. The user has a direct perception of how events take place and which elements are involved without following a textual description and look for the objects which the description is referred to each time.

Hypertext

We consider hypertext as a way to organise information that is transversal to all above mentioned typologies. The term "hypertext" was coined by Ted Nelson around 1965 for a collection of documents (or "nodes") containing cross-references or "links" which, with the aid of an interactive browser program, allow the reader to move easily from one document to another. An extension of hypertext to include graphics, sound, video and other kinds of data is a "hypermedia".

2. Educational Strategies

Of course these techniques have to mixed in a way coherent with a specific Educational Strategy that is defined by the kind of relationship that we manage to set up with the learner. From a practical point of view this is implemented identifying evaluating and self-evaluating techniques that concur to determine a specific sequences of learning units.

As far as the target course is concerned, we think it would be better to resort to active strategies. These consider the learner as an agent able to influence the educational process and to decide in a certain way its procedures.

A language learning course should be based on expositive strategies as well, and these should be used for those parts which seem to be linked to secondary objectives. Even if software will implement expositive strategies, this leaves the tutor free to integrated this basic strategy with whatever strategy and whatever interaction with students. A typical method used is to ask learners to formulate definitions and to predict what will happen before explaining the correct version. Instead of receiving the piece of information passively, the learner will tend to activate a mechanism that will compare what he thought before and what the course is telling at present.

In order to be effective a language learning course should also supports discovery activities. The basic assumption in this case, which dates back to the Socratic model of the dialogue or conversation, is that the learner already knows something or has already his/her own experience about the topic or the activity involved; his/her knowledge may be weak and it may need to be taken on to surface, to be rationalised and widened. Learning discovery may be free or guided. The main difference between guided and free discovery consists in the fact that in the former the learner follows a scheme according to logical/time steps set by the program in advance, along a sequence that reproduces the questions that he/she could be able to ask him/herself to grammar rules and vocabulary.

In the free discovery the sequence is not set in advance. The learner follows the scheme freely according to his/her curiosity, evaluating which are the points that he/she knows best/least.

In the end the choice of an educational technique must take educational strategies into account. The educational technique allow in turn to implement some strategies that otherwise couldn’t be made via computer. However fixed and coded relationship between a specific strategy and a technology doesn’t exist: the same technology may in fact implement different strategies.

3. Navigation styles

To navigate in a hypermedia means "finding one’s way around": the style and type of this navigation depends on the hypertext structure and on the different design principles that may guide the development of the multimedia application. The user may be led on a set path or he may be more free to explore. A brief summary of different hypertext and multimedia structures is hereby provided. The choice of the architecture of a hypermedia teaching application may be relevant if related with the pedagogical needs the instrument is developed for: it’s useful to let the user free to explore an encyclopædia, but what if he has to improve his knowledge about a working procedure? Providing a set path with verification tests might be more useful in such a case.

The linear model

According to the linear model, the contents of the course are divided in units or lessons, from the easier topics to the most difficult.

Intermediate assessment and evaluation of the learner’s achievement are provided through the execution of tests such as simple multiple choice, multiple choice with several answers, true/false.

 



Figure 1 The linear model

 

 

Hierarchical model

This model has a hierarchical structure: one or more main topics allow to access more information in different pages, but each page is separated from the others. The user must turn back to one of the main topics to change the direction of his/her navigation. Such a structure requires a very rigid determination of main topics and pages of the second level.

 



Figure 2 The hierarchical model

 

Modality model

The modality model assures a mixture between a linear and a hypertextual structure based on two or more main "modalities". This structure, largely used in Dida*El courses, implies that different types of pages run parallel dealing with the same topic from different points of view. For example there are courses with four modalities, that may be described as "Descriptive" (what is it, how does it look like), "Operative" (How do I…), "Simulating" (Try to do it ) "Testing" (Answer these questions). This structure may be useful in a language course where there may be "grammar", "listening", "reading", "exercises", etc. The user can therefore:

Navigate by topic, choosing, for each setting or lesson ("In the Office", "Shopping", "Holidays") to start from grammar and finish with exercises;

Navigate by modality, reading only "grammar pages" from one lesson to another, for example.

Skip from one page to another (pages have to be very "well-knit" together).

Giving the user a full navigating freedom in this kind of structure may result rather complicated because different buttons and functions have to be provided: a good interface design is required.



Figure 3 The modality model

 

Mixed hypertextual structure

According to the needs of each multimedia course different mixed solutions may be adopted, at different hierarchical levels. For example the user may be forced to go through a lesson before passing to the next one (linear), but he may be free to choose the listening section before grammar (hypertextual and modal), or he may be forced to go back the "dialogue" page to choose the "reading" section (hierarchical, at a lower level).



Figure 4 The mixed hypertextual structure

 

5. The ISLE demonstrator

In this section we are going to describe how we intend to exploit the experiences done with the first ISLE prototype by producing a full English course embedding spoken language learning activities. As already stated in the previous sectionswe think that in order to achieve satisfactory "learning performances" it is necessary to train the users in all the four fundamental oral skills (listening, reading, writing and talkingxxx, xxx, xxx, and xxx).

The spoken language learning activities will therefore be included in an English course we are currently developing. The course is currently structured in 10 units. Each unit is in turn made of 5 lessons. The lesson is therefore a sort of atomic learning entity that is characterised by a dialogue around which all the learning activities will have to be organised. The contents are organised following a practical approach and are centred around the "adventures" of Paolo during a working travel around the StatesBritain.

It will not be necessary, for the purpose of the project (i.e. validation with the users) to set up exercises related to all the 50 lessons; 10 5 lessons will be enough to support the validation activities.

1. Presentation

The SCENARIOS of the course will be:

• red to indicate "error"
• green to indicate "correct"
• black to indicate "the neutral" messages.

1. Phases of Testbed

When the application is launched, the learner has to choice to follow the course in

1. Composition of MENU: Units and Lessons

Units

The MAIN MENU is a collage of images that reproduce the scenarios developed in each unit. For instance, recall that the first scenario is about the arrival of the leading character of the history to Manchester, the graphical elements associated to this scenario are an image of an aeroplane in flight and the grey colour.

Each image of the MAIN MENU is mouse sensitive: when the cursor passed over the image, a short description of the contents in the unit pops up.

Lessons

Once the learner has clicked on a scenario, he sees the MENU of LESSONS. The background of the menu (and also for all the pages of the unit) is the image that identifies the unit, magnified and rendered opaque. The voices of the lessons are on the side of a textbox. When the mouse passes over a voice, the content of the lesson is displayed in the textbox.

 

2. Interface of the tutorial pages

Once a lesson is selected, the learner has at hand a standard interface for reading the texts, listening to the conversations and doing the exercises. Every PAGE is composed by:

1. Buttons

1. Text / Dialogue

• Access the TRANSLATION;
• Listen again at the PRONUNCIATION
• View the associated GRAMMATICAL NOTES;
• View the associated LINGUISTIC NOTES
• View the associated GENERAL NOTES.

The learner can resume the listening of the dialogue after having seen the text with simple click on the key "Dialogue".

1. The Exercises

To do "Role Play" exercises, the learner plays the role of one of the figures appearing in the displayed image and says his parts of the dialogue. The learner may use two buttons: "Listen" or "Answer". The button "Listen" triggers a question to the student. The text of the question is not displayed. To hear again the question, the learner may push again the button "Listen". When the learner feels it is useful, he may record his answer by pushing the "Answer" button.

1. Feedback connected to the diagnosis functions

When the diagnosis functions are triggered, the feedback given depends on the type of errors diagnosed: PHONETIC ERRORS (Phone Analysis) or ACCENT ERRORS (Stress Analysis). These two kinds of errors are always taken into account in the feedback given. The learner cannot choose to have information only about the phonetic errors or about the accent errors.

After a recording of the learner, if the sentence said is recognised, the application writes it in a text box.

If the answer is correct, the wrongly pronounced words and the wrongly accentuated phonemes are displayed in red and are mouse sensitive. A message is also displayed to encourage the learner to check his or her errors:

"I think you had trouble with the words in red; click on them for more information"

FEEDBACK with PHONE ANALYSIS

Clicking on the red word trigger a pop-up box where the wrong phonemes and words are highlighted in red.



Together with the DISPLAY of the error message, there is also a NOTE and, when available, an access to SPECIFIC FEEDBACK with STRESS ANALYSIS



When the learner click on a red word, a pop-up box is displayed with the erroneous word, highlighting the mistake. The syllable or the letter on which the learner has put the wrong accent is highlighted in red and the correctly accentuated syllable or the letter are highlighted in green.

Together with the DISPLAY of the error message, there is also a NOTE and, when available, an access to SPECIFIC REINFORCEMENT EXERCISES.

2. Notes associated to feedback

1. Training

1. Training to improve the abilities of accent pronunciation.