How NatSpeak Learns

Last Modified: March 22, 2000

People who used DragonDictate for Windows (Dragon's earlier discrete recognition product) will remember that DragonDictate for Windows constantly adapted its acoustic models to your voice after everything that you spoke. This made DragonDictate for Windows continually improve in its recognition accuracy, at the expense of requiring the user to correct every mistake as soon as they occurred.

In Dragon NaturallySpeaking, the product no longer continuously adapts with everything you say. Not only is continuous adaptation somewhat slow, but it implies that the user will be correcting every mistake. In the Dragon NaturallySpeaking, you are not require to correct any mistakes and the system will not get any worse in its recognition performance if you decide not to correct any mistakes.

That said, many people ask whether Dragon NaturallySpeaking will actually adapt further to your voice based on any particular behaviors. The answer is yes. Dragon NaturallySpeaking will adjust its acoustic models of your voice (improving recognition accuracy) under the following circumstances:

1. Dragon NaturallySpeaking adapts whenever you run General Training program. General Training is specifically designed to teach Dragon NaturallySpeaking how you talk. At the end of every General Training session, Dragon NaturallySpeaking will adapt and modify the acoustic models which represent how you speak based on what you said during General Training.

This means that you can improve your recognition accuracy by running General Training again and again. Even after you have initially trained for 18 minutes, you can still go back and run General Training and read any amount of text, and Dragon NaturallySpeaking will learn from that. For example, when I go to a trade show, I often start-up General Training and read from one of the text selections for about five minutes to allow Dragon NaturallySpeaking to adapt to the acoustics of the (often loud) trade show hall.

2. Dragon NaturallySpeaking will also learn about how you speak every time you use the Train Words Dialog. This is the dialog which prompts you to read one word or phrase at a time.

When you speak a word in the Train Words Dialog, Dragon NaturallySpeaking will adapt its acoustic models for that word (or phrase) but also for other words in the vocabulary which share similar sounds. Using the Train Words Dialog is the best way to get Dragon NaturallySpeaking to start recognizing a word which it seems to be having problems with. (For example, I often have trouble getting Dragon NaturallySpeaking to recognize the word "pause" until I train it in the Train Words Dialog at least once.)

The Train Words Dialog has another side effect which none of the other training mechanisms have. In the Train Words Dialog, it is possible to train word so its recognizes when you say something other than what Dragon NaturallySpeaking would normally expect.

For example, you can train the word "hello" so the recognizes when you say "goodbye". Although you can do this more effectively using dictation shorthands (spoken forms in the Vocabulary Editor), this example illustrates important point. When you train a word or phrase in the Train Words Dialog, Dragon NaturallySpeaking will consider changing the pronunciation of that word if you say something completely differently. This is critical if the pronunciation of the word in Dragon NaturallySpeaking's dictionary does not match the way you normally say it.

The dictionary which is supplied with Dragon NaturallySpeaking is very comprehensive in its coverage of various pronunciations. For example, you can pronounce the word "tomato" like "toe-MAY-toe" or like "toe-MAH-toe". Both pronunciations are available in Dragon NaturallySpeaking and both map to the word "tomato". But let's say that there's a word that you pronounce completely differently from how Dragon NaturallySpeaking expects. A classic example is the word "IT". Dragon NaturallySpeaking expects you to pronounce this "it" which rhymes with "bit". But most people actually say "eye-tee".

If you train the word "IT" in the Train Words Dialog, and pronounce it "eye-tee" then Dragon NaturallySpeaking would adjust the pronunciation of this word so that it will recognize it in future. (And the program is smart enough not to fiddle with the pronunciation of "it".

3. Finally, Dragon NaturallySpeaking will adapt whenever you use the Correction Dialog. The Correction Dialog is displayed when you say "Correct That" or "Correct " or you use the Correction Dialog hot key.

If Dragon NaturallySpeaking recognizes something other than what you said, and you use the Correction Dialog to teach Dragon NaturallySpeaking what you actually said, then not only will Dragon NaturallySpeaking correct the text in your document, but it will use that information to learn about how you speak so that it will be less likely to make similar mistakes in future.

For example, say you dictate the sentence "this is a test which has a misrecognition in it" and Dragon NaturallySpeaking types: "this is a test which has a Ms. Recognition and". You say "Correct Ms. Recognition" and select the word "misrecognition" from the list in the Correction Dialog (or type it in). Dragon NaturallySpeaking will then correct your document, but it will also learn from the correction.

Dragon NaturallySpeaking will adapt the acoustic models for your voice based on the actual thing that you said (which you told it in the Correction Dialog). This is why this important to only use the Correction Dialog to correct a misrecognition and not to change your mind.

Dragon NaturallySpeaking will also adapt on the text in the same utterance (phrase separated by pauses) at the same time. This is based on the idea that if you correct part of the utterance, and do not correct another part of the utterance, then the part you do not correct must already be correct (what a mouth-full).

In our example, there were actually two misrecognitions. One was the word "misrecognition" which we corrected above. The other was the word "and" which should have been "in it". Because Dragon NaturallySpeaking will adapt on the entire utterance, you should correct both misrecognitions.

Because there are sometimes two mistakes in a single utterance, Dragon NaturallySpeaking will actually delay adapting until after you make another correction. This means that Dragon NaturallySpeaking will not adapt on the sentence "this is a test which has a misrecognition in it" when you correct "misrecognition" until after you make another correction. If you correct "in it" after you correct "misrecognition" than Dragon NaturallySpeaking will adapt on the entire sentence after the second correction.

Because of this delayed adaptation, Dragon NaturallySpeaking often seems to think for a long time after some corrections which seem easy.

In summary, Dragon NaturallySpeaking will adapt its model of your voice whenever you run General Training, use the Correction Dialog, or use the Train Words Dialog. The more Dragon NaturallySpeaking learns about your voice, the better your recognition performance will be. However, you are not required to correct errors when you dictate in order for Dragon NaturallySpeaking to learn more about your voice. This leaves you free to concentrate on your work instead of on training the system.

This web page (http://www.synapseadaptive.com/joel/HowNatSpeakLearns.html) was last modified on March 22, 2000. The contents of this page are (c) Copyright 1998-1999 by Joel Gould. All Rights Reserved. See Copyright Information for more details.