Getting to Know Talkback, the Onboard Screen Reader

Talkback is the screen reader developed by the Google Eyes-Free Project. It provides spoken feedback, and for users of Android 2.1 and higher, it also includes a soft keyboard and a soft d-pad for on-screen use. While Talkback provides enough functionality to access most of the phones' features, it doesn't allow for all of the control blind and visually impaired users are accustomed to. Also, its soft d-pad doesn't give users access to some important areas of the phone.

What Talkback does:

• Announces Alarms, incoming text messages and emails, and other notifications.
• Speaks the entire contents of a new screen.
• Announces placement in edit fields, characters as they are typed, and text being deleted.
• Launches a soft (on-screen) typing keyboard.
• Reviews text by character, word, line, sentence, paragraph, or entire text in edit fields.
• Displays soft d-pad on screen so that users can scroll to and activate most of the buttons and other elements that can be reached with a physical navigational controller.
• Speaks the names of items with focus as the hard or soft navigational controller is being used, the presence of unlabeled buttons and the state of checkboxes as focus moves over them, the names of soft buttons on the touch screen as they are pressed, and the "Clicked" indication as items or buttons are activated.
• Repeats or spells last utterance in response to keyboard shortcuts.
• Interrupts speech when the proximity sensor is touched. The proximity sensor is often located in the upper left-hand corner of the screen (portrait) or in the lower left-hand corner of the screen (landscape. Oddly, sometimes touching the left edge (portrait) along the center can stop speech as well.
• Speaks battery level, time, and connection status in response to keyboard shortcuts.
• Changes behavior regarding silent mode, system notifications, caller ID, and the proximity sensor via Accessibility Preferences.
• Speaks faster or slower via the phone's Text to Speech settings
• Speaks louder or softer in response to the phones' physical volume control.

What Talkback Doesn’t Do:

• Unable to explore the touch screen without also activating buttons.
• Unable to move through text by character, word, sentence, paragraph, or entire screen outside of edit fields.
• Unable to signal the difference between upper and lower case through changes in pitch or caps announcement during typing or editing.
• Unable to echo by word during typing.
• Unable to use the soft d-pad in status bar notifications, in Recently Used screen, in Android System screens where users choose default apps for specific actions, in screens launched by the Menu key, and in screens launched by long-pressing the selector on an item.

Exceptions::

• The keyboard shortcuts to repeat and spell the last utterance may not work as expected in self-voicing apps like the Eyes-Free Shell. For best results, the Talkback keyboard should be set to Hidden in these cases.
• In K9 Mail by Dog Walkers, users can press the selector in an open message to enter review mode and arrow through a message line by line.
• When using Ideal Web Reader, users can move forward or backward through a web page in chunks, sentences, words, and characters by adjusting the level of granularity. A chunk most often ends with a hard return,.

Turning on Talkback

To turn on Talkback for the first time, sighted assistance is required. If the app is already on the phone, users should search for it on the Android Market and reinstall it to make sure they have the latest version. The new version replaces the existing version, and speech is not lost. For instructions on how to install apps, refer to the post on the Android Market.

To turn on Talkback or any other screen reader, the steps are the following. Only one screen reader should be active at a time; however, it is safe to turn on the second screen reader before turning off the first:

1. Unlock the screen if necessary.
2. Go into Settings.
a. From the stock home screen, press the Menu button; then Find the Settings item and activate it by touching the soft button or by navigating to and pressing the selector on it. Menu may be a soft button on the touch screen or a physical key on the handset.
b. From the Eyes-Free Shell, down-stroke to Applications in the 8 position, type the letter S, scroll to Settings if not there already, and press enter, search, or the selector.
c. From Mobile Accessibility, swipe to Apps, double tap or press the selector, then swipe to and press the selector on Settings.
3. FIND the Accessibility item and activate it by touching the soft button or by navigating to and pressing the selector on it.
4. Make sure the Accessibility option is checked. If it isn't, touch the soft button or scroll to it and press the selector .
5. Find the Talkback option and check it by touching the soft button or navigating to it and pressing the selector .
6. If another screen reader is checked, scroll to it and press the selector to uncheck it. The screen reader behavior usually changes after the second screen reader is no longer checked.
7. Return to the Home screen by pressing the Back button several times or by pressing the Home button once. Back and Home may be soft buttons on the touch screen or physical keys on the handset.

Once Accessibility and Talkback or another screen reader are checked, blind and visually impaired users can operate their Android devices independently.

Setting Talkback Keyboard as the Input Method

In order to use the Talkback soft keyboard or soft d-pad, users must set the Talkback Keyboard as an input method. This is done as for any other keyboard. It's a two-phase process. In the first phase, Talkback Keyboard is set as a possible input method (Steps 1-5 below). In the second, it is set as the input method of choice (Steps 6-10 below).

Once Talkback Keyboard is set, it remains the input method even after the phone is powered down. When the phone restarts or the Talkback keyboard is reset, the keyboard opens in Navigating mode. Information on the Talkback keyboard modes is presented in a later section of this post.

To set the Talkback keyboard as an input method, do the following:

1. From your preferred home screen, go into Settings, using the steps described earlier in this post.
2. Navigate to and press the selector on Language and Keyboard or Locale and Keyboard.
3. Navigate to and press the selector on Talkback Keyboard to check it as an available input method.
4. Navigate to and press the selector on Talkback Keyboard Settings, and make any adjustments you choose. Options are these:
a. Launch Tutorial – Pressing the selector here helps you learn how to use the Talkback Keyboard. Sometimes the tutorial works correctly, speaking the text of each screen; other times, it force closes or fails to read the text of the screens automatically. Press Back to exit and try again if the tutorial doesn't work.
b. Autotyping mode – checking this item opens the soft keyboard when focus moves to an edit field. This is best for phones without physical keyboards. Users who prefer the hardware keyboard may want to leave this unchecked.
c. Haptic Feedback – checking this item causes the phone to emit subtle vibrational bursts when typing on the soft keyboard or when navigating with the soft d-pad.
d. Auditory Feedback – Checking this item causes the phone to make a soft clicking sound when typing on the soft keyboard and when scrolling with the soft d-pad.
e. Auto capitalization – checking this item puts capital letters at the beginning of the text and after final punctuation, like periods and question marks.
f. Show settings key – This is a dropdown list that puts a Settings key on the soft keyboard. Options are Automatic, Always, and Keep Hidden. The Settings key opens the Input method, Talkback Keyboard Settings, and Accessibility screens.
g. Voice input mic – This is a dropdown list that puts a microphone key for voice recognition in the lower left-hand corner of the soft keyboard. Options are On Main Keyboard and On Symbols Keyboard.
5. Press Back several times or Home to return to the Home screen. The first phase of the process is complete.
6. Launch an app with an edit field, and move focus to the edit box. In this example, go to the stock phone application, but any app will do. This is the beginning of the second phase of the process.
a. From the stock launcher, navigate to and press the selector on All Applications or Sliding Drawer. Then navigate to and press the selector on Phone. On both screens, items are laid out in a grid, so it is necessary to arrow up and down and left and right. Once you're in the stock phone app, scroll to the edit field.
b. From the eyes-Free Shell, down-stroke to Applications in the 8 position. Type the letter p, or down-arrow to the first app that begins with P. Continue scrolling down until you hear, "Phone." Then press enter, the selector, or Search. Once you're in the stock phone app, scroll to the edit field.
7. Long-press the selector in the edit field. This means that you should press and hold the selector down for 2 to 3 seconds. In forums and manuals, the word "hold" is often used.
8. Pres the selector on Input Method, when Input Method is spoken. In forums and manuals, the word "touch" or "tap" is often used for pressing or short-pressing.
9. Navigate to and press the selector on Talkback Keyboard to check it as the input method of choice.
10. Press Back several times or Home once to return to the Home screen.

The Talkback Keyboard is in Navigating mode at this point.

Becoming Familiar with the Talkback Keyboard Modes

The Talkback keyboard operates in 3 modes: Typing, Navigating, and Hidden. These are only available when Talkback keyboard is set as the input method.

• Typing mode puts a soft or virtual keyboard on screen. This is the mode to choose when entering text, phone numbers, or URLs in edit fields.
• Navigating mode puts a soft or virtual directional pad on screen. This is the mode to choose when scrolling through items or controls using the touchscreen. Note that, when the phone is restarted or when the Talkback keyboard is set as the input method, the keyboard opens in Navigating mode. Note also that the soft d-pad cannot be used in status bar notifications, in the Recently Used screen, in Android System screens where users choose default apps for specific actions, in screens launched by the Menu key, and in screens launched by long-pressing the selector on an item.
• Hidden mode removes both the soft typing keyboard and the soft d-pad from the screen. This is the mode to choose when using the physical navigational controller or hardware keyboard.

To cycle through the modes, users long-press the volume-up and volume-down keys until the phone emits a short vibrational burst. Long-pressing volume-down repeatedly cycles through the modes in one direction (i.e., Hidden, Typing, Navigating), and long-pressing volume-up repeatedly cycles through the modes in the opposite direction (i.e., typing, Hidden, Navigating). This is convenient when editing, as a single long-press of a volume key activates the desired mode.

If long-pressing the volume keys does not cycle to another keyboard mode, the likely cause is that focus is on a screen where the Talkback keyboard is in active (e.g., on the list of status bar notifications or in a set of menu options).

Using the Talkback D-Pad

When the Talkback keyboard is launched for the first time or when the phone is restarted, the Talkback keyboard is in Navigating mode. This means that a square occupies the lower third of the screen and that users can flick a finger up, down, left, or right to scroll to on-screen controls and edit fields.

In slider phones, when the hardware keyboard is tucked away, the soft or virtual d-pad is above the capacitive buttons at the bottom of the screen (portrait orientation), regardless of whether auto-rotate is turned on or off, but when the hardware keyboard is pulled out, the soft d-pad is above the center of the physical keyboard.

• To explore the boundaries of the soft d-pad, place a finger an inch (3 cm) or so from the bottom of the screen; then slide the finger up or down until the phone vibrates. If you slide the finger to left or right, notice that the phone vibrates along a horizontal line all the way across the screen. These two lines are the top and bottom margins of the soft d-pad.
• To find alternative Back, Home, Menu, and Search buttons, slide a finger along the horizontal lines that mark the top and bottom boundaries of the soft d-pad. At the left and right ends of the top line are the Back and Home buttons, and at the left and right ends of the bottom line are the Menu and Search buttons.
• To activate the alternative Back, Home, Menu, or Search button, slide your finger to the button, and when Talkback announces "back," "home," "menu," or "search," lift your finger. To cancel, slide your finger away before lifting it.
• To scroll with the soft d-pad, flick a finger up or down. If information is presented in a grid, you may need to flick left or right once to move to the next row of information, then flick up or down again to move across the row. A flick is a short, quick sliding gesture, often involving a rolling of the fingertip (e.g., from pad to tip to fingernail) as the finger slides over the screen. For best results, flick near the bottom of the soft d-pad.
• To press an on-screen button while using the soft d-pad, briefly touch the general center of the d-pad area with one finger.
• To long-press an on-screen button while using the soft d-pad, place a finger in the general center of the d-pad area and hold it on the screen for 2 to 3 seconds.

Note that the soft d-pad cannot be used in status bar notifications, in the Recently Used screen, in Android System screens where users choose default apps for specific actions, in screens launched by the Menu key, and in screens launched by long-pressing the selector on an item. This is why blind and visually impaired users must choose phones with physical navigational controllers.

Typing with the Talkback Keyboard

To Type with the on-screen soft or virtual keyboard that is part of Talkback, users must first set Talkback keyboard as their input method, by following the instructions in an earlier section of this post.

Once the Talkback keyboard is the input method, a standard typewriter keyboard appears in the lower third of the screen when focus moves to an edit field or when the Talkback keyboard is put in Typing mode. The soft keyboard is in portrait orientation If there is no physical keyboard or if the physical keyboard is closed, but the soft keyboard is in landscape orientation if the physical keyboard is open.

To use the soft keyboard, do the following:

1. Move focus to an edit field.
2. The phone announces, "Typing," to signal that the keyboard is on the screen. If this doesn't happen, long-press the volume-up/down until the phone vibrates and announces the keyboard mode. Repeat this process if necessary until you hear, "Typing." Note that, for the soft keyboard to open automatically when focus moves to an edit field, go into Settings/Language and Keyboard/Talkback Keyboard Settings and check the autotyping option.
3. Place a finger lightly in the lower third of the screen, and slide it over the letters and other keys, lifting your finger when you hear the character or key you want to type. The phone emits a soft click to signal that the key has been pressed.
4. Continue typing in this manner, touching delete and lifting your finger to erase characters when necessary.
5. When you are done, long-press the volume-down to put the keyboard in Navigating mode; then flick to scroll out of the edit field. The other option is to long-press the volume-up key to put the keyboard in Hidden mode; then scroll out of the edit field with the physical navigational controller.

Note: The Talkback soft keyboard has the standard qwertty layout with the following minor variations.

• The number row is not on the main keyboard, but at the top of the symbols keyboard.
• Comma and period are to the left and right of the spacebar respectively.
• The Return/Enter and the Delete keys are in the lower right-hand corner of the keyboard, near the M and spacebar.
• The Shift, Symbols, Microphone, and Settings keys are in the lower left-hand corner of the keyboard near the Z and spacebar.
• The alt key is on the symbols keyboard, not the main keyboard.

Becoming Familiar with Special Keys on the Talkback Soft Keyboard

A few of the keys on the on-screen keyboard that is part of Talkback have the following special uses or behaviors.

• Alt – The alt key is in the lower left-hand corner of the Symbols keyboard above the symbols key. If you touch it once, the phone announces, "Alt on," and an alternate keyboard with accents and other special characters appears. This keyboard stays on the screen until the alt or the spacebar is touched. Note that there is no quick fix option in the Talkback Keyboard Settings, so typing a tilde followed by an n does not produce an ñ.
• Microphone – The Mic key is in the lower left-hand corner of the screen near the spacebar. After you touch it, the phone vibrates briefly. You have about 30 seconds to speak an utterance. Speak at a conversational rate and say any punctuation you think appropriate as the phone does not insert it automatically (e.g., hi comma how are you question mark). After a few seconds, the phone speaks what it thinks you said and prints the utterance on the screen. You can edit it or press the mic key again to add another sentence. If you edit the text, be sure to move focus to the end before continuing. At this time, only the first sentence begins with a capital letter.
• Settings – The settings key is in the lower left-hand corner of the screen to the left of the Mic key. If you touch it, you are presented with the following 4 options: (1) Select Input Method, which you can use to change the soft keyboard; (2) Talkback Keyboard Settings, which you can use to access the tutorial or change options like Auto Type or Auto Capitalization; (3) Accessibility Settings, which you can use to change your screen reader or enable or disable services like Kickback and Soundback; and (4) Cancel, which you can use to exit the screen.
• Shift – The shift key is in the lower left-hand corner of the keyboard above the Symbols and Settings key. If you touch Shift once, the phone says, "Shift on," and the following letter is capitalized. If you touch shift twice, the phone says, "Shift-lock on," and all subsequent characters are typed in uppercase until you touch the shift again once, prompting the phone to say, "Shift-lock off."Talkback does not indicate that letters are upper- or lowercase during typing or review.
• Symbols – The symbols key is in the lower left-hand corner of the screen to the left of the Settings key. If you touch it once, the phone announces, "Symbols keyboard on," and an alternate keyboard with the number row and commonly used punctuation appears. This keyboard stays on the screen until the Symbols key or the spacebar is touched. If you touch the alt key while in the symbols keyboard, the phone announces, "alt on," and another alternate keyboard with less commonly used punctuation, like the pipe and curly brackets, appears. This additional keyboard stays on the screen until the alt key or the spacebar is touched. In both cases, pressing the spacebar returns to the standard typing keyboard.

Reviewing Text While Typing or Editing

With current versions of Talkback, users of Android 2.1 and higher can review text while in edit fields. This means they can review emails, text messages, and notes during composition or double-check that items are spelled correctly in search fields (.

Text can be reviewed by character, word, sentence, paragraph, or entire text. These are the levels of granularity, Android's term for review levels.

• To review text using hardware controls, simply scroll with the physical arrows, d-pad, trackball, or trackpad. Scrolling left and right moves focus to the previous or next character, word, sentence, or paragraph, which the phone speaks. Scrolling up and down moves focus to the previous or next line; the phone speaks the character with focus, but not the entire line.
• To set a new level of granularity using hardware controls, pres the alt key plus the up or down arrow. Repeatedly pressing up-arrow while holding down the alt key increases granularity, cycling through Character, Word, Sentence, Paragraph, and Entire Text. Repeatedly pressing down-arrow while holding down the alt key decreases granularity, cycling through Entire Text, Paragraph, Sentence, Word, and Character.
• To review text using soft controls, put the Talkback keyboard in Navigating mode with a long-press of the volume keys, and flick your fingers left or right in the soft d-pad area. It is possible to flick while in the soft typing keyboard, but characters are sometimes inadvertently typed. In either case, flicking in the correct area of the screen takes some practice. A flick is a swift sliding gesture, often involving a rolling of the fingertip from a corner of the nail, to the center of the nail, to the opposite corner of the nail as the finger moves across the screen.
• To set a new level of granularity using soft controls, tap or double-tap with two fingers in the soft d-pad area. The fingers must be held more than half an inch or one centimeter apart, and the tap must be a brief touching of the screen without any sliding. This is important because it's easy to activate an on-screen control (by holding the fingers too close together) and even easier to move focus to another on-screen control (by allowing the fingers to slide on the screen). Repeatedly tapping with two fingers in the d-pad area increases granularity, cycling through Character, Word, Sentence, Paragraph, and Entire Text. Repeatedly double-tapping with two fingers in the d-pad area decreases granularity, cycling through Entire Text, Paragraph, Sentence, Word, and Character.

Troubleshooting the Talkback Keyboard

Though the Talkback soft Keyboard and soft d-pad are handy, they have their quirks. The following are the most common problems and likely solutions.

• The hardware navigational controller is sluggish. – If this happens, use the volume keys to put the Talkback keyboard in Hidden mode.
• The hardware navigational controller is sluggish even when the Talkback Keyboard is in hidden mode. – If this happens, restart Talkback by pressing the Restart Talkback item in the All Applications list. For this item to be present, Accessibility Preferences must be installed from the android Market. Another way to resolve the problem is to restart the phone.
• I can not move focus out of an edit field when using the soft keyboard or soft d-pad. – If this happens, use the volume keys to put the Talkback Keyboard in Navigating mode.
• I cannot move focus out of an edit field even when the Talkback keyboard is set to Hidden Mode. – If this happens, press the alt key twice; then press the alt key again once.
• Nothing happens when I flick on the soft d-pad. – If this happens, slide your finger more quickly over the screen, or flick closer to the bottom edge of the screen.
• The wrong letters appear on the screen when I use the hardware keyboard. – If this happens, you probably pressed the alt or shift+alt combination to type numbers or special characters, and you are unable to return to the standard typing keyboard. Pause for a moment. Then press the alt or shift+alt keys twice, then once more. This should bring up the typing keyboard.
• Different letters appear on the screen when I use the soft keyboard. – If this happens, pay attention to what your finger is doing. Be sure to slide it lightly over the screen. Be sure you are not rocking your fingertip from side to side so that it touches adjacent letters. Be sure you're not rocking your fingertip vertically so that the phone doesn't mistake the movement for a lifting of the finger.

Adjusting Speech Rate

To adjust speech rate, do the following:

1. Unlock the screen if necessary.
2. While on the Home screen, press the Menu button. This may be a soft button on the touch screen or a physical key on the handset.
3. Navigate to the Settings item and activate it by pressing the selector.
4. Navigate to Language and Keyboard or Locale and Keyboard and activate it by pressing the selector.
5. Navigate to the Text to Speech Settings item and activate it by pressing the selector.
6. Navigate to the Rate item and activate it by pressing the selector.
7. Navigate through the choices--slow, normal, fast, and very fast--and check the desired rate by pressing the selector.
8. Return to the Home screen by pressing the Back button several times or by pressing the Home button once. Back and Home may be soft buttons on the touch screen or physical keys on the handset.

Adjusting Speech Volume

To adjust the volume of speech, do the following:

1. Unlock the screen if necessary.
2. Locate the physical volume control.
3. Make Talkback speak by opening or closing the phone or by scrolling through a list of applications or options.
4. While the phone is speaking, repeatedly press the volume-down button to decrease loudness or the volume-up to increase loudness.

The volume of speech is not affected if the physical volume button is pressed while the phone is silent. More information about adjusting Talkback and ringer volume is covered in a later post.