Braille IME Helper

Version 2.2

Introduction

Braille IME Helper (BrlIMEHelper) enables users to input Chinese characters directly through the braille keyboard on a braille display. When no braille keyboard is available, the addon can also simulate it using a computer keyboard. With conversion from braille input to IME operations by the addon, users familiar to braille rules can input Chinese characters without learning other input methods and keyboard layouts. So far, the addon is an implementation based on bopomofo braille and 微軟注音 IME commonly used in Taiwan, and its concept can be extended to other braille systems and IMEs in the future.

Features

1. Chinese input (including punctuations and math symbols) through the braille keyboard.
2. Simulating braille keyboard by a computer keyboard.
3. Allowing system and NVDA keyboard shortcuts.
4. Different braille composition modes for different windows.
5. Allowing password input on a web browser by the braille keyboard.
6. Various customization options from users' feedback.

System and environment requirements

Before installation, check the following environment settings:

• NVDA 2017.3 or later.
• The IME must be properly configured, and the detail is presented in the next paragraph.
• If you would like to simulate braille keyboard by a computer keyboard, check that it supports NKRO (N-key rollover). See also: What PC keyboards allow for 6-key braille data entry?
• Please prevent Braille IME Helper from running with other applications or addons related to braille input, such as PC Keyboard Braille Input for NVDA addon and Braille Chewing.
• It is suggested to disable "Report changes to the reading string" option of NVDA's input composition settings to enjoy better experience during quick typing.
• It is suggested to set the braille input table to "English (U.S.) 8 dot computer braille", so that NVDA's behavior is closer to the habit of Taiwanese users.

This addon assumes that the user configures the IME as follows, but he/she can also change related options to suit the actual circumstance.

Manipulation

Gestures

The addon provides both computer-keyboard and braille-keyboard gestures. Before introduction, please comprehend the following terms:

1. Computer keyboard: It contains a main keyboard and a numpad. Names of its keys are enclosed by brackets in this section.
2. Braille keys: 9 keys mapped to dots 1 through 8 and the braille space. According to rules defined by the input translation table, the user can type characters via them.
3. Braille keyboard: A physical braille keyboard consisting of 9 braille keys usually belongs to a braille display. However, the addon also offers a virtual braille keyboard emulated by a computer keyboard to facilitate manipulation by users who prefer braille input.

Computer Keyboard

• NVDA+Ctrl+6: Enable/Disable the feature of simulating braille keyboard by a computer keyboard.
• F, D, S, J, K, L, A, semicolon [;:], and space bar: Dot 1 through 8 and braille space.
• 0 through 9: Reserved for number input and candidate word selection.
• NumPad (optional feature, with Num Lock on): Input of a braille cell dot by dot.
  • 0 through 8: The braille space and dots 1 through 8.
  • 9: Clear uncompleted braille cell.
  • Decimal point [.]: Complete input of one braille cell.
  • Divide [/]: View uncommitted braille cell.
• NVDA+PrintScreen: Copy braille display content (raw text or Unicode braille) to the clipboard.
  • Raw Text mode:
    • Press once: Copy this line.
    • Press twice: Copy this line and the focus context presentation.
  • Unicode Braille mode:
    • Press once: Copy currently displayed braille cells.
    • Press twice: Copy all braille cells of this line.
  • Press three times: Switch between the two modes.

Braille Gestures

A braille gesture consists of the braille space and other braille dot(s). It allows the user to execute some specific function or emulate some key shortcut by the braille keyboard. With these braille gestures, the user may reduce the chance of moving his/her hands away from braille keys, and thus efficiency of operation is enhanced.

The subsequent two subsections introduce braille gestures. The first subsection outlines some common braille gestures and suggestions for the user to quickly remember them. The second subsection provides a table to enumerate all braille gestures defined by this addon. Note that braille gestures executing BrlIMEHelper functions are available via both physical and virtual braille keyboards, but other braille gestures can only be used with the virtual braille keyboard.

To simplify subsequent content of the section, a braille gesture is described only by its braille dot(s), without mentioning the braille space.

Compatibility warning: NVDA versions earlier than 2018.3 disallow braille input from "No braille", so source of gestures provided by the addon is "braille keyboard". If the running braille display driver does not define the same gesture, then braille input generated by its physical braille keyboard may cause execution of them, which does not happen in newer NVDA versions.

Common Braille Gestures

BrlIMEHelper Functions

What a user just enters in native mode is stored in the braille buffer before finish of composition. For example, 135 126 is insufficient for composition, but a review of the braille buffer shows that ㄅㄛ has been entered.

2, 4, 5 + space has an additional effect. It makes NVDA dismiss braille message without update of any control content.

After press of dots 1, 3, 6 + space, the braille input translation table becomes "Unicode braille" immediately. The next press will let it become the original table selected by the user. However, if the original braille input translation table is "Unicode braille", then it will become "English (U.S.) 8 dot computer braille".

Document Editing

Windows System Key Shortcuts

NVDA Operations

All Braille Gestures

Here is a table listing all supported braille gestures.

Before start, pease note that functions of most of the braille gestures can be induced by dot 7 and dot 8. Dot 7 represents that the emulated key shortcut contains [Ctrl], or it is one of function keys [F1] to [F12]. Dot 8 represents that the emulated key shortcut contains either [Alt] or [NVDA] key. Both dots 7 and 8 represents that the emulated key shortcut contains either of [Ctrl]+[Alt] and [Win]. Anyway, proper comprehension of the design principle would help the reader to remember a lot of braille gestures at short notice.

The original proposal of design in Chinese is available in message #3664 of nvda-tw group.

Options

Automatically enable braille keyboard simulation when NVDA starts

If checked, braille keyboard simulation is enabled automatically when NVDA starts.

The key shortcut to toggle IME alphanumeric/native mode

It determines the key command sent by the addon when the user presses dots 4, 5, and 6 along with the braille space. To keep the behavior consistent, the default is left shift.

Note: Windows XP users, please configure the default language mode of 新注音 to native mode if you would like to use [Ctrl]+[Space] to switch between 美式鍵盤 and 新注音.

Disable braille keyboard simulation by default in IME alphanumeric mode

If checked, the user can always type characters by the current keyboard layout in IME alphanumeric mode. The option improves experience of users who are familiar to the standard computer keyboard but new to IME keyboard layout of its native mode.

"Braille Keys" and "Ignored Keys"

Users can determine positions of braille dots, braille space, and ignored (reserved) keys for braille keyboard simulation in BrlIMEHelper settings dialog. Braille keyboard simulation is automatically disabled when either of the "Braille Keys" or the "Ignored Keys" edit control is focused. Exact 9 braille keys are required, but number of ignored keys is unlimited. If a key appears in both options, "Braille Keys" takes precedence. After entering all key positions by single computer keyboard strokes, [Apply] or [OK] button press is necessary to take effect. Occasionally, the "Braille Keys" option may not fully work, because simultaneous transmission of some key commands is unsupported by internal design of your computer (or notebook) keyboard. Please change your configuration to find a set of feasible braille keys.

Free all non-braille keys during braille keyboard simulation in IME alphanumeric mode

If checked, all keys, except the braille keys, are ignored during braille keyboard simulation in IME alphanumeric mode.

Keyboard Mapping

The option corresponds to the configuration of keyboard layout in IME native mode.

Allow dot-by-dot braille input via numpad during braille keyboard simulation

If checked, the user can input braille cells dot by dot via the numpad with Num Lock on.

Indication of manual/automatic toggle of braille keyboard simulation

The two options determines indication of toggle of braille keyboard simulation. Particularly, automatic toggling may occur after change of foreground window and/or options of the addon.

Consistent braille keyboard simulation toggle state for all processes

If checked, there is only one single state of braille keyboard simulation toggle, which is the behavior of all versions earlier than 2.0. Otherwise, the addon logs state of each process independently. A user who allows different IMEs for different windows may uncheck this option to reduce times of toggling braille keyboard simulation. Note that all toggle states of processes become the default state of NVDA starting if this option is changed from unchecked to checked.

Behavior of the simulated braille keyboard

The computer keyboard can emulate braille input from both the current working braille display and "No braille". When conflict happens, the precedence is determined by this option. On default, gestures provided by the addon take precedence.

Remarks

1. In alphanumeric mode, the effect is determined by NVDA's braille input table.
2. The original NVDA behavior of dot 7, dot 8, and dot 7 + dot 8 is preserved in both modes.
3. The addon do not influence other buttons on a braille display, such as buttons for scrolling and positioning.
4. Users may manage all above shortcuts via NVDA input gestures dialog and BrlIMEHelper settings dialog.
5. If no composed character is spoken after composition completion, then IME may get stuck by unreasonable (phonetic) input.

Issues and ways of development

• Known bugs:
  • NVDA braille does not reflect immediately as user input in "Braille Keys" and "Ignored Keys" edit controls.
  • Under some circumstances, the addon cannot successfully update the user configuration.
• Possible improvements in the future:
  • Allow users to customize the default state of the addon.
  • Allow users to customize dot patterns of symbols, and/or to load dot patterns from braille translation tables.
  • Allow the addon to work with other IMEs.

Contributors

• Bo-Cheng Jhan <school510587@yahoo.com.tw>
• 黃偉豪 <hurt.nzsmr@gmail.com>

Acknowledgement

• BrlIMEHelper has been sponsored by Taiwan Visually Impaired People Association <vipastaiwan@gmail.com> from 2020/01/01 to 2021/12/31, hereby express our sincere appreciation.

History of changes

Version 0.0

• The initial version.

Version 0.1

• Refactor the documentation, so that it becomes more readable by users.
• Reset the braille buffer when the foreground window changes.

Version 0.2

• Fixed the wrong behavior of simulation of dot 7 (A) in explorer.
• Split init.py into two files.

Version 0.3

• Enable localization of messages.
• Implement new braille parsing scheme.
• Add "addon_lastTestedNVDAVersion" information.
• Add rules to ensure input of a single bopomofo symbol.
• Correct several input dot patterns of math symbols.

Version 0.4

• Implement compatibility of braille pattern prefixes by delayed input sending.
• Try to substitute the most recent braille input on braille input rejection.
• Change the braille pattern of ∠ into ⠫⠪ (1246-246), i.e. the prefix of ←.

Version 0.5

• Code refactoring.
• Stop simulating braille keyboard by a computer keyboard in browse mode.
• Assign proper documentation and category to each script.

Version 0.6

• Implement a toggle to enable/disable braille input simulation in IME alphanumeric mode.
• Handle bopomofo.json path in str type.
• Do not simulate braille input if some modifier key is still held down on loading BrlIMEHelper keyboard hooks.

Version 0.7

• Add dot pattern variants of some symbols.
• Let braille shortcuts be manageable by NVDA input gesture manager.
• Adjust some coding style of command scripts.

Version 1.0

• Support NVDA 2019.3 based on Python 3.
• Implement a dialog for user configurations.

Version 1.1

• Use a more proper sound effect for the warning of typo.
• Allow users to customize positions of braille keys and ignored keys on a computer keyboard.
• Add an [Apply] button into the settings dialog.

Version 1.2

• Allow users to ignore all non-braille keys during braille keyboard simulation in alphanumeric mode.
• Add "Keyboard Mapping" option corresponding to the keyboard mapping option of 微軟注音 IME.
• Allow braille input of rhymes with ⡼ prefix.

Version 2.0

• Allow dot-by-dot braille input via numpad keys.
• Add feedback manners of audio and none for indication of braille keyboard simulation toggle.
• Allow independent state of braille keyboard simulation toggle for each process.
• Modify keyboard shortcut NVDA+X to NVDA+Ctrl+6.
• Add specification for all options into readme.md.
• Code refactoring.

Version 2.1

• Add various braille gestures to enhance efficiency for users who use virtual braille keyboard.
• Abandon use of NVDA winVersion API to avoid compatibility issue caused by API update in NVDA 2021.1.
• Add "The key shortcut to toggle IME alphanumeric/native mode" option.
• Avoid gettext warning in keyboard.py.
• Update files obtained from AddonTemplate.
• Various corrections and improvements of this documentation.

Version 2.2

• Implement quick switch between two braille input translation tables, the original and the Unicode braille, by dots 1, 3, 6 + space.
• Allow ⡀ and ⢀ during Unicode BRL input by the computer keyboard.
• Dismiss NVDA braille message properly even if "Show messages indefinitely" option is enabled.
• Avoid deprecated usage of "collections" module, "collections.abc" instead.
• Implement "print screen" for the braille display.