Despite of the difference among the Input Methods, the IM Engine supplies the same foundational functions for each kind of the specific Input Method: add / remove input elements for the further combination and retrieve the corresponding pre-editing components or real input character candidates back.

According to the number of input step, Input Methods can be classified into two categories:

l One-Step Input Method: This kind of Input Method only needs one step to get the desired characters of words from the IM Engine. It only uses candidates’ relative interfaces of IM Engine to finish an input cycle.

l Two-Step Input Method This kind of Input Method needs two steps go get the characters of words user desires. Firstly retrieve a pre-editing component corresponding to inputted elements, and then use these components as input to get the real candidates or words (such as Chinese character in Chinese Phonetic Input method) in the second step.

The One-Step Input Methods are common in Latin Language Input Methods, while the Two-Step Input Methods are usually used for phonetic input in those bi-bytes language countries, such as China, Japan and Korea. That phonetic Input Methods involve an intermediary step towards the selection of the desired character. Phonetic input requires the entry of a Roman character or phonetic symbol to, first, arrive at a phonetic spelling of the desired Chinese word. This first stage requires disambiguation of the key press sequence, yielding candidates with the various phonetic spellings of the disambiguated sequence. Second, after the user selects the word with the desired phonetic spelling, another search is invoked using this phonetic spelling to present a list of Chinese characters that are pronounced the same as the phonetic spelling. Finally, tone mark can be added to the input sequence to further narrow the candidate list of Chinese characters. As a result, a relatively small number of Chinese character candidates are presented.

1.2 IM Engine Functional Design

Typically in the current frame work, IM Engine repeats the same communication work with the IM Manager in its life cycle - adding or removing elements, accepting users’ selection, and retrieving the candidates. The Diagram 1-1 sets out this cyclic process between the IM Engine and IM Manager:

Diagram 1-1: Cyclic flow between IM Engine and IM Manager

Reference to the diagram the functions of IM Engine can be defined as the following parts in two main aspects:

1. Input Functions

This aspect mainly focuses on the inputting working and includes these functions:

l Add / Remove an element

IM Engine retrieves an element or elements in unified format from the IM Manger. In each specific kind of the Input Method, the IM Engine converts elements into the indeed format Input Methods really needs, and stores elements in the internal buffer for the following combination. Then IM Engine uses exists elements to produce the pre-editing components or real character candidates. These are necessary both in One-Step Input Methods and Two-Step Input Methods.

l Retrieve pre-editing Components and Make user selections

For Two-Step Input Methods, IM Engine supplies a set of the pre-editing components for the current input elements to IM Manager, and IM Manager let user make a selection in these components. After the selection of pre-editing components is made, the IM Engine uses the selected pre-editing component to replace the real input elements in the internal buffer for the generation of real characters or words. These are only necessary in Two-Step Input Methods. In notice, the count of pre-editing components is notified to IM Engine by the IM Manager when the initialization or the resize event of IM Helper occurs.

l Retrieve indeed candidates and Make user selections

IM Engine supplies a set of the indeed candidates for the current input elements or pre-editing components to IM Manager, and IM Manager let user make a selection in these candidates. These are necessary both in One-Step Input Methods and Two-Step Input Methods. In notice, the count of indeed candidates is notified to IM Engine by the IM Manager when the initialization or the resize event of IM Helper occurs.

l Commit desired characters or word

IM Engine commit a string contains the desired character of word to IM Manager after the selection of indeed candidates is made. These are necessary both in One-Step Input Methods and Two-Step Input Methods.

2. Management functions

This aspect mainly focuses on the management working and includes these functions:

l Management of Input Methods

IM Engine stores all the information of existing Input Methods which are supplied itself, and manages a input context for each Input Method instance at currents. These are necessary both in One-Step Input Methods and Two-Step Input Methods.

l Switch of components and candidates

IM Engine distributes components or candidates into small pages because of display size of IM Helpers and makes switches among these pages. These are necessary both in One-Step Input Methods and Two-Step Input Methods.

1.3 IM Engine Interface Design

To suffice the requirement of functional design, IM Engine interfaces is define as an independent module with five main parts: (The part with dashed line is implemented outside the module)

Diagram 1-2: Components in IM Engine

1. Input Method Descriptions

A single IM Engine can provide more than one kind of Input Method, so that IM Engine should contain a structure to store Input Method information for each Input Method. This structure includes a set of information of Input Method needed by IM Manager. IM Manager uses value in this structure to decide the whole control flow. This structure is named IME_InputMethod in LiPS Input Method Specification.

uuid an unique string to identify this input method. You should define it by using the standardize method.

encoding the character encoding used in the input method. The strings send to IM Manager via any methods should use this encoding. UTF-8 encoding is preferred here, though other local encodings should also be acceptable. IM Manager accords to this encoding to convert the strings to its internal encoding.

name the name of this input method, should be encoded in the encoding specified by the previous encoding parameter.

icon local path of the icon file for this input method.

language all languages supported by this input method. Multiple languages should be separated by comma. The language name is same as the glibc locale name standard (without encoding info). For example: zh_CN,zh_TW,ja_JP,ko_KR.

help a brief help text of this input method, should be encoded in the given encoding.

copyright copyright information of this input method, should be encoded in the given encoding.

version version string of this input method, should be encoded in the given encoding.

capability_mask the capabilities specification of the Input Method. Echo bit of this field indicate a kind of functional feature of the Input Method. The capabilities include:

IME_CAP_TWO_STEP: if this bit is set to 1 indicates the Input E is a Two-Step Input Method

Using a static array to store this structure for each Input Method in the IM Engine is recommended. The content of in this structure is provided to IM Manager through the public interfaces discussed in the below.

2. Engine Core

This part of IM Engine is dependent on different Input Method implementation. It can be constructed with specific Input Method core library directly. For the highly reuse, the implementation derived from any abstract layer on core library is recommended.

3. IM Engine Public Functions

IM Engine uses a series of standardized public interface to communicate with IM Manger. These interfaces covers all the functional features brought forward last section and are gathered in a single structure as a set of call-back function pointers in IM Engine. IM Manager uses these interfaces through this structure named IME_Functions in LiPS Input Method Specification.

l initialize()

Initialize IM Engine, includes loading table, dictionary, user configuration etc.

l finalize()

Clear and release corresponding data, structures of IM Engine.

l get_number_of_input_methods()

Retrieve the number of input methods that IM Engine supports.

l get_input_method_info ()

Retrieve the specific input method information of the Input Method Engine.

l create_input_context ()

Create an Input Context instance. Input Context can use different input method and have different status. Input Method Manager has responsibility for providing “uuid” and “id” to the Input Method Engine, while Input Method Engine see to create corresponding inter structures and initialization. See the definition of the structure IME_InputContext.

l destroy_input_context ()

Destory an Input Context instance. Input Method Engine should release all corresponding data and structures, and do some necessary clean work.

l add_element()

Pass a key input element or touch screen points array element to the Input Method Engine. For the Key Input Method, the element should be a key code defines in this standard. For the Touch Screen Input Method, the element should be array of a mouse trail points which indicates the current character. For the Voice Input Method, the element should be the raw audio data from the microphone.

l del_element()

Delete a passed element from the IM Engine.

l reset ()

Reset current input context to original state.

l select_preedit_candidate ()

In two-step input method, notice IME that user has selected a preediting candidate word.

l select_candidate ()

Notice IME that user has selected a candidate word.

l previous_preedit_candidates_page ()

In two-step input method, notice IME turn preediting candidates to previous page.

l previous_candidates_page ()

Notice IME turn candidates to previous page.

l next_preedit_candidates_page ()

In two-step input method, notice IME turn candidates to next page.

l next_candidates_page ()

Notice IME turn candidates to next page.

l update_preedit_candidates_page_size ()

In two-step input method, when the IMM can’t show all pre-editing candidates words, IMM should IME to reset page size, so that IME can turn to next or previous page correctly.

l update_candidates_page_size ()

When the IMM can’t show all candidate words, IMM should IME to reset page size, so that IME can turn to next or previous page correctly.

The call-back function pointers is filled with the specific interface implementation and passed to IM Manager in IM Engine register module function discussed in the below.

4. IM Engine Public Functions

IM Engine exports public interfaces to IM Manager for the communication start from IM Manager in one direction, and IM Manager also needs to export a set of public interfaces to fulfil the needs of communication starts from IM Engine. These are included in a structure named IMM_Functions in LiPS Input Method Specification. IM Engine only stores a pointer to this structure and it is pointed to the real structure instance when the IM Engine register module function is called. The detail about the structure is described in the following chapter.

5. IM Engine Register Module Function

IM Engine only exports a IM Engine register module function ime_register_module() to the public programmes, and this function register the IME_Functions structure to IM Manager and retrieve the address of global IMM_Functions structure and set it to internal pointer.

2 IM Helper Design

2.1 Overview

IM Helper is a kind of widget to help IM Manager communicate with applications. The displaying and some kind of special inputs are various and heavily dependent on the lower system interfaces. They are not considered as a functional part of IM Manager but as an independent Input / Output parts to help IM Manager’s work. For this design, IM Helper is module that supplies a set of User Interfaces (UI) to help IM Manager deal with displaying or extended inputting work. According to the purpose of IM Helper, it can be categorized into two basic classes:

l IM Input Helper: This kind of IM Helper is mainly response for the extended input work the current Input Method. IM Manager only handles those common input manners directly passed by application such as key-pad or keyboard input. But as the technology development, the classic single input is replaced by new type of input method or even multiple inputs crossed input method. These are including virtual keyboard input, hand-writing touch screen input, or even microphone voice input, etc. IM Manager needs helpers to process and accept these extended then convert into valid information passing to IM Manger.

l IM Display Helper: This kind of IM Helper is mainly response for the output work for the current Input Method. It is simple and clear, including displaying the pre-editing components and real character candidates, indicating the current status of Input Method, etc.

Although Input Helper and Display Helper focuses on the different working, both use the same interface to communicate with IM Manager for unification and the convenience of management.

2.2 IM Helper UI Design

Considering the most popular input method and display custom, the IM Helper UI can be planned as the following patterns.

1. IM Input Helper Design

IM Input Helper mainly focuses on helping the IM Manager to retrieve element from some special kind of input, such as Virtual Keyboard, Touch Screen, and Voice etc. The following pictures show various demo UIs of this kind of Helper we want to implementation

l ID: IMH-INPUT-KB001 (Mask 0x0042)

It is a common virtual keyboard IM Input Helper. Its keys divided into four main categories: Numeric keys, Latin keys, Function keys and Punctuation keys.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	0
IMH_CAP_ INPUT_HELPER	1
IMH_CAP_VKEY_INPUT	1
IMH_CAP_TOUCHSCREEN_INPUT	0
IMH_CAP_VOICE_INPUT	0

l ID: IMH-INPUT-TC001 (Mask 0x0082)

It is a standard IM Input Helper for the Chinese Input Method. It is combined with a Chinese Character Input Area, a Latin Character Input Area, a Punctuation Input Area and four Special Symbol Input Buttons.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	0
IMH_CAP_ INPUT_HELPER	1
IMH_CAP_VKEY_INPUT	0
IMH_CAP_TOUCHSCREEN_INPUT	1
IMH_CAP_VOICE_INPUT	0

l ID: IMH-INPUT-TC002 (Mask 0x0082)

It is a standard IM Input Helper for the Latin Input Method. It is combined with two Latin Character Input Areas, an embedded Punctuation Input Area in the second Latin Character Input Area and four Special Symbol Input Buttons. Two Latin Character Input Area switching accelerates the input speed of Latin words.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	0
IMH_CAP_ INPUT_HELPER	1
IMH_CAP_VKEY_INPUT	0
IMH_CAP_TOUCHSCREEN_INPUT	1
IMH_CAP_VOICE_INPUT	0

l ID: IMH-INPUT-TC003 (Mask 0x0082)

It is a standard IM Input Helper for Full Screen Input Method. Its Input Area is the whole touch screen, so it only has a four Special Symbol Input Buttons combination.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	0
IMH_CAP_ INPUT_HELPER	1
IMH_CAP_VKEY_INPUT	0
IMH_CAP_TOUCHSCREEN_INPUT	1
IMH_CAP_VOICE_INPUT	0

2. IM Display Helper

IM Display Helper needs to display pre-editing components or real character candidates, and it also has to indicate the current Input Method status.

l ID: IMH-DISPLAY-001 (Mask 0x0015)

It is a standard IM Display Helper for a Two-Step Input Method. Pre-editing candidates (PinYin) list and candidates (HanZi) list display, Horizontal display, Numeric label display as superscript, fixed position at the bottom of the client window.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	1
IMH_CAP_ INPUT_HELPER	0
IMH_CAP_NUMERIC_DISPLAY	1
IMH_CAP_VERTICAL_DISPLAY	0
IMH_CAP_TWOSTEP_COMPATIBLE	1
IMH_CAP_OVER_SPOT_DISPLAY	0

l ID: IMH-DISPLAY-002 (Mask 0x0035)

It is a standard over-spot IM Display Helper for Two-Step Input Method. The only different from the IMH-DISPLAY-001, is that its position changes synchronously with the input cursor.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	1
IMH_CAP_ INPUT_HELPER	0
IMH_CAP_NUMERIC_DISPLAY	1
IMH_CAP_VERTICAL_DISPLAY	0
IMH_CAP_TWOSTEP_COMPATIBLE	1
IMH_CAP_OVER_SPOT_DISPLAY	1

l ID: IMH-DISPLAY-003 (Mask 0x0021)

It is a standard over-spot IM Display Helper for the One-Step Input Method. Vertical Display, No pre-editing candidate list, No numeric selection.

*Bit Mask*	*Value*
IMH_CAP_ DISPLAY_HELPER	1
IMH_CAP_ INPUT_HELPER	0
IMH_CAP_NUMERIC_DISPLAY	0
IMH_CAP_VERTICAL_DISPLAY	0
IMH_CAP_TWOSTEP_COMPATIBLE	0
IMH_CAP_OVER_SPOT_DISPLAY	1

2.3 IM Helper Functional Design

IM Helper includes two braches – IM Input Helper and IM Display Helper. IM Input Helper takes charge for input work, while IM Display Helper takes charge for display work. Two kinds and IM Manager form an information cycle in the framework as Diagram 2-1 shows:

Diagram 2-1: Cyclic flow between IM Helper and IM Manager

Basing on this diagram above the functions of IM Helper can be described as the following parts in three main aspects:

1. Input / Output Function

This aspect mainly focuses on the input / output working of IM Helper:

l Add elements

IM Helper receives the raw input data from the corresponding input devices, and converts the data into the standard format needed by IM Engine. For the virtual keyboard, input data is converted into standard key codes. For the touch screen, input data is converted into points array. For the voice device, input data is converted into raw sound format. IM Input Helper use interface supplied by IM Manager pass the converted data to IM Manager and IM Manager use IM Engine’s interface add them into IM Engine’s internal buffer. This is the main work of IM Input Helper.

l Display components and candidates

Each time a new candidates (components) list is produced, IM Engine will pass them to IM Manager. IM Manager calls the interface in IM Helper module to notify IM Helper to update the display matching the latest candidates (components) content. This is the main work of IM Display Helper.

2. Management Function

This aspect mainly focuses on the management working of IM Helpers. IM Helper stores the information of itself which are necessary for management of IM Manager. It is most important to supply a description of the capabilities IM Helper can afford. This description includes type description, layout description, and functional description. Type description indicates whether it’s an Input Helper or a Display Helper. Layout description shows the position and size features of the Helper. Functional description then describes all the rest functional abilities.

2.4 IM Helper Interface Design

To suffice the requirement of functional design, IM Helper interfaces is define as an independent module with five main parts: (The part with dashed line is implemented outside the module)

Diagram 2-2: Components in IM Helper

An IM Helper module can implement to be an IM Display Helper or an IM Input Helper, or even both. To implement both two kinds of IM Helper in the same module is not necessary.

1. IM Helper Description

A single IM Helper can provide one kind or both kinds of IM Helper, so that IM Helper should contain a structure to store IM Helper information for the Helper capabilities. This structure includes all the information of any kind of Helper. Even IM Helper implements both kinds of IM Helper, the capabilities of the two IM Helper are stores in one variable of this structure type. This structure is named IME_Helper_Info in LiPS Input Method Specification. IME_Helper_Info structure contains sufficient information that helps the IM Manager uses all of its functional ability to finish the displaying work.

uuid the unified identifier of the IM Helper. IM Manager uses this field to identify the IM Helper. It is mainly used in the period of switching an IM Engine when IM Manager should select a proper Helper for the current IM Engine.

encoding the character encoding of the IM Helper. IM Manager uses this field to convert the characters from internal encoding of itself to the encoding of the Helper when passing a candidates list to the Helper.

name the name of the IM Helper. IM Manager uses this field to retrieve Input information of the Helper.

description the description of the IM Helper. IM Manager uses this field to retrieve more Input information of the Helper.

icon the path of the IM Helper ICON file. IM Manager uses this field to load and display a icon file when a IM Helper module is being used.

capability_mask the capabilities specification of the IM Helper. Echo bit of this field indicate a kind of functional feature of the Helper. The capabilities include:"capability_mask":

IMH_CAP_DISPLAY_HELPER: if this bit is set to 1 indicates the Helper supplies all the functional feature of an IM Display Helper.

IMH_CAP_INPUT_HELPER: if this bit is set to 1 indicates the Helper supplies all the functional feature of an IM Input Helper.

IMH_CAP_NUMERIC_DISPLAY: If this bit is set to 1 indicates the Helper can display a superscript label with each candidate. Otherwise, not.

IMH_CAP_VERTICAL_DISPLAY: If this bit is set to 1 indicates the Helper display a candidates list vertically. Otherwise, the Helper displays a candidates list horizontally.

IMH_CAP_TWOSTEP_COMPATIBLE: If this bit is set to 1 indicates the Helper can be used in displaying for a Two-Step Input Method. Otherwise, the Helper is only for the One-Step Input Method.

IMH_CAP_OVER_SPOT_DISPLAY: f this bit is set to 1 indicates the Helper moves with the position of input cursor. Otherwise, the Helper is fixed at any side of the client window. For IM Input Helper this bit is always 0.

IMH_CAP_VKEY_INPUT: if this bit is set to 1 indicates the Helper can supplies virtual keyboard input.

IMH_CAP_TOUCHSCREEN_INPUT: if this bit is set to 1 indicates the Helper can supplies touch screen input.

IMH_CAP_VOICE_INPUT: if this bit is set to 1 indicates the Helper can supplies voice input.

This structure is necessary in IM Helper module and is passed to IM Manager when registration of IM Helper occurs.

2. Helper Core

This part includes two sub parts that are Input Helper Core and Display Helper Core. Input Helper Core is response for the low-level implementation of IM Input Helper and is dependent on the specific input devices, while Display Helper Core is response for the low-level implementation of IM Display Helper and is dependent on the specific window system. Any IM Helper module should contain one of these two core implementation at least.

3. IM Helper Public Functions

IM Helper uses a series of standardized public interface to communicate with IM Manger. These interfaces covers all the functional features brought forward last section and are gathered in a single structure as a set of call-back function pointers in IM Helper. IM Manager uses these interfaces through this structure named IMH_Functions in LiPS Input Method Specification. Due to the calling direction, parts of functions in this structure are used for IM Display Helper.

l initialize()

This function is used at first when the Helper module is loaded. It finishes all the initialization work of the Helper, including allocate memory space for all the display widgets, build up the layout, set default display and so on. This function is used for both kinds of IM Helper.

l finalize()

This function is the final step before the Helper module is unloaded. It clears all the memory allocations and other destroys work. This function is used for both kinds of IM Helper.

l show_preedit_candidates_list()

This function is used when Input Method generate a new set of pre-editing candidates or the IM Engine gets the input focus. Its displays or updates its pre-editing candidates list area of the display. It is only available in Two-Step Input Method. If the IMH_CAP_TWOSTEP_COMPATIBLE bit is set, IM Helper must supplies the function to display the pre-editing candidates. If the IMH_CAP_NUMERIC_DISPLAY bit is set, IM Helper should display a number label as a superscript of each candidate. This function is only used for IM Display Helper.

l show_candidates_list()

This function is used when Input Method generate a new set of candidates or the IM Engine gets the input focus. Its displays or updates its candidates list area of the display. If the IMH_CAP_NUMERIC_DISPLAY bit is set, IM Helper should display a number label as a superscript of each candidate.

l hide_all_candidates_list()

This function is used mostly when the IM Engine loses the input focus. It should hide the Helper. If the Helper contains a pre-editing candidates list, this process should hide both of these two kind lists. This function is only used for IM Display Helper.

l nofity_ime_status()

This function is used when the status of Input Method is changed and only available in Two-Step Input Method. If the IMH_CAP_TWOSTEP_COMPATIBLE bit is set, IM Helper should supplies the function to synchronise the status with Input Method. This function is used for both kinds of IM Helper. This function is only used for IM Display Helper.

l notify_numeric_display()

This function is used when IM Manager enable a number selection of candidate. If the IMH_CAP_NUMERIC_DISPLAY bit is set, IM Helper should supplies the function to enable its numeric displaying on demand. This function is only used for IM Display Helper.

l notify_client_size()

This function is used when position of client window or the input cursor if changed. If the IMH_CAP_OVER_SPOT bit is set, when the input cursor moves, IM Manager will call this function to tell the Helper about the new cursor location and size. Otherwise, only when the client window moves, IM Manager will call this function to tell the Helper about the new location and size of the client window. The parameters x, y, width, height have the complete different meanings in the two process. This function is used for both kinds of IM Helper.

4. IM Engine Public Functions

IM Helper exports public interfaces to IM Manager for the communication start from IM Manager in one direction, and IM Manager also needs to export a set of public interfaces to fulfil the needs of communication starts from IM Helper. These interfaces are most used for IM Input Helper and included in a structure named IMM_Helper_Functions in LiPS Input Method Specification. IM Helper only stores a pointer to this structure and it is pointed to the real structure instance when the IM Helper register module function is called. The detail about the structure is described in the following chapter.

5. IM Helper Register Module Function

IM Engine only exports a IM Engine register module function imh_register_module() to the public programmes, and this function register the IMH_Functions structure to IM Manager and retrieve the address of global IMM_Helper_Functions structure and set it to internal pointer.

3 IM Manager Design

3.1 Overview

The IM Manger is the central part of the whole framework. It is response for all of the managing controlling, and communicating work the whole life cycle. According the type of working, three main tasks of IM Manager are defined as follow:

l Management: Load current IM Engine module and IM Helper module into memory, followed by registering all the Input Methods and Helpers. Then group Input Method and IM Helper by choosing the right IM Helper for the Input / Output of the Input Methods in IM Engine. The corresponding destroys work is done at the finalizing period of framework.

l Control: Control the flow and the status of the current Input Method. According to the designs. The flows are dissimilar for different kinds Input Method and also relative to the capabilities of IM Helper bound with the current Input Method.

l Communication: Get elements from input devices of IM Input Helper, and pass them to the IM Engine for the further process. Retrieve the candidates list from IM Engine and pass them to IM Helper for user interact and make selection. All of cooperative work between the IM Engine and IM Helper is finished by the IM Manager.

3.2 IM Manager Functional Design

Main task of IM Manager has been classified into three main categories above. This section organizes design description according to this.

1. Management

Management work of IM Manager occurs at three periods: firstly happens at the very beginning of Input Method Framework loading, then repeats at the switching of the Input Method and finally takes place when the unloading of the framework.

Initially, the framework loading should follow these steps:

l Load IM Engine module then register all of the Input Method and corresponding call-back function in the module;

l Load IM Helper module then register the Helper and corresponding call-back function;

l Initialize the IM Engine with its own initialize call-back functions;

l Initialize the IM Helper with its own initialize call-back functions;

l Switch to the default Input Method of the IM Engine;

When then Input Method switches, the IM Manager should finish all of the transitional work with following regulation:

l Destroy the input context and finalize the current Input Method if there is one loaded;