Or, why Ubuntu touch has will soon have a better Persian keyboard layout than Android?

I finally completed the design of the Persian layout with all of its bells and whistles, and I have proposed the layout for merging into the Ubuntu project. I am excited to share with you some screenshots!

At the end of this post, I will discuss why this layout is even better than the current golden standard for mobile Persian layouts, which is implemented in Android.

Screenshots

Normal mode. Keys in the bottom row from left: symbols shift key, language menu, comma, space, zero-width non-joiner (ZWNJ), dot, and backspace.

---

Normal mode. Extended keys for Persian letter Yeh (ی).

---

Normal mode. Extended keys for dot, from left: Persian percent sign, colon, semicolon, Persian question mark, exclamation mark, slash, and Persian Kashida.

---

Normal mode. Extended keys for comma: Persian- (and French-) style quotation marks, parentheses, and Persian/Araic diacritics.

---

Symbols mode. Keys in the second row from left: decimal separator, thousands separator, and Iranian currency symbol ﷼ (Rial). Note also that the ZWNJ key is now replaced by ZWJ.

---

Symbols mode. Extended keys for Rial (currency) key.

---

Symbols mode, shifted.

---

Why is this better than the Persian layout in Android?

In the design of this new Persian layout for Ubuntu phone, I have addressed the following shortcomings in the Persian keyboard layout in Android. Some of these points are repeated from a previous blog post, while some others are new.

• Android has no question and exclamation marks in its normal mode, not even as extended keys. The user has to switch to symbols keyboard to type either of these frequent characters.
• Android has no decimal or thousands separator (٫ and ٬, respectively). A dot character (.) is no replacement for decimal separator in Persian since it is terribly similar to the Persian zero digit (۰). Many people mistakenly use a slash character as a replacement for decimal separator, and it results in BBC Persian to mis-report earthquake magnitudes.
• In Android, keys from different rows are all aligned horizontally (see a screenshot). This is unlike the US/UK layouts where for example the key “A” is between “Q” and “W” (in the lower row). This perfect alignment will increase probability of mis-typing.
• Android keyboard is too crowded with some consonant letters that are rarely used in modern Persian writing. Based on the results from this letter frequency analysis tool that I developed myself, I have removed the least-used characters from this Persian layout, and people seem to be happy about it.

Remaining issues

There are some minor issues about this keyboard layout that are beyond the layout itself, and should be addressed in the ubuntu-keyboard package as a whole. I am interested to work with other Ubuntu developers to find a proper way to fix these minor issues in the future.

First, the ubuntu-keyboard package does not provide an easy way to define a key where the label of the key is different from what it inserts into the text. The ZWNJ and ZWJ keys are good examples, and they are now working only thanks to a sub-optimal hack.

Second, key labels have an LTR (left-to-right) directionality by default. This forced me to add an RLM (right-to-left mark) character after all directionally-weak characters (e.g. parentheses and brackets) to make them show up correctly on the key label. As a result, these invisible RLMs are inserted into the text whenever the user types e.g. a parenthesis. While this is not a big issue (RLM is invisible), this has to be fixed to avoid possible text-processing issues with some legacy software.

What’s next?

For me, this was a beautiful experience for contributing to the Ubuntu project. I look forward to see this layout merged into the project, and to make more code contributions to Ubuntu.

The Persian alphabet is based on the Arabic script, although the two languages are linguistically very different. The alphabet has 33 main letters (including hamza) and a dozen of diacritics (short vowels) and punctuation marks. To type Persian text in the computer, however, some extra invisible characters are essential. These include the space key (of course!) and the Zero-Width Non-Joiner (ZWNJ) key.

Design goals

• The Persian layout should allow the user to insert anything needed for typing a Persian text. These include Persian letters, numerals, punctuation marks, diacritics, and control characters.
• The layout should be easy to use. This means:
  • Frequently-used letters should be available to type directly, without using a “shift” or “alt” or other modifier keys.
  • Less frequently used letters should be hidden as extended keys to save screen real estate, so that the rest of the keys are larger and hence easier to access.
  • As I wrote in my first post in this series, it is preferred that keys from different rows are NOT horizontally aligned, to reduce chance of mistyping letters. This condition requires neighboring rows in the keyboard to have different number of keys.
• The layout should be familiar for users of the standard Persian layout for the desktop, as defined in the standard ISIRI 9147.

Letter usage frequency in Persian

To decide which characters are used frequently and which are not, I analyzed the text of a free Persian ebook from Project Gutenberg, using a small tool I made with bash and gnuplot (there is also a similar tool as a perl script from Jadi). This is the distribution of letter usage in Persian.

This distribution is of course different from text to text. In this ebook, for example, there is an unusually low number of ZWNJ’s (as shown by ⟘ in the plot above) and not a single Persian numeral. I made the same distribution for a collection of my own posts in Google+, and also for articles in the political page of a mainstream Persian newspaper. In all cases, the following letters were always used the least: ‎ژ, ؤ, ئ, ث, ض, ظ‎ and غ. This is also consistent with what Jadi obtained by analyzing another pile of contemporary Persian text.

Layout design proposals

Based on the above constraints and considerations, I came up with two designs for the Persian layout.

Proposal #1:

First row, 10 keys, extended keys for numerals. Letters غ and ض are NOT hidden to make enough keys for 10 extended numbers
ج ح خ ه ع غ ف ق ص ض 
۰ ۹ ۸ ۷ ۶ ۵ ۴ ۳ ۲ ۱

Second row, 9 keys
م ن ت/ث ا ل ب ی س ش

Third row, 9+1 keys (including backspace)
‏←‏ چ گ ک و پ د/ذ ر ز ط/ظ

Proposal #2:

First row, 9 keys, less frequent letters are hidden
چ ج ح خ ه ع/غ ف ق ص/ض

Second row, 10 keys, extended keys for numerals, letter ث is not hidden
م ن ت ث ا ل ب ی س ش
۰ ۹ ۸ ۷ ۶ ۵ ۴ ۳ ۲ ۱

Third row, 8+1 keys (including backspace)
‏←‏ گ ک و پ د/ذ ر ز ط/ظ

Both of the two proposals look good. I finally decided to go with proposal #1, since it has numerals in the top row (as it is more common in soft keyboards). In the picture at the top of this post I have sketched the final layout (in the normal and extended modes), also including the 4th row.

Preliminary keyboard layout implementation

The following is my implementation of the Persian layout for Ubuntu touch as of now. There is a problem with aligning the backspace key. Also the ZWNJ key shows an empty label (since it inserts an invisible character into the text), while the symbol shift key (?123) still shows the English label.

Next, I am going to fix the small problems I mentioned above. Also, I will start the design of symbols keyboard, i.e., the keyboard that appears by pressing the ?123 (or rather, ‏۱۲۳؟) key.