Design Development i18n Ubuntu

Persian keyboard layout for Ubuntu phone – Part 3: Layout design

The Persian alphabet is based on the Arabic script, although the two languages are linguistically very different. The alphabet has 33 main letters (including hamza) and a dozen of diacritics (short vowels) and punctuation marks. To type Persian text in the computer, however, some extra invisible characters are essential. These include the space key (of course!) and the Zero-Width Non-Joiner (ZWNJ) key.

Design goals

  • The Persian layout should allow the user to insert anything needed for typing a Persian text. These include Persian letters, numerals, punctuation marks, diacritics, and control characters.
  • The layout should be easy to use. This means:
    • Frequently-used letters should be available to type directly, without using a “shift” or “alt” or other modifier keys.
    • Less frequently used letters should be hidden as extended keys to save screen real estate, so that the rest of the keys are larger and hence easier to access.
    • As I wrote in my first post in this series, it is preferred that keys from different rows are NOT horizontally aligned, to reduce chance of mistyping letters. This condition requires neighboring rows in the keyboard to have different number of keys.
  • The layout should be familiar for users of the standard Persian layout for the desktop, as defined in the standard ISIRI 9147.

Letter usage frequency in Persian

To decide which characters are used frequently and which are not, I analyzed the text of a free Persian ebook from Project Gutenberg, using a small tool I made with bash and gnuplot (there is also a similar tool as a perl script from Jadi). This is the distribution of letter usage in Persian.

This distribution is of course different from text to text. In this ebook, for example, there is an unusually low number of ZWNJ’s (as shown by ⟘ in the plot above) and not a single Persian numeral. I made the same distribution for a collection of my own posts in Google+, and also for articles in the political page of a mainstream Persian newspaper. In all cases, the following letters were always used the least: ‎ژ, ؤ, ئ, ث, ض, ظ‎ and غ. This is also consistent with what Jadi obtained by analyzing another pile of contemporary Persian text.

Layout design proposals

Based on the above constraints and considerations, I came up with two designs for the Persian layout.

Proposal #1:

First row, 10 keys, extended keys for numerals. Letters غ and ض are NOT hidden to make enough keys for 10 extended numbers
ج ح خ ه ع غ ف ق ص ض 
۰ ۹ ۸ ۷ ۶ ۵ ۴ ۳ ۲ ۱

Second row, 9 keys
م ن ت/ث ا ل ب ی س ش

Third row, 9+1 keys (including backspace)
‏←‏ چ گ ک و پ د/ذ ر ز ط/ظ

Proposal #2:

First row, 9 keys, less frequent letters are hidden
چ ج ح خ ه ع/غ ف ق ص/ض

Second row, 10 keys, extended keys for numerals, letter ث is not hidden
م ن ت ث ا ل ب ی س ش
۰ ۹ ۸ ۷ ۶ ۵ ۴ ۳ ۲ ۱

Third row, 8+1 keys (including backspace)
‏←‏ گ ک و پ د/ذ ر ز ط/ظ

Both of the two proposals look good. I finally decided to go with proposal #1, since it has numerals in the top row (as it is more common in soft keyboards). In the picture at the top of this post I have sketched the final layout (in the normal and extended modes), also including the 4th row.

Preliminary keyboard layout implementation

The following is my implementation of the Persian layout for Ubuntu touch as of now. There is a problem with aligning the backspace key. Also the ZWNJ key shows an empty label (since it inserts an invisible character into the text), while the symbol shift key (?123) still shows the English label.

Next, I am going to fix the small problems I mentioned above. Also, I will start the design of symbols keyboard, i.e., the keyboard that appears by pressing the ?123 (or rather, ‏۱۲۳؟) key.

Development Ubuntu

Persian keyboard layout for Ubuntu phone – Part 2: Build environment

Or, how to mess with a perfectly working keyboard. Where is the Q, by the way? ?

Before making any new layout, I should first get the relevant source code and see if I can compile the package as it is, and to install the compiled package to see it in action. A quick search for “Ubuntu touch keyboard layout” landed me on this AskUbuntnu exchange, which in turn pointed me to this very useful page about keyboard layouts in the Ubuntu wiki.

The source code

The keyboard in Ubuntu touch comes from a package called ubuntu-keyboard and is hosted on Launchpad. I downloaded the repository for this package by running

bzr branch lp:ubuntu-keyboard add-persian-layout

Layouts for different languages are defined as plugins to the ubuntu-keyboard package. For example, the English layout is defined in

add-persian-layout$ ls plugins/en/*

Keyboard_en_email.qml  Keyboard_en_url.qml
Keyboard_en.qml        Keyboard_en_url_search.qml

database_en.db     englishplugin.json
englishplugin.h  overrides.csv         the_picture_of_dorian_gray.txt


From the above, files with qml suffix define key arrangements. Each of these layout variants have keys specific to their use case. For example the email layout (Keyboard_en_email.qml) has a dedicated “@” key to make it easier to type email addresses.

Compile recipe

I had no idea how to compile the ubuntnu-keyboard package. So I wrote to Mike Sheldon, an Ubuntu developer who has done many recent commits to this package. He nicely helped with the following compile recipe. Thank you Mike!

1. Start a shell session on the device: phablet-shell

2. Make the root partition writable:

sudo mount -o rw,remount /

3. Install the build dependencies:

sudo apt update
sudo apt install devscripts bzr build-essential debhelper doxygen libanthy-dev libchewing3-dev libgl1-mesa-dev libgles2-mesa-dev libglib2.0-dev libgsettings-qt-dev libmirclient-dev libhunspell-dev libpinyin7-dev libpresage-dev libxml2-utils maliit-framework-dev pkg-config python3-flake8 python3-setuptools qt5-default qtbase5-dev qtbase5-private-dev libfontconfig1-dev libfreetype6-dev libudev-dev libmtdev-dev libxrender-dev qtdeclarative5-dev qtdeclarative5-private-dev xvfb presage

4. Checkout your branch on the device:

bzr branch <your branch>

5. Run “debuild” in the root of your source checkout, this will build all the packages and place them in the directory above your source directory

6. Install the new packages:

sudo dpkg -i ../*.deb

7. As this also installs the autopilot tests the first time around you’ll need to also run:

sudo apt -f install

8. Stop the current maliit-server:

stop maliit-server

9. Restart maliit with additional debugging:

MALIIT_DEBUG=true maliit-server

I did all of the above successfully on my Nexus 4. To make sure everything has actually worked, I replaced the key for “Q” and changed it to “W”. This change does show up in the keyboard, as you can see from the image at the top of this post.

Side notes

Insufficient space on root partition

I had problems with installing the development tools (step 3 above): packages failed to install due to insufficient storage in the root partition of my Nexus 4. I uninstalled many apps to free some space. At the end, I had to completely wipe the phone and install a fresh Ubuntu image. Even more, I had to also delete some not-crucially-needed system directories

sudo rm -r /usr/share/locale-langpack/[!en]*

This is a terrible thing to do on a production system. But since I use the Nexus 4 only for development, it just provided me with the much needed free space to install dev tools.

Difficulty in editing files on the phone

Since editing any text file on the phone is very difficult, I did all the actual edits on my desktop machine, synced  it to my ownCloud instance, and downloaded the edited files via wget.

Compile process stopped by the phone OS

Each compile takes about 45 minutes to complete. During compilation, if I switch out of the terminal app, or if the phone is locked after a few minutes of inactivity, the compile process is stopped by the OS. I assume this is done to all app in order to save power. To prevent this, I disabled screen locking in Setting -> Security and Privacy -> Locking and unlocking.

What’s next?

In the next posts, I will continue to document my work on making a Persian layout for Ubuntu phone. Meanwhile, I’d love to hear your comments. If you are expert in Ubuntu touch development, do you see something that I am doing wrong or inefficient? If you are an Ubuntu phone end user, do you need a particular keyboard layout to be added or fixed?

Development i18n Ubuntu

Persian keyboard layout for Ubuntu phone – Part 1: What to expect

Ubuntu touch does not have a Persian keyboard layout (as of December 2016). I thought it would be a valuable contribution to the Ubuntu project if I could make one. I had previously designed and implemented a soft keyboard layout for Android (that is, Persian layout for the excellent Hacker’s Keyboard by Klaus Weidner). I knew that making a new keyboard layout for an existing keyboard system should not be so difficult: as long as I start from an existing keyboard layout that is most similar to Persian (e.g. Arabic), I can replace Arabic characters with Persian ones and the rest would be mostly cosmetic changes.

That’s why I decided to make it.

Android’s Persian keyboard layout

But before I start, let’s take a look at Persian keyboard layout in my Android smartphone running Android 5.1. The following is the Persian layout in Android in its various modes.

normal mode
normal mode, extended letters for Alef
normal mode, extended diacritics symbols
normal mode, the Zero-Width Non-Joiner (ZWNJ) key, together with the extended form, Zero-Width Joiner (ZWJ)
symbols mode
symbols mode, extended Rial (currency) key
symbols mode, shifted

From the above, it is evident that there is a lot going on in the Persian layout of Android 5.1. It requires much more work than to just replacing a handful of characters in the normal mode of a Latin layout and to call it a Persian layout. Most notably,

  • there is no shift key in the normal mode for Persian layout. The Arabic script (which is also used in modern Persian language) does not have a concept of lower case and upper case.
  • The extended keys of the dot key are completely different from a Latin layout. They are filled with diacritics symbols used in the Persian alphabet. Theses are the symbols that represent short vowels in the Arabic script.
  • There is a separate key near the space key dedicated to ZWNJ, and its extended key, ZWJ. These (invisible) special characters are defined in unicode and control how neighboring characters join or not join. The symbols shown on the key buttons are different from what they actually insert into the text.
  • In the symbols mode, Persian digits are used instead of Latin digits (e.g. ۲ instead of 2). Notice also that the same Persian digits are shown as small hints in the normal mode of the Persian layout (the upper row of the keyboard). Also, Persian currency symbol (Rial) is used.
  • When in symbols mode, the key to switch back to normal mode is not “ABC”, but “اب‌پ” (the first three letters of the Persian alphabet).

If I set Android Persian layout as the ultimate goal for Ubuntu touch, it is clear from the above that I should

  • define a separate symbols keyboard, specific to Persian,
  • define special keys for ZWNJ and ZWJ, in the same way that other special keys (shift etc.) are defined,
  • make sure Persian digits are used in symbols mode and are also shown as hints in the normal mode

However, I have some criticisms for the Android layout, and I would like to avoid those (in my opinion) Android’s mistakes. Most importantly,

  • there is no (Persian) question mark (؟) and comma (،) in the normal mode. One has to enter symbols mode to enter them. In Latin keyboards, both such keys are extended forms of the dot (.) key and can be typed very quickly directly from the normal mode.
  • Keys from different rows are all aligned horizontally. This is unlike Latin keyboards where for example the key “A” is between “Q” and “W” (in the lower row). This perfect alignment will increase probability of mis-typing. It would be nice to make a different number of keys in each row of the layout, so that keys from different rows are not horizontally aligned.
  • There are simply too many consonant characters that are hardly used in Persian. I would very much prefer to hide keys Jeh (ژ), Theh (ث) and Thal (ذ) as extensions to keys Zeh (ز), Teh (ت), and Dal (د), respectively. This will save some precious horizontal space, and will make the keys larger, hence easier to type.

In the next post, I will write about the first steps to make changes to the Ubuntu keyboard.