Porting Android to a new device

by Peter McDermott

Executive summary

If
you’re having trouble estimating the time and complexity of porting
Google’s Android mobile platform to your embedded device, our
experience porting Android to a Nokia N810 Internet Tablet may provide
you with useful information.

Android overview

Android is a mobile phone
software platform created by Google. It is based on the Linux kernel, a
number of software libraries written in ‘C,’ a Java-like virtual
machine for running application code, and a full suite of mobile phone
applications.

Android costs nothing and the source code is
freely available. Its license terms (see the Apache license link at the
end) are commercial-friendly, saying, in essence, "Do what you like
with it, just don’t sue us if something goes wrong." The one exception
to this rule is the Linux kernel, which is licensed under the GNU
Public License. Because of this, manufacturers must release their
device’s Linux kernel source code after product shipment.

Taiwan’s
HTC shipped the first Android phone, the G1, in October 2008. Other
mobile manufacturers, including Motorola, LG, and Samsung are all
reportedly creating Android-based phones. Additionally, from NthCode’s
home base in China, we see many startups and smaller manufacturers
leveraging Android in an attempt to gain a foothold in the global
smartphone market.


T-Mobile G1, the first Android phone

The
Android platform provides all the services one would expect in a mobile
phone, including phone calling, networking, messaging, and multimedia.
Our analysis of the source code so far is that these services are
complete and built on top of an excellent architecture, but do not
implement every last feature found in modern mobile phones. This is the
1.0 release, after all.

For application developers, Android
builds on top of the popular open-source Eclipse integrated development
environment. Developers can use Eclipse to write Java applications,
which are then compiled to Java bytecode before being translated for
running on Android’s Dalvik Virtual Machine (VM).

Google
claims their Dalvik VM runs more efficiently in terms of speed and
memory tradeoffs than Java — slyly suggesting that Google wrote their
own VM for performance reasons. However, while we haven’t seen
supporting data, we suppose Google really created Dalvik to escape from
needing to cede control or pay a license fee to Sun, the creator and
steward of the Java platform.

By translating from Java to Dalvik
bytecode, Google is able to leverage the mature tools and large
existing community of Java developers. The downside is that developers
will need to take the additional step of running the Google translation
tool on their compiled Java code before their application will run on
an Android phone (Google automates this step in their Android Eclipse
plug-in, however).


Android emulator screenshot

Google
also provides an Android emulator. The emulator is a virtualized ARM
microprocessor that runs the same system code — and almost exactly the
same Linux kernel — that runs on a device. Google provides an Eclipse
add-on so that Android programs can be compiled and debugged in Eclipse
while running in their emulator. The Android emulator is as complete an
emulator as we have ever seen — just make sure your developers have
enough RAM in their PCs to be able to run Eclipse and the emulator
together.

So what’s in it for Google?

Google has
given mobile phone manufacturers everything they want — a free, open,
mobile operating system — and, in return, only asked that the devices
they create can run Android applications. We can only speculate that
Google expects to eventually earn more from consumers using Google
services on Android devices than it costs Google to develop and
maintain Android.

Overview of the Nokia N810

We
work with embedded Linux systems at NthCode, so we thought it would be
a fun challenge (and would help with a project we’re working on) if we
could port Android to an existing Linux device.

Luckily, we
already had two Nokia N810 Internet Tablets in the office. The N810 is
a handheld mobile device that comes with a high-resolution 4.1-inch
touch screen and a slide-out QWERTY keyboard. The N810 has a 400 MHz
Texas Instruments OMAP 2420 CPU, 128MB RAM, and WiFi networking. The
one thing the N810 can’t do is make phone calls — this is why Nokia
markets it as an Internet Tablet that can surf the web, rather than a
phone. At the time of this writing, the N810 costs about USD $425.

An
active community of enthusiasts enjoys hacking the N810’s Linux-based
software. While some of those enthusiasts had already ported a
pre-release version of the Android SDK to the N810, we discovered that
no one had yet ported the Android 1.0 release to it. So we decided to
see how long it would take us, and what we could learn in the process.

Development steps

As
mentioned earlier, the Android system software runs on top of a Linux
kernel. This Linux kernel provides services to applications — such as
file access, process scheduling, and inter-process communication.

Google
made a number of modifications to the standard Linux kernel for
Android. Likewise, Nokia modified the standard Linux kernel to support
their hardware, such as the keypad, touch screen, and file system.

We
quickly discovered that Nokia’s N810 changes are made against an
earlier 2.6.21 Linux kernel. And, unlike earlier Android prereleases,
the Android 1.0 modifications are made against a later 2.6.25 Linux
kernel. We investigated and discovered that between these two versions,
a year had elapsed and the Linux community had made thousands of
changes to kernel source code.

So, to make both Nokia’s and
Google’s modifications work together on the N810 we either needed to
port the N810 changes to work on the newer 2.6.25 Linux kernel, or port
the Android changes to the N810’s earlier 2.6.21 Linux kernel.

What did Google change in the kernel?

We
checked the differences between the Android kernel and the standard
Linux kernel and found that Google had changed 75 files and added an
additional 88. We have prepared an annotated list of changed files at
the end of this document, and a brief summary here.

  • Goldfish — 44 Files
    — The Android emulator runs a virtual CPU that Google calls Goldfish.
    Goldfish executes ARM926T instructions and has hooks for input and
    output — such as reading key presses from or displaying video output
    in the emulator.

    These interfaces are implemented in files
    specific to the Goldfish emulator and will not be compiled into a
    kernel that runs on real devices. So we safely ignored these files in
    our work.

  • YAFFS2 — 35 Files — Unlike PCs,
    which store files on disks, mobile phones store files in sold-state
    flash memory chips. The HTC G1 uses NAND flash, a type of flash memory
    that is becoming more popular due to its combination of high density
    and low cost.

    YAFFS2 is an acronym for "Yet Another Flash File
    System, 2nd edition." It provides a high-performance interface between
    the Linux kernel and NAND flash devices. YAFFS2 was already freely
    available for Linux. However, it is not part of the standard 2.6.25
    Linux kernel, and so Google added it to Android.

  • Bluetooth — 10 files
    — Google made changes to 10 files in the Bluetooth communications
    stack. These changes fix apparent bugs related to Bluetooth headsets,
    and add Bluetooth debugging and access control functions.

  • Scheduler — 5 files
    — The Android kernel also contains slight changes to the CPU process
    scheduler and time-keeping algorithms. We don’t know the history of
    these changes, and the impact was not evident based on a cursory
    examination.

  • New Android Functionality — 28 files
    — In addition to bug fixes and other small changes, Android contains a
    number of new subsystems that are worth mentioning here, including the
    following:

    • IPC Binder — The IPC Binder is an
      Inter-Process Communication (IPC) mechanism. It allows processes to
      provide services to other processes via a set of higher-level APIs than
      are available in standard Linux. An Internet search indicated that the
      Binder concept originated at Be, Inc., and then made its way into
      Palm’s software, before Google wrote a new Binder for Android.

    • Low Memory Killer
      — Android adds a low-memory killer that, each time it’s called, scans
      the list of running Linux processes, and kills one. It was not clear in
      our cursory examination why Android adds a low-memory killer on top of
      the already existing one in the standard Linux kernel.

    • Ashmem
      — Ashmem is an Anonymous SHared MEMory system that adds interfaces so
      processes can share named blocks of memory. As an example, the system
      could use Ashmem to store icons, which multiple processes could then
      access when drawing their UI. The advantage of Ashmem over traditional
      Linux shared memory is that it provides a means for the kernel to
      reclaim these shared memory blocks if they are not currently in use. If
      a process then tries to access a shared memory block the kernel has
      freed, it will receive an error, and will then need to reallocate the
      block and reload the data.

    • RAM Console and Log Device
      — To aid in debugging, Android adds the ability to store kernel log
      messages to a RAM buffer. Additionally, Android adds a separate logging
      module so that user processes can read and write user log messages.

    • Android Debug Bridge
      — Debugging embedded devices can best be described as challenging. To
      make debugging easier, Google created the Android Debug Bridge (ADB),
      which is a protocol that runs over a USB link between a hardware device
      running Android and a developer writing applications on a desktop PC.

      Android
      also adds a new real-time clock, switch support, and timed GPIO
      support. We list the impacted files for these new modules at the end of
      this document.

  • Power Management — 5 files
    — Power management is one of the most difficult pieces to get right in
    mobile devices, so we split it out into a group separate from the other
    pieces. It’s interesting to note that Google added a new power
    management system to Linux, rather than reuse what already existed. We
    list the impacted files at the end of this document.

  • Miscellaneous Changes — 36 files
    — In addition to the above, we found a number of changes that could
    best be described as, ‘Miscellaneous.’ Among other things, these
    changes include additional debugging support, keypad light controls,
    and management of TCP networking.

  • NetFilter — 0 files
    — Finally, our change list showed Netfilter as having 22 changed
    files. However, examination showed the only difference was the
    capitalization of the filenames (xt_DSCP.c vs. xc_dscp.c). The contents
    of the files were all identical. So we ignored these files in our port.

Port the N810 changes forward or the Android changes backward?

As
you can see from the above, Android makes significant changes to the
standard Linux kernel. Additionally, Google’s system software checks to
make certain the kernel version is 2.6.25 or newer. This led us to
wonder if Android requires features in the standard 2.6.25 Linux kernel
that are not in earlier releases.

On the other hand, the N810
kernel changes were only to add hardware support. Nokia did not change
core operating system services or add new interfaces.

We decided that we would have a higher chance of success by porting the N810 changes forward to the 2.6.25 Linux kernel.

So, without further ado…

Step 1: Port the N810 kernel changes to the 2.6.25 Linux kernel

This work took one developer seven days to complete, and included the following steps:

  • Downloaded the latest Linux-OMAP-2.6 kernel from Montavista
  • Selected the 2.6.25 version from the kernel download (used ‘git reset –hard v2.6.25-omap1’))
  • Patched the 2.6.25 kernel with the following patches from the N810’s 2.6.21 kernel:
    • N810 Multi Media Card (MMC) driver
    • N810 I2C serial bus controller chip driver
    • N810 touch screen driver
    • N810 keyboard mapping
    • N810 audio and video DSP driver
    • N810 DMA driver
    • N810 board revision patch

After
a number of software build/flash/test/debug cycles, our new kernel was
able to boot the N810’s default software and function well.

One
issue we did have is that the kernel driver for the CX3110 WiFi chip is
provided as a binary-only kernel module, not source code that we could
recompile on the new kernel. This meant that we were not able to make
our new kernel support WiFi on the N810.

This is an important
note for those trying to do similar ports: if you have binary-only
drivers, make sure they’ve been compiled against the kernel baseline
that you plan to use.

Step 2: Add the Android kernel patches to the 2.6.25 N810 Linux kernel

With
our new N810 Linux kernel up and running, we could now apply the
Android patches. This was mostly straightforward and took one developer
two and a half days of applying patches, flashing the kernel, and
verifying the patches worked.

If you are using a kernel baseline
different from the one in the Android SDK, you should plan to spend
more time and resources to make it work correctly.

Luckily,
the Android and N810 patches do not touch any of the same kernel parts.
This made merging these two change sets much easier.

Step 3: Bring-up Android on our new kernel

After
two weeks of patching and testing, we were finally ready to try to boot
the full Android system. Software development would not be any fun if
everything worked the first time, so, luckily, there were issues (that
is a joke).

Here is the list of what we worked through:

  • The N810’s jffs2 file system lacked a feature Android needs in the mmap system call
  • So we switched to using the ext2 file system, but it reported data corruption
  • So we switched to using the ext3 file system, and the corruption went away
  • The system hung while booting, and while we were trying to debug, it would reboot
  • We discovered the watchdog timer was rebooting the system, so we disabled it
  • After that, the system sometimes successfully booted, but the screen was blank
  • We could see graphics after we applied a double-buffered video patch to the N810
  • After this, we could sometimes boot to the home screen, but the system often crashed
  • We recopied some system libraries to the N810, and the crashes went away
  • We enabled the touch screen by changing a power setting in the system software
  • We enabled the keypad by fixing a keymap file in the system software

And,
finally, afer 19 days, it worked. The Android robot would show while
powering up, and then we could see the main menu and use the
applications.


Android 1.0 booting on the N810
(Click to play)

Step 4: Play Around

This,
of course was the pay off. If you get this far, we recommend that you
take a photo and email it to your nerdiest friends and colleagues to
show off your accomplishment.


Android 1.0 running on Nokia’s N810 Internet tablet
(Click to enlarge)

Next steps

We
want to be able to create our own Android applications and run them on
the N810. To do that, we will need to add support for the Android Debug
Bridge, fix an intermittent system crash, and see if we can enable
battery charging when the power supply is plugged in. Additionally, our
application needs Internet access, so we either need to make the WiFi
chip work, or enable USB Ethernet to send network data through the USB
cable to a host PC, which can then route the N810’s data to the
Internet.

Finishing up

Before we’re done, we will
release the changes we made back to the Android and N810 communities so
that they can take advantage of the work we have done and have a bit of
fun.

Things to consider in your project

If you’re porting Android to a new device, we recommend you consider the following points:

First,
this is a just a milestone along the way to any kind of shipping
product. We still have work to do before the N810 will be good enough
for us to develop custom applications. Likewise, the N810 isn’t a
phone, so we didn’t have to spend time debugging the call stack. And
then there are the pieces that don’t work — power management, WiFi,
etc. — that could all take more time to make work than we spent on
this initial port. So the hard work is ahead, not behind.

Second,
the N810 is a shipped commercial product that has gone through Nokia’s
strenuous quality control processes and is well loved by lots of
hackers. We have worked on other projects with flakey hardware and no
vendor support (yeah, that was a huge mistake — and it was made even
worse by doing it fixed-bid). If you are porting Android to your own
hardware, be sure to factor time and resources into dealing with
hardware risks. We highly recommend that you make sure there are
software people assigned to making the hardware drivers work.

Third,
figure out which baseline kernel version you will use at the start.
This way, you can verify or add tasks for making sure your hardware has
the kernel drivers developed for that version of the Linux kernel. And
since an outside vendor often provides binary drivers, make certain
that you will have those drivers compiled against your Linux kernel
when you need them.

Fourth, use the right tools and processes —
preferably the same ones that the Linux and Android developers use.
We’ve worked with big companies who import open-source software into
closed-source development models. While this makes it easier to get
started, these teams later pay a price when they are not able to easily
merge new versions of open-source software into their code base.
Additionally, if your developers want to use Windows at all, we
recommend you push them out of their cocoon and into the Linux world. A
highly effective team would be one running Linux on their desktop, git
for source control, and vim or emacs to edit ‘C’ code.

Finally,
assemble the right team. While we’re new to Android, we’ve been hacking
embedded Linux systems for a long time. So check your teams’ experience
and plan accordingly.

Saying thanks

First, I’d
like to thank Tang Yongjun, the NthCode developer who actually ported
Android to the N810. I don’t know that I would have been as patient as
he was.

He and I would both like to also thank PenguinBait,
B-man, and QWERTY-12 (yes, awesome names), who made the original port
of the pre-release Android SDK to the N810. We were able to follow many
of their steps to make our port of the Android 1.0 release easier.

Finally,
this is a living document. If you spotted a mistake, or think others
would benefit some piece of piece of information we don’t have here,
please email me so I can consider the change.


Download

We
have placed our kernel patch, a complied kernel image, a compiled
userspace image, and a few notes at the following SourceForge.Net page:
Android-N810 Downloads.

Appendices

Annotated index of Android kernel source diffs

Resources

Apache Open Source License
GPL License
Montavista’s Linux-OMAP Repository
The Original Android N810 port
Elinux.org’s Wiki for Android on OMAP
Maemo Wiki About Updating N810 Firmware

This paper was originally published by NthCode, here. It is reproduced with permission.


About the author
— Beijing resident and American expatriate Peter McDermott founded
NthCode in 2005 to tap into the talent he saw in his Chinese co-workers
while creating some of the world’s first Linux-powered mobile phones.
Peter enjoys running NthCode to this day and maintains a personal
weblog, Beijing Analog to Digital where he discusses technology, business, and life in Beijing. Click the photo at left for an NthCode staff photo.