cloud goes social

Disclaimer

I am not responsible if anything goes bad. Ever. In particular if your devices will be unusable after following this tutorial! Mine didn’t :)

Introduction

I’ve recently been involved in making some changes to the Linux kernel. Not any Linux kernel but the Android Linux Kernel. I’ll share what I believe is a pretty good workflow for everyone interested in playing with it both on a NexusOne (with the stock OS, i.e. WITHOUT CyanogenMod) and on a HTC Magic (with CyanogenMod installed). They are quite similar scenarios, and many others have been writing about it. I always had to change a few things from others descriptions and therefore here I report what I found to be a working solution for the latest patched updated versions. This is an aggregation work of others spread all around the interwebs. I like things accessible in one place :)

Throughout this “tutorial” I will be assuming that you are working on a Linux environment (i.e. Ubuntu). Furthermore I work on a 32-bit system, if you are not in this situation try it out, but further changes might be needed.

Preliminary Settings

Let’s start with having all the necessary software which we will need later installed and operative. If you are already an Android and/or a Kernel developer you most probably have this already done. Skip this section.

1. adb is needed. Install the latest Android SDK – I personally install it in

   ~/Bin/android-sdk-linux_x86

   Remember to add the tools/ directory to your $PATH variable.

2. ndk is needed. Install the latest Android NDK – I personally install it in

   ~/Bin/android-ndk-VERSION

   For example the latest is version r5 and the directory will be:

   ~/Bin/android-ndk-r5

   . Remember then to add the toolchains/arm-eabi-4.4.0/prebuilt/linux-x86/bin directory that is present inside the ndk to your $PATH variable.

3. fastboot is needed. Grab it and download it inside your sdk/tools directory or anywhere that is in your $PATH
4. Software is needed. Install it.

   apt-get install git-core gnupg sun-java6-jdk flex bison gperf libsdl-dev libesd0-dev libwxgtk2.6-dev build-essential zip curl libncurses5-dev zlib1g-dev
   

   If you are on a 64-bit machine also install:

   apt-get install ia32-libs lib32z1-dev lib32ncurses5-dev gcc-multilib g++-multilib
   

In the end your ~/.bash_profile, ~/.bash_rc, … would look along the lines of:

export PATH="~/Bin/android-sdk-linux_x86/tools:~/Bin/android-ndk-r5/toolchains/arm-eabi-4.4.0/prebuilt/linux-x86/bin:${PATH}"

Now let’s dwell into the interesting things. First of all have your directory structure well planned ahead. Or you will end up losing a lot of time just remembering what is what.

mkdir ~/android
mkdir ~/android/bootimg_builder
cd ~/android

Now we clone the git repositories for the Google kernel and for the CyanogenMod kernel. We will have two directories with the sources for the two kernels: cm-kernel and google-kernel

git clone git://github.com/CyanogenMod/cm-kernel.git cm-kernel
git clone git://android.git.kernel.org/kernel/msm.git google-kernel

This is where I will first go into explaining the procedure for the HTC Magic with CyanogenMod installed and afterwards I will explain how to make it work for the NexusOne.

HTC Magic with CyanogenMod

Let’s start by switching to our own branch, in case we want to modify anything and keep track of the changes without polluting the master branch. The latest CyanogenMod (6.1) uses the kernel version 2.6.35, we keep on using that.

cd cm-kernel
git checkout -b personal_branch origin/android-msm-2.6.35-froyo-stable

Now we need to pull the configuration file off from the device, have it plugged to the PC through the USB port.

adb pull /proc/config.gz config.gz
gunzip config.gz && mv config .config

Now we update the .config that will be used while compiling the kernel.

ARCH=arm CROSS_COMPILE=arm-eabi- make oldconfig
ARCH=arm CROSS_COMPILE=arm-eabi- make menuconfig

The last command will open up an interactive menu. Change what you want (i.e. nothing if you don’t require particular stuff…) and then get to Exit (and select Yes if you changed something, if you didn’t you won’t be prompted).

It is time to actually compile the kernel.

ARCH=arm CROSS_COMPILE=arm-eabi- make

The first time this will be a long procedure, the resulting file will be stored in

arch/arm/boot/zImage

and you can test it on your device directly by going into fastboot mode (hold the BACK button while powering up the device). Then issue this command:

fastboot boot arch/arm/boot/zImage

The phone should boot up normally and you can confirm that it’s running your kernel by going into Settings-About-Scroll down to the Kernel section. Note that if you reboot the phone it will go back to using the pre-installed kernel. More on flashing the kernel later on.

NexusOne with standard Android OS

First we need to unlock the NexusOne to accept new roms to be installed, have it in fastboot mode by holding the TRACKBALL down while booting up and issue this command

fastboot oem unlock

Note that this will void your warranty!

We also need the NexusOne to be rooted: rooting goes beyond the scope of this tutorial. Have your phone rooted in any way that you want (i.e. by using the SuperOneClick tool (it requires a Windows machine or luck with a Linux one).

This one is much more interesting! And while being very similar it also has some differences. First of all from our ~/android/ directory let’s move to the google-kernel directory and again create our own branch to work on.

cd ~/android/google-kernel
git checkout -b personal_branch origin/android-msm-2.6.32

Note that in this case we’re using the kernel version 2.6.32 which is the one that’s installed by default on the NexusOne. The latest 2.6.35 should work but I haven’t really tested.

Next we create the usual .config file

ARCH=arm CROSS_COMPILE=arm-eabi- make mahimahi_defconfig
ARCH=arm CROSS_COMPILE=arm-eabi- make menuconfig

Hit Exit and save if prompted (you shouldn’t!)

We now manually modify the .config file with your favorite text editor open the file and search for these strings

nano .config
uncomment CONFIG_USB_ANDROID_ACM=y (add the =y part and remove what's there)
comment CONFIG_MSM_SERIAL_DEBUGGER
comment CONFIG_MSM_FIQ_SUPPORT

We’re ready to compile the kernel

ARCH=arm CROSS_COMPILE=arm-eabi- make

This will take some time, the resulting product will be stored in arch/arm/boot/zImage and you can test it on your device directly by going into fastboot mode (hold the TRACKBALL down while powering up the device). Then issue this command:

fastboot boot arch/arm/boot/zImage

The phone should boot up normally and you can confirm that it’s running your kernel by going into Settings-About-Scroll down to the Kernel section. Note that if you reboot the phone it will go back to using the pre-installed kernel. More on flashing the kernel later on.

NexusOne enable my WiFi, please

The WiFi kernel module that is on the NexusOne is not compiled to work with this kernel that we are using therefore WiFi won’t work. There’s a work around but it requires you to have root privileges on your device. Rooting goes beyond the scope of this tutorial. Have your phone rooted in any way that you want (i.e. by using the SuperOneClick tool (it requires a Windows machine or luck with a Linux one).

First you copy the compiled kernel module to the phone /sdcard

cd ~/android/google-kernel
adb push drivers/net/wireless/bcm4329/bcm4329.ko /sdcard/

Then you need to make the phone /system partition writable, back up the old module and install the new one

adb shell
su
mount -o remount,rw -t yaffs2 /dev/block/mtdblock3 /system
cd /system/lib/modules/
cat ./bcm4329.ko > ./bcm4329.ko-orig
cat /sdcard/bcm4329.ko > ./bcm4329.ko
mount -o remount,ro -t yaffs2 /dev/block/mtdblock3 /system

At this point your WiFi should be working again! :)

On flashing for awesome later use

Now that you tested that your compiled/(modified) kernel is actually working you may want to have it installed also after rebooting the phone. To achieve this you need to flash it to the phone internal storage. It is advisable that first you backup everything using a recovery tool (Amon RA 2.0.0 for the NexusOne works great with the nandbackup tool).

First grab two utilities that will be needed for this step and unzip them into the ~/android/bootimg_builder directory that we have created previously.

Throughout this part of the tutorial I will give commands for both the Magic and the NexusOne, please use only the ones that are useful for your case.

We first need to grab the boot.img file from the running device, have the device running and connected to the PC via USB then issue the command

cd ~/android/bootimg_builder
adb shell
cat /dev/mtd/mtd2 > /sdcard/mtd2.img
exit
adb pull /sdcard/mtd2.img ./nexus-boot.img

OR for the MAGIC

adb pull /sdcard/mtd2.img ./magic-boot.img

We now extract the boot.img-kernel and ramdisk by using the supplied perl script

./split_bootimg.pl nexus-boot.img

OR for the MAGIC

./split_bootimg.pl magic-boot.img

The output of this command will be along the lines of:

Page size: 2048 (0x00000800)
Kernel size: 2226104 (0x0021f7b8)
Ramdisk size: 164628 (0x00028314)
Second size: 0 (0x00000000)
Board name:
Command line: no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5
Writing nexus-boot.img-kernel ... complete.
Writing nexus-boot.img-ramdisk.gz ... complete.

We now need to copy the compiled kernel to our working directory discarding the one extracted previously

cd ~/android/bootimg_builder
cp ../google-kernel/arch/arm/boot/zImage ./nexus-boot.img-kernel
OR for the MAGIC
cp ../cm-kernel/arch/arm/boot/zImage ./magic-boot.img-kernel

Now we combine everything into just one boot.img that we can then flash to the device

./mkbootimg --kernel nexus-boot.img-kernel --ramdisk nexus-boot.img-ramdisk.gz -o boot.img --cmdline 'no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5' --base 0x20000000

OR for the MAGIC

./mkbootimg --kernel magic-boot.img-kernel --ramdisk magic-boot.img-ramdisk.gz -o boot.img --cmdline 'no_console_suspend=1 console=null'

At this point we’re ready to flash the new image to the device, have it in fastboot mode (TRACKBALL on the NexusOne, BACK on the Magick while booting up) and issue the command

fastboot flash boot boot.img
fastboot reboot

If everything has gone well enjoy your new kernel installed definitively on your device. You did make a nandbackup before in case you want to go back, right? Well, I told you to :D

Conclusion and Acknowledgments

Hope you enjoyed this full tutorial on everything needed to compile, test and install a kernel on a NexusOne or on a Magic with CyanogenMod installed. I would like to thank you all that have done the hard work of writing tutorials on how to do so before [here|here], the guys that maintain CyanogenMod [here|here] and those who wrote the perl script to extract the two images from the boot.img file [here]. Mine has been just a work of aggregation. But I wanted to have everything working and described in a single post.

Cheers.

~C

Last week, on Friday, I attended the rescheduled-at-the-last-hour Google Android Developers Lab in Zürich and, as a result, I started learning about Android application development, the Android Google SDK and, as a gift, I came back home with a shiny new NexusOne, which is not available in Europe, yet.

I’m now making a list in the typical fashion of a good ol’ Italian-directed western movie both from a (advanced, I daresay) user and a developer perspective. I hope no-one will feel bad about what I write as most of it, to me, are easily acknowledgeable facts, rather than my only suppositions.

Let’s start up without further waiting.

THE GOOD

NexusOne Overall

I must say that is the best non-iPhone iPhone-competitor that I’ve seen, tried and played with (played as in using, not played as in playing mobile games, for which I don’t have time). The size is right about the same as the iPhone (yes, don’t give me shit it’s one millimeter (I DO use the metric system, and all of you should, as well) less thick, and stuff like that). So, for any iPhone user it just fits good in your hand/pocket/wherever-you-carry it. The screen is bright, the manufacturing quality seems good, it has a replaceable battery with all the pros and cons of that (i.e. it fell and of course the shell opened up, the battery came off, etc, I don’t have to tell you all of this, right?). The screen finishing seems different from the iPhone one and oily fingers DO leave more visible traces than on the iPhone. But that’s just noticeable while the screen is switched off.

Android OS 2.1 updated

The OS is stable, it’s fast and it’s good. I like the interface elements, even though they are much different from the iPhone ones. Everything that’s shipping with/on the phone is good, fast and reliable. A few things could be changed, perhaps, but nothing too radical or that would have a major impact on the overall OS. The “notification system” is a well structured way of informing the user of changes or if something happened. Maybe having the date always on on the top bar while no notifications are available would be a nice thing, rather than having to touch the bar to display the date. But maybe that’s even an option, I do not know.

SDK

At the Android Developers Lab we had the pleasure to have a few talks by one of the Android Application Development Gurus: Reto Meier (The guy that wrote this book). And coming from the wonderful iPhone/MacOSX SDK I was impressed by the quality put into the Android SDK. It surely is easy to learn, easy to code against and quite good. A few very good ideas are Alarms (I smell cronjob around them! And I like it!) and Intents for inter-app communication. Please Apple, if you are reading this have something like it on the iPhone, as well!

THE BAD

The Keyboard
It just badly suck. Especially if you come from the iPhone. First of all, like many European user, we tend to know at least two languages. One being our mother tongue, the other being English. And that’s saying “at least two”, many do speak even three or four. It is very common for me, for example, but for many, many (yes Google, if you’re reading this: MANY) other to write in different languages. And sorry to say it but switching dictionary-based-not-so-well-working-word-recognition on the Android is even harder than on old mobile phones (and I DO mean old mobile phones). Basically it just has ONE language for input AND for the system. To change that you need to go all the way to the preferences. It just resolves into not caring about the language-input-recognition. Which isn’t a very good system, either, yet.

Secondly the keys are too packed together. Or the touch sensitivity isn’t that great. Or I do not know what, but on the iPhone I rarely rarely mistype a word, and in that rare occasion it just gets (properly) corrected. On the NexusOne Android phone I keep, keep, keep mistyping words. And they don’t get corrected, or get corrected in the wrong/weirdest way (for Italian input, at least).

Just two examples, first in Italian, second in English. In Italy we use accents. Quite a lot. The future first person singular of the verb “to go” is “andrò”.

Android I type: Andro – it suggests: Andro, androne, Andria, Andromeda, andrò – for it I have to scroll, as well, on the suggestion.

iPhone I type: Abdro – it suggests: andrò

Let’s see with English, where there’s much use of the “‘” character for shortening things around. Let’s write “you’ve”.

Android I type: Youve – it suggests: nothing

iPhone I type: Youve – it suggests: you’ve

Now, this is just two examples, and both use quite annoying things, such as accents, for which normally you would have to tap-and-keep-pressed a letter for the options to come up or the “‘” character which requires on both phones to “switch” keyboard to the punctuation and then back to the letters (which the iPhone is smart enough to do on his own, anyway, getting more points for it, as well), but I happen to use such features much. much. MUCH. And I do bet most of the user will find this “problem” quite annoying.

Enough said: Google, please fix the keyboard: changing input language, understanding what I would’ve wanted to write, user-interaction needed to write.

This is the ONE BAD THING about the Android OS. Unfortunately it is also quite a big issue for smartphones where we usually “write” more than “speak”. Fortunately it is a software-fixable issue. So just fix it! :)

THE UGLY

Apps

Ok, the iPhone has been out for longer, has less fragmentation, whatnot. But come on, some “bigs” can really do a better job with their apps. First of all, the Facebook app, if compared to the iPhone companion is like 1 era away. I won’t say much more, just update it! Shazam is at the same level, what a pity. And that’s just to cite two GOOD apps. The Market is plagued by bad applications that suck much. And they suck more than most iPhone applications, which already suck quite a lot (many of them, at least). This of course has nothing to do with Google, which is actually delivering quite GOOD applications. Keep the good work up, Google. Step it up, developers!

Just one thing related to Application development. InterfaceBuilder, for the iPhone GUI has no rivals. Just think of something, writing GUI for Java is THE nightmare, at the current state of things.

Battery Life

I haven’t played around too much with the phone. But battery draining seems quite fast. I’ve been mostly texting/writing emails and the thing (brand new!) lasts only two/three days. My iPhone 3G which is now one year and a half old, used exactly in the same way + for listening to some music lasts the same or at times more. This is NOT a good signal. So please, engineers, work on this issue. Please.

WRAPPING IT UP

The NexusOne is definitely a good phone. Possibly some Android OS updates which I hope won’t come too late in the future will make it even a better one. The openness of the Market is surely a good thing, but at this time I would still suggest buying and iPhone. That said, I will wait to see enhancements been thrown down the pipe and update you on the situation. I’m sure the Android OS has a bright future, and I’m happy to be using it and to be developing for it. As much as I’m happy that the iPhone is around, so I don’t waste hours to write an email :P

These are just my thoughts.

~C