Using an emulator
We just saw how useful it can be to have all the developer kits' rootfs on the host, but what if we can execute all the programs directly on the host? Referring to the earlier example with the Hello World program, we mean the possibility to compile it on the host and then executing it on the host too.
It is quite obvious that the advantages in this case are minimum, but consider the case where we have a complex program to compile with tons of libraries. Of course, this approach has some disadvantages. First of all, the fact that our x86 CPU has no idea about how to execute the ARM code, so we need a program that emulates the ARM CPU over the x86 one. This emulation needs a lot of CPU resources, and most probably, the execution time is slower than the original one. However, in some circumstances, it may be preferred to emulate the ARM CPU. A very powerful embedded system may have two 4 GB RAMs whereas a real powerful host PC may have 32 GB, without considering the fact that the host's disk can be 10 times quicker than a microSD or flash memory.
Despite wishing that we use an emulator, let's see how we can use it.
The emulator we're going to use is QEMU, a generic machine emulator (and virtualizer). If we take a quick look at QEMU's wiki site at http://wiki.qemu.org/Main_Page , we read the first lines:
When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance.
It's exactly what we need!
Executing a program
As the first (and simple) example, let's see how we can execute the Hello World program on the host. We already cross-compiled it, and we got an ARM executable:
# file helloworld helloworld: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dy namically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=9d36da7eb92d0d552bc04a7771f5ebbb14d04497, not st ripped
OK, now, we need to install the QEMU program, which is split into several packages:
$ apt-cache search qemu | grep '^qemu' qemu-block-extra - extra block backend modules for qemu-system and qem u-utils qemu-kvm - QEMU Full virtualization qemu-slof - Slimline Open Firmware -- QEMU PowerPC version qemu-system - QEMU full system emulation binaries qemu-system-arm - QEMU full system emulation binaries (arm) qemu-system-common - QEMU full system emulation binaries (common) qemu-system-mips - QEMU full system emulation binaries (mips) qemu-system-misc - QEMU full system emulation binaries (miscelaneous) qemu-system-ppc - QEMU full system emulation binaries (ppc) qemu-system-sparc - QEMU full system emulation binaries (sparc) qemu-system-x86 - QEMU full system emulation binaries (x86) qemu-utils - QEMU utilities qemu-efi - UEFI firmware for virtual machines qemu - fast processor emulator qemu-guest-agent - Guest-side qemu-system agent qemu-launcher - GTK+ front-end to QEMU computer emulator qemu-user - QEMU user mode emulation binaries qemu-user-binfmt - QEMU user mode binfmt registration for qemu-user qemu-user-static - QEMU user mode emulation binaries (static version) qemubuilder - pbuilder using QEMU as backend qemuctl - controlling GUI for qemu qemulator - transitional dummy package to virtualbriks
However, we just need the qemu-user and libc6-armhf-cross packages, one that stores the emulator itself and the libc libraries for ARM. So, let's install them:
$ sudo aptitude install qemu-user libc6-armhf-cross
When the installation is finished, we can go back where the helloworld program is placed and execute it with QEMU:
# cd /opt/armhf-rootfs-debian-jessie/root/ # qemu-arm -L /usr/arm-linux-gnueabihf/ helloworld Hello World
Tip
The root's privileges are not required by QEMU. We need them just because the directory where the program is located is owned by root.
As we can see, this is nice, but if limited to this usage, it can be quite useless. However, this is just the beginning, since QEMU can do much more. In particular, it can emulate a complete embedded system, that is, emulating the whole hardware (CPU, memories, and peripherals), or it can act as a generic CPU that uses the host PC resources.
The differences between these two different approaches is that the former needs a more complete support by QEMU (since it must emulate not only the CPU but all the other peripherals too), while the latter just needs the CPU and system calls emulations. This is just how the qemu-arm programs works, but we want more. We want to avoid specifying special paths for external libraries and/or other kind of dependencies. We'd like to execute a program or, better, a whole root filesystem as an ARM CPU does.
Tip
Due to space reasons and since this is not the main target of this book, we'll show the latter operation mode only. You may wish to emulate a whole ARM system and can take a look at the QEMU documentation.
Entering into an ARM rootfs tree
If we take a look at the qemu-user packages, we get three kinds of them:
$ apt-cache search qemu-user qemu-user - QEMU user mode emulation binaries qemu-user-binfmt - QEMU user mode binfmt registration for qemu-user qemu-user-static - QEMU user mode emulation binaries (static version)
What is really interesting for us is the last one, qemu-user-static. Here the package's description:
$ apt-cache show qemu-user-static Package: qemu-user-static Priority: optional Section: universe/otherosfs ... Description-en: QEMU user mode emulation binaries (static version) QEMU is a fast processor emulator: currently the package supports ARM, CRIS, i386, M68k (ColdFire), MicroBlaze, MIPS, PowerPC, SH4, SPARC and x86-64 emulation. By using dynamic translation it achieves reasonable speed while being easy to port on new host CPUs. . This package provides the user mode emulation binaries, built statically. In this mode QEMU can launch Linux processes compiled for one CPU on another CPU. . If binfmt-support package is installed, qemu-user-static package will register binary formats which the provided emulators can handle, so that it will be possible to run foreign binaries directly. ...
So, using this package, we can launch Linux processes compiled for one CPU on another CPU. We can also use this static version, and we don't need any external native (x86) libraries. So, we can suppose to use chroot in the ARM rootfs and then start working as we were on an ARM CPU!
Tip
For those that doesn't know what the chroot command does, let me suggest to take a look at its man pages.
OK, maybe these concepts are quite obscure for a newbie, so let's do a demonstration of this fantastic feature.
First of all, let's install the needed package:
$ sudo aptitude install qemu-user-static
Then, we have to copy the binary of the ARM rootfs. Here, we can check that the binary is compiled statically using the ldd program:
$ ldd /usr/bin/qemu-arm-static not a dynamic executable $ sudo cp /usr/bin/qemu-arm-static /opt/armhf-rootfs-debian-jessie/usr/bin/
Then, we need to mount some special filesystems on the ARM filesystem before doing chroot. In particular, we need the /dev, /proc and /sys directories due to the fact that they are needed to most of the Linux's standard commands. To do this without mounting them twice, we can use the bind option of the mount command, so we can have the devfs, procfs, and sysfs in more than one place at time.
So, let's duplicate the needed filesystems in our ARM rootfs:
$ for fs in dev proc sys ; do \ sudo mount -o bind /$fs \ /opt/armhf-rootfs-debian-jessie/$fs ; \ done
Then, we have to add some tempfs too (this is Debian specific):
$ sudo mount -t tmpfs -o 'rw,nosuid,nodev,mode=755' tmpfs /opt/armhf-rootfs-debian-jessie/run $ sudo mkdir /opt/armhf-rootfs-debian-jessie/run/lock $ sudo mount -t tmpfs -o 'rw,nosuid,nodev,noexec,relatime,size=5120k' tmpfs /opt/armhf-rootfs-debian-jessie/run/lock
We should copy some networking settings:
$ sudo cp /etc/resolv.conf /opt/armhf-rootfs-debian-jessie/etc/resolv.conf
As the last step, we have to jump into the ARM rootfs! In the following commands, we used the uname command before and after the jump in order to show you that we effectively changed the running platform:
giometti@ubuntu1510:~$ uname -a Linux ubuntu1510 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 U TC 2016 x86_64 x86_64 x86_64 GNU/Linux giometti@ubuntu1510:~$ sudo chroot /opt/armhf-rootfs-debian-jessie/ root@ubuntu1510:/# uname -a Linux ubuntu1510 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 U TC 2016 armv7l GNU/Linux
Great! We are running an ARM rootfs on an X86 PC!
Tip
Note that for better readability, we didn't remove the prompt as we did on every command executed into the host PC. So, you can notice that before chroot, the prompt is giometti@ubuntu1510:~$ while it becomes root@ubuntu1510:/# later on.
We can verify that all filesystems we created for this target are in place:
root@ubuntu1510:/# mount udev on /dev type devtmpfs (rw,nosuid,relatime,size=1005904k,nr_inodes =251476,mode=755) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755) tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=51 20k)
We can now execute our helloworld program as we were on the Wandboard:
root@ubuntu1510:/# cd root/ root@ubuntu1510:/root# ls helloworld helloworld.c root@ubuntu1510:/root# file helloworld helloworld: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dy namically linked, interpreter /lib/ld-linux-armhf.so.3,for GNU/Linux 2 .6.32, BuildID[sha1]=9d36da7eb92d0d552bc04a7771f5ebbb14d04497, not str ipped root@ubuntu1510:/root# ./helloworld Hello World
If we correctly set up the networking support, we can proceed in installing new packages as on every ARM machine. So, we can update the current repositories:
root@ubuntu1510:/root# apt-get update Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB] Ign http://httpredir.debian.org jessie InRelease Get:2 http://repos.rcn-ee.com jessie InRelease [4350 B] Get:3 http://httpredir.debian.org jessie-updates InRelease [142 kB] Get:4 http://security.debian.org jessie/updates/main armhf Packages [2 92 kB] Get:5 http://repos.rcn-ee.com jessie/main armhf Packages [375 kB] Get:6 http://httpredir.debian.org jessie Release.gpg [2373 B] Get:7 http://httpredir.debian.org jessie Release [148 kB] Get:8 http://httpredir.debian.org jessie-updates/contrib armhf Package s [20 B] Get:9 http://httpredir.debian.org jessie-updates/main armhf Packages [ 9276 B] Get:10 http://security.debian.org jessie/updates/contrib armh f Package s [994 B] Get:11 http://security.debian.org jessie/updates/non-free armhf Packag es [20 B] Get:12 http://httpredir.debian.org jessie/main armhf Packages [8834 kB ] Get:13 http://httpredir.debian.org jessie-updates/non-free armhf Packa ges [450 B] Get:14 http://httpredir.debian.org jessie/contrib armhf Packages [44.6 kB] Get:15 http://httpredir.debian.org jessie/non-free armhf Packages [74. 5 kB] Fetched 9992 kB in 1min 2s (160 kB/s) Reading package lists... Done
Tip
Note that all downloaded repositories are based on the armhf platform instead of x86.
Then, we can install a native ARM compiler:
root@ubuntu1510:/root# apt-get install make gcc
Then, we can natively recompile our Hello World program:
root@ubuntu1510:/root# make helloworld cc helloworld.c -o helloworld root@ubuntu1510:/root# file helloworld helloworld: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dy namically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=e124a6c84b518908a8c6e25365169fc18890dfde, not st ripped root@ubuntu1510:/root# ./helloworld Hello World
Tip
Of course, this demonstration is just a brief introduction about what we can do with QEMU and how we can use this operation mode to develop a complex application. To present all QEMU's features and all modes of functioning, I need to write a dedicated book.
As a final note, we have to remark that when we wish to close this ARM root filesystem emulation, we can simply type the exit program as follows:
root@ubuntu1510:/root# exit exit giometti@ubuntu1510:~$
Then, we have to unmount all the previously mounted filesystems:
$ for fs in dev proc sys run/lock run ; do \ sudo umount /opt/armhf-rootfs-debian-jessie/$fs ; \ done