How u-boot pass parameters to Linux kernel

Linux (kernel) takes parameters when it boots up. This resembles an application started in a console with command line parameters. So we call kernel parameters "kernel command line", although kernel is not really started the same way as a command line application.

Linux kernel defines quite a few boot parameters. The kernel-parameters.txt file under .../Documentation has a complete list of all supported kernel parameters for the specific kernel version. Some examples are boot_delay, console, initrd, nfsroot, etc. These parameters affect the way the kernel boots up and behaves when it's up running.

Linux is typically booted (run) by a bootloader. For some bootloaders such as u-boot, they are linux aware and provide mechanism to pass command line to linux when it boots up the kernel. But for others that are not Linux aware and no way is available to pass command line, the kernel command line could be hardcoded in source code and compiled into the kernel image.

u-boot passes kernel command line by bootargs environment variable. All the key=value pairs in bootargs are passed over as command line to kernel. So if you want to pass certain kernel parameters, you put them in the bootargs environment variable (note this is u-boot environment variable). When u-boot starts kernel code by bootm for example, it will pass the start address and end address of the command line string, and kernel takes it from there.

There are many details involved in the process of passing command line from bootargs to kernel, and how the kernel parse, process and make use of the command line. But I think this is already enough to customize a bootable system.

Deadlock and how to avoid it

Deadlock is a situation where two processes are each waiting for the other to release a resource it held, or more than two processes are waiting for resources in a circular chain, so that no one can have the resource required and all stop running.

Deadlock can happen between processes, thread or task (in vxWorks). And it can happen on any kinds of shared resources. I use process here for discussion.

There are three conditions in order for a deadlock to happen:
1. Each of the processes involved access multiple shared resources.
2. Each of the processes involved hold some shared resources while requiring other shared resources.
3. A circular waiting chain potentially possible.

To handle a deadlock, basically we have to break one or more of the conditions above. There are four ways to avoid it as far as I know.
1. All the processes apply the same coding pattern. And the pattern is, all the processes require (semWait for example) the shared resources in the same order, so that circular chain will be formed.
This method is suitable for smaller scale application, where all the shared resources can be listed and ordered.

2. Back-off algorithm. Each of the processes either have all the shared resources it need before proceed or none of them. In reality, the first semTake() can use WAITFOREVER, and the subquential semTake() use NOWAIT. And if one of the subsequential semTake() fail, it will back off and release all the resources it already holds.
This method increase the coding complexity. And it's not suitable for realtime system as the back off takes unpredictable time.

3. Avoid processes access multiple resources. This can be done by redesign the software's structure, algorithm or data structure. For example, a client-server model can be used so that only the server manage the shared resources, while client access the resources through the server.

4. Use some sort of watchdog to monitor the processes and if a deadlock is detected, reset the system or the processes.
Since a deadlock situation rarely happens, and if built-in mechanism is too expensive, then a third party monitor maybe suitable. For example, Linux kernel totally ignore deadlock and pretend it will never happen! (Supprising, isn't it? But that's real. When a deadlock indeed happens, the system just reboot.)

Embedded linux application remote debugging using GDB

As we all know, GDB is a pretty good debugging tool. Using it to debug applications running on the same Linux machine on which you run the GDB debugger is somewhat different from debugging an application running on a different architecture. This is called remote debugging, i.e, the application is running on a Linux base embedded system (with a different CPU), and the GDB debugger is running on the development host machine.

Starting from version 5.3 (am I right?), GDB supports remote debugging (even multi-threaded program).

Here is how to set up GDB for remote debugging. I'm using m68k as an example.(I know it's pretty old, but for other architecture, the process is pretty much the same).

1. Build and install GDB cross debugger
Download GDB from http://ftp.gnu.org/gnu/gdb, then make and install it.

tar zxvf gdb-6.6.tar.gz
cd gdb-6.6/
mkdir build
cd build/
../configure --program-prefix=m68k-uclinux- --target=m68k-elf --disable-werror
make
sudo make install

2. Build and install gdbserver. (Some linux distributions have gdbserver as an application in the rootfs. Some toolchains have it in the package. If it's not available there, you have to cross compile it)
gdbserver comes with the GDB package. So to cross compile it you can do:

$ cd gdb-6.6/gdb/gdbserver
$ export CC=m68k-ulinux-gcc // if required
$ export LD=m68k-ulinux-ld // if required
$ ./configure --host=m68k-uclinux --target=m68k-uclinux
$ make

3. Build the application with proper CFLAGS, LDFLAGS etc.
For CFLAGS, I believe you need at least -g. And if possible, give -O0 to disable optimization. An example of mytest.c will be:

m68l-uclinux-gcc -g -O0 -Wall -o mytest mytest.c

There are 2 executables generated, one is mytest stripped off symbols, and the other is mytest.gdb with all the symbols. It'll be used by the debugger to load symbol table.

4. Copy gdbserver and mytest over to target (using nfs for example)

5. On the target side, run application by gdbserver. I use network as the communication interface, but you can use serial port too.

gdbserver :3000 mytest

The 3000 is the port number on which gdbserver is listening for gdb commands.

6. On the client side, run cross debugger gdb, connect to gdbserver. And once connected, you can use any gdb commands for debugging.

m68k-uclinux-gdb 192.168.1.100:3000 ./mytest.gdb

192.168.1.100 as you imagine is the target's IP address. And if you want to be able to "list" source code in gdb, you have to put the source code (mytest.c) together with mytest.gdb in the same dir. (You can set it up if you really want to put the source code in other dirs by setting environment variables)

And that's it. There are some details I missed here, they can be found online or by gdb help.

Linux System.map file and its use.

System.map file is the kernel's symbol table created everytime the kernel is compiled. It has the address information of all the symbols (variables, functions, etc) used in kernel. These address information can be useful for trouble shootting and debugging. For example, last time my uClinux kernel doesn't boot up, I used System.map to find out that the RAM end address is not correctly set, and from there I finally solved the problem.

Here is a good article talking about System. It's from http://rlworkman.net/system.map/. But I copy it here for ease of access.

---------------------------------------------------------------------------------
This page is a mirror of Peter Jay Salzman's System.map Explanation, and the only modification made by me is the addition of this note. --rworkman

The system.map File
There seems to be a dearth of information about the System.map file. It's really nothing mysterious, and in the scheme of things, it's really not that important. But a lack of documentation makes it shady. It's like an earlobe; we all have one, but nobody really knows why. This is a little web page I cooked up that explains the why.

Note, I'm not out to be 100% correct. For instance, it's possible for a system to not have /proc filesystem support, but most systems do. I'm going to assume you "go with the flow" and have a fairly typical system.

Some of the stuff on oopses comes from Alessandro Rubini's "Linux Device Drivers" which is where I learned most of what I know about kernel programming.

What Are Symbols?
In the context of programming, a symbol is the building block of a program: it is a variable name or a function name. It should be of no surprise that the kernel has symbols, just like the programs you write. The difference is, of course, that the kernel is a very complicated piece of coding and has many, many global symbols.

What Is The Kernel Symbol Table?
The kernel doesn't use symbol names like BytesRead(). It's much happier knowing a variable or function name by the variable or function's address, like c0343f20. Humans, on the other hand, do not appreciate addresses like c0343f20. We prefer to use symbol names like BytesRead(). Normally, this doesn't present much of a problem. The kernel is mainly written in C, so the compiler/linker allows us to use symbol names when we code and allows the kernel to use addresses when it runs. Everyone is happy.

There are situations, however, where we need to know the address of a symbol (or the symbol for an address). This is done by a symbol table, and is very similar to how gdb can give you the function name from an address (or an address from a function name). A symbol table is a listing of all symbols along with their address. Here is an example of a symbol table:

c03441a0 B dmi_broken
c03441a4 B is_sony_vaio_laptop
c03441c0 b dmi_ident
c0344200 b pci_bios_present
c0344204 b pirq_table
c0344208 b pirq_router
c034420c b pirq_router_dev
c0344220 b ascii_buffer
c0344224 b ascii_buf_bytesYou can see that the variable named dmi_broken is at the kernel address c03441a0.

What Is The System.map File?
There are 2 files that are used as a kernel symbol table:

/proc/kallsyms
System.map
There. You now know what the System.map file is.

Every time you compile a new kernel, the addresses of various symbol names are bound to change.

/proc/kallsyms is a "proc file" that is created on the fly when a kernel boots up. Actually, it's not really a disk file; it's a representation of kernel data which is given the illusion of being a disk file. If you don't believe me, try finding the filesize of /proc/kallsyms. Therefore, it will always be correct for the kernel that is currently running.

However, System.map is an actual file on your filesystem. When you compile a new kernel, your old System.map has wrong symbol information. A new System.map is generated with each kernel compile and you need to replace the old copy with your new copy.

What Is An Oops?
What is the most common bug in your homebrewed programs? The segfault. Good ol' signal 11.

What is the most common bug in the Linux kernel? The segfault. Except here, the notion of a segfault is much more complicated and can be, as you can imagine, much more serious. When the kernel dereferences an invalid pointer, it's not called a segfault -- it's called an "oops". An oops indicates a kernel bug and should always be reported and fixed.

Note that an oops is not the same thing as a segfault. Your program (usually) cannot recover from a segfault. The kernel doesn't necessarily have to be in an unstable state when an oops occurs. The Linux kernel is very robust; the oops may just kill the current process and leave the rest of the kernel in a good, solid state.

An oops is not a kernel panic. In a panic, the kernel cannot continue; the system grinds to a halt and must be restarted. An oops may cause a panic if a vital part of the system is destroyed. An oops in a device driver, for example, will almost never cause a panic.

When an oops occurs, the system will print out information that is relevent to debugging the problem, like the contents of all the CPU registers, and the location of page descriptor tables. In particular, the contents of the EIP (instruction pointer) is printed. Like this:

EIP: 0010:[<00000000>]
Call Trace: []

What Does An Oops Have To Do With System.map?
The information given in EIP and Call Trace is not very informative. Since a kernel symbol doesn't have a fixed address until after the kernel is booted, c010b860 can point to any kernel symbol. Kernel developers wouldn't have the faintest clue where to begin looking for the bug if you simply reported an address. They need a symbol name to begin hunting for the bug.

To help understand cryptic oops output, a daemon called klogd, the kernel logging daemon, is used to perform symbol-address translation. When an ooops occurs, klogd intercepts the oops report, translates addresses into symbol names (e.g. translating c010b860 into BytesRead()), and logs the event with the system logger, usually syslogd,

To perform kernel symbol-address resolution, klogd uses System.map.

There. Now you know what an oops has to do with System.map.

Fine print:
There are actually two types of address resolutions performed by klogd.

Static translation, which uses the System.map file.
Dynamic translation, which is used with loadable modules. These translations don't use System.map and is therefore not relevant to this discussion, but I'll describe it briefly anyhow:
Klogd Dynamic Translation
Suppose you load a kernel module which generates an oops. An oops message is generated, and klogd intercepts it. It is found that the oops occured at d00cf810. Since this address belongs to a dynamically loaded module, it has no entry in the System.map file. klogd will search for it, find nothing, and conclude that a loadable module must have generated the oops. klogd then queries the kernel for symbols that were exported by loadable modules. Even if the module author didn't export his symbols, at the very least, klogd will know what module generated the oops, which is better than knowing nothing about the oops at all.

Where Should System.map Be Located?
System.map should be located wherever the software that uses it looks for it. It's the only answer possible until some standards board (or someone of clear authority) mandates exactly where System.map should be located and what its name should be. With that in mind, let's look at some software packages and where they expect System.map to be.

klogd
If klogd isn't given the location of System.map as a command line option with the -k switch, it uses the following string array (as of version 1.4.1) to search for it (see source code file ksym.c:

static char *system_maps[] =
{
"/boot/System.map",
"/System.map",
#if defined(TEST)
"./System.map",
#endif
(char *) 0
};
klogd looks for both "System.map" and "System.map-release" in these directories where "-release" is your kernel version. This is an intelligent search: if klogd finds a System.map for a kernel version that is different from the currently running kernel, it'll keep searching.

Although the klogd man pages and source code comments claim that /usr/src/linux is in the search path, I can't find any reference to it. I've reported this to Debian BTS and to Dr. G.W. Wettstein (the author of ksym.c).

Device Drivers
System.map isn't just useful for debugging kernel oopses. A few drivers need System.map to resolve symbols since they're linked against kernel headers instead of glibc). They won't work correctly without the System.map for the particular kernel currently running. This is NOT the same thing as a module not loading because of a kernel version mismatch, which has to do with the kernel version, not the kernel symbol table which changes between kernels of the same version!

ps
ps uses a different (more general) search array than klogd:

*sysmap_paths[] = {
"/boot/System.map-%s",
"/boot/System.map",
"/lib/modules/%s/System.map",
"/usr/src/linux/System.map",
"/System.map",
NULL
};
where %s gets replaced by the currently running kernel version.

What else uses (or doesn't use) the System.map
At one point (May 2003), I thought that lsof and dosemu used System.map, but from looking at the source code (May 2007) they don't appear to anymore (or perhaps I was mistaken).

What Happens If I Don't Have A Healthy System.map?
Suppose you have multiple kernels on the same machine. You need a separate System.map file for each kernel. If you run a kernel with no (or an incorrect) System.map, you'll periodically see annoying warnings like:

System.map does not match actual kerneleverytime you use ps. Also, your klogd or ksymoops output will not be reliable in case of a kernel oops.

TODO
Look at ps more closely to determine if the man page is really in error, and if so, report it to Debian BTS and the ps maintainers.
Fix the CSS: my markup sucks ass. I'm really bad at webpage design. Need a better way to distinguish between sections and subsections. Fixed font content should have a smaller a font size.


Acknowledgements
Rickey Page (28 May 2003) for doing my heart good.
Mauro Giachero (May 2007): Read the klogd source code and determined that the klogd man page and source code comments are wrong about the System.map man page search path. He also provided information about the System.map usage (or lack thereof) of lsof, ps, and dosemu.

uClinux 2.6.x ROM based image with only data, init and bss in RAM

Recently I've been trying to build a uClinux (2.6.x kernel)ROM based image for Coldfire uc5272, with the kernel executing (XIP) from within ROM, .data, .init and .bss segments in the RAM, and ROMfs as rootfs in ROM. This arrangement saves memory as only these changable parts of the system are in RAM.

There are three files to change to build such an image:
1. vmlinux.lds.S
This is the linker script file for 2.6.x kernel. It describes the memory layout of the image. The .romvec and .text segments are defined in ROM, and .data, .init, .bss segments are defined in RAM.

2. Head.S
This is the second bootloader for 2.6.x kernel. It't the entry point of the kernel. In this file, some codes are needed to move .data, .init into RAM, and clear out .bss segments.

3. uclinux.c
This is the mapping driver for ROMFS partition support. Here the starting address of romfs has to be given.

I have made all the changes, load the image into ROM, and issue a "go" command. What I got first is:

go 0x10c20000 ...

Then it just stopped there. After digging down for a while, I found the problem. The _ramend variable is 0, while it's supposed to be the end of the RAM. But I clearly set a value for it in head.S. And here is the code snap:

GET_MEM_SIZE /* macro code determines size */
addl %a7,%d0
movel %d0,_ramend /* set end ram addr */

And here is the code for moving .data, .init into RAM:

lea _etext, %a0
lea _sdata, %a1
lea _sbss, %a2
_copy_data:
movel (%a0)+, (%a1)+
cmpal %a1, %a2
bhi _copy_data

How come _ramend is 0? By the way, the logs at __log_buf (0x3df44 of RAM in my case) indicates that there was a memory access violation, which coincides a _ramend with wrong value.

The answer is really simple. But finding the answer took me some time. It turned out that the order of setting _ramend and moving .data into RAM is significant and I did it in a wrong order.

The right order is to move .data into RAM first and then set _ramend. Why? because when the image is burnt into ROM, the .data segment is in ROM, with all the variables in it have their init values. For _ramend, it's 0. But _ramend's address is actually in RAM (0x2000c in my case). So if _ramend is set before .data is moved, the set value will be over taken by the init value from ROM. So the .data needs to be moved into RAM first, then set the value for _ramend, the value will be kept and for later use.

How do I find the address of _ramend? I found it from System.map. The first 4 variables in .data segment are:

00020000 D _rambase
00020000 D _sdata
00020004 D _ramvec
00020008 D _ramstart
0002000c D _ramend

Linux kernel and glibc

I have this question for a while. What's the dependency of kernel and system libraries such as glibc? Can kernel codes call functions in glibc? Do we need glibc when we compile kernel? Or do we need kernel code to compile glibc?

This is what I found from http://kernelnewbies.org/FAQ/LibraryFunctionsInKernel:

Q: Can I use library functions in the kernel ?
A: System libraries (such as glibc, libreadline, libproplist, whatever) that are typically available to userspace programmers are unavailable to kernel programmers. When a process is being loaded the loader will automatically load any dependent libraries into the address space of the process. None of this mechanism is available to kernel programmers: forget about ISO C libraries, the only things available is what is already implemented (and exported) in the kernel and what you can implement yourself.

Note that it is possible to "convert" libraries to work in the kernel; however, they won't fit well, the process is tedious and error-prone, and there might be significant problems with stack handling (the kernel is limited to a small amount of stack space, while userspace programs don't have this limitation) causing random memory corruption.

Many of the commonly requested functions have already been implemented in the kernel, sometimes in "lightweight" versions that aren't as featureful as their userland counterparts. Be sure to grep the headers for any functions you might be able to use before writing your own version from scratch. Some of the most commonly used ones are in include/linux/string.h.

Whenever you feel you need a library function, you should consider your design, and ask yourself if you could move some or all the code into user-space instead.

So, the idea is that the kernel doesn't (and actually can't) refer to external libraries. Kernel and glibc can be compiled seperately. However, glibc seems to have to compile against a particular kernel. Why? because it uses some kernel headers. Also, when a new kernel comes out, it's new features can't be handles by the old APIs, new libraries are needed and they have to be compiled against the new kernel.

An endianess problem with uClinux on uc5272

I have been struggling in the past few days trying to run a uClinux cramfs image on a Coldfire uc5272 board. I know, the uc5272 is pretty old, with its uCbootloader at version 1.7.7. I'm trying to run a 2.6.x kernel image on it, so it's interesting to see how a pretty new kernel running on a pretty old hardware.

uCbootloader has special support for cramfs image. Unlike u-boot or most of the other bootloaders, uCbootloader understands cramfs and can "ls" its contents, can "cat" file contents. And the actual kernel file, linux.bin, is embedded in the rootfs "/" directory. uCbootloader will uncompress the image, retrieve linux.bin out of the rootfs and run it, and the kernel in turn mount the fs.

u-boot and most other bootloader requires that the kernel file is out of the rootfs, usually on top of rootfs in the image. The kernel got loaded first, and then the rootfs got mounted by the kernel.

I have tried a few builds but none of them works. "ls" shows an error "Not a valid file system", and "go" doesn't uncompress the image. After a while I realized that either the image is not valid, or the uCbootloader somehow doesn't like the image, one way or the other. Nothing to do the kernel, because it's not even there yet.

I started to doubt the endianess of the image. I used "od" to display the magic number of the image, it's 0x28cd3d45, little endian, which is the host's endianess. It's supposed to be right. There is a convention that cramfs image is always little endian, even for big endian CPU. The kernel is supposed to do the swapping.

So what's going wrong, what about the uCbootloader? It doesn't complain "wrong magic number" etc. But it's old right? So I did a little experiment, I change the order of the first 4 bytes of the image to make it looks like a big endian image (the first 4 bytes as a long, is the magic number of cramfs). And "ls" works! Though it's all messy codes, but it seems to accept it as cramfs.

Then I used /sbin/mkfs.cramfs on my Ubuntu 9.10 to build a big endian image. This one support different endianess by a "-N" option. The image is downloaded and programed into flash, and "ls" shows its contents correctly. Beautiful! And "go" command uncompress the image, load the kernel from "/" dir and run. It stops there though, complaining illegal instructions. This is not supprising , because the kernel is built in little endian.

So, it's a good exercise at least. I have 2 things to go further, one is update the uCbootloader to 1.7.8. And the other is to build a big endian kernel.

"static" in C and C++

This post will talk a little bit about the C/C++ keyword "static" and its use.

static as a key word can be used in three distinct occasions in C:

1. a static local variable declared within a function. The value of the static variable remains between function invocations.
2. a static variable declared within a module (a file for example), but outside any function bodies. Such a variable can only be accessed by functions within the module, but not functions in other modules. It's a localized global variable.
3. a static function declared in a module. Such a function has a scope of the module, with only functions in the module in which it's declared can call it.

For embedded Linux, a program's static variables (and global variables) are actually in the program's data segment, and exist until the program terminates. (Since compiler and linker know their existence and their types, they are pre-allocated.) To be more accurate, these with initial values are in the data segment, while these without initial values are in the program's BSS segments with "0" as initial values.

This is different for local variables declared in a function (auto variables), and params passed over to functions, they got their storage in the thread's stack(usually every thread of the program has a stack). And memory allocated by malloc() is from the program's heap(there's typically one heap for the program).

Now for static in C++. There are:

1. static data members of class. static data members are associated to a class instead of an object of the class. There is only one memory copy of a static data member for all the objects.
2. static methods. Again, a static method is associated to a class instead of an object. A static method is just like a regular function. The only difference is that it can access private and protected static data members of the class. And if you make a static method public, it can be called just like a function, with the class's name and a double "::" in front of it.

Linux/Unix file system inode

Every objects (file, directory,etc) in a Unix style file system is represented by an inode. An inode is a data structure which contains information about the file object, such as user and group ownership, access mode (read, write, execute permissions) and type of file etc. An inode is again identified by a inode number, which serves as the index of the inode.

The harddrive or a partition containing a file system usually has three parts, a super block, an area for inodes, and the largest part, blocks for the actual file contents.

The inode number indexes a table of inodes in a known location on the device. From the inode number, the kernel can access the contents of the inode, including the data pointers, and so the contents of the file.

The kernel doesn't really know the name of the file in the file system. All it knows is inodes. The directories (which is a file object itself) translate file names to
inodes. Its contents is a mapping table of file names and corresponing inode.

Some commands can be used to see the inode of a file.

ls -i
example:
# ls -i /etc/passwd
# 32820 /etc/passwd

stat
example
# stat /etc/passwd
# File: `/etc/passwd'
Size: 1988 Blocks: 8 IO Block: 4096 regular file
Device: 341h/833d Inode: 32820 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2005-11-10 01:26:01.000000000 +0530
Modify: 2005-10-27 13:26:56.000000000 +0530
Change: 2005-10-27 13:26:56.000000000 +0530

Again, file name is just a human readable representation of the file. If a hardlink is created on a file, a "ls -l" command will show the file has 2 links, meaning the file has 2 names now. And if a symbolic link is created on a file, a "ls -l" command will show that the file has only 1 link. The symolic link itself is a newly created file, with its contents pointing to the original file. A symbolic link has an inode associated with it.

There are also APIs to manipulated file by inodes. There are mainly for system/application development purpose.

Linux and uClinux rc file and inittab file

On a linux system inittab file, rc scripts and relavant config files in /etc play key roles at system startup. Inittab file is read and parsed by init process, to decide what level the system will run at and which processes are started at bootup and at normal operation. Then the rc script will be run at that level.

There are system V and BSD styles to the arrangement of inittab, rc scripts and config files. And there are differences between Linux distributions. But the key point is, the system will startup at a predefined level most likely defined in inittab, and rc scripts will be run for that level to spawn all the necessary processes.

In uClinux there are inittab and rc file as well. But there are no run levels. An embedded system usually runs at just one level. So be carefull, uClinux inittab and rc are different from that of standard Linux.

In uClinux, rc is executed BEFORE inittab! rc file basically contains commands to prepare the system environment such mount /var, make directories, startup network etc. Inittab can be used to startup programs and services, such as serial terminals, IP servers, etc.

Something about the terminal. The line for terminal is usually like this:

ttyS0:vt100:/bin/agetty 9600 ttyS0

agetty is an excutable binary in /bin directory. It opens a tty port, prompt a login name and invokes /bin/login command. The above is the simplest format of an agetty command. It actually has quite a few options to change its behaviors. For example this one:

agetty -L 9600 ttyS1 vt100

This will force the line to be a local line with no need for carrier detect. This can be useful when you have a locally attached terminal where the serial line does not set the carrier detect signal. Try this if your terminal just sleeps instead of giving you a password: prompt.

Another thing is that agetty will optionally display the contents of /etc/issue. You can use a -f to display an alternative file, or you can use -i to ignore any issue file, not to display the contents of /etc/issue (or other) before writing the login prompt.

One more note, if there is no agetty in inittab, and you choose init with "enable console shell" when you configure core applications, then the uClinux will start without login at all.

MJPG-streamer, a light weight video streaming server for embedded system

A short post, about a light weight video streaming server. I've been looking around for a small but robust realtime video streamer that can stream video from a camera over IP network to a standalone viewer or browser. And of course it has to be able to run smooth on a resource constrained embedded system such as mini2440(ARM9 + 64M ram + 128M flash).

mjpg-streamer looks like a good fit. It's an open source project at http://sourceforge.net/projects/mjpg-streamer/. And it's compiled for mini2440 at http://code.google.com/p/mjpg-streamer-mini2440/.

So I installed the binary for mini2440, it works pretty good. It works with a cheap USB camera, it works with a CMOS camera too. Only thing is, the frame rate is not high enough, 5 frames/sec, with 1-2 seconds delays. CMOS camera is obviously better than USB camera regarding to frames rate and delays. I haven't tried a camera with some compression capability yet, but it's supposed to have better result.

But it's pretty good overall. There are a few things I want to do with it (too busy now to do it now) in the near future:
1. A better camera
2. Make it part of the rootfs
3. Look into the code to see how it works with the camera driver.
4. Some modifications, such as embedded data within the pictures.

How come a ROM based ROMFS image can run from within RAM?

Background: On a Coldfire uC5272 platform, a ROM based ROMFS image(linux.bin + romfs) supports XIP(execution in place), meaning the image can be burnt into Flash and boot and run from within ROM, with only the .data and .bss section moved into RAM by second bootloader(crt0_rom.S for 2.4.x kernel).

The image I built just works fine. But I decided to go a bit further. What's gonna happen if I run the image from within RAM? It's not supposed to be able to boot.

To my supprise, when I set uCbootloader environment varible "RAMIMAGE=yes" and type "go" command, the image get copied into ram, boot and run with staring address at 0x0020000, exactly as "RAMIMAGE=no"(which means boot from ROM at 0x10c20000).

Here is my explanation. For uCbootloader, when we type "go" to boot, the loader will fetch the first 32bytes of the image (the .bvec section of linux.bin) as the SP and IP content, and execute from the address pointed to by IP (the entry point of second bootloader). When the image get copied into RAM starting at 0x0020000, and when we "go" from there, the first 32 bytes starting from 0x0020000 are fetched into SP and IP. But, here is the point, the content of IP is still pointed to the entry address of second bootloader in ROM, so the execution is still go back to ROM.

To prove it I erase the Flash so there is no image in ROM at all. Then I download the ROM based image into RAM. Now I "goram" at 0x0020000, system just stop there without booting up, because the address pointed by IP is empty this time.

Conclusion, in a uClinux image, once generated, all the addresses of symbols (global varibles, functions, etc) are arranged and fixed. As long as the execution can jump to the right address, it will execute normally.

How to make putty private key authenticated

Once in a while I need to access my ubuntu9.10 at home from my Windows laptop. This is pretty easy. Once I have a ssh client (putty) installed on my Windows, open a NAT for port 22 on my router, I can login my account by username and password.

This is very insecure as everyone know, a username/passwd authentication in clear text is easy for attackers too. So I changed the authentication to RSA public/private key authentication. Here is a link that gives a pretty good explanation about how to do that. https://help.ubuntu.com/community/SSH/OpenSSH/Keys. But it's talking about clients running linux.

In order for the windows client putty to work, these extra steps are needed:
1. On ubuntu (the ssh server), copy the newly generated public key id_rsa.pub to authorized_keys, and uncomment the "AuthorizedKeysFile %h/.ssh/authorized_keys" in sshd_config. Of course restart sshd by "/etc/init.d/ssh restart"

2. Copy the private key, id_rsa to windows, load it into puttygen and save it as private key in .ppk format. (You need to download puttygen.exe from putty website)

3. Change your session to use private key authentication, by loading the private key into your session's configuration. Don't forget to save your session's configuration!

And the next time you login, you will be asked for a passphrase for your private key. After that, you're logged in private key authenticated. And you feel secured!!!

Linux zigbee checkout

Linux zigbee is an open source project implementing ieee802.15.4 standard and partially zigbee stack in kernel. Its source code can be downloaded by "git clone git://linux-zigbee.git.sourceforge.net/gitroot/linux-zigbee/linux-zigbee", according to http://sourceforge.net/projects/linux-zigbee/develop

As you can see, the project is managed by git, a revision control system used by some open source project.

I'm new to git so I download the source code by following the command, everything is fine. But then when I browse the source code, I found that my local repository doesn't have certain files and directories which exist when I browse them online from here: http://linux-zigbee.git.sourceforge.net/git/gitweb.cgi?p=linux-zigbee/kernel;a=tree;h=d6fac23360241750fcae3d9e2f1012af79934c48;hb=d6fac23360241750fcae3d9e2f1012af79934c48

After a little digging out I realize what's checked out in my local directory is the master branch while the one has more files and directories is devel branch. Some commands can be used to catch up the latest updates from the devel branch:

$git branch -r --->shows all the remote branches git is tracking

origin/HEAD -> origin/master

origin/devel

origin/devel-wpan-phy

origin/master

You can see there are three branches with the HEAD pointed to origin/master.

$git branch --->shows existing branches, with the current branch highlighted with an asterisk.

master

* origin/devel

I have 2 branched with origin/devel as the current one. (I have switched to origin/devel already, otherwise the asterisk will be with master)

$git checkout -b origin/devel --->switch to origin/devel branch, and bring in all the latest code from devel.

With the last command my current branch is switched from master to devel. And I have all the latest updates from devel in my local directory, such as macieee802154, drivers for atmel tranceiver, cc2420, etc.

From here, the following commands can be used to synchronize local repository with remote:

$git pull

or

$git fetch

How is romfs moved above .bss in uClinux?

Platform: Coldfire uc5272 dev board running uClinux 2.4.x

For a ram based image, the .bss section doesn't exist in the image and has to be created by the second boot loader before kernel starts to run. Since the .bss section will be created right after .data section, taking part of the romfs' position, the romfs has to be moved up to make room before .bss is created and initialized.

The assembly file crt0_ram.S acts as the second bootloader. It takes over control from bootloader and do the actual moving of romfs. The following code snapshot is where the moving happens.

85 #ifdef CONFIG_ROMFS_FS

86 /*

87 * Move ROM filesystem above bss :-)

88 */

89 lea.l _sbss, %a0 /* Get start of bss */

90 lea.l _ebss, %a1 /* Set up destination */

91 move.l %a0, %a2 /* Copy of bss start */

92

93 move.l 8(%a0), %d0 /* Get size of ROMFS */

94 addq.l #8, %d0 /* Allow for rounding */

95 and.l #0xfffffffc, %d0 /* Whole words */

96

97 add.l %d0, %a0 /* Copy from end */

98 add.l %d0, %a1 /* Copy from end */

99 move.l %a1, _ramstart /* Set start of ram */

100

101 _copy_romfs:

102 move.l -(%a0), %d0 /* Copy dword */

103 move.l %d0, -(%a1)

104 cmp.l %a0, %a2 /* Check if at end */

105 bne _copy_romfs

106 #endif

The third longword in the romfs image is the size of itself in bytes. So after line 93, register d0 (%d0 hereafter) has the length of the whole romfs. But you may wonder why the length is added a 8 at line 94? and then the last two binary digits are zeroed out at line 95?

The comments explain it but may be too brief to get a full understanding.

As we know, when we align a memory boundary, we align it to the nearest longword position, ie, an address that is divisible by 4. By adding 4 to the address and then zeroed out the last 2 binary digits, which could be 0,1,2,3, we got the nearest longword position.

For example, when we align 0x0101, we hope it is aligned at 0x0104. 0x0101 + 4 = 0x0105, then 0x0105 AND 0xfffc = 0x0104. (it's equal to minus 1).

Then why adding 8 instead of 4 in the snapshot? Because at line 102, 103, the address is minus 4 first before a longword is moved. So we need %a0 and %a1 to point to 1 longword above the actual address.

So there may be up to 4 bytes (if the boundary is already aligned, then it's 4 bytes) that are not part of romfs but get moved. But it doesn't matter, because these extra bytes are from free memory.

A Linux tool for downloading kernel and rootfs to mini2440

The manufacturer of mini2440 provides a MS Windows tool DNW.exe for downloading kernel image and root file system into the board. It works great but I'd like to have a similar tool on my dev Linux box so that I don't have to switch back and forth. Fortunately core_rui wrote a small command line program doing exactly the job. (Even though I prefer a program with GUI, but this is good enough for now).

Its usage is simple:

dnw2 suppervivi-128M

dnw2 zImage_T35

dnw2 root_qtopia_128M

Here is the source code:

/* dnw2 linux main file. This depends on libusb.
*
* Author: Fox
* License: GPL
*
*/
#include <stdio.h>
#include <usb.h>
#include <errno.h>
#include <sys>
#include <fcntl.h>
#include <unistd.h>

#define QQ2440_SECBULK_IDVENDOR 0x5345
#define QQ2440_SECBULK_IDPRODUCT 0x1234

struct usb_dev_handle * open_port()
{
struct usb_bus *busses, *bus;

usb_init();
usb_find_busses();
usb_find_devices();

busses = usb_get_busses();
for(bus=busses;bus;bus=bus->next)
{
struct usb_device *dev;
for(dev=bus->devices;dev;dev=dev->next)
{
if( QQ2440_SECBULK_IDVENDOR==dev->descriptor.idVendor
&& QQ2440_SECBULK_IDPRODUCT==dev->descriptor.idProduct)
{
printf("Target usb device found!n");
struct usb_dev_handle *hdev = usb_open(dev);
if(!hdev)
{
perror("Cannot open device");
}
else
{
if(0!=usb_claim_interface(hdev, 0))
{
perror("Cannot claim interface");
usb_close(hdev);
hdev = NULL;
}
}
return hdev;
}
}
}

printf("Target usb device not found!n");

return NULL;
}

void usage()
{
printf("Usage: dnw2 nn");
}

unsigned char* prepare_write_buf(char *filename, unsigned int *len)
{
unsigned char *write_buf = NULL;
struct stat fs;

int fd = open(filename, O_RDONLY);
if(-1==fd)
{
perror("Cannot open file");
return NULL;
}
if(-1==fstat(fd, &fs))
{
perror("Cannot get file size");
goto error;
}
write_buf = (unsigned char*)malloc(fs.st_size+10);
if(NULL==write_buf)
{
perror("malloc failed");
goto error;
}

if(fs.st_size != read(fd, write_buf+8, fs.st_size))
{
perror("Reading file failed");
goto error;
}

printf("Filename : %sn", filename);
printf("Filesize : %d bytesn", fs.st_size);

*((u_int32_t*)write_buf) = 0x30000000; //download address
*((u_int32_t*)write_buf+1) = fs.st_size + 10; //download size;

*len = fs.st_size + 10;
return write_buf;

error:
if(fd!=-1) close(fd);
if(NULL!=write_buf) free(write_buf);
fs.st_size = 0;
return NULL;

}

int main(int argc, char *argv[])
{
if(2!=argc)
{
usage();
return 1;
}

struct usb_dev_handle *hdev = open_port();
if(!hdev)
{
return 1;
}

unsigned int len = 0;
unsigned char* write_buf = prepare_write_buf(argv[1], &len);
if(NULL==write_buf) return 1;

unsigned int remain = len;
unsigned int towrite;
printf("Writing data ...n");
while(remain)
{
towrite = remain>512 ? 512 : remain;
if(towrite != usb_bulk_write(hdev, 0x03, write_buf+(len-remain), towrite, 3000))
{
perror("usb_bulk_write failed");
break;
}
remain-=towrite;
printf("r%d%t %d bytes ", (len-remain)*100/len, len-remain);
fflush(stdout);
}
if(0==remain) printf("Done!n");
return 0;
}

To compile: gcc dnw2.c -o dnw2 -lusb

dnw2 can be placed in /usr/bin for ease of use.

uClinux memory layout

georges@menie.org gave a good explanation of uClinux memory layout here:

http://www.menie.org/georges/DragonEngine/memory-map.html

Some thing I want to add, how is the image.bin (which is the executable image we usually download to the hardware board to execute) generated.

Like George said:
The uClinux build system is building first the linux kernel, then the user applications and the root filesystem. The last step is usually to extract the text, data and init segments, to convert them in a flat binary file and to add the root file system at the end.

For Arcturus ColdFire uC5272, if you take a look at the Makefile at .../vendors/Arcturus/uC5272, you will find something like this:

$(CROSS_COMPILE)objcopy -O binary --remove-section=.ramvec\
--remove-section=.eram \
--set-section-flags=.romvec=CONTENTS,ALLOC,LOAD,READONLY,CODE \
$(ROOTDIR)/$(LINUXDIR)/linux $(IMAGEDIR)/linux.bin

cat $(IMAGEDIR)/linux.bin $(ROMFSIMG) > $(IMAGE)

Basically objcopy remove .ramvec, .eram sections, and set section flags of linux, and copy the rest into linux.bin. Then linux.bin and romfs.bin are cat'ed into image.bin file.

Open source IEEE802.15.4/Zigbee implementation

This is a summary of my recent research about IEEE802.15.4/Zigbee open source implementation.

1. Linux-Zigbee:

URL: http://sourceforge.net/apps/trac/linux-zigbee/

An implementation of IEEE802.15.4 standard and several protocols running on the top of IEEE 802.15.4 (like ZigBee) for Linux. It contains two parts, kernel protocol stack and userspace utilities.

It's a good candidate for embedded linux system functioning as a Zigbee coordinator, gateway, control center or similar devices. There are also efforts to put it into Freescale MC1322x Platform in a package. (MC1322x is basically one signal SoC with both RF and an ARM7 mcu)

2. Using the Freescale MC1322x Series ARM7 Processor with integrated 802.15.4 by Mariano Alvira:

URL: http://mc1322x.devl.org/

It's a good cand The purpose is to develop Open Source tools (namely Linux related) and hardware for the Freescale MC13224V processor.

This is interesting because it potentially makes it possible for Linux-Zigbee to work on MC1322V. Then we will have 802.15.4/Zigbee sensors running Linux (uClinux?)

3. Open-ZB:

URL: http://www.open-zb.net/

• Implementation of beacon-enabled mode of the IEEE 802.15.4 protocol stack developed in nesC, under the TinyOS operating system v1.1.15 and version 2.0 (new) for the CrossBow MICAz and TelosB motes;
• Implementation of the ZigBee Network Layer (including the IEEE 802.15.4) supporting the Cluster-tree topology, featuring our proposed mechanism to schedule de beacon transmission, the Time Division Beacon Scheduling

It's open source but not Linux. It's a good candidate for implementing 802.15.4 and Zigbee sensors.

(I know company who implement a Zigbee to Ethernet gateway using TinyOS too)

4. Zigbee for PIC18, PIC24 by Microchip

URL: http://www.microchip.com/stellent/idcplg?IdcService=SS_GET_PAGE&nodeId=2680&dDocName=en538049

It's proprietory but provides full source code. It's only for Microchip PIC series microcontrollers.

How to bring up WIFI connection on Embedded Linux

This is how I bring up the wlan connection up on a mini2440 dev board running Linux by using command lines.

The hardware board is a Sumsung ARM9 based dev board running Linux kernel 2.6.29. The wifi card is a TP-Link WN321G usb adapter.

First of all, I have to make sure the driver is compiled into the kernel. The driver for this adapter is "Ralink rt2501/rt73 (USB) support". This is what I chose when I configure the kernel before building it.

When the system boots up, I see something like this:

[root@FriendlyARM /]# usb 1-1: new full speed USB device using s3c2410-ohci and

address 2

usb 1-1: New USB device found, idVendor=148f, idProduct=2573

usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0

usb 1-1: Product: 54M.USB.......

usb 1-1: Manufacturer: Ralink

usb 1-1: configuration #1 chosen from 1 choice

wmaster0 (rt73usb): not using net_device_ops yet

wlan0 (rt73usb): not using net_device_ops yet

This is enough to tell that the card is detected by the system.

Then the following commands are used to bring the connection up:

ifconfig eth0 down --->bring down the ethernet interface

ifconfig wlan0 up --->bring up the wifi interface

iwlist wlan0 scanning | grep ESSID ---> scan the essids of available wifi network (option)

iwconfig wlan0 key "ABCD..." --->this is the key for secured wifi network. I'm using a 128bit WEP, so the key is a 26 digits HEX. For text key, you can use:

iwconfig wlan0 key s:"text passwd"

iwconfig wlan0 essid "Home AP" --->set the wifi network to connect to

iwconfig wlan0 ap auto --->connect to the wifi network

ifconfig wlan0 192.168.1.15 ---->set the IP of the wifi interface. (don't have to do this if you are using dhcp)

After all of these, use ping to see if you are connected or not.

Looks like the WN321G is not very sensitive to signals. So if you are not connected and you are sure you have followed the commands, try a different location where signal is stronger.

These process can be intergrated into the system's startup scripts. It'll be in /etc/init.d/rcS file for mini2440.

Linux device driver

As we know, every device in a linux system is a file. A program read/write to a device through the file. How is this possible? It's made possible by device driver.

Device driver model found in Unix and Linux is a natural partitioning of functionality between application code and hardware or kernel devices related code. It isolates user's application from directly access to critical kernel data structure and hardware devices. It hides the operation details of particular hardwares. And a loadable device driver ease the embedded system development and field updates/upgrades.

Note: When we talk about device drivers, loadable kernel module (LKM), loadable module, and module, we usually refer to the same thing, a loadable kernel device driver module.

Device driver is running in the kernel space. It could be statically compiled into the kernel. This is done by selecting the specific driver when we configure the kernel in the build process. For example, if we want a system to boot from a NFS root system, we have to compile the network-related drivers such as TCP/IP, ethernet driver into the kernel image, so that they are available during system boot from a remote root file system.

In many cases device drivers are loadable modules. They are compiled into seperate binaries files (against the kernel on which it will be running), and can be installed/uninstalled after the kernel has booted. They can be loaded by startup scripts, can be "demand loaded" when needed, or can be loaded manually by user space commands.

The following are the mostly used commands:

lsmod ---> list the modules that are inserted into the kernel

insmod ---> insert a module into a running kernel, it takes the module's file name.

rmmod ---> removes a module from a running kernel

modprobe ---> most powerful tool, load/unload a module, togather with dependent modules, etc.

There are a few files that are working togather with loadable modules. They are:

/etc/modprobe.conf ---> configuration file for modprobe. It enables a system developer to associate devices with device drivers. For simple embedded system, modprobe.conf might be empty or might contains very few lines. The modprobe utility is compiled with a set of default rules that establish the defaults in the absence of a valid modprobe.conf.

../modules.dep ---> in the same location where modules are installed. It contains a list of all the modules that the kernel build system is configured for, along with dependency info for each. It's used by modprobe to decide dependencies.

uClinux flat file format

In order for a OS to execute a program, the OS kernel has to load the program into RAM (or partial of the program for XIP). The kernel loader needs to know how and where to load each segments of the program.

Where does this imformation come from? It comes from the file, which is in a format that the kernel loader supports and understands. When we build a program, the linker will put address location information in the file in the specific format, and these information is read by kernel loader to locate the program into RAM.

uClinux uses a Binary Flat format commonly known as BFLT. It is a relatively simple and lightweight executable format based on the original a.out format. It has seen two major revisions, version 2 and more recently version 4. Version 2 is used by the m68k-coff compilers and is still in use with the ARM elf2flt converter while version 4 is used with the m68k elf2flt converter. Like many open source projects worked on by many individuals, little features have creped into versions without the revision field changing. As a consequence what we detail in each version may not strictly be the case in all circumstances. Both the 2.0.x and 2.4.x kernels’ bFLT loaders in the CVS support both formats. Earlier kernels may need to have binfmt_flat.c and flat.c patched to support version 4, should version 4 binaries report the following message when loading

bFLT:not found.
bad magic/rev(4,need 2)

Each flat binary is preceded by a header of the structure shown below in listing 1. It starts with 4 ASCII bytes, “bFLT” or 0x62, 0x46, 0x4C, 0x54 which identifies the binary as conforming to the flat format. The next field designates the version number of the flat header. As mentioned there are two major versions, version 2 and version 4. Each version differs by the supported flags and the format of the relocations.

The next group of fields in the header specify the starting address of each segment relative to the start of the flat file. Most files start the .text segment at 0x40 (immediately after the end of the header). The data_start, data_end and bss_end fields specify the start or finish of the designated segments. With the absence of text_end and bss_start fields, it is assumed that the text segment comes first, followed immediately by the data segment. While the comments for the flat file header would suggest there is a bss segment somewhere in the flat file, this is not true. bss_end is used to represent the length of the bss segment, thus should be set to data_end + size of bss.



Figure 1 : Flat File Format

struct flat_hdr {
char magic[4];
unsigned long rev; /* version */
unsigned long entry; /* Offset of first executable instruction with text segment from beginning of file */
unsigned long data_start; /* Offset of data segment from beginning of file */
unsigned long data_end; /* Offset of end of data segment from beginning of file */
unsigned long bss_end; /* Offset of end of bss segment from beginning of file */

/* (It is assumed that data_end through bss_end forms the bss segment.) */

unsigned long stack_size; /* Size of stack, in bytes */
unsigned long reloc_start; /* Offset of relocation records from beginning of file */
unsigned long reloc_count; /* Number of relocation records */
unsigned long flags;
unsigned long filler[6]; /* Reserved, set to zero */
};
Listing 1 : The bFLT header structure extracted from flat.h

Following the segments’ start and end pointers comes the stack size field specified in bytes. This is normally set to 4096 by the m68k bFLT converters and can be changed by passing an argument (-s) to the elf2flt / coff2flt utility.

The next two fields specify the details of the relocations. Each relocation is a long (32 bits) with the relocation table following the data segment in the flat binary file. The relocation entries are different per bFLT version.

Version 2 specified a 2 bit type and 30 bit offset per relocation. This causes a headache with endianess problems. The 30 bit relocation is a pointer relative to zero where an absolute address is used. The type indicates whether the absolute address points to .text, .data or .bss.

#define FLAT_RELOC_TYPE_TEXT 0
#define FLAT_RELOC_TYPE_DATA 1
#define FLAT_RELOC_TYPE_BSS 2
struct flat_reloc {
#if defined(__BIG_ENDIAN_BITFIELD) /* bit fields, ugh ! */
unsigned long type : 2;
signed long offset : 30;
#elif defined(__LITTLE_ENDIAN_BITFIELD)
signed long offset : 30;
unsigned long type : 2;
#endif
Listing 2 : Version 2 relocation structures - Not for use in new code.

This enables the flat loader to fix-up absolute addressing at runtime by jumping to the absolute address specified by the relocation entry and adding the loaded base address to the existing address.

Version 4 removed the need of specifying a relocation type. Each relocation entry simply contains a pointer to an absolute address in need of fix-up. As the bFLT loader can determine the length of the .text segment at runtime (data_start - entry) we can use this to determine what segment the relocation is for. If the absolute address before fix-up is less that the text length, we can safety assume the relocation is pointing to the text segment and this add the base address of this segment to the absolute address.

On the other hand if the absolute address before fix-up is greater than the text length, then the absolute address must be pointing to .data or .bss. As .bss always immediately follows the data segment there is no need to have a distinction, we just add the base address of the data segment and subtract the length of the text segment. We subtract the text segment as the absolute address in version 4 is relative to zero and not to the start of the data segment.

Now you could say we may take it one step further. As every absolute address is referenced to zero, we can simply add the base address of the text segment to each address needing fix-up. This would be true if the data segment immediately follows the text segment, but we now have complications of -msep-data where the text segment can be in ROM and the data segment in another location in RAM. Therefore we can no longer assume that the .data+.bss segment and text segment will immediately follow each other.

The last defined field in the header is the flags. This appeared briefly in some version 2 headers (Typically ARM) but was cemented in place with version 4. The flags are defined as follows

#define FLAT_FLAG_RAM 0x0001 /* load program entirely into RAM */
#define FLAT_FLAG_GOTPIC 0x0002 /* program is PIC with GOT */
#define FLAT_FLAG_GZIP 0x0004 /* all but the header is compressed */
Listing 3 : New flags defined in Version 4

Early version 2 binaries saw both the .text and .data segments loaded into RAM regardless. XIP (eXecute in Place) was later introduced allowing programs to execute from ROM with only the data segment being copied into RAM. In version 4, it is now assumed that each binary is loaded from ROM if GOTPIC is true and FLAT_FLAG_GZIP and FLAT_FLAG_RAM is false. A binary can be forced to load into RAM by forcing the FLAT_FLAG_RAM flag.

The FLAT_FLAG_GOTPIC informs the loader that a GOT (Global Offset Table) is present at the start of the data segment. This table includes offsets that need to be relocated at runtime and thus allows XIP to work. (Data is accessed through the GOT, thus relocations need not be made to the .text residing in ROM.) The GOT is terminated with a -1, followed immediately by the data segment.

The FLAT_FLAG_GZIP indicates the binary is compressed with the GZIP algorithm. The header is left untouched, but the .text, .data and relocations are compressed. Some bFLT loaders do not support GZIP and will report an error at loading.

Note: most of the info above is from http://www.beyondlogic.org/uClinux/bflt.htm

What's portmap and it's use

In a uClinux system, there is a program "portmap" in the /bin directory. And in the system start script rc file, sometimes you see a line like "/bin/portmap &". This means portmap will be started as part of the system startup. But what's portmap, and what's it for? In a uClinux system, when should it be included?

portmap is a server that converts RPC program numbers into DARPA protocol port numbers. It must be running in order to make RPC calls.

When an RPC server is started, it will tell portmap what port number it is listening to, and what RPC program numbers it is prepared to serve. When a client make a RPC call, by a program number, it will first contact portmap on the server machine to determine the port number where the RPC packets should be sent to.

In a embedded uClinux system, sometimes we want to mount a NFS file systme in the target board, and NFS service is based on RPC. That why we need to start portmap when we start up the system. (Of course you can start the server manually in a shell).

romfs - ROM file system

What is rom fs:

romfs is a space-efficient, small, READ-ONLY filesystem originally for Linux and used by some Linux based projects. It is a block-based filesystem, that means it makes use of block (or sector) accessible storage driver (like disks, CDs, ROM drives, RAM disk, etc).

romfs itself, although it may sound weird at first, cannot handle ROMs of any kind. It is just a file system, working over a block device, including ROM disk or RAM disk. In order for romfs to work on ROM or RAM, you need drivers to make them appear as disk (ROM disk or RAM disk).
Where romfs is used:
There are at least two main uses where romfs, despite its limitations, is almost perfect.

1. The first is normal Linux systems, where you want some of the files to take the least possible space, and you can't afford or need integrated compression. This is typical in some boot procedures. You can have your initial ram disks in romfs format, and even if you don't use it after the boot, its code is still very small to waste memory on. This is also the single case when compression is easily achievable, as the initial ramdisk loader recognizes gzip compressed images. Currently no romfs code supports compression.

2. The other possible use when you need to use a Linux kernel on a device which has significant amount of read-only memory, and so it makes little sense to have a writable filesystem on it. This is the case with many embedded systems. uClinux is this case.

What the other file system:
There other file systems that are widely used in embedded linux system, such as ext2, ext3, cramfs(RAM based file system), JFFS2, NFS(we can even put root file system on a NFS server), and pseudo file systems such as proc, sysfs etc. They have different advantages and are chosen according to the specific applicaton.

uClinux rc and inittab files

There are two files playing key roles in system startup, rc and inittab. They are referenced by the first user space process "init", which is spawned by the kernel afte completion of the boot process.

/etc/inittab contains system configurations and directives for each run level, as well as directives that apply to all run levels. For a standard Linux system, the entries in inittab will always like this:

id:runlevel:action:process

For example:
l1:1:wait:/etc/rc.d/rc 0 ----> at runlevel 1, wait until /etc/rc.d/rc 0 start and finish
x:5:respawn:/etc/X11/prefdm -nodaemon ----> start xdm at level 5

However, since uClinux has only one runlevel, the inittab usually is very simple. (An example shows there is no "action" field". Am I right?)

The rc file contains all the scripts to run at system startup. This is where different file system are mounted, network interfaces are configured, etc. We can add our own scripts to start specific applications.

It's simple once they are understood. But they can turn to be complex in a real live system I believe.

How to add new user application to uClinux

In order to add your own user application to uClinux-dist, do the following:

1. Create a sub-directory in .../uClinux-dist/user directory, this sub-directory is corresponding to your application.
2. Copy all your source files into the sub-directory, including .c, .h and Makefile. You can originize your files anyway you like within the sub-dir.
3. Edit and modify these 3 files:
uClinux-dist/user/Makefile ---- add make rule for your application
uClinux-dist/config/config.in ---- add configuration for your application
uClinux-dist/config/config.help ---- add configuration help for your app, this help will show up when you click on the "Help" button in "make xconfig" submenu.
4. Build new image (romfs)
make xconfig ---- customize user application settings and select your newly added apps.
make

Once the image.bin is built, downloaded and running in the target, you can find your apps in the /bin directory, along with other existing applications such as ping, web server, ftp, etc.

How to write/modify Makefile is key here, there are more to know about.

Make wifi work on Ubuntu 9.10

My ubuntu was installed and setup, but wireless doesn't work so it's not connected to internet yet.

First of all I tried to make a linksys WUSB11 v1 work. This is the 802.11b usb adapter connected to the mechine and being used by WinXP system. Ubuntu 9.10 default installation has WUSB11 driver installed but unfornately the driver is for v2.6 and up. After trying for a while I decided to buy a new one , hopefully a plug and play one.

I got a TP-Link WN353G PCI card for cheap from pccyber. According to the hardware support document from https://help.ubuntu.com/community/WifiDocs/WirelessCardsSupported, the card should be work out of the box.

No it's not. The bootloader GRUB will stop in the middle and the mechine just freeze. Don't know why. Maybe what I got is WN353G and the document says WN353GD?

There seems to be a native driver for this card, but it's a lot of work to do. So I decided to go the path of using ndiswrapper, which wraps up windows driver for Linux. The following is how I make it work.

1. Download ndiswrapper packages.
Since the mechine is offline, so I download them on another mechine and copy them over using USB disk.
http://packages.ubuntu.com/karmic/misc/ndiswrapper-common
http://packages.ubuntu.com/karmic/misc/ndiswrapper-utils-1.9
http://packages.ubuntu.com/karmic/net/ndisgtk

2. Install the packages in the order
sudo dpkg -i ndiswrapper-common_*.deb
sudo dpkg -i ndiswrapper-utils-*.deb
sudo dpkg -i ndisgtk_*.deb

3. Install windows driver on Ubuntu
After installed ndiswrapper (all 3 packages) , open Windows Wireless Drivers from System->Administration menu.Then click Install New Drivers and select your Windows card driver file to install. The driver file should be *.inf from the disc accompanied with the card. I chose the one for windows2000.

Finally,the installed network card driver will list in the left box.You can click the network manager icon at right-top of your screen and find wireless network.

After the above installation, the system reboots successfully and my wifi AP is autimatically detected. After configuring the wireless network such as security, the system is connected!

Run uClinux on cf5272 board

Now I have build 2 kernels, one from Redhat and the other is from Ubuntu. I'll try both on cf5272.

The lab configuration is simple. cf5272 connects to a linux host through RS232, and connect to the network through ethernet cable. The linux host acts as a stage server to download image, operate the board using minicom.

I made all the files ready on the stage server, download image.bin into the board through RS232, run the OS and monitor the output from minicom. Now the board is alive by responding to commands in minicom.

The image from redhat seems to have less commands than the one from Ubuntu. I must have select fewer when I configure the kernel's applications (Busybox).

By using the image from Ubuntu, I have the mkdir, cd, cat etc command to mount a nfs dir (on my laptop linux) to a target dir under /var, and see my files on the target.

Few things to mount a nfs files system:
1. On nfs server, export the dir by editing /etc/exports.
2. Run "/etc/init.d/nfs" stop to stop nfs service, and re-start it with the new exported dir by running "/et/init.d/nfs start"
3. On client (in this case the cf5272 board), run "mount -t nfs -o nolock :/ /var/"

Build uClinux on Ubuntu 9.10

I have successfully build a uClinux kernel on Redhat Linux. Now I'm trying to setup a Ubuntu and build it again on it.

I have spent the past few days trying to setting up a Ubuntu 9.10 box and finially got it installed and running on a Dell dimension 2400 desktop. (The system will boot to Win XP or Ubuntu by changing the BIOs. For Win XP, turn Ubuntu disk off; for Ubuntu, turn Win XP disk off).

When I install m68k took chain for uClinux by running:
m68k-elf-tools-20031003.sh

I got error: tail: cannot open input `+43' for reading, no such file ...

Some smart people give a solution, by running the following command in sequence:
1. tail -n +43 m68k-elf-tools-20031003.sh > m68k-elf-tools-20031003.tar (this retrieve the tar ball file from within the .sh file)

2. tar xvf m68k-elf-tools-20031003.tar (untar the toolchain to ./usr/local dir. So if you want to install the tool chain to /usr/local, you can run the tar command from root dir)

This is where they are talking about this: http://www.linuxquestions.org/questions/linux-newbie-8/fc-6-gunzip-stdin-invalid-compressed-data-format-violated-535785/

Now install the uClinux distribution ... smooth!

make xconfig, make dep, and make. images are successfully generated!

BTW: looks like a newer version of mk68k tool chain installation doesn't has the above problem.

The make of uClinux

Configuring kernel is to generate .config file as far as I know. The .config will be used by "make" to generate vmLinux, the kernel proper. As this point of time, I'm not quite sure about whether "make xconfig" or the actual "make" prepares the source code tree for compile.

Something interesting from a book:
"Most kernel software modules also read the configuration indirectly via the .config file. During the build process, the .config file is processed into a C header file found in the .../include/linux directory, called autoconf.h. For each entry in the .config file, a corresponding entry is created in autoconf.h. Many kernel source files include this file directly using #include preprocessor directive. This is how the cource files in the kernel source tree reference the kernel configuration."

Run "make" and I get this error after the make runs for a while:


"The file /usr/src/linux/Makefile is missing!
Perhapes your kernel source is broken?"


The "kernel source" is the compiling host's kernel source, or the uClinux's kernel source? uClinux distribution doesn't include all the Linux kernel files? Is it just a patch to the standard kernel?

Anyway, I downloaded the Linux 2.4.8-20 (which is the version of the host) source code and untar it into the /usr/src/ dir. And re-make.

Fine, it's compiling until it's stopped by other errors. (can't remember exactly ...)

This evening in the class, asked the teacher and it turned out that I chose a wrong uClibc! It should be "uC-libc" instead of "uClibc". uC-libc is specific and optimised for Coldfire 68k and the other one is more generic.

Ola, make flys and image.bin is built in .../images dir!

Note: the image.bin file is the uClinux kernel plus root file system, it is the one that's going to be download into cf5272 and run.

Something else:
1. the uClinux distribution (source code) is complete to build the kernel, no dependency on any other source codes such as the source code of the host linux.
2. the tool chain for the specific architecture is complete and no dependency on host's gcc.

Solve the "make: wish: Command not found" error

Since it's a Redhat linux system, I'm looking for a rpm package for tk 8.3. The system has tcl 8.3.5 installed so I'm looking specifically tk 8.3.5 rpm package.

Googling can't find it. The closest rpm I can find is tk 8.3.2. Tried to install it, but fail since it's looking for tck 8.3.2.

Then I tried to build it from source code, failed again. I think if I build from source, I have to build both tcl and tk.

Getting a bit frustrated! Gave up doing it and went to bed.

Getting up this morning, tried different key words searching and found one at: http://www.rpmfind.net/linux/RPM/index.html I have ever had toubles finding some rpm. I'll consider other Linux distributions later.

I've had touble installing the rpm again. The rpm command just freeze!

Searching on internet and solved the problem by "rm -f /var/lib/rpm/__db*".

Finally got "make xconfig" running!

Getting started

I finally started my blog on embedded system development. I'm gonna use this blog to keep a record of whatever I'm experiencing during my embedded system development. I hope this can be a valuable resource for whoever are interested in embedded system learning and development.

If I have a contract, I'll be just working on the contract. If I don't have any on hand, I'm just doing whatever I feel like to do. So the blogs will not be in any specific order, and usually there will be no dependencies from one topic to another.

So I'll start from the thing I'm doing now, to compile a uClinux kernel and run it on a Freescale Codefire dev board uC5272. The spec of the board can be found at: http://www.arcturusnetworks.com/products/uc5272/

The build process is straight forward based on the instruction. But when I configure the kernel as the first step, I got problem.

The make xconfig give me the error:
"... make: wish: Command not found ..."

It turned out that my Linux doesn't have tk 8.3 intalled. Strange! I have to find it and install it.