Tweaking the GoboLinux LiveCD: a tutorial

This is a quick guide to the internals of how the GoboLinux Live CD is constructed. It will get you up to speed with the tools that build the CD, allowing you to make customizations. With this we hope to lower the entry barrier to potential contributors to the distro's development.

Prerequisites

This tutorial assumes you're running a GoboLinux system, with the most recent version of the Scripts package installed. Your kernel needs to support UnionFS, SquashFS and CramFS. You also need the SquashFS-Tools installed. All of this is available from the GoboLinux repositories.

This tutorial assumes all steps are performed with superuser privileges (become superuser with su or use sudo). This is because the tools need to create device nodes, unpack files keeping privileges and suid bits and perform mounts using UnionFS. As usual, care must be taken when performing tasks as superuser, when running pre-release software downloaded through SVN and especially, when doing both, as it's the case here. Proceed at your own risk.

The tools

To get started, you need to install the BuildLiveCD package. You can install it by typing:

 
Compile buildlivecd

A number of other GoboLinux tools and packages will be involved in the process of building the CD, but this is the only one you need to install in your /Programs directory. We'll get to them in detail one by one, but here's an overview:

InitRDScripts - this is the "Scripts" package for the mini-GoboLinux environment in which the CD kernel is loaded
LiveCD - scripts that are specific to the Live CD environment, having to do mostly with its initialization.
ConfigTools - some scripts that tweak configuration files to match the system where they're running.
Installer - the actual installation program, that copies the contents of the CD to the hard disc
AbsTK - Abstract Tool Kit: the interface abstraction library that allows the Installer to have both text-mode and graphic interfaces.

The CD mastering environment

Once you have BuildLiveCD installed in your system, you can set up the CD mastering environment. A single command creates the necessary directories and downloads all needed packages: UpdateEnvironment. Run it in an empty directory; in our examples we'll use /Depot/LiveCD.

 
mkdir /Depot/LiveCD 
cd /Depot/LiveCD 
UpdateEnvironment

Note that this will download hundreds of megabytes worth of packages, so this may take a while.

For GoboLinux devs: This script downloads some files from SVN and stores a local copy of the SVN repository under LocalSVN/. To ease development, you may want to have a copy with write access enabled, instead of the default anonymous SVN copy. To do this, pass your Savannah username as an additional parameter to UpdateEnvironment.

 
UpdateEnvironment hisham

If your username in the local machine is not the same as your Savannah username, you can pass it as a second parameter (since you're running as superuser, the script will "sudo" to your user account when running svn).

Building the CD

The whole process of building the CD is wrapped by the CompressAndBuildISO script. In your first run, you can simply call it with no parameters and it will go through the whole process:

 
CompressAndBuildISO

The script is composed of a few stages -- when you make a modification that affects only one of the later stages, you can pass a parameter to have it skip the earlier ones, in order to save time. The stages are:

rolayer - Constructs the directory tree with the contents of the LiveCD: uncompresses packages building a /Programs tree, files into /Files, and generates a /System directory with the necessary symbolic links and system files.
squashfs - Compresses this newly-constructed system into several SquashFS images: one for /Files, a few ones for sections of the big /Programs tree, and one for the rest (/System, /Mount, /Users...). This is done so that modifications on part of the filesystem do not trigger a full regeneration of the entire system's "squashed image" (a process which takes several minutes even on a 2+GHz machine).
initrd - Because the actual system is stored in the CD in a compressed format, its contents cannot be directly accessed by the disc bootloader. The bootloader launches instead an InitRD ("initial RAM-disk") image, a mini-GoboLinux system that serves the sole purpose of hosting the kernel, mounting the SquashFS system, and booting into it. This step of CompressAndBuildISO creates the CramFS image of this mini-GoboLinux to be used as the InitRD.
iso - In the final step, the ISO 9960 file system is generated, containing the bootloader, the kernel, the InitRD image and the SquashFS files.

A walkthrough of the Live CD boot process

The CD bootloader

To be written

The InitRD environment

To be completed

The InitRD environment is a mini-GoboLinux system created by the MakeInitRDTree script, which was called when you run CompressAndBuildISO. You will find it at the Output/InitRD directory.

Do not modify Output/InitRD directly: its contents are rewritten when CompressAndBuildISO runs through the initrd step. To tweak the InitRD scripts, modify instead the local SVN copy of InitRDScripts, located at LocalSVN/tools/InitRDScripts.

Currently, this part of the build environment is not very friendly to customized versions. If you want to keep a custom version of the InitRD, you need to replace LocalSVN/InitRD with your own copy after running UpdateEnvironment.

For GoboLinux devs: To have your changes done permanently, there are a few different places where they may need to be applied:

Changes to /Programs/InitRDScripts should be comitted to the GoboLinux SVN. If your LocalSVN/ copy was created with write access, it's only a matter of running svn commit.
Other entries in /Programs are fetched from the initrd-packages/ repository (see the MakeInitRDTree script for details).
The directory structure and other configuration files are generated by MakeInitRDTree, so if you change that, modify it accordingly so that a clean rebuild creates the files and structure as you intend it.

At the end of the startGoboLinux script of the InitRDScripts package, the SquashFS images are mounted, the pivot operation to make it the root directory is performed and init is called to proceed with the boot process.

Inside SquashFS

From there on, we're inside a GoboLinux system very much like the one that's installed in the hard disk. In fact, the same packages are used. When init runs, it calls the BootDriver from the BootScripts package. The BootDriver detects it is running from the Live CD, so instead of running its usual bootscripts, it calls the StartLiveCD script from the LiveCD package. That script is responsible for the rest of the initialization.

The SquashFS structure

The main LiveCD environment (i.e., the directory tree you see once the LiveCD booted) is stored as a series of SquashFS images. These images are built from two layers: one read-only layer that is the actual filesystem, (Programs, Files, System, etc.), and a read-write layer that sits on top for debugging purposes, containing assorted files that you may have created while logged in using ChrootEnvironment.

The main, first layer is generated at Output/ROLayer. This is built out of packages listed in Archives/Packages that are combined to build /Programs, additional files under Archives/Ext that are unpacked to their proper locations, plus a series of files and directories that are generated on-the-fly by the BuildRoot script. BuildRoot is called by CompressAndBuildISO during the rolayer step.

Keep in mind that when you make modifications that alter the LiveCD environment itself, you need to regenerate the SquashFS image. To do this, pass an additional parameter to CompressAndBuildISO:

 
CompressAndBuildISO squashfs

For GoboLinux devs: To have your changes done permanently, you need to consult your changes in the Output/RWLayer and apply them to the relevant recipes which will generate new packages.

The package set for the CD is currently at the livecd-packages repository (see UpdateEnvironment for details). When modifying the package set, you also need to update the /Programs/BuildLiveCD/Current/Data/Packages-List-* files.

After modifying packages, a full rebuild starting from rolayer is needed to keep your Output directory in sync with the package repository:

 
CompressAndBuildISO rolayer