ChrootCompile - an overview

ChrootCompile is a wrapper that creates an isolated execution environment for running Compile. This isolated environment (a "chroot" jail) serves three main purposes:

  1. it allows you to build packages using different tools than you currently have installed in your live system;
  2. it protects your system from any problems during any step of the compilation process;
  3. and it helps you ensure that the dependency information of your recipe is correct.

When you run a recipe through ChrootCompile, it will read its dependency tree, and based on that, it will construct a directory containing a minimal, yet fully-working GoboLinux system, containing the basic packages required by the Compile toolchain (such as a shell, a compiler and basic Unix commands) and the dependencies listed by your recipe -- nothing else. Then, it will run your recipe through Compile in this isolated system, producing, if everything goes well, a binary package by the end of the process.

This way, you can build packages using, for example, different versions of GCC and Glibc than your system uses. In its default configuration, ChrootCompile fetches the basic set of packages of its toolchain from a fixed repository; ensuring that the basic ABI of packages built with it remains consistent. This step is performed by a separate script, SetupChrootEnv.

SetupChrootEnv creates an environment in which ChrootCompile can run (if executed in an empty directory, ChrootCompile calls SetupChrootEnv automatically the first time). This environment will contain packages necessary for creating chroot jails. On each execution of ChrootCompile, a new chroot jail is created. This ensures that the effects of a previous compilation do not affect the next compilation. The main goal of ChrootCompile is to obtain reproduceable compilations: no matter how different the configuration of two machines may be, when running ChrootCompile on the same recipe, they should ideally produce the same results.

An important aspect for obtaining this reproducibility is that a compilation using ChrootCompile does not perform network access. All files necessary for a successful compilation have to be provided beforehand, either by the SetupChrootEnv script (base toolchain), or by the user (recipes and additional dependency packages). Source tarballs are downloaded before entering the chroot. This way, one is forced to verify explicitly which packages are being used to run the compilation, instead of the best-effort results provided by FindPackage in usual Compile runs. Also, changes in remote systems (files in servers, CVS updates, etc.) have no effect during the compilation.

Using ChrootCompile has the advantage not only of controlling what goes in when building, but also controlling what stays out. Occasionally, a compilation succeeds in a live system because of the existence of a package that is not listed in the dependency file, or because of a tweak fixing a dependency package that was forgotten about the recipe builder. This recipe is then sent to the store and it fails to work in other systems that lack the dependency or the tweak (which should have resulted in a fixed recipe for the dependency package). With ChrootCompile, compilations do not succeed accidentally. A recipe that works with ChrootCompile has much greater chances on working in other systems with regular Compile -- and is certain to work with ChrootCompile using the same set of dependency packages.

ChrootCompile has been crucial in the development of GoboLinux 013. Virtually all packages were compiled using it (with the notable exceptions of, Firefox and Thunderbird, which are packed from their official binary builds). This allowed the development team to reproduce bugs in recipes more easily, to debug recipes without the risk of running them in a live system, and most prominently, to painstakingly fix the dependency files of lots of recipes.

ChrootCompile can be installed in GoboLinux systems by typing:

InstallPackage ChrootCompile

© 2002-2021
gobo AT gobolinux DOT org