apt-build

Optimize Debian packages for your system

Julien Reveret

<shaddai@debian-fr.org>

Julien Danjou <acid@debian.org> (english translation)

Introduction

You are new in the Linux world, or an old warrior. Anyway, you don't know well the way that Debian package system works, and you wonder how to install a program in a way where upgrades, dependancies are handled correctly by the system. Possible methods to install a software are the following: the first one is by downloading the tarball, uncompressing it, running configure scripts, and then launching make install. This is an easy task for every Linux user, but this is not adpated for the Debian package management system. The second solution is to use stow, to have a better way to handle upgrades. But stow does not implement dependancies management. The ultimate solution is to use apt-build to recompile a software already packaged for Debian.

Try it and you will adopt it

On the first day, it created the repository

To install apt-build, do as usual:

[shaddai@patator]~ $ sudo apt-get install apt-build 
    

Once installed, the post installation script will ask you several questions, about the build directory, the options to give to gcc, the processor architecture. This information will be written in the /etc/apt/apt-build.conf file. If you made a mistake, you will be able to go back and fix it. Here is an example:

[shaddai@patator]~ $ cat /etc/apt/apt-build.conf
build-dir = /var/cache/apt-build/build
repository-dir = /var/cache/apt-build/repository
Olevel = -O3
march = -march=pentium2
mcpu = -mcpu=pentium2
options =
[shaddai@patator]~ $
    

About the build-dir and repository-dir parameters, be sure to enter a directory that is on a partition with a lot of free space. Most of the packages (all ?) are built with a level 2 optimization, so if you choose level 3 and indicate the architecture your processor is part of, we can expect a performance increase. Now, Gentoo fans will ask me "what about flags ?" ; sorry for us, but there are not. By flags, Gentoo users mean libraries with which software are linked to. Here, it is out of the question for a simple user to change package dependancies. Indeed, if you do this, you could make mistakes that don't exist with the original package. No need to fill a bug report in this case, you are not using the official Debian package anymore, but yours. Finally, you will need an additional line in your /etc/apt/sources.list, if it does not already exist:

deb-src ftp://debian_mirror/ [stable|testing|unstable] main contrib
    

Run apt-get update to resynchronize your package index files. Now you are ready to use apt-build !

And now, what do we do?

I'll begin to detail the main commands of apt-build. In fact, like apt-get, there are some keywords, like update, upgrade, install, source, clean-build.

First, the update command ; it updates the index packages list, files belonging to /var/lib/apt/lists. The upgrade command, simply upgrades all package. Let's deal with install with an example:

[shaddai@patator]~ $ sudo apt-build install memstat
-----> Installing build dependencies (for memstat) <-----
Reading Package Lists... Done
Building Dependency Tree... Done
0 packages upgraded, 0 newly installed, 0 to remove and 0  not upgraded.
-----> Downloading memstat source (memstat) <-----
Reading Package Lists... Done
Building Dependency Tree... Done
Need to get 22.4kB of source archives.
Get:1 ftp://127.0.0.1 1.0.4/main memstat 0.4-1 (dsc) [482B]
Get:2 ftp://127.0.0.1 1.0.4/main memstat 0.4-1 (tar) [21.9kB]
Fetched 22.4kB in 0s (322kB/s)
dpkg-source: extracting memstat in memstat-0.4
-----> Building memstat <-----
...
...
...
dpkg-genchanges: binary-only upload - not including any source code
dpkg-buildpackage: binary only upload (no source included)
-----> Moving packages to repository <-----
-----> Updating repository <-----
Using: -O3 -mcpu=pentium2 -march=pentium2
...
...
Reading Package Lists... Done
Building Dependency Tree... Done
Reading Package Lists... Done
Building Dependency Tree... Done
The following NEW packages will be installed:
  memstat
    

We can see that apt-build makes several steps: installation of the necessary packages for the compilation, download of sources, then compilation and installation of the built package. When you want to repackage yourself, dpkg-buildpackage will not install dependancies, as apt-build does, and the repackaging can become hard and can need skills that we do not have. Until now, we have seen a simple recompilation, in order to optimize software, but if we want to change sources, modify the rules file or other files in the debian/ directory, apt-build can be useful, and bring to you the view of the core of a package. However, I recommend reading the maintainer guide.

What else?

Let's take a look on that command: source. It does nothing magical, it downloads the source package and the diff file containing the debian/ directory in which you will find necessary files to build the package (rules, control, changelog files, etc), it puts them in the build directory, then uncompresses it in the source package and finally patches it with the diff file. Example:

[shaddai@patator]~ $ sudo apt-build source ethereal
-----> Downloading ethereal source (ethereal) <-----
Reading Package Lists... Done
Building Dependency Tree... Done
Need to get 6170kB of source archives.
Get:1 ftp://127.0.0.1 1.0.4/main ethereal 0.10.4-2.1 (dsc) [821B]
Get:2 ftp://127.0.0.1 1.0.4/main ethereal 0.10.4-2.1 (tar) [6154kB]
Get:3 ftp://127.0.0.1 1.0.4/main ethereal 0.10.4-2.1 (diff) [14.6kB]
Fetched 6170kB in 1s (3533kB/s)
dpkg-source: extracting ethereal in ethereal-0.10.4
   

We can notice that a third file is downloaded, the package description. You now have the software sources, with the debian/ directory and the files needed for the package building, which you can modify as you want, in order to add or remove options. Once the modifications are made, you can build and install the package using the command apt-build install modified_package. But, like said before, you are using your own package, your modifications can create new errors. Modifiying package can be a dangerous game if you don't know what you do.

The main options have been seen, but there are still some small options which make all the difference, like: clean-build, which is to apt-build what the clean command is to apt-get, it cleans the build directory, but not the repository. You can delete a package with the remove command, or obtain information on sources with the info one.

Finally, take a look on the apt-build manpage which contains some interesting options. But I kept the best for the end.

With if we can redo the world, with apt-build too!

The devil is among us!

The last command which I will talk about is world, it should remember something to people loving the little red fellow. Indeed, on BSD systems, you can recompile all the ports, you just have to change directory to /usr/ports and type make world. This concept has been used by the apt-build author, and this is what happens when you want to rebuild the world:

[shaddai@patator]~ $ sudo apt-build world
-----> Rebuilding the world ! <-----
You should read README.Debian first
[shaddai@patator]~ $
   

The reading of this famous README seems to be mandatory, and we learn that we have to build a package list of what we want to rebuild and install instead of actual packages, list from which we should remove sensible packages like gcc, libc6, or vital ones. Let's build this list:

root@patator:~ # dpkg --get-selections | awk '{if ($2 == "install") print $1}' > /etc/apt/apt-build.list
   

Once this list edited, it will be possible to do apt-build world to recompile and optimize all our system.

And now, this is dramatical !

You used apt-build in order to build your own packages, and when you upgrade with apt-get dist-upgrade, one of your beautiful package is replaced by an official one, what a pitty. You could have avoided this by modifying the pinning of your built packages with apt-build. To put a higher priority for packages built by apt-build, you will have to edit the /etc/apt/preferences file like this:

   Package: *
   Pin: release o=apt-build
   Pin-Priority: 990 
   

Thus, you won't be worry by a dist-upgrade which would replace your package by the official ones. To check repositories priorities, use the command apt-cache policy, which will give a result like this:

[shaddai@kavado]~ $ apt-cache policy 
Package Files:
 100 /var/lib/dpkg/status
     release a=now
 500 http://ftp2.de.debian.org sid/non-free Packages
     release o=Debian,a=unstable,l=Debian,c=non-free
     origin ftp2.de.debian.org
 500 http://ftp2.de.debian.org sid/contrib Packages
     release o=Debian,a=unstable,l=Debian,c=contrib
     origin ftp2.de.debian.org
 500 http://ftp2.de.debian.org sid/main Packages
     release o=Debian,a=unstable,l=Debian,c=main
     origin ftp2.de.debian.org
 990 file: apt-build/main Packages
     release o=apt-build,a=apt-build,l=apt-build,c=main
Pinned Packages:
   

This solution, use package pinning, is clean. However, be careful to follow updates. Indeed, if we decide to rebuild a lib of which several programs depend on, and if we don't upgrade it, it can block the upgrade of packages. So, take always a look on the updates of the packages you care about.

What time is it? 15 past midnight ?

It's time to quit, I hope I gave you envy to use apt-build, which is a useful and interesting tool. I also want to warn you, the author carefully invites you not to recompile gcc or others. Neither me nor him are responsible for your silliness. So if you kill your Debian, do not write to me (except if you want to make me laugh). Note also that apt-build does not invent new packages, and if the software you want to install is not packaged for Debian, apt-build won't be able to download sources from a Debian mirror and compile them (you will have to ask for packaging and wait until it is accepted).

Changelog

17/03/2005
Julien Reveret

30/12/2004
Julien Reveret

27/10/2004
Julien Reveret

Copyright 2004 par Julien Reveret
Ce document est protégé par la GNU Free Documentation Licence version 1.1, ou toute future version publiée par la Free Software Foundation