Observations, Musings & Gadgets

...things I find interesting

Archives for October 2017

Creating your own deb files

Our automated Ubuntu installer needs to be able to handle what we call "unpackaged" software. By this we mean, that obscure piece of software that only exists as a tarfile of compiled binaries.

In the past, our first incarnation of salt controlled installs used salt's cmd.run to perform an rsync from a file server to the client machine. This was far from ideal, as even if the package was installed, the rsync would run to compare the two filesystems, which was both slow and uneccesary.

Then we discovered salt's native archive module. So now we use tar.gz files to push out these unpackaged bits of software.

It goes something like this. Touch a file on the target machine, install the software then use find / -newer marker.file >> filelist.txt. Then you weed out all the obviously unneeded files from your list of files (ie /sys /var/cache etc) and use that list to build a tar file. The we add other actions to our salt state to set additional paths (by adding files to /etc/profile.d) and other actions (a desktop icon, or symlink to /usr/bin perhaps).

We put the resulting tar file on our file server and use salt's archive module to extract, which has an unless clause to prevent unarchiving if it's already in place. You do have to create a hash file though.

You with me so far?

Last week, an install failed - java wasn't installed, which broke a few other package installs. Turns out the third party PPA we use to install Oracle Java hadn't updated in a while and the tar file it referenced on Oracle's servers had been removed. So I set about "repairing" the deb file. Having never worked with deb files, I was pleasantly surprised at how easy it was.

dpkg-deb -R original.deb somedir

This command will unpack the deb file into the directory specified.

somedir-
       |-DEBIAN
       |-usr
       |-bin

Inside the DEBIAN directory are the control files. The most important one (and infact the only required one) is the control file. We will look at this later.

For this particular deb file, the relevant bits were in the postinst script. This was actually the part of the process that grabbed the tar file from Oracle's site and installed it. I changed the script to pull a local copy of the tar file.

To rebuild the deb package is just as easy.

dpkg-deb -b somedir new.deb

Obviously, if you are hacking an existing deb file like this, you should probably change the name of the package to prevent possible conflicts. In this case, we kept the same version, with the thinking that, when the PPA updated, the newer version would overwrite the existing (hacked) version.

That was relatively painless. So what about creating a package from scratch? It's actually quite simple.

Armed with the information in the deb-control man page, I set about trying to create one.

Here's an unpackaged piece of software. Clojure is a general purpose programming language, that is supplied in a zip file. We typically install unpackaged software into /opt. So how do we go about that?

Firstly, we'll create the structure we need. Remember we want the software to be installed in /opt/clojure.

cd /tmp
mkdir clojure
mkdir clojure/{DEBIAN,opt}

By the way, if you haven't seen the curly braces used this way before, it's a sort of short hand. It means repeat this command, substituting each item in the list. So mkdir clojure/{DEBIAN,opt} is the same as typing..

mkdir clojure/DEBIAN
mkdir clojure/opt

If I wanted to be really lazy, I could have also typed mkdir -p clojure/{DEBIAN,opt} which would have made all three directories.

Now, where was I? That's right, creating the structure for the deb file. If we ignore the DEBIAN directory for the moment, all the files inside the deb will be copied into place on the target computer, relative to /. By placing our files in clojure/opt they will appear in /opt after we install our deb file.

The zipfile for clojure, when unzipped, creates a folder called clojure-1.8.0. We'll unpack that in the right place and rename it.

cd clojure/opt
unzip /some/path/to/clojure-1.8.0.zip
mv clojure-1.8.0 clojure
cd ../..

If you wanted to include other files, to be placed in other parts of the filesystem, just create the neccessary folders inside the clojure directory.

The only other file that is required is the control file, clojure/DEBIAN/control

This file specifies information about the package. There are four required items; Package, Version, Maintainer and Description.

There are also some optional items. The dpkg-deb program complains if you don't set the Architecture, so we will set that as well.

Package: clojure
Version: 1.8.0
Architecture: all
Maintainer: thats me <me@somewhere.com>
Description: Clojure is a robust, practical, and fast programming language with a set of useful features that together form a simple, coherent, and powerful tool.

That should be sufficient to make the new deb file.

dpkg-deb -b clojure clojure-1.8.0.deb

That's it. Deb package successfully built.

For anything more complicated than that, there are also scripts that fire before and after the install of the files, called preinst and postinst. If you do use these scripts, you should seriously consider implementing the prerm and postrm scripts to undo the actions of those scripts in case the package is uninstalled.

These scripts give you a lot of scope. If you want to find out how mainstream packages work, it's really easy to unpack existing deb files to see how their maintainers have made use of these scripts.

tagged as ubuntu, salt, deb

Archives