Creating your own Signed APT Repository and Debian Packages

We create a lot of our own debian packages at Aframe where I work, and until recently have been keeping them in a flat repository without properly signing any of our packages. However, since there’s a possibility some of those packages may be released publicly (since Opscode may not be providing debian lenny packages for Chef anymore), I decided that it was high time to properly organise the repository and sign all our packages plus the repository itself.

As anyone who has tried this will no doubt have found, there is a large amount of conflicting information out there as to how exactly this can be achieved. To ease the burdens of my fellow sysadmins, I thought I’d gather all the necessary info together into one easy post.

The first thing we’re going to need to do is to generate a GPG key to sign our repository and all of our packages with. If you’ve already done this, please skip this section. If not, you can follow these simple steps!

Creating a GPG Key for Signing Stuff

  • Run the following command: gpg --gen-key
  • Select Option 5 to generate an RSA Key
  • Choose your keysize from the options given (doesn’t really matter for our purposes
  • Enter your name and email address when prompted

There, we now have a GPG key we’re going to use to sign everything. The next stage is to produce a ASCII formatted public key file which we can distribute to people so they’ll be able to tell apt to trust our repository.

Creating a ASCII Formatted Public Key File

By default, the gpg utility’s export function produces a binary formatted output of our public key. If you’re planning to distribute your public key via the web, this isn’t very handy so we’re going to use the gpg utility’s --armor option to produce an ASCII formatted version.

You’ll want to substitute the email address of the key you’re exporting, and the output filename as appropriate in the following command.

  • gpg --armor --export --output

Save this keyfile somewhere, we’ll be making it available over the web for people to add to their apt keychains – this is what’ll say that our repository is trusted for package installs.

Signing Some .deb Packages

I’m going to assume that you already know how to create .deb packages, by way of keeping this blogpost short. This section will simply cover signing a package you’re creating and resigning an existing package.

The good news is that if you’ve already generated a GPG key as detailed above, your packages will be automatically signed as long as the name and email address in your package’s changelog file are the same as that of the GPG key you created. This means that simply running dpkg-buildpackage will now give you signed packages. If you’re unsure of how to build debian packages at all, there’s plenty of information out there on doing this. I might write a blog post on that soon 🙂

If you want to resign an existing debian package, for example if you’re setting up your own backport of a package (as with my usecase, backporting Chef 0.9.16 into debian), then this is very easy too if you already have a GPG key set up. We use a tool called dpkg-sig.

Install the dpkg-sig tool by running the command

  • apt-get install dpkg-sig

Then run the following command to sign the package, substituting the name of the deb package as appropriate.

  • dpkg-sig --sign builder mypackage_0.1.2_amd64.deb

The “builder” name mentioned is a debian convention, indicating that the builder of the package signed it. The GPG key used will be the one you set up above, providing you’re running the command as the same user you set up the key with.

Creating a Signed APT Repository

OK, so you’ve got a bunch of signed packages, but now you need some way to make them available to the world. Preferably one which doesn’t throw up authentication warnings every time they try to install from your repository. What you need is a signed repository!

The first step to creating your repository is to create a base directory for it to live in. Doesn’t matter where, any directory will do. Now inside that directory, create a folder called conf.

Inside your conf folder, you’re going to create a text file called distributions. Below is the one I’ve used for my repository:

Label: apt repository
Codename: lenny
Architectures: amd64 source
Components: main
Description: Aframe debian package repo
SignWith: yes
Pull: lenny

Some of the above will be self explanatory, for example if you’re not packaging for debian lenny, you’d replace all occurrences of lenny with squeeze, for example. Additionally, if you’re going to package for more than one distribution, you’ll want to copy the above section for each distro. It all stays in the same file, you just change the “lenny” parts as applicable.

Also, since I’m only creating 64-bit packages, I’ve said that my repository will only contain the amd64 and source architectures. If you’re packaging for i386 or other architectures, remember to add them!

The important line in the above example is the one that says SignWith: yes. That line states that we want to sign the repository with our GPG key. Again, if you’re running commands as the same user you used to create your GPG key, this should work fine.

Now that we have the descriptions file all ready to go, we’re going to use a tool called reprepro to add our packages to the repository. You can install reprepro by running the following command:

  • apt-get install reprepro

The nice thing about this tool is that it will create all the structure it needs inside the repository automatically so you don’t need to worry about it. Here’s an example of how to add a package to our repository. PLEASE NOTE you have to run the below command from your repository directory, ie the directory containing the conf folder.

  • reprepro --ask-passphrase -Vb . includedeb lenny /home/joncowie/libjavascript-minifier-xs-perl_0.06-1_amd64.deb

So what does this command mean? Well, the --ask-passphrase part tells it to prompt you for the passphrase for your GPG Key. You did set one, right? The -Vb . includedeb lenny part tells the command to be verbose, and set the base directory for the command as the current directory. It then says we’re going to be passing the command a deb file (that’s the includedeb part) and then says that we’re going to be adding it to the lenny distribution in our repository. The last part is the absolute path to the .deb package we’re adding.

When you run this command, you’ll see a bunch of output telling you that various folders have been created, and once you’ve entered the password for your GPG key your package will be tucked away nicely in your properly structured, properly signed debian repository. Running the above command on a further package or packages will add the new package(s) into the existing structure. You’ll probably now notice that your repository directory is structured much more like the official debian repositories – that’s because it’s now structured “The Debian Way”.

Making your Repository Available to the World

The final section in this blog post is how to make your wonderful new repository available to the rest of the world. This comes in two parts. The first is making your repository directory available over the web. I’m going to assume if you’re creating packages you can probably do this part by yourself, so I’m going to skip over it 😉

The second part is making our public GPG key available to the world so they can mark our repository as trusted. I’d suggest keeping the public GPG key we created above in the root of your repository, to make it easy for people to find. Just make sure you only store the *public* part of the key there. That’s the part we created using the gpg --armor --export command.

I’d also recommend publishing a simple command for people to download your public key and import it into their apt keychain in one step – this makes it nice and easy for them to get up and running with your repo. This command is as follows (change the URL to your key as appropriate):

wget -O - | sudo apt-key add -

Once your users have run the above command, they can add your nicely-formatted repo to their /etc/apt/sources.list file by adding the following line (change URL and distribution as appropriate):

deb lenny main

Then they just run apt-get update and they’re all ready to use your repository containing all of your signed packages – and not an authentication warning in sight.

Produce a members report for all your Mailman lists

I recently had cause to produce a report on the membership of all our Mailman mailing lists, so rather than doing it manually I knocked together the following handy bash script…change mailman location and output file as desired 🙂

CURRMONTH=`date +%m-%Y`
LISTS=`/usr/local/mailman/bin/list_lists  | awk '{print $1}' | grep -v [!0-9]`
echo "Mailman Report for ${CURRMONTH}" > ${OUTPUTFILE}
echo >> ${OUTPUTFILE}
for x in ${LISTS}
        echo "Members of List ${x}:" >> ${OUTPUTFILE}
        LIST_MEMBERS=`/usr/local/mailman/bin/list_members ${x}`
        for mems in ${LIST_MEMBERS}
                echo ${mems} >> ${OUTPUTFILE}
echo >> ${OUTPUTFILE}
/bin/mail -s "Mailman_Report_for_${CURRMONTH}" -c < ${OUTPUTFILE}

Shared Network Storage with iSCSI and OCFS2

So we got a bunch of new hardware at work recently to build a crunch farm for all our heavyweight data processing. Part of that system is two very beefy servers which share a SAN (This one for those interested) for the majority of their disk storage. The SAN uses iSCSI, which was fairly straightforward to set up (I’ll document it here anyway) so I got that all set up and then made a nice big ext3 partition for the servers to share. All so far so good, the servers were talking to the SAN, could see the partition, read and write to it etc. The only problem seemed to be that when one server changed a file, the other server wouldn’t pick up the change until the partition had been re-mounted. What I hadn’t accounted for was that ext3 doesn’t expect multiple machines to share the same block device so it wasn’t synching changes.

I knew that filesystems designed for exactly this sort of sharing were available but hadn’t done much with them but after investigating for a bit, it seemed like the Oracle Clustered File System (linky) was the best option as it was already supported by the Linux kernel and was pretty mature code. The main problem I had in setting all of this up was that the available documentation was very much geared towards people who already had in-depth experience of OCFS whereas I’d never used it before. Hence this blog post, which details setting up iSCSI and then configuring both servers to talk to the same OCFS partition. The instructions are written for Ubuntu Server, but will work on any distro which used apt. Packages are also available for rpm distros, the only instructions you need to change are the package fetching ones.

Setting up iSCSI

* Install Open-iSCSI

apt-get install open-iscsi

* Edit the Open-iSCSI configuration file

The default configuration file could be located at /etc/openiscsi/iscsid.conf or ~/.iscsid.conf. Open the file and set the parameters as required by your iSCSI device. I’ve included the (mostly default) options I used for reference

node.startup = automatic node.session.timeo.replacement_timeout = 120 node.conn[0].timeo.login_timeout = 15 node.conn[0].timeo.logout_timeout = 15 node.conn[0].timeo.noop_out_interval = 10 node.conn[0].timeo.noop_out_timeout = 15 node.session.iscsi.InitialR2T = No node.session.iscsi.ImmediateData = Yes node.session.iscsi.FirstBurstLength = 262144 node.session.iscsi.MaxBurstLength = 16776192 node.conn[0].iscsi.MaxRecvDataSegmentLength = 65536

* Save and close the file. Restart the open-iscsi service:

/etc/init.d/open-iscsi restart

Now you need to run a discovery against the iscsi target host which basically finds all the iSCSI targets the SAN can give us:

iscsiadm -m discovery -t sendtargets -p ISCSI-SERVER-IP-ADDRESS

Finally restart the service again:

/etc/init.d/open-iscsi restart

Now you should see an additional drive on the system such as /dev/sdc. Look in the /var/log/messages file to find out device name:

Next, you need to use fdisk to create a blank partition on the device. This is pretty well documented so I’ll skip these steps, other than to say that I’ll assume the device was called /dev/sdc, and the new blank partition is called /dev/sdc1 for the remainder of this post. So now we’re talking to our iSCSI device and we’ve got a blank partition all ready to format as an OCFS drive. Next, how exactly we do that!

To be continued…

Creating DMG Files Without MacOS X

I’ve put together a script for creating DMG files without using OS X…it requires Linux, I’ve tested it on Kubuntu 7.10 but it should work on anything recent. The process will also be Wiki’d, but in the meantime, instructions are below for the curious!

Run the following commands:

# This gets and builds a patched version of Apple's diskdev_cmds package which will work on Linux
tar xzf diskdev_cmds-332.14.tar.gz
bunzip2 -c diskdev_cmds-332.14.patch.bz2 | patch -p0
cd diskdev_cmds-332.14
make -f Makefile.lnx

# Create symlinks to the mkfs and fsck commands for HFS+
sudo cp newfs_hfs.tproj/newfs_hfs /sbin/mkfs.hfsplus
sudo cp fsck_hfs.tproj/fsck_hfs /sbin/fsck.hfsplus

# Get and enable the hfsplus kernel module
sudo apt-get install hfsplus
sudo modprobe hfsplus

Now that's done, you can use the following handy bash script (must be run as root) I've written to create a DMG file which contains the contents of a directory you specify on the command line.


# DMG Creation Script
# Usage: makedmg <imagename> <imagetitle> <imagesize (MB)> <contentdir>
# imagename: The output file name of the image, ie foo.dmg
# imagetitle: The title of the DMG File as displayed in OS X
# imagesize: The size of the DMG you're creating in MB (Blame Linux for the fixed size limitation!!)
# contentdir: The directory containing the content you want the DMG file to contain
# Example: makedmg foo.dmg "Script Test" 50 /home/jon/work/scripts/content
# Author: Jon Cowie
# Creation Date: 02/04/2008

if [ ! $# == 4 ]; then
	echo "Usage: makedmg <imagename> <imagetitle> <imagesize (MB)> <contentdir>"

	if [ ${USER} != "root" ]; then
		echo "makedmg must be run as root!"
		echo "Creating DMG File..."
		dd if=/dev/zero of=${OUTPUT} bs=1M count=$FILESIZE
		mkfs.hfsplus -v "${TITLE}" ${OUTPUT}

		echo "Mounting DMG File..."
		mkdir -p ${TMPDIR}
		mount -t hfsplus -o loop ${OUTPUT} ${TMPDIR} 

		echo "Copying content to DMG File..."

		echo "Unmounting DMG File..."
		umount ${TMPDIR}
		rm -rf ${TMPDIR}

		echo "All Done!"

Hope it’s useful!

Jings, Crivvens and Help M’Boab!

Yon chils at Wikipedia hev pit up a Scots vershun o’ the site…’er isnae ‘at mich on ee noo but wit is there is bonnie ‘an guid…ah’m sure ee bonnie lads and lassies north o ‘ee border will fix ‘at richt quick!

Ee Link

Groovy Virtualisation Hardware

I came across some interesting news today (Linky) that said Neterion is releasing a fairly hardcore network card designed for offloading VM Network management from the Hypervisor. I think this has the potential to be quite an interesting field in the future…in small scale Virtualisation deployments it’s not such a big deal that the Hypervisor has to do all the legwork for IO, but when you scale up to much larger deployments, network IO has the potential to be a significant bottleneck. You can mitigate this somewhat by utilising the physical network card, but this in turn shifts the load onto the host OS. It should be interesting to see in future what else hardware manufacturers come up with along similar lines: VM aware disk & memory controllers, for example…Intel have already made strides towards VM aware CPUs with VT as well.

Being a rather geeky type, I’m quite excited to see what all these clever hardware types come up with in the next few years…wouldn’t it be nice to be able to buy a server full of VM aware kit which lets you run multiple VMs as quickly as if you were using just one host OS? I can’t see virtualisation going away any time soon, it’s just too damn useful – so I reckon this might be something just around the corner.

Funky $PS1 Script

I happened across a really cool script for doing some clever stuff with your Bash prompt….check out the link, and if you like it then sling it into your .bashrc file 🙂 Clicky…