Rookery
 

Main Page

    

Hatch you own little penguin.

The rookery is a build system that you can use to take a collection of source rpms and turn it into a distribution. It takes care of figuring out the build order and seeing the build through to completion. IT REQUIRES YOU TO HAVE AN EMPTY SCRATCH PARTITION. It will erase everything in the build partition as a normal course of business.

Download
rookery-0.8.5-1.src.rpm (~63K 04/07/05)
rookery-0.8.5.tar.gz (~59K 04/07/05)
Beta code to create a live CD: livecd-0.2.2-1.src.rpm (~3.3M 04/07/05)
(If you use the system and have problems or suggestions, please contact me.)

Setting Up
Before using it, you have to do some setting up.

1) You must have an empty partition. I would also recommend making it as big as you can, 8GB works nicely for me. Get its device name using mount. For example, /dev/sda3 for partition 3 on the first scsi drive. Or /dev/hda5 for the fifth partition on the first IDE drive.
2) Locate or create a directory you want to use for the mount point. For example, /mnt/target. Go ahead and put it in /etc/fstab so that it gets mounted when you boot up.
3) As root, run the utility rookery-setup. Follow its questions. I recommend using its defaults except for the above mentioned scratch partition and mount point.
4) After it finishes, run rookery. It requires dialog to be installed on your system. In rookery, go to the configure menu and run "edit global settings". Be sure to save. The rookery-setup program only writes a partial configuration, rookery writes a full configuration file. EVEN IF YOU DON"T CHANGE ANYTHING, SAVE THE CONFIG. See Image:

conf-menu.png (37654 bytes)

5) Run the "test global settings" menu item. Follow its suggestions and keep running it until it says everything looks OK. If you are upgrading to a new version of rookery, run this command too. It will help you find/fix things that have changed.
6) Populate the /opt/repo/source/core/srpms directory with all the files that constitute the core of your distribution. You can separate srpms and keep them in their own subdirectory. Use the Create item in the repo menu of configure to setup additional repositories. In step 5, you should have populated the setup and tools directories.
7) In the compile menu, run the "prepare to do a build" menu item. See image:

build-menu.png (48929 bytes)

8) Run the "Check the build preparations" menu item. Fix everything it complains about. You will not get a successful build if there are still issues that it complains about at this step. VERY IMPORTANT - Everytime you fix things, go back to step 7 and re-run the "prepare to do a build" menu item. This causes the build dependencies and build schedule to get regenerated.
9) Run the "Build first pass tools"  menu item.
10) In the Begin menu, run the "Build All" menu item. It took about 22 hours to do a full build on my dual P3-700 machine with the source list found below.

Notes

Experiment with it. If you see improvements, please let me know. This project has been 7 months in the making and it reliably builds a distribution for me. I know all its quirks and I'm trying to make it more generic so that other people can use it.

The suite of utilities has extensive capabilities. For example, it can work out circular dependencies. The first one you will run across is in glibc. You must take perl out of the BuildRequires. The problem is glibc needs perl which needs db4 which needs perl. You can take perl out of the db4 BuildRequires here too. The reason is that rpmbuild needs perl, so it will be installed for the build. No package needs to require perl unless it is a perl module and needs to wait until perl itself is built before being built.

The rookery works by wiping clean the scratch partition, restoring its directory structure, installing the rpms/files to satisfy rpm. Installs the files to satisfy rpmbuild. Then installs the files to satisfy dependencies recursively for the source rpm being built. It does not install anything extra so that problems can be found. This lean approach means that it kind of bootstraps itself.

The rookery creates a clean, sterile environment and chroots into it to perform the build. In this way, it will map out the exact requirements for building a srpm. I've turned in many many bugzilla reports correcting the BuildRequires for the Fedora repository. It is in pretty good shape now. The way that the rookery works, it will build a repository against itself. I followed some ideas from Linux From Scratch as I wrote the rookery build system.

The rookery also has some unique features:

  • It stores away all log files from the build. You can later run a report against the whole build and see what kind of errors/warnings it found.
  • It can let you that a build will fail due to incomplete dependencies before you ever start building.
  • It has the ability to create graphs of the build dependencies of the whole repository ( Fedora Core 3 ), or just one package ( mkinitrd ).
  • It is menu driven.
  • It can do a test install to the scratch partition so you can subsequently boot into it for testing.

Testing

I'm testing against the srpms that will become Fedora Core 4 using a Fedora Core 3 build machine. Here's a list of files that you can download to experiment with - the version numbers may be slightly different since I grabbed these from rawhide, but they illustrate a closed dependency list. You can also see my repository layout by looking at this file. You can certainly add more packages to this, but this is a good starting point.

Here's a file that shows all the patches I had to make to satisfy dependencies and break circular dependecies. You can see the technique by trying the tools against the original srpms. Note: I normally put the original downloaded srpm into the orig directory for each repo if and only if I modify the srpm. You also may find it easier to use the command line tools at this stage (This stage being step 8 above):

As root, run mksindex.
Then as build, run mkdepindex
analyze-deps /etc/rookery/dep.index | less

When all diags say everything is OK, you can go back to the GUI and continue at step 9.

Live CD

So you've compiled everything and have a bunch of rpms. Now what do you do with them? You could make a Live CD with them. Here's how:

1) Download the livecd srpm and put it into a repo so that the rpms get picked up later for installation into a Live CD. For example, copy it into the /opt/repo/source/core/srpms directory and then, as root, run /opt/rookery-scripts/build-one /opt/repo/source/core/srpms/livecd-0.2.1-1.src.rpm.
2) Install ONLY the livecd-devel rpm. The other rpms it produces will be installed in the CDROM image.
3) As root, run the rookery command. Go into the Install menu of the Rookery and select Live CD.
4) Go into LCDCONFIG menu item and fill in the information about your distribution. When you are finished save it.
5) Go into the SELECT menu item. To automatically fill the list with potential rpms, go into the RESET menu item.
6) If you do not want development rpms in the list, go into the FILTER menu item. This is recommended.
7) Go into the SEL menu item and finalize the selection. Uncheck the items you don't want. When you press OK, they are removed from the list. You cannot add them back unless you either restart from the RESET command or manually edit /opt/livecd-devel/conf/install-rpm-list.txt. Remember you have a finite amount of room and you have to be judicious with your selections.
8) Leave the select menu area and go into the BLANK menu item. This creates the blank CD-Rom image.
9) When that's done, choose INSTRPMS to install your rpms into the image. If it fails, troubleshoot and come back to this same step. It erases the contents of the image everytime - so you can run this as many times as you want. (Troubleshooting will probably involve running the /opt/livecd-devel/scripts/livecd_2_rpm_install program from the command line.) Either add rpms to correct dependencies or remove rpms because of lack of space. You may also experiment with making a bigger image by editing /opt/livecd-devel/conf/buildroot.conf and changing the buildroot_mklodisk_size parameter (if you do this, go back to step 8). Make it too big and it won't fit on the CD. If you need to access the image, you can do it by:
umount /mnt/target
mount /opt/livecd-devel/images/livelinuxcd.vhd /mnt/target/ -o loop
ls /mnt/target or any other command including chroot /mnt/target
Don't forget to umount /mnt/target when you are through. I'd also like to take a moment to recommend you experiment using a re-writable CD. As you can probably guess, you'll go through many iterations getting the recipe just right. This can lead to a lot of test disks to throw away...unless you use a re-writable disk.
10) When the image is filled with your rpms, you may want to go into the image and look around. Maybe you want to tidy up, change permissions, etc. Just remember that all your changes will be lost next time you install the rpms. So, I would make a script if you do much customizing and run it.
11) Run the FINALIZE menu item to change the image into its form ready to be made into an iso. The program will continue by creating the iso. It will be located at /opt/livecd-devel/images/disc1.iso.
12) Burn it. You can use the BURN menu item or any other program. You may want to adjust the cdrom parameters in /opt/livecd-devel/conf/buildroot.config.

If you don't want to compile your own distribution and would rather use precompiled...
1) In rookery, create at least 1 repository. Copy the rpms into the rpm directory of that repository. For example, in rookery - you create a FC3 repository. Then you will need to copy the precompiled rpms to /opt/repo/source/FC3/rpms.
2) Start from the beginning of the first set of Live CD instructions.


This page was last updated 7 April, 2005