Package Management: The Return of the Hybrid

While that in itself isn't a particularly impressive feat -- after all we were already able to build the complete gcc 2 part before -- the interesting aspect is how we are doing it. First of all the build system no longer "cheats" when building a hybrid image. For building a hybrid image in the current Haiku master branch separate gcc 2 and gcc 4 builds have to be configured and the build system then "borrows" files from the other configuration. Also the needed optional packages of the other configuration are simply reused.

With package management we separate the hybrid's primary and secondary architecture cleanly ("architecture" in anticipation of things like x86-64 with x86 support; currently it's just two incompatible compilers for the same hardware architecture). The build is configured with the compilers for both architectures and the build system knows what to build with which. Furthermore there are dedicated packages for the secondary architecture. E.g. the ncurses package built with gcc 4 for a gcc2/gcc4 hybrid differs from the ncurses package built with gcc 4 for a gcc4/gcc2 hybrid. This is necessary not only because the files in the primary and secondary architecture package would otherwise clash, there may also be things like data files that we don't want to duplicate and therefore leave out in the secondary architecture package.

Commands, libraries, and add-ons for the secondary architecture live in an architecture subdirectory of their usual location. For commands it's just to avoid clashes, for libraries and add-ons the directories are also significant to the runtime loader. It will only search libraries and add-ons in the directories matching the architecture of the program.

haikuporter has grown support for building packages for the secondary architecture and the build recipes for the required software have been adjusted accordingly. There are still some open questions with respect to organizing certain types of files in secondary architecture packages (documentation, certain development files), but those will be answered when we have gained more practical experience.

But back to the build system: Besides having support for hybrid builds it also retrieves packages from the HaikuPorts repository. Yes, there is one now! In fact there are already multiple versions of it. Whenever packages are added to it, removed, or updated a new version is generated (currently it has to be done manually). This way it will be possible to check out older revisions of Haiku and still have the matching repository for it. Those repositories are fixed. For official Haiku releases we'll have mutable repositories in which packages get updated over time. The freshly built Haiku is pre-configured with the matching repository, so that the package manager is immediately usable.

And finally we've also added support for bootstrapping Haiku to the build system. If the build has been configured respectively, a jam -q @bootstrap-raw builds a Haiku bootstrap image. The process will not download any pre-built packages. Instead haikuporter is used to cross-compile bootstrap versions of the essential packages -- all that's needed to run Haiku (ICU, zlib,...) as well as the basic build tools. It also builds source packages for all software needed for a regular Haiku image and adds them to the bootstrap Haiku image. This way the bootstrap Haiku can be used to build the regular packages. We still have to do exactly that for the architectures for which we don't have any packages yet (x86 gcc 4 and x86-64).

Once that's done I think we should very soon merge the package management branch back into the master branch. Package management isn't "complete" yet, but it should be reasonably usable and there are only a few smaller known issues/regressions it introduces. It would definitely benefit from wider exposure at this point. So, everyone please feel encouraged to test it already and shout (via Trac), if you find your favorite feature or use case broken. I've started a wiki page that lists the changes users can expect by switching to a package management Haiku.

On a related note, Stephan Aßmus has started working on HaikuDepot, a GUI package manager application which will also provide functionality like screenshots and user ratings. He has posted a first screenshot of the GUI.