Cake Package Tutorial

Introduction

C and C++ development lacks a fundamental tool: the package manager. Adding a new library to your project which you've just found on GitHub should be no more than copying its URL into your project file. Managing the master project of your application which links to 10-20 in-house and third-party dependencies should not cost more than a few minutes of your precious development time.

There are great initiatives, like biicode, CPM, fips and Hunter (these are the CMake-based ones I'm aware of). I'd like to present you yet another tool, Cake, which is based on CMake and git and which differs from those approaches in the following ways:

Cake is lightweight, non-intrusive solution, minimal change in your CMake files, no need to commit yourself to a new ecosystem
It adds only two CMake functions to manage your dependencies: quick to learn
Both kinds of packages are supported:
- External packages: fetched, built and installed before the main project
- Subprojects, added with CMake's add_subdirectory
Cake can be used with existing, CMake-enabled packages (no need to modify the library for use it with Cake)
Supports optional dependencies (think of a --with-sqlite switch in autoconfig)
Helps to manage master projects: provides a new solution to the git-submodule problem
Implemented as a platform-neutral CMake script.

And the what it does not do:

Does not discover dependencies of your project automagically
No file-level dependency management, a package is the entire repository of the library
No high-level package descriptor files: versioning is controlled by asking for specific git commits/tags.

I'd like to show you how to use Cake and encourage you to download it and try it on your own projects. I'd also like to invite developers to improve Cake with new features.

As a quick demonstration, the CMakeLists.txt of a project using libpng looks like this:

cmake_minimum_required(VERSION 3.1)
project(pngtest)

include(${CAKE_ROOT}/Cake.cmake)

cake_find_package(PNG REQUIRED URL git://git.code.sf.net/p/libpng/code) # the official libpng repo

include_directories(${PNG_INCLUDE_DIRS})
add_definitions(${PNG_DEFINITIONS})
add_executable(pngtest main.cpp)
target_link_libraries(pngtest ${PNG_LIBRARIES})

The cake_find_package() macro clones, builds and installs libpng and its dependency, zlib, in configure-time and calls find_package(PNG).

Overview

The package manager is a collection of CMake scripts. The repository can be found at https://github.com/tamaskenez/cake. It contains a CMake module which can be downloaded/included in your CMakeLists.txt files. It provides 3 CMake functions: cake_pkg, cake_find_package, cake_add_subdirectory and two helper shell commands: cake and cakepkg.

Let's see the basic operations you can do with Cake:

Install a package without dependencies from the command-line

The recommended way to install packages is to call Cake commands from your project's CMakeLists.txt. However in certain scenarios you may prefer to install the before configuring your project with CMake. Cake provides a shell command cakepkg, to download and install packages manually or in an init script of your repository.

Let's use the cakepkg to install zlib:

cakepkg INSTALL URL https://github.com/madler/zlib.git # official zlib repo

This command clones the zlib repository then builds and installs it with CMake. The location of the local copy of the zlib repo and the build and install directories are controlled by Cake's configuration variables, more about these later.

You can control the details of the git-clone operation using the URL query string:

cakepkg INSTALL URL https://github.com/madler/zlib.git?branch=develop&depth=1

The query items are parsed by Cake and forwarded to git-clone.

You can also forward variable definitions to the cmake command which builds the library:

cakepkg INSTALL URL https://github.com/madler/zlib.git?-DAMD64=1

This is similar to the --with-* flags feature of autoconf.

Install a package with dependency from the command-line

A package manager's main feature is to fetch and install the dependencies recursively. This command which install libpng, which is dependent on zlib:

cakepkg INSTALL URL git://git.code.sf.net/p/libpng/code # official libpng repo

This command clones the libpng repository, discovers that it is dependent on zlib, so clones that one, too. Then installs zlib and then libpng.

How does Cake know libpng is dependent on zlib? There are three methods to specify this.

The first method is to be used with third-party project you can't or don't want to modify (like libpng). For such projects the dependency information is described in a file, cake_pkg_db.cmake in the root of the Cake installation.
The second method can be used if you can add files to the library project but don't want to mess with its CMakeLists.txt. In this case you need to put a file cake-depends.cmake in the root of the library's repository, which installs the required dependencies.
The third, recommended method is to simply replace the find_package() commands in your library's CMakeLists.txt with cake_find_package(... URL ...) calls.

In case of libpng we need to use the first method so we can use the official, unmodified libpng repository. We'll describe all three methods in detail later.

Install and use a package from your CMakeLists.txt

This is the scenario already shown in the introduction:

cmake_minimum_required(VERSION 3.1)
project(pngtest)

include(${CAKE_ROOT}/Cake.cmake)

cake_find_package(PNG REQUIRED URL git://git.code.sf.net/p/libpng/code) # the official libpng repo

include_directories(${PNG_INCLUDE_DIRS})
add_definitions(${PNG_DEFINITIONS})
add_executable(pngtest main.cpp)
target_link_libraries(pngtest ${PNG_LIBRARIES})

The cake_find_package() macro downloads and installs libpng and its dependency, zlib, in configure-time. Of course, if you call cmake-configure the second time, git-clone will be skipped. Also, Cake stores the SHA1 commit-id of the last successful builds of zlib and libpng. If the repositories' HEADs are still on those commits no build will be triggered, the command will take very little time..

Note that you only need to replace find_package with cake_find_package in your main project. Neither the libpng's CMakeLists.txt nor the FindPNG.cmake module (which calls find_package(ZLIB...)) have to be changed.

After the cloning/building phase the cake_find_package command calls the usual find_package() with rest of the arguments. You may object that it would be a cleaner API design to provide a Cake command which performs the clone/build/install phase and you would call find_package explicitly:

cake_pkg(INSTALL URL git://git.code.sf.net/p/libpng/code) # the official libpng repo
find_package(PNG REQUIRED)

The cake_pkg actually exists and this code does work. However, using cake_find_package is still better in some cases:

Typically use want to use a library as an external dependency (built in its own source tree) when you're not modifying it. On the other hand, when your project is broken down into libraries you're working on it's more convenient to add them as subdirectories into a master project.

When using library 'A' in library/executable 'B' you don't want to care about the fact whether 'A' comes from a separate source tree or built in the same master project as 'B'. You will just need to call cake_find_package(A URL ...) in the CMakeLists.txt of project 'B' and it will work in both cases. This is demonstrated in the next example:

Add a package as a subproject

When you're using dependencies you need to edit and debug it's much more convenient if you add them as subprojects with the CMake add_subdirectory command.

The tricky part to add a package as a subproject is, as opposed to an external project, that it won't be installed before your main project calls find_package for it. Moreover, we would like to use the package's headers from their original location in the source tree and not from their install location.

Fortunately CMake has a solution for these problems. There are two things to keep in mind:

Instead of the classic add_include_directories, add_definitions commands, the dependent package must use the target_* command family to describe these properties (target_include_directories, target_compiler_definitions, target_compile_options, target_link_libraries, target_compile_features). It also may need to use the target_include_directories command's BUILD_INTERFACE/INSTALL_INTERFACE generator expressions to allow the headers to be used from their original location in the source tree.
In order to be used transparently as an external project, the dependent package must install a config-module which creates an import library.

Let's see the CMakeLists.txt of a project which includes a package optionally as a subdirectory:

cmake_minimum_required(VERSION 3.1)
project(subprojtest)

include(${CAKE_ROOT}/Cake.cmake)

option(ADD_AS_SUBPROJECT TRUE)

if(ADD_AS_SUBPROJECT)
    cake_add_subdirectory(subprojlib URL git://github.com/tamaskenez/cake-sample-subprojlib.git?depth=1)
endif()

cake_find_package(subprojlib REQUIRED
    URL git://github.com/tamaskenez/cake-sample-subprojlib.git?depth=1)

target_link_libraries(subprojtest subprojlib)

Depending on the value of ADD_AS_SUBPROJECT this CMakeLists.txt will add subprojlib either as an external dependency (cloned and installed in configuration-time) or as a CMake subdirectory:

If ADD_AS_SUBPROJECT is FALSE this example works just like the previous one, except that we expect subprojlib to provide an import library. So instead of

include_directories(${subprojlib_INCLUDE_DIRS})
add_definitions(${subprojlib_DEFINITIONS})
target_link_libraries(subprojtest ${subprojlib_LIBRARIES})

it's sufficient to call only

target_link_libraries(subprojtest subprojlib)

If ADD_AS_SUBPROJECT is TRUE then cake_add_subdirectory(subprojlib ...) clones the subprojlib into the specified subdirectory and calls the CMake add_subdirectory command.

Later, the cake_find_package(subprojlib ...) detects that subprojlib has already been added as a subdirectory and does not attempt to clone it again and neither it calls find_package.

Of course, if the cake_find_package(subprojlib) is in the same master CMakeLists.txt as the corresponding cake_add_subdirectory(subprojlib) command you don't really need to use cake_find_package at all since it will be a no-op. But usually the cake_add_subdirectory calls are in the master project and all cake_find_package calls are in the independent subprojects.

Specifying Dependencies

As we said earlier, there are three methods to specify the dependencies of a package.

Specifying dependencies without modifying the package's repository

This is the option we used with libpng. The Cake repository contains a file cake-depends-db.cmake which defines a package dependency database. The line which describes libpng's dependency, zlib is:

cake_depends_db_add(URL git://git.code.sf.net/p/libpng/code SCRIPT "cake_pkg(INSTALL URL https://github.com/madler/zlib.git?depth=1)")

The command cake_depends_db_add adds an entry to the database for the libpng repository. It assigns a script which will be invoked before building libpng:

cake_pkg(INSTALL URL https://github.com/madler/zlib.git?depth=1)

Of course the script can be more complex as we can see in the next section:

Specifying dependencies without changing the package's CMakeLists.txt

In certain cases you can add a file to the library's repository but does not want to mess with its CMakeLists.txt (so it can be still be built without Cake). In this case you can add a file cake-depends.cmake into the root of the library's repository which contains the same dependency-script as above. For example, if we choose to add cake-depends.cmake to the libpng repository:

# git.code.sf.net/p/libpng/code/cake-depends.cmake
cake_pkg(INSTALL URL https://github.com/madler/zlib.git?depth=1)

These dependency-script receives the build configuration variables you specify in the query part of the package URL. For example, in case of a hypothetical library which has an optional SQLite dependency:

cake_find_package(mylib URL https://github.com/myaccount/mylib?MYLIB_WITH_SQLITE=1)

The cake-depends.cmake dependency-script of mylib could be this:

# github.com/myaccount/mylib/cake-depends.cmake
if(MYLIB_WITH_SQLITE)
    cake_pkg(INSTALL URL https://github.com/myaccount/sqlite-cmake)
endif()

The CMakeLists.txt of mylib would use the plain find_package and would not depend on Cake:

# github.com/myaccount/mylib/CMakeLists.txt
...
if(MYLIB_WITH_SQLITE)
    find_package(SQLite REQUIRED)
    add_definitions(-DHAVE_SQLITE)
endif()
...

Specifying dependencies with `cake_find_package`

If you don't mind modifying the library's CMakeLists.txt and that it will depend on Cake, the simplest and recommended method is to use cake_find_package instead of find_package. In this case neither you need an entry in code-depends-db.cmake nor you need to add cake-depends.cmake (which just duplicates the references to your dependencies). The CMakeLists.txt of the mylib example above would look like this:

# https://github.com/myaccount/mylib/CMakeLists.txt
...
include(${CAKE_ROOT}/Cake.cmake)
...
if(MYLIB_WITH_SQLITE)
    cake_find_package(SQLite REQUIRED URL https://github.com/myaccount/sqlite-cmake)
    add_definitions(-DHAVE_SQLITE)
endif()
...

Cake Configuration Variables

When you install a package with cakepkg (or cake_pkg, cake_find_package), Cake eventually calls the cmake command to configure, build and install the package. The parameters Cake passes to cmake can be controlled by the Cake configuration variables: CAKE_PKG_CMAKE_OPTIONS, CAKE_PKG_CMAKE_NATIVE_TOOL_OPTIONS, and CAKE_PKG_CONFIGURATION_TYPES. You can set them in the shell environment or in your CMakeLists.txt.

CAKE_PKG_CMAKE_OPTIONS is the most important one. This variable contains the cmake's command line options the package will be configured with. It usually contains a -DCMAKE_INSTALL_PREFIX=<...> definition to specify where the package will be installed to. Related Cake-specific directories, like local copy of the repository and build directory will be also created under this directory (under ${CMAKE_INSTALL_PREFIX}/var).
The options in CAKE_PKG_CMAKE_NATIVE_TOOL_OPTIONS will be added to the end of the cmake --build command after the '--'.
CAKE_PKG_CONFIGURATION_TYPES usually contains a list like "Debug;Release". It controls the configurations the packages will be built in.

As an alternative to set the Cake configuration variables you can also create a CMake file like this:

set(CAKE_PKG_CONFIGURATION_TYPES Debug Release)

set(CAKE_PKG_CMAKE_OPTIONS

    -DCMAKE_INSTALL_PREFIX=${CMAKE_CURRENT_LIST_DIR}/_install

    -DCMAKE_PREFIX_PATH=${CMAKE_CURRENT_LIST_DIR}/_install)

And set the environment variable CAKE_PKG_FILE point to this file.

Roadmap

The following features are expected to be added in the future:

- Finer control of the git-clone, git-pull, CMake config/build operations (like forced clean build, global clone depth setting).

- Features to help managing master projects with multiple subrepositories (like googles git-repo-tool or gclient)

- zip/tgz, Mercurial and SVN support

History

This is the initial revision.