CMake: copying shared libcrystax, libgnustl and libboost_* to build- and install-tree (and their semantics)

[intelfx]

I was wondering what is the "intended" workflow of using CMake build system and shared runtime libraries (libcrystax.so, libgnustl_shared.so and libboost_*.so ahd others); and, more generally, what are the "intended" semantics of build- and install-trees when using CMake with Android NDK.

Cc: @rpavlik — you are the author of the only Android/CMake toolchain file which at least somehow considers boost, so I'd like to hear your comments as well.

Part 1. Shared runtime libraries and CMake

So far, the CrystaX CMake toolchain file copies libcrystax.so and libgnustl_shared.so into the build-tree, and that's all. This has three problems:

1. Bundled third-party libraries (Boost, at least) are simply left out. A project which uses CMake and shared boost is non-functional after building.
2. This behavior depends on the ${LIBRARY_OUTPUT_PATH} variable which is deprecated and not even set by default. Considering that ${ANDROID_STL} defaults to gnustl_shared and libcrystax.so is available only as a shared library, a project built with CMake with default settings is non-functional after building (!).
3. This contradicts the workflow described below (see below).

Part 2. Build- and install-tree semantics with Android

In the CMake toolchain file I find some rudimentary code that sets ${LIBRARY_OUTPUT_PATH} to something containing libs/${ANDROID_NDK_ABI_NAME}. This means that the authors intended the build-tree to become the final location of the libraries. That is — if the CMake project is correctly placed within the Android project, you can just cmake . && make and the libraries will end up in the correct places for Android packaging. (I hope I am clear here.)

However, I find this pretty unusable. First, any typical cross-compilation effort will include multiple projects which will try to find each other. By default (with existing toolchain file), CMake won't search libraries in ${LIBRARY_OUTPUT_PATH}, but it will search in ${CMAKE_INSTALL_PREFIX}/lib. Same holds for find_package(CONFIG).

Second, almost no projects are written with Android/CMake in mind, so vast majority of existing projects will install their libraries in $prefix/lib.

Hence, to make projects find each other (without extensively patching their build systems), I typically build in a temporary directory and then install to a common prefix, with libraries installed in $prefix/lib and exported configs in $prefix/lib/cmake/Project/ProjectConfig.cmake. Then I symlink $prefix/lib from $android_project/libs/$ABI and run Android packaging.

The described workflow allows to cross-compile any "chains" of projects without altering their build systems (provided that they can build for Linux).

With all this in mind, there should be a way to install the shared runtime libraries alongside the project's library destination, typically ${CMAKE_INSTALL_PREFIX}/lib. I'd expect a parameter of kind ANDROID_SHARED_HELPER_DESTINATION, which, when set, will make the toolchain file install the shared runtime libraries to ${CMAKE_INSTALL_PREFIX}/${ANDROID_SHARED_HELPER_DESTINATION}.

This parameter should be honoured by the toolchain file itself (to install libcrystax.so and libgnustl_shared.so), and custom Find*.cmake modules should be written for all bundled 3rd-party projects (boost, ...) which will chain to real find modules and install the found libraries afterwards (an example of such "chaining" already exists in the tree as FindBoost.cmake).

Comments? Maybe I'm horribly wrong somewhere and using CMake with shared runtime already works well?

[rpavlik]

FWIW, the same sort of issue (bundling dependencies for runtime) crops up on Windows as well since neither has a system-wide package manager handling dependencies. On Windows, https://github.com/OSVR/OSVR-Core has to copy OpenCV runtime dlls. On Android, we need to copy libcrystax, etc. and the boost libraries used (on Windows we statically link to boost).

There's substantial support in CMake for extending the behavior of stock Find* modules transparently - that's how the Android-CMake fork I've put together gets the built-in FindBoost.cmake module to find Boost in CrystaX - but somewhat less support for arbitrary bundling of runtime dependencies on platforms with no notion of (or no active/respected) RPATH - there's bundle utilities but I don't typically use that much anymore, rather using imported targets, etc. to get the dependencies copied manually.

The chaining could work, I suppose, at least here in the Android case where you can say "only installed builds will work" (getting it to work on Windows in build trees as well as install trees is... joyful, through no fault of CMake) though you'd have to know where people want things installed. You almost never want to actually explicitly say ${CMAKE_INSTALL_PREFIX} in an install command, either - just use a relative path since the install prefix/destdir can be changed at install time.

[rpavlik]

Note also that there are more resilient methods than LIBRARY_OUTPUT_PATH to find out where things are getting built to (namely generator expressions) but they may require a user to run an extra command (crystax_copy_deps or something) with the target name - I'm not sure if they can be passively injected by a toolchain file.

Microsoft's new vcpkg tool does do something that seems a little sketchy to perform copying of stuff on add_executable calls - I don't understand how it works exactly (Wasn't aware you could override built-in commands and call their built-in implementations) but it presumably must (Haven't personally tested it) - see https://github.com/Microsoft/vcpkg/blob/master/scripts/buildsystems/vcpkg.cmake

[ras0219-msft]

Yeah, the vcpkg mechanism is a bit smelly. It's unfortunately not composable either -- you can only override builtin functions once as far as I can tell.

However, basically the idea is to hook into every add_executable() directive and add a post-build step to run our AppLocal.ps1 script. This script uses dumpbin to analyze the runtime dependencies of the output executable and copy all dependencies vcpkg is aware about to the output directory. I believe you should be able to do a similar process when targetting android by using objdump, then finally editing RPATH to be relative to the appliation directory?