CMake migration modular cross platform by jolavillette · Pull Request #304 · RetroShare/libretroshare

jolavillette · 2026-06-02T14:03:34Z

Implement full modular cmake compilation on linux, windows and macos

Works with the following PRs in RetroShare and i2p/libsam3
RetroShare/RetroShare#3229
i2p/libsam3#30

Example of script to compile everything:

#!/bin/bash

Exit immediately if any command fails

set -e

Clean build directories if the "clean" argument is passed

if [ "$1" == "clean" ]; then
echo "=== Cleaning build directories ==="
rm -rf libretroshare/Build
rm -rf retroshare-service/Build
rm -rf retroshare-friendserver/Build
rm -rf retroshare-gui/Build
fi

Detect number of CPU cores for fast parallel compilation

NPROC=$(sysctl -n hw.ncpu 2>/dev/null || nproc 2>/dev/null || echo 4)

Detect Qt5 path on macOS if Homebrew is installed

QT_PREFIX=""
if command -v brew &> /dev/null; then
QT_PREFIX=$(brew --prefix qt@5)
fi

echo "=== Starting RetroShare Compilation ==="

1. Compile libretroshare (core library) and librnp automatically

echo ">>> Step 1: Compiling libretroshare..."
mkdir -p libretroshare/Build && cd libretroshare/Build
cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DRS_LIBRETROSHARE_STATIC=OFF -DRS_LIBRETROSHARE_SHARED=ON ..
cmake --build . -j${NPROC}
cd ../..

2. Compile retroshare-service (headless daemon)

echo ">>> Step 2: Compiling retroshare-service..."
mkdir -p retroshare-service/Build && cd retroshare-service/Build
cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 ..
cmake --build . -j${NPROC}
cd ../..

3. Compile retroshare-friendserver

echo ">>> Step 3: Compiling retroshare-friendserver..."
mkdir -p retroshare-friendserver/Build && cd retroshare-friendserver/Build
cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 ..
cmake --build . -j${NPROC}
cd ../..

4. Compile retroshare-gui (graphical user interface)

echo ">>> Step 4: Compiling retroshare-gui..."
mkdir -p retroshare-gui/Build && cd retroshare-gui/Build
cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5
-DCMAKE_PREFIX_PATH="${QT_PREFIX}"
-DENABLE_GUI=ON
-DENABLE_QT_TRANSLATIONS=ON ..
cmake --build . -j${NPROC}
cd ../..

echo "=== Compilation Successfully Completed! ==="

The PR also provides an orchestrator
See "make help"

…, and versioning

…dows

…of local submodule

…dows

…DEPRECATED to OFF

…dows

…d symbols to avoid PE limit

…dows

… build dirs

…rary

… privately via absolute paths

G10h4ck · 2026-06-02T19:44:06Z

libnrp does use CMake too? In that case it shouldn't be needed to compile it manually before, if added properly as a dependency CMake will deal with compiling of that too automatically just like the other dependencies (ex: restbed)

G10h4ck

See my inline comments

G10h4ck · 2026-06-02T19:45:56Z

 /******************** notify of new Cert **************************/

+#if defined(WINDOWS_SYS) && defined(X509_NAME)
+// Workaround: Windows headers (<wincrypt.h>) define X509_NAME which conflicts with RetroShare types.


Is RetroShare which define X509_NAME as a type? In that case RS shouldn't pollute the namespace like that, and the type should be renamed to RS_X509_NAME

X509_NAME is actually not a RetroShare type; it is an internal OpenSSL struct. We cannot rename it to RS_X509_NAME because it is defined in OpenSSL's headers. The #undef is the only safe workaround to allow using OpenSSL's X509_NAME if Windows <wincrypt.h> (which defines it as a macro) was previously included.

double check if this still happens with WIN32_LEAN_AND_MEAN

Done. I removed the #undef workaround and rely on WIN32_LEAN_AND_MEAN instead, which keeps <windows.h> from pulling in <wincrypt.h> (where the colliding X509_NAME macro comes from). It was already defined globally for the qmake build and for the CMake shared target, but not for the CMake static target — which is the default (RS_LIBRETROSHARE_STATIC=ON) — so I moved the definition out of the RS_LIBRETROSHARE_SHARED block to cover both. The only explicit #include <wincrypt.h> is in rsserver/rsloginhandler.cc (for DPAPI), and that file doesn't include sslfns.h, so no translation unit ever sees both. I also fixed the misleading comment: it's an OpenSSL type, not a RetroShare one.
Compilation OK linux, windows, macos

G10h4ck · 2026-06-02T19:46:08Z

+#undef X509_NAME
+#endif
+#if defined(WINDOWS_SYS) && defined(X509_EXTENSIONS)
+// Workaround: Windows headers (<wincrypt.h>) define X509_EXTENSIONS which conflicts with RetroShare types.


See previous comment

Same as above. X509_EXTENSIONS is an OpenSSL internal type, not a RetroShare type. We must undefine the Windows macro to be able to use OpenSSL properly.

double check if this still happens with WIN32_LEAN_AND_MEAN

Same as above — removed together with the X509_NAME workaround. With WIN32_LEAN_AND_MEAN now applied to the default (static) CMake target as well, <wincrypt.h> is never included in a translation unit that uses the OpenSSL types, so the X509_EXTENSIONS macro collision can no longer occur.

G10h4ck · 2026-06-02T19:52:08Z

 };

+#if defined(WINDOWS_SYS) && defined(ERROR)
+// Workaround: Windows headers (<wingdi.h>) define ERROR as 0, which breaks enum members named ERROR.


Being this a public header I think we should not mess with undefining a windows symbol, because that would happen also for any library downstream user. It is strange this is happening bacause the enum is scoped enum class. If failing anyway better investigating where the actual error is happening rstor.h doesn't include <wingdi.h> directly so it should not happen there. Another workaround could be to rename the enum values, but that should be a last resort.

Good point. Undefining ERROR in a public header is dangerous for downstream users. I have removed the #undef ERROR hack entirely from rstor.h. Instead, I added WIN32_LEAN_AND_MEAN globally in CMakeLists.txt (as you suggested below), which completely prevents <wingdi.h> from polluting the namespace.

G10h4ck · 2026-06-02T19:55:07Z


+#if defined(WINDOWS_SYS) && defined(ERROR)
+// Workaround: Windows headers (<wingdi.h>) define ERROR as 0, which breaks enum members named ERROR.
+#undef ERROR


Being this someway internal implementation here is less dangerous, but still why this used to work and now it doesn't? I remember to compile on windows a general thing must be set, it was called something "Windows lean and mean headers" or something like that so it stop polluting all around, please check that instead of undeffing things around, which is quite ugly and be done just as a last resort.

I have removed the #undef ERROR hack here as well, and I added target_compile_definitions(${PROJECT_NAME} PUBLIC WIN32_LEAN_AND_MEAN) in CMakeLists.txt for Windows builds. This cleanly solves the macro pollution issue without touching the headers.

G10h4ck · 2026-06-02T19:57:27Z

+	if(WIN32)
+		set_target_properties(${PROJECT_NAME} PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)
+		if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+			target_link_options(${PROJECT_NAME} PRIVATE "-Wl,--export-all-symbols,--exclude-libs,libcrypto.a:libssl.a:libsqlite3.a:libopenpgpsdk.a:libbitdht.a:librestbed.a:librestbed-static.a:libminiupnpc.a:librnp.a:libsexpp.a:libbotan.a:libbotan-3.a:libbotan-2.a:libz.a:libbz2.a:libjson-c.a")


This doesn't smell good at all, hardcoding all those library here, is basically unmaintenable code, CMake is here to do this kind of things automatically, so if a new library get added it automatically sets the proper flags, same goes if a library dependency get removed

I have replaced that hardcoded list with a simple -Wl,--exclude-libs,ALL. This cleaner approach prevents any statically linked library from blindly re-exporting its symbols on Windows, automatically handling any future dependency changes.

G10h4ck · 2026-06-02T20:13:31Z

+			${PROJECT_NAME} PUBLIC "${RESTBED_DEVEL_DIR}/source/" )
 		target_link_directories(${PROJECT_NAME} PUBLIC "${CMAKE_CURRENT_BINARY_DIR}/../supportlibs/restbed")
-		target_link_libraries(${PROJECT_NAME} PRIVATE restbed)
+		target_link_libraries(${PROJECT_NAME} PRIVATE restbed-static)


This sounds it will breaks all the targets that where working before (tens of Linux distributions, Android...) double check this thing and investigate how to avoid hard-coding static here

You are right, hardcoding restbed-static was too restrictive and broke systems that don't install a system-wide librestbed.so. Simply reverting it to restbed caused ld: library 'restbed' not found locally because the fetched restbed project does not create an alias named restbed, so CMake was emitting -lrestbed instead of linking the local target. I have updated the logic to dynamically resolve the correct target (restbed-shared or restbed-static) based on BUILD_SHARED_LIBS, and link to that specific target using a ${RESTBED_TARGET} variable.

G10h4ck · 2026-06-02T20:14:59Z

 		)
 		FetchContent_MakeAvailable(restbed)

+		if(TARGET restbed-shared)


Static/dynamic linking hard-coding again, is this really necessary? Do it properly not hard-coding

I have removed the hardcoded EXCLUDE_FROM_ALL and static/shared checks. We now rely strictly on the generic BUILD_SHARED_LIBS property and seamlessly link the resolved ${RESTBED_TARGET}.

G10h4ck · 2026-06-02T20:15:57Z


 if(NOT RS_WARN_DEPRECATED)
-	target_compile_definitions(${PROJECT_NAME} PRIVATE RS_NO_WARN_DEPRECATED)
+	target_compile_definitions(${PROJECT_NAME} PUBLIC RS_NO_WARN_DEPRECATED)


Why making it PUBLIC ?

That was a mistake. I have changed it to PRIVATE so we don't suppress deprecation warnings for downstream targets.

jolavillette · 2026-06-02T21:57:00Z

about librnp:
"libnrp does use CMake too? In that case it shouldn't be needed to compile it manually before, if added properly as a dependency CMake will deal with compiling of that too automatically just like the other dependencies (ex: restbed)"

You are completely right. We previously avoided adding it as a native CMake dependency because statically linking the librnp CMake target re-exported all of its internal dependencies (Botan, json-c, etc.) and broke the MinGW 65535 exported symbol limit on Windows. However, now that we have implemented the -Wl,--exclude-libs,ALL fix from your other comment, this limitation is gone. We can absolutely integrate librnp properly via CMake

- Build librnp via add_subdirectory under RS_RNPLIB, saving/restoring BUILD_SHARED_LIBS so libretroshare stays shared while librnp is static. - Windows: propagate UNICODE/_UNICODE to subprojects, link winmm/iphlpapi, attach socket libs to whichever restbed target exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…the sslfns.h X509_NAME/X509_EXTENSIONS #undef workaround

jolavillette · 2026-06-04T13:20:47Z

These definitions must be PUBLIC (not PRIVATE) because downstream targets (retroshare-gui, retroshare-service) include libretroshare's public headers, which in turn include <windows.h> and <openssl/x509.h>. If these flags are only PRIVATE, the macro conflicts reappear in every downstream target that includes our headers.
Each of these solves a specific, documented compilation failure on MSYS2/MinGW:

UNICODE / _UNICODE: libbitdht/src/util/bdfile.cc:47 calls MoveFileExW() with wchar_t* arguments. Without UNICODE, the Windows headers resolve the call to MoveFileExA() which expects char*, producing: error: cannot convert 'const wchar_t*' to 'LPCSTR'.

NOGDI: <windows.h> includes <wingdi.h> which does #define ERROR 0. This macro silently replaces every occurrence of ERROR in RetroShare's logging code (e.g. RsLog levels in rsdebug.h). Note: WIN32_LEAN_AND_MEAN does not exclude <wingdi.h>, so NOGDI is needed explicitly. This replaces the #undef ERROR hacks that were previously in rstor.h and rsdebug.h (removed per your earlier review comments #4 and #5).

NOMINMAX: <windows.h> defines #define min(a,b) and #define max(a,b) macros that break every call to std::min() / std::max() throughout the codebase.

NOCRYPT: <windows.h> includes <wincrypt.h> which does #define X509_NAME ((LPCSTR) 7) and #define X509_EXTENSIONS ((LPCSTR) 5). These macros shadow OpenSSL's X509_NAME and X509_EXTENSIONS structs, making any #include <openssl/x509.h> fail. This is the clean replacement for the #undef workaround that was in sslfns.h (removed per your review comments #2 and #3).

--exclude-libs and LTO are independent, and disabling IPO/LTO on Windows is still required: the build does not link with LTO enabled on Windows (MinGW/GCC LTO codegen fails with multiple-definition / thunk errors), regardless of the symbol exclusion.

The exclusion you're referring to is a separate fix, on the shared/DLL build only: it had to be narrowed from --exclude-libs,ALL (which broke the build) down to just the heavy third-party archives (libbitdht, librnp, libsexpp, libudp-discovery, librestbed, libsqlcipher, libbotan-2/3, libz, libbz2) — enough to stay under the PE/COFF ~65535 exported-symbol cap while keeping libretroshare's own API exported. The default static build is unaffected.

So AND NOT WIN32 should stay.

G10h4ck · 2026-06-04T17:23:34Z

@jolavillette please keep replying comment by comment, I have very little time, so try to optimize for the reviewer time ;-)

G10h4ck · 2026-06-04T17:34:43Z

+	# <wincrypt.h> (whose X509_NAME / X509_EXTENSIONS macros collide with the
+	# OpenSSL types of the same name used in pqi/sslfns.h).
+	target_compile_definitions(${PROJECT_NAME} PUBLIC WIN32_LEAN_AND_MEAN)
+endif()


I think WIN32_LEAN_AND_MEAN should go at line 278 or so, this also would avoid another if block and code lines to maintain

ping @jolavillette

I don't know why I can reply here and not under the next comments...

I have moved WIN32_LEAN_AND_MEAN to the main if(WIN32) block alongside the other Windows definitions, and removed the redundant if(WIN32) block entirely.

"Done. I have added inline comments right above each definition inside the CMakeLists.txt file explaining exactly what header pollution or compilation error they prevent (MoveFileExW translation, <wingdi.h> ERROR macro collision, min/max collision, and OpenSSL X509 types shadowing).

You're right, that find_library(BOTAN_LIBRARY ...) line was dead code leftover from before we introduced add_subdirectory(librnp). Now that librnp handles its own dependencies internally through CMake, we don't need to manually find Botan in libretroshare's CMakeLists anymore. I have completely removed it.

I have just added a detailed comment above the --exclude-libs line explaining that we use an explicit list instead of ALL because ALL inadvertently hides symbols from CMake's internal objects.a archive, preventing libretroshare from exporting its own core C++ symbols.

Regarding WIN32_LEAN_AND_MEAN and the inline comments for the definitions, this was already addressed and fixed in my latest commit. The flags are now cleanly grouped together with proper comments for each of them.

G10h4ck · 2026-06-04T17:35:41Z

-		${CMAKE_CURRENT_BINARY_DIR}/../supportlibs/librnp/Build/src/lib
-		${CMAKE_CURRENT_BINARY_DIR}/../supportlibs/librnp/Build/src/libsexpp
-	)
+	find_library(BOTAN_LIBRARY NAMES botan botan-3 botan-2 libbotan-3 libbotan-2 REQUIRED)


ping @jolavillette

G10h4ck · 2026-06-04T18:34:42Z

+	# <wincrypt.h> (whose X509_NAME / X509_EXTENSIONS macros collide with the
+	# OpenSSL types of the same name used in pqi/sslfns.h).
+	target_compile_definitions(${PROJECT_NAME} PUBLIC WIN32_LEAN_AND_MEAN)
+endif()


ping @jolavillette

- Add detailed comments explaining the necessity of an explicit --exclude-libs list over ALL (due to CMake objects.a export stripping). - Group WIN32_LEAN_AND_MEAN with other Windows compile definitions. - Add inline comments explaining each Windows compile definition (_UNICODE, NOGDI, NOMINMAX, NOCRYPT). - Remove dead find_library(BOTAN_LIBRARY) code now that librnp is a CMake subdirectory.

G10h4ck

Legit

jolavillette added 25 commits May 18, 2026 16:01

build(cmake): minimal fixes for transitive restbed headers, i2pcommon…

ffcfda9

…, and versioning

SAVE: Toutes les modifications courantes (Universel + Windows)

5edeb5f

Fix jsonapi generator and OpenSSL macro collisions

df1642b

Fix RNP include paths and jsonapi recompilation cache in CMake

ef67686

Merge Windows changes: integrate WS2/IOCP linkage and dlfcn support

31efa37

build(cmake): support dynamic plugin linkage by exposing RNP directories

39df2f2

Merge branch 'CMakeMigration-Modular' into CMakeMigration-Modular-Win…

39e4dad

…dows

Build: WIN32: use FetchContent restbed 4.8 with bundled ASIO instead …

ed94151

…of local submodule

build: encapsulate RNP and PGP dependencies as PUBLIC targets

60e127e

Merge branch 'CMakeMigration-Modular' into CMakeMigration-Modular-Win…

4ccd1b5

…dows

build: use target_compile_options instead of global CMAKE_CXX_FLAGS

c9804e8

Merge branch 'CMakeMigration-Modular' into CMakeMigration-Modular-Win…

7a5d75b

…dows

build: fix deprecation warning encapsulation and set default RS_WARN_…

ad0dcf6

…DEPRECATED to OFF

Merge branch 'CMakeMigration-Modular' into CMakeMigration-Modular-Win…

0d5f63a

…dows

build: disable LTO on Windows to bypass GCC 15 thunk link bug

dc0347b

cmake: configure libretroshare as DLL on Windows and restrict exporte…

72041cf

…d symbols to avoid PE limit

Merge branch 'master' into CMakeMigration-Modular

62d1c60

Merge branch 'CMakeMigration-Modular' into CMakeMigration-Modular-Win…

f4434ef

…dows

Merge branch 'master' into CMakeMigration-Modular-Windows

31a5b27

CMake: Dynamically find RNP include paths to support macOS and custom…

f98b61f

… build dirs

Merge branch 'master' into CMakeMigration-Modular-Windows-MacOS

b61af2c

fix(build): conditionally compile I2P SAMv3 support and link sam3 lib…

b60973c

…rary

fix(cmake): add missing target_include_directories for system rapidjson

e7086b2

Address PR review: link heavy cryptographic dependencies (RNP, Botan)…

e38be45

… privately via absolute paths

Address PR review: set RS_WARN_DEPRECATED back to ON

6a1d720

jolavillette mentioned this pull request Jun 2, 2026

C make migration modular cross platform RetroShare/RetroShare#3229

Open

jolavillette changed the title ~~C make migration modular cross platform~~ CMake migration modular cross platform Jun 2, 2026

jolavillette mentioned this pull request Jun 2, 2026

build(cmake): minimal fixes in CMakeLists.txt for transitive restbed headers, i2pcommon, and versioning #298

Closed

tempory fixes following gio comments on github

1f54e95

G10h4ck requested changes Jun 2, 2026

View reviewed changes

jolavillette force-pushed the CMakeMigration-Modular-CrossPlatform branch from a705f77 to 0c3a395 Compare June 2, 2026 21:53

jolavillette force-pushed the CMakeMigration-Modular-CrossPlatform branch from 0c3a395 to 8f55057 Compare June 2, 2026 22:11

tempory fixes RetroShare#2 following gio comments on github

2225487

jolavillette force-pushed the CMakeMigration-Modular-CrossPlatform branch from 8f55057 to 2225487 Compare June 2, 2026 22:40

jolavillette and others added 2 commits June 3, 2026 13:21

fix(cmake): apply WIN32_LEAN_AND_MEAN to the static lib too and drop …

8d09daf

…the sslfns.h X509_NAME/X509_EXTENSIONS #undef workaround

G10h4ck requested changes Jun 3, 2026

View reviewed changes

Comment thread CMakeLists.txt

Comment thread CMakeLists.txt

Comment thread CMakeLists.txt

fix(cmake): use explicit --exclude-libs list instead of ALL on Windows

29a6dd3

G10h4ck requested changes Jun 4, 2026

View reviewed changes

G10h4ck approved these changes Jun 5, 2026

View reviewed changes

G10h4ck merged commit e13bc4d into RetroShare:master Jun 5, 2026
1 check failed

jolavillette deleted the CMakeMigration-Modular-CrossPlatform branch June 5, 2026 05:35

Conversation

jolavillette commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Exit immediately if any command fails

Clean build directories if the "clean" argument is passed

Detect number of CPU cores for fast parallel compilation

Detect Qt5 path on macOS if Homebrew is installed

1. Compile libretroshare (core library) and librnp automatically

2. Compile retroshare-service (headless daemon)

3. Compile retroshare-friendserver

4. Compile retroshare-gui (graphical user interface)

Uh oh!

G10h4ck commented Jun 2, 2026

Uh oh!

G10h4ck left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jolavillette Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jolavillette Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jolavillette commented Jun 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jolavillette commented Jun 4, 2026

Uh oh!

G10h4ck commented Jun 4, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jolavillette commented Jun 2, 2026 •

edited

Loading

jolavillette Jun 2, 2026 •

edited

Loading

jolavillette Jun 2, 2026 •

edited

Loading