Verify Msys2 build on Windows

[MGN-GIT]

Hi Developers

Environment

• Platform: Windows x86
• Build: MSYS compiled from source
• Test suite: winsup test suite

Background

I built MSYS from source on a Windows x86 machine and want to validate my build.

Questions

1. How can I validate my build? Is there a recommended way to verify that a source build is correct and complete on Windows x86?
2. Is there a built-in test suite I can run to validate the build? I found the winsup test suite and ran it — is this the right one to use, or are there others I should be running?
3. Are the observed failures expected? I'm seeing 130 failures out of 282 tests. Are these known/expected on Windows x86, or do they indicate a problem with my build?

---

Full Test Log

PASS: winsup.api/checksignal.exe
PASS: winsup.api/crlf.exe
FAIL: winsup.api/devdsp.exe
PASS: winsup.api/devzero.exe
PASS: winsup.api/iospeed.exe
PASS: winsup.api/mmaptest01.exe
PASS: winsup.api/mmaptest02.exe
PASS: winsup.api/mmaptest03.exe
PASS: winsup.api/mmaptest04.exe
FAIL: winsup.api/msgtest.exe
PASS: winsup.api/nullgetcwd.exe
PASS: winsup.api/resethand.exe
FAIL: winsup.api/semtest.exe
FAIL: winsup.api/shmtest.exe
PASS: winsup.api/sigchld.exe
PASS: winsup.api/signal-into-win32-api.exe
FAIL: winsup.api/systemcall.exe
PASS: winsup.api/user_malloc.exe
PASS: winsup.api/waitpid.exe
FAIL: winsup.api/ltp/access01.exe
FAIL: winsup.api/ltp/access03.exe
FAIL: winsup.api/ltp/access04.exe
FAIL: winsup.api/ltp/access05.exe
PASS: winsup.api/ltp/alarm01.exe
PASS: winsup.api/ltp/alarm02.exe
PASS: winsup.api/ltp/alarm03.exe
PASS: winsup.api/ltp/alarm07.exe
FAIL: winsup.api/ltp/asyncio02.exe
PASS: winsup.api/ltp/chdir02.exe
FAIL: winsup.api/ltp/chdir04.exe
FAIL: winsup.api/ltp/chmod01.exe
FAIL: winsup.api/ltp/chmod02.exe
FAIL: winsup.api/ltp/chown01.exe
FAIL: winsup.api/ltp/close01.exe
PASS: winsup.api/ltp/close02.exe
FAIL: winsup.api/ltp/close08.exe
FAIL: winsup.api/ltp/creat01.exe
FAIL: winsup.api/ltp/creat03.exe
FAIL: winsup.api/ltp/creat09.exe
FAIL: winsup.api/ltp/dup01.exe
FAIL: winsup.api/ltp/dup02.exe
FAIL: winsup.api/ltp/dup03.exe
FAIL: winsup.api/ltp/dup04.exe
FAIL: winsup.api/ltp/dup05.exe
FAIL: winsup.api/ltp/execl01.exe
FAIL: winsup.api/ltp/execle01.exe
FAIL: winsup.api/ltp/execlp01.exe
FAIL: winsup.api/ltp/execv01.exe
FAIL: winsup.api/ltp/execve01.exe
FAIL: winsup.api/ltp/execvp01.exe
PASS: winsup.api/ltp/exit01.exe
FAIL: winsup.api/ltp/exit02.exe
FAIL: winsup.api/ltp/fchdir01.exe
FAIL: winsup.api/ltp/fchdir02.exe
FAIL: winsup.api/ltp/fchmod01.exe
FAIL: winsup.api/ltp/fchown01.exe
FAIL: winsup.api/ltp/fcntl02.exe
FAIL: winsup.api/ltp/fcntl03.exe
FAIL: winsup.api/ltp/fcntl04.exe
FAIL: winsup.api/ltp/fcntl05.exe
FAIL: winsup.api/ltp/fcntl07.exe
FAIL: winsup.api/ltp/fcntl07B.exe
FAIL: winsup.api/ltp/fcntl08.exe
PASS: winsup.api/ltp/fcntl09.exe
FAIL: winsup.api/ltp/fcntl10.exe
FAIL: winsup.api/ltp/fork01.exe
PASS: winsup.api/ltp/fork02.exe
PASS: winsup.api/ltp/fork03.exe
FAIL: winsup.api/ltp/fork04.exe
PASS: winsup.api/ltp/fork06.exe
FAIL: winsup.api/ltp/fork07.exe
FAIL: winsup.api/ltp/fork09.exe
FAIL: winsup.api/ltp/fork10.exe
PASS: winsup.api/ltp/fork11.exe
FAIL: winsup.api/ltp/fpathconf01.exe
FAIL: winsup.api/ltp/fstat01.exe
FAIL: winsup.api/ltp/fstat02.exe
FAIL: winsup.api/ltp/fstat03.exe
FAIL: winsup.api/ltp/fstat04.exe
FAIL: winsup.api/ltp/fsync01.exe
FAIL: winsup.api/ltp/ftruncate01.exe
FAIL: winsup.api/ltp/ftruncate02.exe
FAIL: winsup.api/ltp/ftruncate03.exe
PASS: winsup.api/ltp/getegid01.exe
PASS: winsup.api/ltp/geteuid01.exe
PASS: winsup.api/ltp/getgid01.exe
PASS: winsup.api/ltp/getgid02.exe
PASS: winsup.api/ltp/getgid03.exe
PASS: winsup.api/ltp/getgroups01.exe
PASS: winsup.api/ltp/getgroups02.exe
PASS: winsup.api/ltp/gethostid01.exe
PASS: winsup.api/ltp/gethostname01.exe
PASS: winsup.api/ltp/getpgid01.exe
PASS: winsup.api/ltp/getpgid02.exe
PASS: winsup.api/ltp/getpgrp01.exe
PASS: winsup.api/ltp/getpid01.exe
PASS: winsup.api/ltp/getpid02.exe
PASS: winsup.api/ltp/getppid01.exe
PASS: winsup.api/ltp/getppid02.exe
PASS: winsup.api/ltp/getuid01.exe
PASS: winsup.api/ltp/getuid02.exe
PASS: winsup.api/ltp/getuid03.exe
PASS: winsup.api/ltp/kill01.exe
PASS: winsup.api/ltp/kill02.exe
PASS: winsup.api/ltp/kill03.exe
PASS: winsup.api/ltp/kill04.exe
PASS: winsup.api/ltp/kill09.exe
FAIL: winsup.api/ltp/link02.exe
FAIL: winsup.api/ltp/link03.exe
FAIL: winsup.api/ltp/link04.exe
FAIL: winsup.api/ltp/link05.exe
FAIL: winsup.api/ltp/lseek01.exe
FAIL: winsup.api/ltp/lseek02.exe
FAIL: winsup.api/ltp/lseek03.exe
FAIL: winsup.api/ltp/lseek04.exe
FAIL: winsup.api/ltp/lseek05.exe
FAIL: winsup.api/ltp/lseek06.exe
FAIL: winsup.api/ltp/lseek07.exe
FAIL: winsup.api/ltp/lseek08.exe
FAIL: winsup.api/ltp/lseek09.exe
FAIL: winsup.api/ltp/lseek10.exe
FAIL: winsup.api/ltp/lstat02.exe
FAIL: winsup.api/ltp/mkdir01.exe
FAIL: winsup.api/ltp/mkdir08.exe
FAIL: winsup.api/ltp/mknod01.exe
FAIL: winsup.api/ltp/mmap001.exe
FAIL: winsup.api/ltp/mmap02.exe
FAIL: winsup.api/ltp/mmap03.exe
FAIL: winsup.api/ltp/mmap04.exe
FAIL: winsup.api/ltp/mmap05.exe
FAIL: winsup.api/ltp/mmap06.exe
FAIL: winsup.api/ltp/mmap07.exe
FAIL: winsup.api/ltp/mmap08.exe
FAIL: winsup.api/ltp/munmap01.exe
FAIL: winsup.api/ltp/munmap02.exe
PASS: winsup.api/ltp/nice05.exe
FAIL: winsup.api/ltp/open02.exe
FAIL: winsup.api/ltp/open03.exe
FAIL: winsup.api/ltp/pathconf01.exe
PASS: winsup.api/ltp/pause01.exe
PASS: winsup.api/ltp/pipe01.exe
PASS: winsup.api/ltp/pipe08.exe
PASS: winsup.api/ltp/pipe09.exe
PASS: winsup.api/ltp/pipe10.exe
PASS: winsup.api/ltp/pipe11.exe
PASS: winsup.api/ltp/poll01.exe
FAIL: winsup.api/ltp/read01.exe
FAIL: winsup.api/ltp/read04.exe
FAIL: winsup.api/ltp/readdir01.exe
FAIL: winsup.api/ltp/readlink01.exe
FAIL: winsup.api/ltp/readlink02.exe
FAIL: winsup.api/ltp/readlink03.exe
FAIL: winsup.api/ltp/rename01.exe
FAIL: winsup.api/ltp/rename02.exe
FAIL: winsup.api/ltp/rename08.exe
FAIL: winsup.api/ltp/rename10.exe
FAIL: winsup.api/ltp/rmdir01.exe
FAIL: winsup.api/ltp/rmdir04.exe
FAIL: winsup.api/ltp/rmdir05.exe
FAIL: winsup.api/ltp/sbrk01.exe
FAIL: winsup.api/ltp/select01.exe
FAIL: winsup.api/ltp/select02.exe
FAIL: winsup.api/ltp/select03.exe
PASS: winsup.api/ltp/setgid01.exe
XFAIL: winsup.api/ltp/setgroups01.exe
PASS: winsup.api/ltp/setpgid01.exe
FAIL: winsup.api/ltp/setregid01.exe
FAIL: winsup.api/ltp/setreuid01.exe
PASS: winsup.api/ltp/setuid01.exe
XFAIL: winsup.api/ltp/setuid02.exe
FAIL: winsup.api/ltp/signal03.exe
FAIL: winsup.api/ltp/stat01.exe
FAIL: winsup.api/ltp/stat02.exe
FAIL: winsup.api/ltp/stat03.exe
FAIL: winsup.api/ltp/stat05.exe
FAIL: winsup.api/ltp/stat06.exe
FAIL: winsup.api/ltp/symlink01.exe
FAIL: winsup.api/ltp/symlink02.exe
FAIL: winsup.api/ltp/symlink03.exe
FAIL: winsup.api/ltp/symlink04.exe
FAIL: winsup.api/ltp/symlink05.exe
PASS: winsup.api/ltp/sync01.exe
FAIL: winsup.api/ltp/sync02.exe
PASS: winsup.api/ltp/time01.exe
PASS: winsup.api/ltp/time02.exe
PASS: winsup.api/ltp/times01.exe
PASS: winsup.api/ltp/times02.exe
PASS: winsup.api/ltp/times03.exe
FAIL: winsup.api/ltp/truncate01.exe
FAIL: winsup.api/ltp/truncate02.exe
PASS: winsup.api/ltp/umask01.exe
PASS: winsup.api/ltp/umask02.exe
FAIL: winsup.api/ltp/umask03.exe
PASS: winsup.api/ltp/uname01.exe
FAIL: winsup.api/ltp/unlink05.exe
FAIL: winsup.api/ltp/unlink06.exe
FAIL: winsup.api/ltp/unlink07.exe
XFAIL: winsup.api/ltp/unlink08.exe
PASS: winsup.api/ltp/vfork01.exe
PASS: winsup.api/ltp/wait02.exe
PASS: winsup.api/ltp/wait401.exe
PASS: winsup.api/ltp/wait402.exe
FAIL: winsup.api/ltp/write01.exe
FAIL: winsup.api/ltp/write02.exe
FAIL: winsup.api/ltp/write03.exe
PASS: winsup.api/pthread/cancel1.exe
PASS: winsup.api/pthread/cancel10.exe
FAIL: winsup.api/pthread/cancel11.exe
PASS: winsup.api/pthread/cancel12.exe
PASS: winsup.api/pthread/cancel2.exe
PASS: winsup.api/pthread/cancel3.exe
PASS: winsup.api/pthread/cancel4.exe
PASS: winsup.api/pthread/cancel5.exe
PASS: winsup.api/pthread/cancel6.exe
PASS: winsup.api/pthread/cancel7.exe
PASS: winsup.api/pthread/cancel8.exe
PASS: winsup.api/pthread/cancel9.exe
PASS: winsup.api/pthread/cleanup2.exe
PASS: winsup.api/pthread/cleanup3.exe
PASS: winsup.api/pthread/condvar1.exe
PASS: winsup.api/pthread/condvar2.exe
PASS: winsup.api/pthread/condvar2_1.exe
PASS: winsup.api/pthread/condvar3.exe
PASS: winsup.api/pthread/condvar3_1.exe
PASS: winsup.api/pthread/condvar3_2.exe
PASS: winsup.api/pthread/condvar3_3.exe
PASS: winsup.api/pthread/condvar4.exe
PASS: winsup.api/pthread/condvar5.exe
PASS: winsup.api/pthread/condvar6.exe
PASS: winsup.api/pthread/condvar7.exe
PASS: winsup.api/pthread/condvar8.exe
PASS: winsup.api/pthread/condvar9.exe
PASS: winsup.api/pthread/count1.exe
PASS: winsup.api/pthread/create1.exe
PASS: winsup.api/pthread/create2.exe
PASS: winsup.api/pthread/equal1.exe
PASS: winsup.api/pthread/exit1.exe
PASS: winsup.api/pthread/exit2.exe
PASS: winsup.api/pthread/exit3.exe
PASS: winsup.api/pthread/inherit1.exe
PASS: winsup.api/pthread/join0.exe
PASS: winsup.api/pthread/join1.exe
PASS: winsup.api/pthread/join2.exe
PASS: winsup.api/pthread/mainthreadexits.exe
PASS: winsup.api/pthread/mutex1.exe
PASS: winsup.api/pthread/mutex1d.exe
PASS: winsup.api/pthread/mutex1e.exe
PASS: winsup.api/pthread/mutex1n.exe
PASS: winsup.api/pthread/mutex1r.exe
PASS: winsup.api/pthread/mutex2.exe
PASS: winsup.api/pthread/mutex3.exe
PASS: winsup.api/pthread/mutex4.exe
PASS: winsup.api/pthread/mutex5.exe
PASS: winsup.api/pthread/mutex6d.exe
PASS: winsup.api/pthread/mutex6e.exe
PASS: winsup.api/pthread/mutex6n.exe
PASS: winsup.api/pthread/mutex6r.exe
PASS: winsup.api/pthread/mutex7.exe
PASS: winsup.api/pthread/mutex7d.exe
PASS: winsup.api/pthread/mutex7e.exe
PASS: winsup.api/pthread/mutex7n.exe
PASS: winsup.api/pthread/mutex7r.exe
PASS: winsup.api/pthread/mutex8e.exe
PASS: winsup.api/pthread/mutex8n.exe
PASS: winsup.api/pthread/mutex8r.exe
PASS: winsup.api/pthread/once1.exe
PASS: winsup.api/pthread/priority1.exe
PASS: winsup.api/pthread/priority2.exe
PASS: winsup.api/pthread/rwlock1.exe
PASS: winsup.api/pthread/rwlock2.exe
PASS: winsup.api/pthread/rwlock3.exe
PASS: winsup.api/pthread/rwlock4.exe
PASS: winsup.api/pthread/rwlock5.exe
PASS: winsup.api/pthread/rwlock6.exe
PASS: winsup.api/pthread/rwlock7.exe
PASS: winsup.api/pthread/self1.exe
PASS: winsup.api/pthread/self2.exe
PASS: winsup.api/pthread/threadidafterfork.exe
PASS: winsup.api/pthread/tsd1.exe
XFAIL: winsup.api/samples/sample-fail.exe
PASS: winsup.api/samples/sample-pass.exe
FAIL: mingw/cygload
============================================================================
Testsuite summary for Cygwin 0
============================================================================
# TOTAL: 282
# PASS:  148
# SKIP:  0
# XFAIL: 4
# FAIL:  130
# XPASS: 0
# ERROR: 0

[dscho]

@Greenie0701 which revision did you try to build? If it is anything newer than 3.3.*, x86 is simply no longer supported. That's an upstream Cygwin decision, nothing we here at MSYS2 can change.

[MGN-GIT]

Hi @dscho
Thanks for the clarification. I actually built the 64-bit target on a Windows x86_64 host system, not a 32-bit MSYS build itself.
The build completed successfully, but when running the winsup test suite I observed around 130 failures out of 282 tests, so I wanted to understand whether that level of failure is expected for MSYS2/Cygwin builds or indicates an issue in my build environment/configuration.
Apart from the winsup test suite, is there any other recommended way to validate that the build is correct and functioning properly?

[dscho]

I actually built the 64-bit target on a Windows x86_64 host system, not a 32-bit MSYS build itself.

Oh. You will want to get your nomenclature straight, then. x86 has never meant anything but 32-bit in MSYS2's context. Highly confusing.

The build completed successfully, but when running the winsup test suite I observed around 130 failures out of 282 tests, so I wanted to understand whether that level of failure is expected for MSYS2/Cygwin builds or indicates an issue in my build environment/configuration.

I encourage you to look at https://github.com/msys2/msys2-runtime/blob/msys2-3.6.9/.github/workflows/build.yaml#L93-L94 and https://github.com/msys2/msys2-runtime/actions/runs/26057101212/job/76609386389 to see how MSYS2 runtime validates new builds.

As to the test suite: It had been broken for Cygwin for a long time, but IIRC last year a large push concluded toward letting it pass again. This push has not permeated to the MSYS2 runtime yet.

If you can investigate, and even fix, a couple of those tests, that would be a cool contribution to the project.

[MGN-GIT]

Hi @dscho

Thanks for the clarification regarding the test suite status. I did some more investigation into the failing tests and found that many of them seem to be related to the MSYS2 installation root detection logic rather than the tests themselves.
I looked at init_cygheap::init_installation_root() in winsup/cygwin/mm/cygheap.cc. Around line 223, under MSYS, the code walks back two additional directory levels from the DLL location:

#ifdef __MSYS__
  /* Back two folders to get root as we have all stuff in usr subfolder */
  for (int i = 1; i >= 0; --i)
#endif

In my build layout, this causes the installation root to be resolved one extra directory back. I modified the logic that only two directories are stripped in total instead of three.
After making that change, the test results improved significantly:

• Before: ~148 passing tests out of 281
• After: ~258 passing tests out of 281

This makes me think that many of the failures were caused by incorrect installation root/path resolution rather than testsuite runtime issues.

I have a few questions before I continue investigating the remaining failures:

• Are some of the winsup tests Cygwin-specific and therefore expected to fail under MSYS2?
• Or is the long-term goal for MSYS2 to pass most (or all) of the same tests that pass in Cygwin?
• Does my observation about the installation root logic sound like something worth investigating further?
• Would you be interested in a patch and a list of the tests that started passing after this change?

If useful, I can share the exact patch and a breakdown of the affected tests.

[dscho]

This makes me think that many of the failures were caused by incorrect installation root/path resolution rather than testsuite runtime issues.

That makes sense. An even easier fix than patching the level number would be to choose the location when running tests accordingly.

• Are some of the winsup tests Cygwin-specific and therefore expected to fail under MSYS2?

Quite possible. I have not looked at them lately, but I seem to remember that there were some tests of cygserver, which might not even be compiled by default in MSYS2 (that I don't remember).

• Or is the long-term goal for MSYS2 to pass most (or all) of the same tests that pass in Cygwin?

The goals of MSYS2 and Cygwin are slightly different. In many ways, you could say that MSYS2's goals enclose Cygwin's (to provide a POSIX emulation layer, a toolchain to target that, an associated package management ecosystem). However, MSYS2 also wants to play better with non-MSYS programs, whereas Cygwin really strongly prefers to interact only within the Cygwin ecosystem.

• Does my observation about the installation root logic sound like something worth investigating further?

Sure!

• Would you be interested in a patch and a list of the tests that started passing after this change?

I'd welcome this. Maybe we'll even end up being able to run this in PR/CI!

If useful, I can share the exact patch and a breakdown of the affected tests.

That would be nice. I'd recommend starting by reinstating the cygwin.yml GitHub workflow, opening a PR, to have a baseline of everything that fails, then push fixes to demonstrate that the PR build fails "less and less".

Sound like an idea?

[aswin-mcw]

Hi @dscho @Greenie0701,

Following up on this — picked up the investigation and suggestion above and opened
two PRs to act on it:

• Cygwin: align testsuite tmp directory with runtime path resolution #343 — Cygwin: align testsuite tmp directory with runtime path resolution
  Fixes the installation-root resolution issue @Greenie0701 identified
  (init_cygheap::init_installation_root() stripping an extra directory level),
  by aligning the testsuite's tmp directory with the runtime's path resolution.
• github-actions: add Windows CI for Cygwin testsuite #344 — github-actions: add Windows CI for Cygwin testsuite
  Reinstates the CI workflow to run the testsuite on every PR, plus XFAIL
  annotations for tests that fail due to known platform limitations
  (chmod semantics, symlink support gaps) and a couple of syscall/test-setup
  fixes (UNC-style path generation, busybox setup).

With both applied, the testsuite results on MSYS2/x64 are now:
TOTAL: 282
PASS: 264
SKIP: 0
XFAIL: 14
FAIL: 4
XPASS: 0
ERROR: 0

The remaining 4 failures all appear to be related to cygserver and mingw cross toolchain — I am still
debugging those and will follow up once we have more clarity. Will also link
both PRs here for tracking.

Thanks again for pointing us toward reinstating the CI workflow as a way to
demonstrate the build "failing less and less" — that framing made it a lot
easier to structure the work.

[aswin-mcw]

Hi @dscho,
Just following up on this — wanted to check if you've had a chance to look at the two PRs (#343 and #344) linked above. Happy to make any changes needed, or clarify anything about the approach. Let me know if there's anything required from my end.
Thanks!