Added multithreaded lock test.

[korjaa]

This should detect issues with uneven locking and releasing.

Test flow:

1. Create 10 threads.
2. Each thread calls mbed_tracef() with different delays 10 times.
3. Join 10 threads.
4. Repeat 10 times.

[korjaa]

Now fails on:

CHECK((mutex_wait_count - mutex_release_count) > 0) failed

[jupe]

@tommikas could you check this out

[korjaa]

A bit worried that https://github.com/ARMmbed/mbed-trace/blob/master/test/Test.cpp#L31-L38 wait and releases are just counters and not real locks.

[korjaa]

The missing mutexes is going to be an issue, if this test is taken to use some modifications need to be done for the test mutexes.

[tommikas]

Now fails on:

CHECK((mutex_wait_count - mutex_release_count) > 0) failed

Off the top of my head I'm guessing that there are interleaved calls from more than one thread. When the first one exits the trace call it rolls back all the "mutex calls" from both threads. This shouldn't happen with a real mutex.

The missing mutexes is going to be an issue, if this test is taken to use some modifications need to be done for the test mutexes.

Yeah, with multiple threads it needs a real mutex. Should be no problem just initializing one and adding the calls to the wait and release functions. The counters should still work.

The original test just ensured that for all the tested calls the library releases the mutex as many times as it locks it.

[tommikas]

@korjaa Even if the original problem has been resolved it wouldn't hurt to have this kind of test in. Would you mind adding the mutex to the wait and release functions?

[korjaa]

"Off the top of my head I'm guessing..."
Yes, the test was flawed to begin with :) And the counters should work with the correct mutexes.

"...to have this kind of test in"
I think in the case of deadlock the test will just get stuck and not return FAIL. Which I guess then would need more restructuring. Feels complicated. :(

[tommikas]

A deadlock shouldn't be too hard to catch. Use a "ready to join" flag for each thread to know when to call pthread_join on them and catch a deadlock either by simple timeout or by polling the counters for a (lack of) change from main thread. A timeout could also catch a livelock, if that could somehow happen.

[korjaa]

Awesome, you already seem to have an idea how to solve this ;)

[jupe]

looks amazing