Operation with OpenWebRX

[eroyee]

Dobrý deň Erik,

Thank you for your work on this, and that of Edouard as well.

Just letting you know that I have created an experimental module to incorporate SoapyMiri into OpenWebRX. It is working, but appears to have some minor issues which could be a result of my hardware, my module implementation, or possibly some feature of libmirisdr-4/SoapyMiri.

This message relates my experience to date: https://groups.io/g/openwebrx/message/5653

The repository is: https://github.com/eroyee/openwebrx_E (develop branch)

I have only been trialling this a short time, but it seems quite stable at 8MHz sample rate. I would be pleased to find an answer to the issues I've mentioned in my post - if you have any ideas it would be good hear, and likewise if I gain any insight I will pass it on.

Čau

[nmaster2042]

Hi.

I just make tests with this SoapyMiri and a MSI.SDR.

@eroyee here is my reply, let me know if I get a good understanding of your issues.

It's the firs time I'm using this driver and the libmirisdr-4.

My SDR software is Cubic, a reference in supporting SoapySDR.

1. I tried all sample rate: only one is causing issue, the 12 Mhz (RSP max seem to be 10) so I think it's something normal.
2. There is some spurious signals that can depend of the device or environment
3. My device seems to receive well at center frequency, see below.

So from my tests, the only issue it the 12Mhz SR, perhaps some other MiriSDR based devices support it.

I didn't tried with openWebRX right now but it's always a good idea to try with other softwares to see how the lib and driver are working.

I hope this will help you.

[ericek111]

Hello, guys! Thanks for reporting these issues and the thorough testing.

Indeed, the driver and the dongle are most stable at 8 MHz sampling rate. 10 MHz works reliably for me (for everyday SDR usage) and 12.096 MHz has been reported to work, too. It's usable for me, though there are some dropped samples. There's little benefit anyway, because the built-in filter only lets 8 MHz of RF through.

With my blue MSI.SDR dongle, there is a significant DC spike. That's completely normal and natural. To avoid it, you can use offset tuning.

I absolutely love this library. With this module, it made the world of SDR so much more enjoyable for me. I will definitely test it with OpenWebRX. I haven't used it in a while.

[nmaster2042]

@ericek111 : thank you very much for your work on this driver.

As SDRplay will drop support for RSP1 end of this year, it's very usefull to have your driver to continue to work with older devices.

I think I found something not working properly.

AGC isn't actually give any effect, tested on CubicSDR and SDR++.

By default it's activated be gain is very low.
Disable / enable AGC doesn't make any change.
The only way to get good reception is to disable AGC and manually set gain.

I don't know if it's a driver issue or lib.

[nmaster2042]

@eroyee : for for DC spike, I don't have on Cubic SDR but get one with SDR++.

I check iq correction and I can receive what is on center freq.

So I think it depends on SDR software used.

Does OpenWebRx has IQ correction implemented ?

If not, you'll need to do what @ericek111 suggested.

[eroyee]

Good morning, thanks for responding @nmaster2042 @ericek111 👍

That screenshot is useful. I wanted to trial this (blue MSi) dongle with another front-end software to see my issue was down to OpenWebRX, or something else, so spent some time yesterday with SDRAngel. However it kept failing with a mirisdr error when compiling, so maybe I'll try CubiSDR for now.

That should allow me to confirm my suspicions that the artifacts I'm seeing are mostly likely due to my hardware, not software - perhaps with the exception of the DC spike that is indeed manageable by offsetting.

Unfortunately I'm not especially well versed with how OpenWebRX manages IQ correction etc. I've done nothing with that part of the code, most of my work has been at the front end, and now a rather crude module (which will need some more work) to incorporate Erik's Soapy driver.

FYI did some more testing with this a few hours ago and now that I understand the correct settings I find that lower sample rates are generally ok, which is good as I suspect 8MHz may be a bit much for the Raspberry Pi that many people are using. I will see if I can trial that on a Pi sometime soon to confirm.

Overall this is a great thing; as well as SDRPlay dropping support for the RSP1 I imagine many people are using the SDRPlay API with their non-SDRPlay hardware, which is probably not the decent thing to do. Between the work of Edouard, Leif, Miroslav with libmiri, and now Erik implementing a Soapy driver there is no longer a reason for this.

Also, from past experience, my feeling is that with OpenWebRX this implementation does not fail when switching bands the same way as the SDRPlay API did. To be fair that's not a direct comparison since that was with a much earlier version of OpenWebRX, but so far it's been a fairly positive experience.

[eroyee]

Just following up with some further observations after fairly extensive testing between myself and another station. Apologies for some unscientific comments, sometimes that all that will do!

CubicSDR works fine, thanks, as far as I can tell it appears to display less artifacts than OpenWebRX, the audio is 'nicer' and there are less evident birdies over the spectrum. It will be evident that I don't know alot about this end of things but if I were to drill down the audio issue I'd say OpenWebRX sounds as if it's using a significantly lower sample rate, it's quite 'buzzy' in comparison.

However another OpenWebRX station, using the SDRPlay driver, also has better audio, and appears to generate fewer birdies.

Bear in mind that at this point my implementation of a soapyMiri module for OpenWebRX does nothing other than use the driver defaults, it doesn't set buffer sizes, nor the format type. I believe that, unless set, neither does the OpenWebRX SDRPlay module.

So, if I probe the soapyMiri driver I see, amongst other things:

Full-duplex: NO
Supports AGC: YES
Stream formats: CS16, CF32
Native format: CU16 [full-scale=4095]

Whereas the sdrplay driver shows:

Full-duplex: YES
Supports AGC: YES
Stream formats: CS16, CF32
Native format: CS16 [full-scale=32767]
(Interestingly is also shows ' Corrections: DC removal')

So I guess, without knowing exactly what CubiSDR passes to Soapy, nor how it goes about it's processing, what I'm wondering at this point is whether the native format/settings are what is responsible for the difference between the two OpenWebRX stations.

Anyway that's where I've got to right now. I need to get other things done so won't be able to get back to this until tomorrow at the earliest, but thought I'd pass on what I'd learnt at this point.

[eroyee]

So I've done a little more with trying to identify the source of the regular spurious signals I'm seeing with OpenWebRX and SoapyMiri (but not with CubicSDR and SoapyMiri, or OpenWebRX and SoapySDRPlay). I suspect these signals are probably related to the slightly 'buzzy' audio.

My knowledge of how this end of an SDR system works is very limited, and I've never really done anything with C++, but I tried to force CF32 (which is what SoapySDR reports it is using with OpenWebRX and SoapySDRPlay), unfortunately to no avail. That is to say it didn't make a difference, but then I don't know if I did it right.

I am not at all sure if the problem is due to the native format, or maybe a result of the DC signal (could that introduce artifacts?), or something else completely unrelated, but I wonder if you've had a chance to try this with OpenWebRX @ericek111 ? I think it needs an expert's ear :-)

[ericek111]

Indeed, the native format value was wrong. But I don't think that has any effect on the issues here mentioned. Anyway, I've fixed it (I think :D). Also the reported sample rates were incorrect. I think it depends on the individual apps to call either listSampleRates or getSampleRateRange, so some apps may have reported slightly incorrect sample rates.

I know that CF32 works reliably without any buzzing or crackling -- SDR++ uses it. But when I was still experimenting with this library, I observed these issues when samples were incorrectly copied into buffers (and were hitting max. values). I suspect it may be caused by OpenWebRX selecting CS16 (which hasn't been tested yet) for its sample format.

Some drivers do DC offset removal in software, for example the gr-osmosdr library: https://gist.github.com/ericek111/4e59b94ecfc20a182c2bb1c3fab5fa42

I don't think these features should be implemented in the layer that connects the driver to your app. Any digital-domain signal conditioning should either be done in the driver (and exposed in Soapy), or in the app.

[eroyee]

@ericek111 - thanks, I've compiled the changed version on a Pi4 I have here for testing. A Soapy probe now reports the native format as "CS16 [full-scale=65536]"

I agree with you that it wasn't likely to be the issue, however I've been surprised before! Also quite agree re the DC blocking, I see the SDRPlay library does that (which is what Soapy reported), whereas libmirisdr-4 doesn't.

Anyway I have to report that the changes don't seem to have altered anything. It's a struggle to report exactly what it's like - and it may well be a libmirisdr difference, and nothing you can do with the driver, but here's a couple of screenshots taken a couple of minutes apart. The only difference is the sdrplay api via SoapySDRPlay, or the libmirisdr library via SoapyMiri:

SoapySDRPlay

SoapyM

iri

As you'll see the spectrum is somewhat different. The significant signal around 3.37MHz in the SDRPlay screenshot is actually unwanted (ie. it's a harmonic or some sort of mixed/rectified signal from a local broadcast station), but it's quite missing with the Miri driver. Also the response is different, the spikes seem sharper with the Miri driver vs the SDRPlay driver, and there's more 'haze' to the spectrum.

For the purpose of figuring this out I'd like to divorce libmirisdr-4 from SoapyMiri but, unfortunately, there's no direct connect facility with OpenWebRX for miri_sdr like there is for rtl_sdr so I can't remove SoapyMiri from the mix at this point. I did try rewriting the current OpenWebRX direct rtl_sdr connector to work with miri_sdr, however C++ isn't my thing and I've come to a halt on that...

It's possible @f4exb may have some insight on this, otherwise I'm out of ideas at this point, sorry. The good news is that in every other respect it's working quite well 👍

[ericek111]

Good job with the SoapyMiri driver! It works out of the box for me. However, I believe there shouldn't be yet another layer. OWRX should interface with Soapy and use its functions. If you could abstract all libmirisdr-related stuff away and query the Soapy module for any capabilities and settings, that would be better.

I managed to get OpenWebRX running (after updating all its dependencies in the AUR) and I don't see any significant difference in the FM broadcast band between sdrplay and soapyMiri. Except one uses the gain reduction method (so the gain is "inverted" -- 10 is more gain than 30).

What sample and gain mode for SDRplay are you running, please? Honestly, in the screenshot it looks like the native driver has set the filter to maybe one level lower. If you could test it in SDR++, that would be perfect.

[eroyee]

Thanks, pleased that module worked ok for you. I've implemented all the additional args mentioned in the info text (offset_tune, bufflen, buffers, asyncBuffs) but haven't really messed with them - or the gain - much.

I may have not been clear with what I meant earlier, sorry. For OpenWebRX there is a separate rtl_sdr connector that works directly with librtlsdr - SoapySDR is not part of this. I was thinking that if I could have ported that connector over to work directly with libmirisdr-4 (thus leaving SoapySDR out of the loop) then I'd have been able to see if the issues I was experiencing were anything to do with the Soapy stack (incl SoapyMiri), or maybe something related to libmirisdr-4. At the moment I remain uncertain of where it's at.

With the sample rate I have tried 1.536, 5, and 8 MHZ, and left the gain mode entirely alone (for both SoapySDRPlay and SoapyMiri). I did do something with gain at one point but it was early on and I don't now recall the outcome, I shall try that again. Will also look SDR++, but am presently trying to get sdrangel going, which is taking some time!

As an aside, I ran some load tests on a Pi4 here. I doubt they're of much use in diagnosing anything but they may be of some interest to you?:

PID USER        PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND   

rtl_sdr via Soapy,  1.024MHz sample rate, listening 3.76MHz:
26397 openweb+  20   0 1742740  76632  34720 S  35.5   1.0   0:43.59 openwebrx                                                                                                             
26432 openweb+  20   0  381408  40356  28652 S   4.6   0.5   0:05.21 soapy_connector  

rtl_sdr *not* via Soapy,  1.024MHz sample rate, listening 3.76MHz:
22216 openweb+  20   0 2057432  88940  48048 S  50.8   1.1   7:06.64 openwebrx                                                                                                             
22259 openweb+  20   0  195124  14424  13928 S   5.9   0.2   0:49.31 rtl_connector 

rtl_sdr via Soapy,  1.536MHz sample rate, listening 3.76MHz:
26397 openweb+  20   0 1758872  79776  34548 S  49.0   1.0  16:57.16 openwebrx                                                                                                             
28112 openweb+  20   0  389604  40500  28800 S   7.2   0.5   1:30.32 soapy_connector   

rtl_sdr *not* via Soapy,  1.536MHz sample rate, listening 3.76MHz:
32668 openweb+  20   0 1289616  74768  34984 S  50.0   0.9   0:41.38 openwebrx                                                                                                             
32696 openweb+  20   0  195124  14452  13952 S   9.5   0.2   0:05.12 rtl_connector  

sdrplay, via Soapy, 1.536MHz sample rate, listening 3.76MHz:
25223 openweb+  20   0 1207688  75760  34760 S  63.5   0.9   1:01.87 openwebrx                                                                                                             
25262 openweb+  20   0  458068  66092  28504 S  34.2   0.8   0:29.47 soapy_connector 

soapyMiri, via Soapy, 1.536MHz sample rate, listening 3.76MHz:
24431 openweb+  20   0 1209624  73308  33872 S  76.8   0.9   1:47.43 openwebrx                                                                                                             
24451 openweb+  20   0  372584  33848  27848 S  42.5   0.4   0:58.86 soapy_connector  

sdrplay, via Soapy, 1.536MHz sample rate, listening 3.67MHz:
69733 openweb+  20   0 1207688  75456  34440 S  63.2   0.9   0:50.95 openwebrx                                                                                                             
69770 openweb+  20   0  458072  65992  28404 S  33.6   0.8   0:23.75 soapy_connector   

'new' soapyMiri, via Soapy, 1.536MHz sample rate, listening 3.67MHz:
70356 openweb+  20   0 1207432  73852  34176 S  77.3   0.9   1:07.07 openwebrx                                                                                                             
70369 openweb+  20   0  372440  33832  27856 S  43.1   0.4   0:35.21 soapy_connector  

All with OpenWebRX, lsb, same hardware etc.