Cheapo HDMI to USB Capture Card not usable

[mofosyne]

Got a cheapo HDMI to USB capture card

Bus 001 Device 014: ID 345f:2109 MACROSILICON 

It's viewable under Cheese photo app, when selecting for 2109 (V4L2) in Preferences.

But in this app, i see only two video resolutions under CV_8U/16 in Option/"Video Devices"

System Configuration (please complete the following information):

• OS: Linux Mint 22.3 x86_64
• Version [e.g. v1.5.2]

Additional context

In the console for kvm-gui -v I can see:

Enumerating cameras (checking indices 0-4)...
[ WARN:0@7.719] global cap_v4l.cpp:914 open VIDEOIO(V4L2:/dev/video1): can't open camera by index
[ WARN:0@7.719] global cap.cpp:478 open VIDEOIO(V4L2): backend is generally available but can't be used to capture by index

So sounds like it's trying to enumerate the V4L2 hdmi to usb card, bt is somehow failing to be indexed.

This is no issue with the virtual keyboard control however... was able to confirm using CHEESE gnome app to view the screen and then used the kvm-gui to send the keystrokes (even when i can't see the screen via kvm-gui)

[sjmf]

Hey - thanks for the report. I'm currently away and can't experiment, but this sounds like it may be a little difficult to address without further information about your setup.

My first thought was a permissions error: it's seeing /dev/video1 but can't open it? But that doesn't make much sense given that Cheese can open it. Could you check whether Linux Mint 22.x has separate device accessibility permissions like Ubuntu 24.x does? (That is – if I'm remembering that correctly: I have used both distros, but not as a daily driver).

The other thought is that it's enumerated at a higher index for some reason, and perhaps tweaking the constant in video.py upwards might successfully enumerate it on another index? If you're using the python version, this should be as simple as editing:

kvm-serial/kvm_serial/backend/video.py

Line 13 in 96f456b

CAMERAS_TO_CHECK = 5 # Reduced from 10 - most users have 0-2 cameras

up to a higher value. That said – again that doesn't make much sense to me, as the error says: "can't open camera by index".

What we need, ideally, is a minimal working example using OpenCV, to attempt to open your device. Are you confident with Python to be able to create such an example?

[mofosyne]

Tried to figure out cv2 camera stream detection... seems rather limited. Also unfamiliar with cv2.

Decided to get LLMs to suggest some code and it suggest talking directly to video4linux kernel module and suggested below code (Added comments to make it easier to read):

import cv2
import fcntl
import struct
import os

# VIDIOC_QUERYCAP exact hex value is from https://github.com/jerome-pouiller/ioctl/blob/master/README.md
VIDIOC_QUERYCAP = 0x80685600

def v4l2_query(path):
    try:
        with open(path, 'rb') as f:

            # https://www.kernel.org/doc/html/v4.8/media/uapi/v4l/func-ioctl.html
            buf = bytearray(104)
            fcntl.ioctl(f.fileno(), VIDIOC_QUERYCAP, buf, True)

            # https://github.com/torvalds/linux/blob/master/include/uapi/linux/videodev2.h#L470
            '''
            struct v4l2_capability {
                __u8	driver[16];    // driver:       name of the driver module (e.g. "bttv")
                __u8	card[32];      // card:         name of the card (e.g. "Hauppauge WinTV")
                __u8	bus_info[32];  // bus_info:     name of the bus (e.g. "PCI:" + pci_name(pci_dev) )
                __u32   version;       // version:      KERNEL_VERSION
                __u32	capabilities;  // capabilities: capabilities of the physical device as a whole
                __u32	device_caps;   // device_caps:  capabilities accessed via this particular device (node)
                __u32	reserved[3];   // reserved:     reserved fields for future extensions
            };
            '''
            driver   = bytes(buf[0:16]).rstrip(b'\x00').decode(errors='replace')
            card     = bytes(buf[16:48]).rstrip(b'\x00').decode(errors='replace')
            bus_info = bytes(buf[48:80]).rstrip(b'\x00').decode(errors='replace')

            version  = struct.unpack_from('<I', buf, 80)[0]
            caps     = struct.unpack_from('<I', buf, 84)[0]
            dev_caps = struct.unpack_from('<I', buf, 88)[0]

            return driver, card, bus_info, version, caps, dev_caps

    except OSError:
        return None


devices = []

for i in range(10):
    path = f'/dev/video{i}'
    if not os.path.exists(path):
        continue

    info = v4l2_query(path)
    if not info:
        continue

    driver, card, bus_info, version, caps, dev_caps = info

    cap = cv2.VideoCapture(i, cv2.CAP_V4L2)
    if not cap.isOpened():
        continue

    w   = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    h   = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
    fps = cap.get(cv2.CAP_PROP_FPS)
    cap.release()

    devices.append({
        'index': i,
        'path': path,
        'card': card,
        'driver': driver,
        'bus_info': bus_info,
        'width': w,
        'height': h,
        'fps': fps
    })

print(f"Found {len(devices)} capture device(s):\n")

for d in devices:
    print(f"  {d['path']}  [{d['index']}]")
    print(f"    Card    : {d['card']}")
    print(f"    Driver  : {d['driver']}")
    print(f"    Bus     : {d['bus_info']}")
    print(f"    Default : {d['width']}x{d['height']} @ {d['fps']}fps")
    print()

Above will output:

[ WARN:0@0.301] global ./modules/videoio/src/cap_v4l.cpp (902) open VIDEOIO(V4L2:/dev/video1): can't open camera by index
[ WARN:0@0.721] global ./modules/videoio/src/cap_v4l.cpp (902) open VIDEOIO(V4L2:/dev/video3): can't open camera by index
[ WARN:0@1.100] global ./modules/videoio/src/cap_v4l.cpp (902) open VIDEOIO(V4L2:/dev/video5): can't open camera by index
Found 3 capture device(s):

  /dev/video0  [0]
    Card    : Logitech BRIO
    Driver  : uvcvideo
    Bus     : usb-0000:0e:00.0-5
    Default : 640x480 @ 30.0fps

  /dev/video2  [2]
    Card    : Logitech BRIO
    Driver  : uvcvideo
    Bus     : usb-0000:0e:00.0-5
    Default : 340x340 @ 30.0fps

  /dev/video4  [4]
    Card    : UVC Camera (345f:2109): USB Vid
    Driver  : uvcvideo
    Bus     : usb-0000:0e:00.0-6.2
    Default : 640x480 @ 30.0fps

Apparently there is a specific library to query it via this python module v4l2py which would be more robust than relying on whatever OpenCV is trying to do.

If you got a script, I can test further but hope this gives extra clues.

[sjmf]

This is great – thanks so much for investigating. It looks like none of the video devices on your system were openable by CV2, even though they are enumerable. Video4Linux is certainly an avenue to explore, but the reason to use openCV is that it works cross platform. The less platform-dependent code we have to write, the better (this is Python after all, but it is true that it's in these cases where the abstraction breaks down). I wonder: is your user in the `video` group? Can you try: `sudo usermod -a -G video $USER` and re-run?

[mofosyne]

I wonder: is your user in the video group? Can you try: sudo usermod -a -G video $USER and re-run?

Ran kvm-gui again, no luck.

[sjmf]

It's possible that PR #34 might fix this issue too, by re-writing the video device code to use platform-specific functionality. Currently in testing phase.

[sjmf]

v1.5.5 is out! And it switches the video backend out entirely.

@mofosyne - Could you test your device against that release, and see if the problem you're having here is resolved?

[mofosyne]

Video feed finally working, but looks rather low resolution.

vs cheese app

Video is finally working. I do notice a half second lag between mouse and the motion in the screen but I think that's on the serial cable side as I see the lag on the test laptop as well.

Here is the log in case it helps

$ kvm-gui
/dev/ttyS31 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS30 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS29 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS28 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS27 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS26 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS25 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS24 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS23 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS22 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS21 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS20 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS19 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS18 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS17 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS16 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS15 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS14 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS13 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS12 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS11 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS10 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS9 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS8 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS7 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS6 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS5 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS4 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS3 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS2 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS1 could not be opened: Could not configure port: (5, 'Input/output error')
/dev/ttyS0 could not be opened: Could not configure port: (5, 'Input/output error')
Found serial ports: ['/dev/ttyUSB0']
Unable to query the parameter info: QCameraImageProcessingControl::WhiteBalancePreset : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::ColorTemperature : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::SharpeningAdjustment : "Invalid argument"
Found 3 cameras via QtMultimedia.
Found video devices: ['Logitech BRIO', 'Logitech BRIO', 'UVC Camera (345f:2109): USB Vid']
Using 19 cached resolutions for device at position 0
Using 19 cached resolutions for device at position 0
Camera Logitech BRIO: no supported viewfinder settings for 1920x1080; letting Qt choose
Auto-detected keyboard layout: en_GB
Verbose logging disabled.
Opened serial port /dev/ttyUSB0 at 9600 baud
Initialised keyboard and mouse operations
Settings loaded from configuration file.
Camera UVC Camera (345f:2109): USB Vid: no supported viewfinder settings for 1920x1080; letting Qt choose
Selected video device: UVC Camera (345f:2109): USB Vid (1920x1080)
Using 11 cached resolutions for device at position 2
Unable to query the parameter info: QCameraImageProcessingControl::WhiteBalancePreset : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::ColorTemperature : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::SharpeningAdjustment : "Invalid argument"
Video view focused - keyboard capture enabled
Mouse LEFT pressed at 449,464
Mouse LEFT released at 448,465
Video view unfocused - keyboard capture disabled
Camera UVC Camera (345f:2109): USB Vid: no supported viewfinder settings for 1920x1080; letting Qt choose
Selected resolution: 1920x1080
Unable to query the parameter info: QCameraImageProcessingControl::WhiteBalancePreset : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::ColorTemperature : "Invalid argument"
Unable to query the parameter info: QCameraImageProcessingControl::SharpeningAdjustment : "Invalid argument"
Video view focused - keyboard capture enabled
Mouse LEFT pressed at 447,466
Mouse LEFT released at 447,466
Mouse LEFT pressed at 199,48
Mouse LEFT released at 199,48
Mouse LEFT pressed at 199,48
Mouse LEFT released at 199,48
Video view unfocused - keyboard capture disabled
Video view focused - keyboard capture enabled
Video view unfocused - keyboard capture disabled
Video view focused - keyboard capture enabled
Closed serial port connection
Video view unfocused - keyboard capture disabled

Is there also a way to decrease the mouse lag?

[sjmf]

Per the status bar, it's running at 720x480. Can you go into the Options->Resolution menu and select a higher resolution? The latest release adds in the ability to select this by hand if the resolution that Qt selects doesn't match the best resolution your device supports.

The mouse lag is a bit frustrating. I've never noticed serial lag before when using such devices.

That said – is there mouse lag in Cheese too? I've found that super cheap HDMI USB capture devices really do just have a significant video lag in hardware, and it can be worth spending a bit more. I've documented that in README.md:

The more you spend on a capture device, the more responsive your video feed will likely be (to a point).

The OpenCV backend I've switched out had hardware-accelerated rendering via a separate QThread, which improved performance greatly. The Qt5 QVideoPainter approach I've switched to only accepts a reduced range of formats reliably, meaning it's possible the update may have killed performance of the video view on some systems? That said, it seemed to be working reasonably well for me during testing, and the benefit was that we get all the device information without having to write an enormous library of per platform code to handle AVFoundation, DirectShow, and V4L (which wasn't working out well and would have introduced a pile of technical debt).

Qt6 has improved the video rendering situation a lot - there's a possible future migration to that - but I don't think there's much in terms of easy wins for improving performance right now. I could possibly expose video formats supported by the camera in another Options menu in case that helps on some systems, but there's a trade-off between usability and configurability, and most people probably won't care as long as it works.