HDMI grabbing?
These are devices you connect you a computer (might be a tiny one like a Raspberry Pi), and which convert HDMI input from a cable into something useful for further processing. I’m sharing here details from my experiments with 2 grab devices, and using Linux for processing.
So practically, what can I do with this?
My usecase is that I have just a single monitor, with just a single HDMI connector. I saw that HDMI grab devices came down to 1200円(~10€), I wondered how useful they were to grab/display/record games running on the Nintendo Switch.
Along my journey, I met many more use cases:
- Remote hands: Using a Raspi with HDMI-grabber, to control a nearby system: you can connect the Raspi also via serial connection, and via USB you can have the Raspi emulate USB-keyboard and maybe mouse. TinyPilot is such a project.
- Streaming:: grabbing HDMI from game consoles, one can stream to the internet, for example to Twitch. Grabbing the HDMI signal of your reflex camera and using it for video conferencing, you can get better quality than from the buildin camera of your laptop.
- Recording/processing: Digitilize videotapes in playing then on something with HDMI output, and record that. Also, retro game emulators now got OCR features which read the screen and translate to different languages. You could do this also with you HDMI stream, so for movies and so on.
The Hardware
My 2 devices:
- The H2C-RPI-B01, an addon board for the Raspi. With the Toshiba TC358743, it can directly encode to x264. I was not able to grab audio with this one. Price tag: ~1500円/12€.
- A USB-grabber, 1190円/10€ from Aliexpress. I have no more specific name for it, it’s available from multiple vendors. usb id 534d:2109.
Setup and Limitations
Both grabbers, with 2 HDMI sources, and a system receiving the grabbed data:
- MIPI CSI-2: protocol for point-to-point transmissions. Ribbon cable, for normal Raspi 2 data lanes, each up to 1Gbit/s. The H2C-RPI-B01 can encode to x264 directly, putting less traffic onto the CSI-2 cable - but driving latency up.
- Raspi4 connectors: Raspi4 has GBit ethernet (800mbit/sec at 0.3ms roundtrip time), and USB3 via usb-c connector (180mbit/sec at 0.3ms roundtrip time) details are here.
H2C-RPI-B01 board experiences
This was my first device, connected to the Raspi4 via ribbon cable, also Raspi cameras get connected like this. The device is cheap, but took many weeks to arrive from China. Getting started is complicated by the 2 modes of operation which are possible: firmware- and kernelbased. Kernel based operation is newer/more promising, it requires to manually load the EDID data with the modes which the board offers to the HDMI source.
Grabbing is limited to 1920×1080@25Hz - which not all HDMI sources support. Also when using a Linux system as HDMI source, you might have to define an own display mode with ‘xrandr’, details to this and the other H2C-RPI-B01 details are in the wiki.
The Toshiba chip on the board can do x264 encoding, that keeps the load on the Raspi cpus down. In that combination, latency goes up though.
Several combinations for encoding, streaming and encoding are possible. I played with ffmpeg/ffplayer, mencoder/mplayer, vlc and yavta. First one should be clear about the goal: streaming? recording? Is latency relevant? Is it ok to have artifacts in the picture? The limiting resources are bandwidth between the single components which are involved (see graphic above)’, resolution/screen frequency of the HDMI source. For me, biggest limitation of the H2C-RPI-B01 was missing audio: it seems like one has to solder extra pins on the board, and connect them to the Raspi. Me and others also then got no sound, with amplification or such it might be doable - details are in in the wiki.
For recording to file, yavta proved
useful, for streaming ustreamer.
ffplay (from ffmpeg) or a webbrowser can be used as client.
Be sure to have a proper power source, this setup seems to need more than you get via usb-c
USB-grabber experiences
This was my second device - to also get audio. The software stacks and experiences from the H2C-RPI-B01 mostly apply again, but this device can not directly encode to x264. Thanks to a patch from @marcan42, also stereo works now. Same restriction to 1920x1080@25Hz here, and USB can here become bottleneck. usbtop can be used to monitor. This device is now so mainstream, also engadget reported - but for Wintendo only.
Latency wise, ~120ms are doable with the device. Further details regarding config are in the wiki.
Summary
I got the setup I was aiming at implemented, but latency turned out to high for playing. For many other use cases, the grabber devices run quite well with Linux. The config details for the devices are here and here.
Many HDMI sources like the Nintendo Switch can also provide 720p@60Hz, which both devices can deal with. If you need more than 1920x1080@25Hz, watch out for other devices.
Potential alternatives I haven’t yet played:
- 4k grabber device, @30Hz: aliexpress
- 1080p@60Hz: aliexpress, driver is now in upstream kernels
- USB3/4k/1080p@60Hz: aliexpress (via marcan)
If you have a Raspi, do not need audio and want to encode to x264 then the H2C-RPI-B01 can do that directly. For pure encoding the NVidia Jetson Nano 2GB ($60) has some impressive specs, encoding 4kp@30 live.
I keep a list of alternatives here. The next article has details regarding streaming and measuring latency.
Last modified on 2020-11-07