What is this about?

Rockbox is an OpenSource operating system for mp3 players. I had many mp3 players which were lacking functions which I would really have liked. Trivial things like allowing playback without pause between tracks, or loudness normalization. It does not stop there: Doom has been ported, one can play mod and sid files from the Commodore Amiga/C64 era, and much more.

Rockbox is a replacement firmware for various mp3 players. Much time for research is required to port to new mp3 players, some time ago I got a FiiO M3K when the Rockbox port appeared. My Rockbox/M3K notes are here.

Why code something for an mp3 player?

We see a trend of people collecting and coding for retro computers. The hardware running Rockbox is mostly to weak to run even Linux. So I wondered how much effort it would take me to so some basic graphical and sound things on the M3K.

Bad apple

..is in some areas of the internet almost a classic to be run on “unusual” devices, just do a video search for the term. The “default” animation is this, and it did not exist for the M3K - so I tried to port this.

Building Rockbox

Installation of Rockbox, if you just want to use Rockbox and not code, is simple: one has to install a bootblock on the mp3player, unzip a file onto a microsd-card, and that’s all.

For writing code, I started with doing an own build, so producing the contents of that zip-file for my mp3player myself from the sourcecode. Also quite easy with the instructions from the homepage:

• First one builds a toolchain for crosscompiling. My system here is x86_64/Fedora34, and the cpu in the M3K is mips.
• Then, checkout of the code via git, having PATH point at the just built crosscompiler
• run configure: this is asking for which target (mp3 player) you want to build for. Then run ‘make zip’, and the build of Rockbox appears.

Coding environment

When running “configure”, I noticed that also a simulator target was offered! So, after building that, I could run “rockbox” on my x86_64 system, it would present me an SDL-application, just like the real hardware! One can then on the emulated screen run code, just like on the real player. Only pitfall: performance on x86_64 is higher, which is relevant when playing animations.

Next, I had to choose between 2 pathes.

• Coding with lua: this is an interpreted language, one deploys a file.lua on the player, and executes it without extra compile steps. So, I could open a file in vim on the x86_64 system, make changes, and then run the script on the emulator without extra compiling. Lua is slower than pure C, but I found the functionality I needed like setting pixels and playing music, so I was going with this.
• Direct C: faster, needs extra compile step

First code

So, I started with modifying existing lua scripts from Rockbox, that worked nicely. Setting single pixels, check. Towards getting an animation onto the player, I implemented a simple picture displayer. Format of choice: bmp. Wikipedia has the details. Basically, in the file header one reads the X- and the Y-resolution for the encoded picture. Then the encoding starts, for all pixels, one after the other, the RGB-values, as one byte. Seeing the first bmp picture rendered on the M3K was nice :)

Towards animation

Next step towards the goal: showing pictures as fast as we can after each other, and see which speed we get with that. For this, I fetched above video with youtube-dl, separated sound into an mp3 file, and rendered all frames into single bmp pictures with ImageMagick. The players display is 240 pixels wide, so I converted the ~6500 frames into 240x120, and had the code show this as animation. The native video runs at 30 frames per second (fps), as expected, the result in the emulator was: 10 fps, and on the real hardware even slower. Well ok, this was just the first try.

Real animation

First idea for optimization: so far I was opening ~6500 files and closing them.. that should come down. Also something interesting: looking at the bmp files with “hexdump -C”, I noticed that all 3 bytes for RGB were at the same value, so storing that just one time would help. So, next try: putting all the pictures into one big file. I was basically writing an own data format here, so I also wrote C-code on the x86_64 system to convert these pictures into one big animation-file. When opening files in Lua, I had settled with reading the whole file and then parsing it - that did not work for this huge animation-file, out-of-memory.

Next idea: making “macro image files”, so taking 4 or 8 single frames, and putting them into a macro-image-file. Why 4 or 8? I found that files should stay below 500 or 700kb to not cause trouble when opening. The result was working, but not really giving more fps.

I experimented with dropping every second frame, but it was still to slow.

Diffs

Most pixels are actually not changing to the next frame. So next idea was to basically write a real animation file format, which is not painting a complete picture each time, but just processing a “list of changes from the last frame”. Conveniently, the first frame is also black. So with new C-code, I created 6500 diff-files, each one describing the diffs to the previous frame. Then describing 8 frames per file helped further. I did never look deeper into the common video formats, but I recall that in the past you could select via options after how many frames “a complete frame” should be used instead of “diffs”.

Sound

Animation got a bit better now, so I wanted the sound in. Simple example code playing mp3 worked. My animation code too, but not together.. turns out that memory was the issue, I had to take out optimizations which did cost memory.

Cutting resolution

I really did not want to.. but going down to 120x90 pixels made things really smooth, and while I have not implemented sleeps or framedrops to get the animation perfectly into sync, that works now at least to some degree.

A video of the result is here, you can get the code and instructions how to setup from github.

If anybody is interested into writing code for somewhat ressource constrained hardware, mp3 players and Rockbox are recommended :)

Last modified on 2021-08-09