Obscure audio codec is a perfect fit for the N64

phoboslab.org

264 points by frostbyte7 2 days ago


midnightclubbed - 2 days ago

This codec would have been great 27 years ago! I suspect it wouldn't have gotten past the patent lawyers in 1999 though. It seems cute today but the entire codec space was a patent minefield, the N64 mp3 decoder was implemented twice (once based on the reference codec, once clean) with Nintendo/Rare then paying RAD game tools to license their decoder... of which none of their code was used but they had a license from the mp3 patent holder (Fraunhofer) which then shielded Nintendo from lawsuits. Fraunhofer wanted something like $250k to license directly which wasn't going to fly.

I'll also note the frame rate issues in Conker were mainly down to raw RDP fillrate issues, which was for sure the weakest part of the N64 hardware, it could draw beautiful triangles (for the time) but oh so so slowly. There was a decent amount of speech during gameplay using the RSP mp3 codec. The cartridge size was still an issue, 64Mb got a good amount of mono speech but was still very much at a premium.

jonas21 - 2 days ago

In case anyone else got to the demo thinking the quality is awful -- the vocals are incomprehensible and keep cutting out -- it turns out it's like that in the original track:

https://www.youtube.com/watch?v=i1gVxKhdGPs&t=57s

hxorr - 2 days ago

Great read, thanks!

Pardon my ignorance but one thing I'm curious about, if the decoder fills almost the entire 4k RSP instruction memory (IMEM) - wouldn't that be something games need for graphics? If so, then is this still usable for say a game?

Sesse__ - 2 days ago

I wonder if this would also be a great fit for the MAS3507D! Supposedly you can replace the program with your own in a 1-kiloword RAM (the default MP3 decoder is in ROM, and probably larger). If you can fit this codec in 1024 MIPS instructions, perhaps you can do the same in 1024 of these DSP instructions…

Too bad there is seemingly zero public documentation on the format of these programs, though (and no emulator or similar).

gatane - 2 days ago

It was all great until they mentioned LLM usage.

The demoscene and homebrew scene is dying as we speak.

pona-a - 2 days ago

Beautiful! I long wanted to write my own audio codec, this might be a nice challenge.

pferdone - 2 days ago

Sorry but what? Half-Life on an N64?? it looks incredible!

peter_d_sherman - 14 hours ago

>"Time-domain codecs (such as ADPCM) need significantly higher bitrates for this quality. Subband codecs, such as MP2, have much worse quality at comparable bitrates, and more “modern” transform codecs, such as MP3, usually need at least an 80–100 MHz CPU. For this GBA demo, no existing codec would fit the bill.

So Aikku93 not only wrote a decoder in ARM assembly, but also created a whole new codec: The Ultra Low Complexity Codec (ULC).

ULC is a transform codec and somewhat similar to MP3. ULC transforms slices of PCM data into the frequency domain. Like MP3, it uses the Modified Discrete Cosine Transform (MDCT) to do so. But unlike MP3, it does not split the signal into multiple bands (ULC's MDCT is “full-band”). Where MP3 uses classic entropy encoding (Huffman specifically), ULC does… not. Instead, transformed coefficients are stored with a very simple nibble-based syntax.

In short: ULC is what MP3 could have been. The same core idea, but without the cruft around it."

Nice!

This would make a great audio compressor / decompressor to run on older computers, on older operating systems, on older more resource-constrained hardware (not just the N64) !

Anyway, really nice!

Related:

https://en.wikipedia.org/wiki/Modified_discrete_cosine_trans...

neuroelectron - a day ago

Didn't Top Gear Overdrive (1998) use MP3 tracks? It had several tracks by Grindstone that would play during races. The audio quality isn't that great but lyrics are mostly comprehendible.

https://www.youtube.com/watch?v=kyXsWd7DpEs

Wipeout 64 had some music by Fluke and Propellerheads but I'm pretty sure it used some tracker format to re-create the tracks from short recordings and honestly, they sound awful, not just because the audio quality but because the repetitiveness and lack of progression.

lanmu - 2 days ago

[flagged]

blenser4 - 2 days ago

[flagged]