Yes, this is Cortado running on Netscape 4.79:



Basically this means Cortado can be made run even on, uh, bad and slow Java virtual machines. No, the JVM included with Netscape isn't fast enough for smooth video playback even on this 3 GHz machine, but sound isn't crackling either.

This is mostly a quick response to http://www.kenpardue.com/blog/2009/07/25/back-on-open-video/ which e.g. reads "What I don’t understand, and what irks me so badly, is why H.264 is demonized so badly by the FLOSS community." Well, I'll try to shed some light on this.

• The problem with H.264 is much more severe than just the costs of the licensing fees. The exact pricing for 2011 and onwards may not yet be decided upon, but its absolutely problematic that there's a single entity (MPEG-LA) with the power to enforce licensing terms and prices. Even if the licensing fees in 2011 happen to be "cheap" there's zero guarantee it'll stay "cheap". When reading the currently available documents on H.264 licensing it appears clear that the overall strategy is to "be cheap now, then get more expensive over time" and while the milking part may be pushed to a later date it'll eventually happen.
• On the internet 100000 free-units is a joke. Every download counts, most downloads don't actually generate a new "customer" and a mildly interesting software will easily hit the 100000 downloads limit.  To make matters worse: How to count downloads if you don't restrict distribution of your software to your own sites?
• H.264 licensing terms are unfair to small players. There's a cap for annual licensing fees ($5 million) that big companies will easily hit, meaning each product shipping after the cap is basically free, while small companies not hitting the cap will bleed for every product delivered. This shifts market balance in favor of big players, hindering competition and innovation. The cap was $3.5 million in 2005-2006 and $4.25 million in 2007-2008 - so over time it got harder for small players to hit the maximum annual royality, not easier.
• The licensing terms are absolutely incompatible with free (speech) software. There are open-source implementation of MPEG formats, meaning you can download, alter and redistribute the source-code - but without a license from MPEG-LA you're not allowed to actually use that software, so you're denied a very essential FOSS right. Even if you had the right to use the open-source MPEG-compliant code you cannot transfer this right to somebody downloading your open-sourced code. You're always dependent on MPEG-LA licensing usage rights. Being at the mercy of a monopolistic 3rd party isn't exactly what FOSS is about.
• Given that that MPEG licensing is incompatible with FOSS the widespread use of such encumbered formats questions the viability of FOSS on the desktop. Surfing the internet and watching embedded videos isn't possible within an intact FOSS environment in an MPEG-dominated world, users always would depend on proprietary and non-open components components like Flash carrying a MPEG license.
• Currently everything hints at per-content licensing fees being considered at MPEG-LA. This means not only technology providers are at the mercy of MPEG-LA but also content producers and providers.

For me that's enough to answer the question of why we need to push free-for-all-under-all-circumstances-without-paperwork media  formats - and the list above most likely isn't even complete.

The current state of the Wikipedia article on Theora states:

I'd like to point out that

• Theora has lower computational complexity than H.264, thus the need for hardware acceleration may not be as dire
• Current netbooks usually have no H.264 acceleration at all due to Intel's choice of hardware components, meaning the computational complexity of H.264 directly impacts on those poor little machines
• Many software players don't use hardware acceleration even where available

"Hardware" acceleration is a much hotter topic in the realm of digital media players or mobile phones than it is in the "usual" computing environment. From what I understand work to optimize Theora for mobile applications has begun.

The Magazin für Computertechnik (c't) printed a 4-page long article about Thusnelda. Preparing the article and actually seeing it in stores was/is an awesome experience and I hope it helps making Theora a bit more widely known. I'd like to thank Volker Zota from c't magazine for making this possible.


edit: There are thumbnails of the article online. Personally I like what they did to the "Xiph.org scared fish(tm)" logo ;-)

Archive.org now has all moving images content also available as Ogg Vorbis + Theora. This is great!

This was a massive reencoding effort and the details of what tools were used are available at http://internetarchive.wordpress.com/2008/11/25/fast-and-reliable-way-to-encode-theora-ogg-videos-using-ffmpeg-libtheora-and-liboggz/

A setup to get the job done. However, after having had a closer look I discovered that the Vorbis encodings were done with e.g.

ffmpeg -y -i CapeCodMarsh.avi -vn -acodec vorbis -ac 2 -ab 128k -ar 44100 audio.ogg

Ooops. This means there's massive amount of content, encoded with libavcodec's Vorbis encoder. The only problem: That encoder is rather primitive and produces significantly inferior audio quality compared to libvorbis. And yup, the Archive.org encodings sound rather unpleasant despite not starving on bitrate.

How I wished ffmpeg would loudly complain whenever using unfinished/experimental/not-state-of-the-art encoders.

(updated again to reflect recent changes)

I have frequently been asked where to get that nice litte Ogg Theora/Vorbis streaming applet. Well, the version I'm working on used to be its source hosted in the Wikimedia SVN, but development recently was moved to http://git.xiph.org/?p=cortado.git and there's a prototype project page at http://www.theora.org/cortado/

A signed binary for Cortado is available at http://www.theora.org/cortado.jar - this one should always be rather recent.

By now I patched Wikimedia's Cortado/jheora fork to play back any valid Theora bitstream with 4:2:0 subsampling. This is mostly a matter of properly consuming bits from the bitstream used for non-VP3 features (e.g. per-block qi indices).

Also, I hardened jheora and the jst-component using jheora to not simply stop playback whenever an exception occurs within the decoder. Upon losing sync of the bitstream (e.g. when the bitstream is damaged) corrupted values may end up getting used as array indices later on - which of course usually lead to an ArrayOutOfBoundsException. The decoder now catches all exceptions and returns appropriate error codes (well, in fact just BADPACKET). The playback component can then decide to continue (the next packet may be fine anyway) or not. The new default behaviour is to just continue decoding. I tested this "error recovery" by randomly consuming bits in the decoder during various stages of decoding, thus wrecking the bitstream on purpose. This results in colorful block-storms and all imaginable sorts of corruption - but playback won't stop, which I consider a much better behavior than just giving up.


Anyway: Thusnelda, the Java world is prepared for you, so don't hesitate to replace the old encoder! ;-)

As many of you know there's a Java applet ("Cortado") for playback of Ogg audio and video which originally was developed at Fluendo. Wikimedia is perhaps the best known "customer" of this applet and they since have forked it ( http://svn.wikimedia.org/viewvc/mediawiki/trunk/cortado/ ) as apparently Fluendo isn't really doing active development on Cortado anymore.

The Java Theora decoder used in Cortado is called "jheora". It is a ported version of an old C reference decoder, meaning it doesn't support the whole Theora 1.0 specification. This means that it won't decode Theora-streams created by "advanced" encoders like Monty's "Thusnelda" encoder.

Thusnelda is most likely going to become the new reference encoder, so we have a problem here.

I had a look into many options to somehow get an updated Java Theora decoder (e.g. port the new C reference decoder, which supports the complete Theora specification), but in the end I decided that just tacking missing Theora features onto the old jheora decoder may actually be the quickest way to get a robust decoder.

I'm positive that this may be the way to go and I was able to write a (biggish) patch to enable jheora to correctly decode Thusnelda streams: https://bugzilla.wikimedia.org/show_bug.cgi?id=15119

This patch is basically a complete rewrite of how quantization matrices are computed (interpolation, scaling etc. etc.) and implements proper handling of 4MV for partially coded macroblocks.

Still missing from jheora, but currently not used by any relevant encoder:

• support for subsampling other than 4:2:0 (personally I think that this isn't relevant for web-video, but it may be a nice-to-have).


• support for per-block quantizers. The patch already doing some decoding related to that (decode what quantizers are used within a frame) and it "just" needs finishing (decode what block is using what quantizer). Should be pretty straightforward to implement and debug once I have a fittingly encoded file.

To put it into a nutshell: jheora isn't completely unmaintained anymore ;-)

With an updated Cortado applet it should be pretty easy to bring HTML5 video (and audio) support to browsers not supporting Ogg codecs (e.g. Apple is shipping Java by default on OS X, so we can enable Ogg Theora/Vorbis playback with a tiny bit of JavaScript embedding the Cortado-applet instead).

As the work on finding a consensus on a baseline codec for HTML5 <video> is either stalled or at least not visibly (from the outside) moving along, browser vendors have the difficult task to ensure <video> is not an interoperability nightmare.

One frequent idea how to cope with the multitude of formats is to have a codec download system, which would link into the system's media framework (Gstreamer, DirectShow, Quicktime) and install a fitting codec to deal with any media format. This is e.g. discussed over at
http://weblogs.mozillazine.org/gerv/archives/2008/06/html5_video_codec_downloader.html. In case of Mozilla it seems the idea of shipping at least one set of free codecs may be dropped. This would make Mozilla completely depend on the media framework of the underlying system.

Now, I see several problems with this approach:

• Not every user has the permission to install additional codecs.
• There are quite a few systems out there with broken media frameworks (some users apparently love to install about every "codec pack" they can find, which can break things horribly).
• There are widespread formats for which no free (and legal everywhere) codecs are available for certain environments. The classic example would be Windows Media, for which Linux users would have to buy Gstreamer components with Microsoft's blessing or resort to unlicensed components, which may be illegal depending on where you live.
• People wouldn't know if using a free playback component for a given format is legal where they live, so any "click here to install if you're aware of the legal risks"-dialog would mostly read "Make page work? y/n". One shouldn't push the nastyness of codec licensing onto the users and expose them to risks they cannot evaluate.
• It opens the door for any proprietary format. Windows users will most likely produce Windows Media content (after all it would work on the "majority of systems"), while Mac users may use an old toolchain to produce content which only Quicktime may be able to decode.
  

Without at least one codec (Theora, Dirac, OMS Video (if latter actually surfaces and is completely free) or any other free alternative) being shipped as default one cannot even credibly point out that there is at least one free alternative which is worth producing content for that ensures compatibility at least within the Firefox population. Given that Firefox is by far the most popular browser amongst those that implement <video> this would have some weight.

In my ongoing series of recycling semi-smart posts that'd otherwise bitrot in non-public archives, I hereby present my view on Theora's bitrate management. The question originally asked sorta boils down to "The last comparisons are a couple of years old, why not do new comparisons?".

Those tests were done with "DVD backups" in mind. That is: The transcoded video was supposed to fit onto a single CD, which means bitrate management has to be enabled.

(Overstatements following for dramatic effect)

Theora's VP3-inherited bitrate management to my knowledge is what one may call a "disaster area". From what I hear it works more by luck and by consuming kittens than by design. Even if it gives you a file of fitting size it'll have its revenge by climbing onto a nearby hill and screaming "Ha Ha, I misplaced bits here and there and royally screwed up quality!".

So albeit there have been bugfixes in other areas things will still look pretty nasty if one dares to activate bitrate management.

General consensus is that the bitrate management has to be ripped out and replaced with something completely different. Here, again, Thusnelda is supposed to come to our rescue, by doing a proper management based on rate distortion theory (IIRC).

Now, there are two lessons to be learned:

1. If you want maximum Theora quality possible *now* you should avoid using the bitrate management and try a quality setting matching your bitrate target (if possible).
2. Once the new encoder is actually in a releasable state Theora may become much much more suitable for web video, which often carries bitrate constraints.

I for sure am curious how much better the new bitrate management will actually be, but until the new encoder is actually available to the public sceptics have every right to remain sceptic. It would be no fun otherwise, I guess.

Well, I think I did never mention how great FOMS is, did I? Nope, I wasn't there. But I'm subscribed to the FOMS mailing list, which is a very nice "place" with a lot of cool people trying to push open/free multimedia technology to the next level. One actually nice property of this list is isn't just populated by us "the usual suspects" Xiph.org volunteers - there are also people from ffmpeg or Mozilla and other great communities.

Perhaps mostly thanks to the HTML5 <video> tag there's renewed interest in free video codecs. The two best known free video codecs would be Dirac and Theora. And when it comes to Theora sooner or later there's the "so, how good is it?" question. Some claim it's bad, others will say it's the best thing ever.

The FOMS mailing list archive is non-public, so I'll just copy & paste what I wrote on that topic:

Well, depending on where you come from you reach different conclusions - e.g. that *all* those mentioned codecs [H.261, H.263, H.264 and Theora] are in fact old technology. They're all DCT based codecs with block based motion compensation with different levels of sophistication to squeeze some more compression out of the same basic scheme, which dates back to the 1980ies.

As for Theora, I feel people like to confuse Theora (the reference implementation as shipped in alpha and beta versions) and Theora (the bitstream specification). The current reference implementation in terms of coding efficiently is (from my evaluations, which were not conducted in a scientific way, so anyone can trust or distrust my findings) superior to H.261 and/or MPEG-1. It is somewhere in the same league as H.263 and/or MPEG-4 Part 2, with robust and well-tuned MPEG-4 Part 2 encoders (e.g. XviD) beating Theora. There are different sweetspots (for example I tend to think Theora does sorta catch up at low bitrates as it doesn't tend to get too blocky) but in general good MPEG-4 Part 2 encoders are more polished. H.264/MPEG-4 Part 10 is yet another notch better. It doesn't do miracles, but it's pretty nice and the most powerful video compression scheme currently available.

Now, comparing bitstream features I don't see why Theora wouldn't be able to catch up with the best MPEG-4 Part 2 encoders. In fact, it has a few (as of now unused) tricks which may make a nice difference.

As for the comparison with H.264 I'll leave that to the more experienced coders [what I meant to say: People with more experience in coding technology]. I feel that Theora overall is the "lighter" format, being able to deliver good quality at modest computational complexity, while some feel that H.264's complexity isn't always justified by the gains in compression efficiency. H.264 seems to have more headroom overall, though.

As for VP3 being "old technology" because of VP6 and VP7 being there: Although this suggests VP3 is three or four generations behind it seems that in fact On2 just released finer tunings of "the same theme" at a regular intervals. VP4 is not a new "generation" over VP3. It appears to be a better tuned and rearranged VP3 (in the doom9 codec shootouts VP4 did mostly show the same strengths and weaknesses as VP3, but an overall  a tad-bit better tuning). I'd guess VP3 to VP7 would roughly be equivalent to the transition of H.263 to H.264, but with more evolutionary steps in between (after all every new iteration of On2's technology will bring new licensing income).

That'd be similar to DivX. Version 3.11 was a hacked MPEG-4 Part 2 derivate, Verion 4 a "proper" MPEG-4 Part 2 codec and the now current version 6.8 is.... still MPEG-4 Part 2. Still of course each version improved in quality - and XviD (also MPEG-4 Part 2) is considered to be even better and remains competetive to the "new" H.264 generation depending on what you're searching for. When VP3 was released it was competetive with the MPEG-4 Part 2 of those days (there was mostly just DivX 3.11 and/or the rather lousy DivX 4), but while the MPEG-4 Part 2 encoders improved (without changing the bitstream!) Theora up to this day is still using the ca. 1999 VP3 encoder without major changes. Work is underway to change just that, but as of now the outdated VP3/Theora encoder (which has plenty of well-identified implementation weaknesses)  continues to distract from what Theora as a format is capable of.

Thus, in my opinion, deducing that a "new generation" of bitstreams automatically outclasses "previous generation" bitstreams can be misleading.

So yeah, I think that Theora can be pretty valuable codec, provided the encoder receives some attention. It's not a secret that there's a new encoder under development (Thusnelda). We'll see how big the actual improvements are once the new encoder is ready - I have big confidence that Theora quality will be noticably better, but until then people of course have every right to remain sceptical.