[nmadrane]

I have developped a software-only video decoder, written entirely in pure J2ME MIDP/2.0. It decodes a 176x144 video at 12 fps. The codec is proprietary and has been developped specificly for that purpose. With this stuff, it is possible to play a video from a midlet (HTTP streaming supported) without relying on the video player installed on the phone (Oplayo, RealPlayer, etc...). The quality is not bad, and is similar to 3GPP. Encoding rates vary between 120 and 170 Kbit/s. It supports real-time overlays with transparency, so that you can put a translucent image on top of the video. Landscape mode is also supported.

I would like to have your opinion about this. Do you think this is something that can be valuable ? Do you see any immediate market for this ? Or do you think that this kind of initiative can't be seriously used since everybody today is focusing on MPEG4/3GPP. I am currently thinking on bringing this technology to the open source world but I need your ideas guys.

[dowlu]

Does this video decoder written without using MMAPI (JSR 135) ? If so can you list the features of this software ?

[nmadrane]

Yes, this video decoder is written without using MMAPI (JSR 135). It decodes the video in software and uses basic J2ME drawing functions to display the frames. The algorithm is proprietary and its speed is partially due to the fact that it doesn't use floating point arithmetics. So, even with java, the speed is enough to decode frames in real-time.

Since my initial post, the decoder has been improved : we can now achieve 20 fps on a 264x216 video. We also managed to play 2 video streams at the same time (picture-in-picture mode).

As i said, the decoder supports real-time translucent (alpha) overlays. It also supports HTTP streaming. The compiled code is about 70 KB, and can be easily integrated with any existing J2ME application that would like to :
- play video streams on any phone, regardless how good its MMAPI implementation is.
- play video streams with a complete control over the GUI (overlays for example).

The drawback of the new version is its encoding rate : 300 kbit/s. This is too much for 3G but ok for HSDPA. I am currently trying to improve that rate : 150 kbit/s would be good.

I also need to test this stuff on many many phones, create a web site for this project and release a beta version of the SDK....

[somatik]

do you have a commercial version available including server side encoding? url?

[drhu00]

is the sound support in your SDK?

[nmadrane]

Well, I finally managed to create a web site demonstrating this technology. It is available at http://www.versatile-media-format.com . There are several demo clips and I will soon post a free SDK so that the video decoder can be integrated into any J2ME application.

There are interactive samples (where video plays and proposes 4 commercial actions at the end), and also multiangle samples (the user can switch between 2 camera angles).

[skjolber]

I am quite impressed over the effort, but will not this software-only approach deplete the battery rather quickly?

Video coding is *so hard* that I have a hard time believeing your codec is competitive (unless you're some kind of genious and are using n-to-m dimensional mappings for image compression).

I think you should offer any samples without a login - and with fair comparisons (PSNR) to the H264 levels available in the WURFL database.

Regards

Thomas

[nmadrane]

Hi Thomas,

I understand your point of view. I should insist on the primary goal of this codec (and where it is supposed to be used) :

- applications that need new ways to present/interact with the video (semi-transparent overlays for menus for example, multiangle video, possibility to play several streams at the same time).

- applications that need an HTTP streaming solution (which is a nightmare on mobiles)

- applications that need to play small video clips on any phone, especialy phones that present implementation flaws/problems with their embedded player.

In these 3 areas, you will have problems with traditinal codec/players. This codec tries to bring a solution in these particular cases and doesn't try to present itself as a competitor of 3GPP for example (in terms of PSNR). The comparison wouldn't be fair. In one case, everything is implemented with floating point operations, in C langage (which allows to implement good decompression techniques on a phone). In the other side, this codec tries to do what he can do with the J2ME virtual machine. And it works : 15 fps on most phones.

You are right, we use something similar to n-to-m dimensional mappings. Our compression ratios are not as good as 3GPP (and I am trying to improve that). But the codec could still be useful.

Concerning the battery, I played a video clip in loop during 12 hours, on a fully charged phone. And the battery level decreased by 50%. I don't know how to interpret this, if it is ok or not.

Concerning registration, is is a quick procedure. The site just needs an email and a username. This is important for me (the site is still beta) so that I can track any problem directly with each registered user. Another reason is that the site proposes an option to send you the player package (JAR + video) via a Wap Push SMS, which costs money.

Best regards,

Nabil.

[nmadrane]

By the way, I put a phone emulator on the web site, so that you can try the samples without having to install them on a real phone.

Thomas, could you please give me a link to the H264 levels available in the WURFL database ?

Thanks

[skjolber]

Hi Nabil,

the points you make about the video overlay and flaws in the embedded playback implemententations are valid, although we have been seeing increasingly mature embedded playback implementations the last year.

I have not tried, but would not a custom datasource solve progressive HTTP playback issues?

The H264 profiles are "10 1b 11 12".

At least the online midlets look quite good, I'll be awaiting the benchmarks.

And please note that the video codecs game is one of standarization and volume - good codecs are available for very low license fees already.

Best regards,

Thomas

[nmadrane]

You are right, a custom data source for HTTP progressive download is the right solution to overcome the problems that most people see when they try to play with RTSP streaming.

By the way, I am now working on integrating this video codec with SVG : the idea is to be able to overlay SVG animations on top of the video stream (for translucent dynamic menus for example). This is quite easy to do, since this codec actually decodes every video frame into an RGB buffer, so we can put anything we want on top of it before actually rendering the frame on the screen.

I can either use Mobile SVG (JSR 226) or use the excellent library called Tinyline. Any ideas before I dig into the code ?

[grahamhughes]

Your figures for 12/20 fps... on what device(s) are these measured?

[skjolber]

nmadrane,

can you not make this optional? What if I wanted to have some 3D graphics instead?

I do not know much about SVG implementations, but the question is whether you want to rely on implementations for SVG - the same implementations which you do not rely on for video?

Also, I guess with TinyLine, the combined footprint will be rather on the big side.

No easy choice here..

[nmadrane]

Hi,

On the web site, there is a demo for more than 100 phones. We have tested the codec on 60 phone models. On all of them the fps is between 12 and 20 for a 220x180 video size. The complete list is on the site.

Here is a list of supported phones in the worst case : multiangle video. When we play multiangle video the player has to decode 2 streams simultaneously (in order to be able to switch between angles instantaneously). On all the following phones we achieve 12 fps (for each stream, so this gives 24 fps combined frame rate) with a screen size between 174x144 and 220x180 :

Nokia/5300, Nokia/5310_XpressMusic, Nokia/6120, Nokia/5610_XpressMusic, Nokia/6500_classic, Nokia/6680, Nokia/E61i, Nokia/E61, Nokia/N70, Nokia/N73, Nokia/5500, Nokia/N82, Nokia/N95, Nokia/6110, Nokia/6233, Nokia/6280, Nokia/E65, Nokia/N93, Nokia/N76, Nokia/N81, Nokia/N91, Nokia/6131, Nokia/E51, Nokia/6600, Nokia/3109c, Nokia/3110c, Nokia/7500, Nokia/8600, Nokia/6300, Nokia/6630, Nokia/5200

Sony-Ericsson/K530, Sony-Ericsson/K770, Sony-Ericsson/K800i, Sony-Ericsson/K800, Sony-Ericsson/K810i, Sony-Ericsson/W200, Sony-Ericsson/W610i, Sony-Ericsson/W800i, Sony-Ericsson/W800, Sony-Ericsson/W810i, Sony-Ericsson/W810, Sony-Ericsson/V640, Sony-Ericsson/W880, Sony-Ericsson/W880i, Sony-Ericsson/W910, Sony-Ericsson/W960, Sony-Ericsson/P1, Sony-Ericsson/P1i, Sony-Ericsson/Z520, Sony-Ericsson/Z610, Sony-Ericsson/Z750, Sony-Ericsson/W580i, Sony-Ericsson/W580, Sony-Ericsson/C902, Sony-Ericsson/K600, Sony-Ericsson/K610i, Sony-Ericsson/W660i, Sony-Ericsson/W850i, Sony-Ericsson/K850i, Sony-Ericsson/P990i, Sony-Ericsson/W300, Sony-Ericsson/W890i, Sony-Ericsson/W950i, Sony-Ericsson/K750i

[nmadrane]

[Can you not make this optional? What if I wanted to have some 3D graphics instead?]

Yes, 3D would be great. As soon as you can render 3D graphics on an offline surface, you can superpose anything on top of the video surface. I have never played with JSR-184 and I am wondering if this is actually possible.

[Also, I guess with TinyLine, the combined footprint will be rather on the big side.]

Yes, I am afraid it will be too slow if I use a software-only SVG implementation like TinyLine. My codec has a JAR footprint of 16 KB and a memory footprint of 700 KB for decoding a 174x144 video. Putting an additional layer with complex SVG animations may kill the virtual machine... Well, I will try this week-end and see the results :)