videoblogging from notting hill carnival
http://www.showstudio.com/projects/jay/jay_start.html
I spent the bank holiday weekend mostly working on this. SHOWstudio arranged for two photographers to go out each day of carnival to send back the experience via mobile phones. I've already developed the software to publish photographs to the web, some simple perl scripts. But for this we also wanted to use a 3G video phone.
First of we got sent some NEC e606 handsets. Whilst these were quite compact and OK to use, there is no email client in them, so getting them of required getting them to another phone and then offloading them to a computer. Done using some windows software supplied by NEC, whcih was a bit awkward. I wanted to automate the process so we got hold of some NEC e808 handsets which have an email client that can send attachments. So we could get video to the server no problem.
A few experiments at a Squarepusher gig proved that whilst the video was interesting the sound wasn't so hot when the volume was high. We also had music that we wanted to use and so it seemed like a good idea to strip out the sound from the MP4 files that we received from the phone. I thought this would be pretty trivial as I could just load the files into QuickTime and strip them - this is easy to do in QuickTime Player. Obviously that's no good on Linux though.
So I looked at what was available for Linux and, as expected, was disappointed. There are several solutions for playing video and loading codecs, but the libraries are scarecly documented [that I could find] and few utilities exist. I naively assumed that the quicktime file structure would be quite simple and therefore easy to manipulate, but some investigation has proven me wrong. Having since delved more into QuickTime's API and features I can understand why it might not be worth trying to do more than video playback.
I dove into the QuickTime Java API thinking that the solution might lie there, and initial reads made me feel quite confident that I could write a Java app to do the work. So I set about working out how to get a Mac server that I could run the app on. I ended up putting my G4 laptop into Internet Vision's rack at RedBus for a few days.
And set about writing QTDemux.java ... I've discovered a few things about QuickTime and QuickTime for Java along the way:
QuickTime is a very powerful toolset.
The document format is overly complex.
The Java API is very poorly documented.
But I managed it.
And then strung together some front end interfaces using SMIL, and discovered that it was too heavy at the front end. QuickTime Player seems to suffer quite badly when you are trying to do quite trivial SMIL work. I can see this being down to the difference in model between SMIL and it's own document format. The thing is that these should essentially be the same thing, and therefore I can't understand why it's not. I think I maybe have a DOM mode of the QuickTime in my head, one that could be simplified, and would map to the DOM that exists in SMIL as well.
There have been several itterations of what I wanted to do with the videos. I wanted to get across an idea of the 'everything going off' feeling I've had when I've gone to carnival. I decided to arrange the videos in a grid and control them using SMIL. I'd alternate between still images [pulled out by the QTDemux app] and video. Maybe play 2 videos at once. They were only small [<12 seconds, 174 x 144, 2.5 fps, MP4, <100k]. And a soundtrack.
That's asking too much. It sort of works. But audio doesn't. Trying to play music along side just stuttered on my iBook. And I reckon that's probably going to be the case with most users. So I changed my approach and removed the music. But as the number of clips increased, quicktime carried on slowing down. I tried to stagger the loads, so that images loaded at different times, but eventually resigned to the fact that - as I knew already, but hoped the situation had improved - I needed to simplify it. So we've now just got the videos pulled in reverse chronological order with a random soundtrack. Simple, but I think it's still a nice mood.
Gerald Jenkins was given the NEC e808 video phone, and a Nokia 3650 for stills. He was also given a back stage pass to the Good Times stage, so that he could film Norman Jay and what was going on around him. His stills are the monotones on the site, taken as colour and then passed through Perl::Magick. Gerald was around the stage on both days of the carnival, and towards the end was starting to feel limited with the video phone towards the end, and I can understand that. Whilst downloading and playing back videos on the phone is quite impressive, the video functions aren't all that yet for recording, and the frame rate can be annoyingly low. That combined with cramped conditions and a quite ugly handset [the silver e808 looks like a compact. And I mean the kind for make up]. It's going to take some getting used to. The other stills on the site were taken with a Nokia 7650, published straight out to the site. I enjoyed the contract between the different photographers on the project. That's something you'll have to take in by looking at it.
Easy peasy once the pieces are in place.
Posted by
Dorian Moore
on Tuesday, August 26, 2003 at 23:57 GMT
Last modified Thursday, October 20, 2005 at 20:22 GMT
