VJing with Flash + Processing: Onyx, FLxEr, and Other Free, Lightweight, Homebrew Efforts

Flash and Processing have some powerful features for animation, interaction, I/O control, video, mixing, and filtering. The natural next step? Turn these into VJ tools. True, there are many wonderful VJ apps out there, but many of them are variations on a simple 2-channel mixer theme. Custom tools promise to open up new visual possibilities.

Flash

VJing with Flash wouldn’t have been such an exciting possibility to me in the past. It’s easy enough to drop simple Flash animations in other applications (Resolume, Jitter, etc.), and up through Flash MX 2004 (aka Flash 7), Flash’s own internal facilities had limitations in terms of capabilities and performance. I’ve seen lots of great VJ sets using stark Flash animations, but always in other programs. Then along came Flash 8 with all sorts of bitmap operations and blend modes, and things started to get more interesting.

Among various VJ tools on multiple platforms, FLxER appears to be the most well-developed:
FLxER

(Much of that site mixes Italian and English, but you should be able to follow it.) 27 kb(!) total download, and it incredibly runs not only on Mac, Windows, and Linux, but Mac OS 9 and PocketPC, too. (Bless you, Flash!) 7 channels, 800×600 output, DV input, live text, recording and playback, broadcast, and all the effects and layers you’d expect. A lot of those features wouldn’t be nearly as practical on other platforms, and the whole thing is free.

Of course, part of the real appeal of working in Flash is that you can relatively quickly build your own, custom VJ app.

Quasimondo, aka Mario Klingemann, is one of the celebrity coders of both the Flash and Processing world. He gave a lecture on building a VJ app using Flash in the spring at FITC and flashconference, and he’s posted the full notes online. A quick skim should give you some ideas, even though he’s decided not to publish the full source.

“Making a VJ Tool With Flash” Lecture Notes [Quasimondo]

Mario got some help from Processing, as well, for MIDI support (using the ProMIDI Processing library) and sound processing (using Sonia). The combination should be quite potent, even though I wish there were better MIDI and sound support inside Flash. Mario also writes with haiku-like minimalism about the essence of VJing, as seen below:

While these options are based on Flash 8, Flash 9 offers improved performance and a much better display model that for me makes it the first version of Flash truly worth devoting some time to. (More on that soon.) Daniel Hai is working on a full-blown ActionScript 3-based Flash video mixer called Onyx. It’s already capable of some sophisticated layering of bitmaps and other cool tricks, and though it has a way to go yet, it already looks promising. Best of all, Daniel says he intends to release it open source via osflash. A new release was due yesterday, but sounds like we’ll have to wait. Read Daniel’s call for development help below and give Onyx a try:

Looking for a partner in crime [Daniel Hai blog]
Onyx VJ site (including live demo; Flash Player 9 required)

See also Daniel’s excellent discussion of the gap between Flash 8 and Flash 9 (which, to me, also illustrates something of why Flash 9 is cool).

Processing

Aside from giving an assist to Flash on sound and MIDI chores, Processing has appeared as a VJ tool in its own right. That’s an appealing concept, given the various visuals you can create in Processing — you could very quickly go far beyond the usual 2-channel video mix and redefine what VJing is. (Hence CDMo’s campaign to change the term to “visualist.”)

The best-known project so far is Robert Hodgin’s processing LIVE project, which set up an array of Griffin PowerMates for hands-on control. Details and visuals (still/motion) on the flight404 site:

processing LIVE

Of course, building your own application isn’t so easy. Ideally, you’d want flexible MIDI control and (most importantly) the ability to drop a bunch of sketches in and use them at will. Here’s a blog post up that talks about some of the challenges of building a VJ Processing app, with a link to code:

Processing VJ Application

… though I’m not clear who actually wrote that entry. (You out there?) The results are interesting, though mainly just a start, layering sketches:

More Ideas?

Anyone else worked on a project? Got ideas for how to make a tool that works well with Flash, Processing, or both? We’d love to hear them. Hopefully CDMotion can keep a running list of current VJ apps built in these tools. Now, back to my own development work …