They’re a recent addition to popular OS X live visual app VDMX. But now is the perfect time to talk about the magic of shaders, generally.
Standard “fragment shaders” are capable of performing feats of visual wonder as both generators and effects, using geometry and pixels. Written in GLSL, they are portable to any OpenGL-compliant graphics card, which now includes a range of modern hardware on both mobile and desktop devices. And since they run on the GPU, they do all of this magic on optimized hardware – without straining your CPU.
Despite Apple’s reputation for being proprietary, they’ve been consistent in making standard OpenGL the basis of both iOS and OS X, and the success of that has added a lot to the momentum of OpenGL, generally, including on other platforms like Linux, Android, and even Windows (despite having DirectX as an alternative). So, this makes sense in a Mac-only app like VDMX. But you can test this out on those platforms, too, right in your browser.
For a glimpse of what you can do, have a gander at:
https://www.shadertoy.com/
https://glsl.heroku.com/
There, you can try out lots of hypnotic examples, with or without any actual knowledge of code – though this is the perfect way to start learning.
VDMX adds native support for GLSL as a generator or effect in the VJ-friendly app. (Developers Vidvox tell CDM they even tried the examples on the sites above to test their own implementation.)
Now, since GLSL is portable, it’d be nice to drop those GLSL files anywhere you like. Vidvox has a spec for that. David Lublin tells us:
We’ve started working on a new specification called ISF (“Interactive Shader Format”) which adds onto the fragment shader a small JSON blob that can be used to declare published parameters and other metadata information that can be used by the host application.
It’s a pretty simple implementation, adding some simple text metadata as JSON; it doesn’t break the way GLSL works normally. Their work looks potentially promising for a lot of applications – and perhaps for other environments, from VJ apps to custom-coded environments.
Vidvox goes through the pros and cons of this particular format, and how it works, in a detailed blog post:
Introducing ISF Video Generators and FX
And if you want to try it yourself, give the latest VDMX a try – a free demo is available:
http://vidvox.net/download/vdmx_demo
I’d be curious to hear what visualists and VJs do with the latest VDMX and shaders – record those sets and demo reels, people! And from developers and creative coders, I’d be eager to hear if the new format is useful.
I’ve been converting GLSL shaders to work with my Processing VJ setup – they’re super easy to use via the new Processing 2.x PShader object. The performance for post-processing effects and texture generation is orders of magnitude faster than what’s been possible in the past. GLSL shaders are super portable, and I recommend looking through the GPUImage (an iOS library) project for tons of GLSL filters that can easily be converted to Processing, WebGL and I’m sure VDMX: https://github.com/BradLarson/GPUImage/tree/master/framework/Source
ShaderToy and glsl.heroku.com are great resources for sure, and I’ve been borrowing and learning from them. GLSL is an entirely different way of thinking about graphics programming, and it’s not easy to learn.
Here are some shaders in action in my most recent DJ/VJ set – mostly just in the first 20 seconds or so, but I use a vignette filter throughout most of the show. I’ve synced them up to update their uniform values via incoming MIDI signals from Ableton live.
https://www.youtube.com/watch?v=LZyu9GKpAE0&feature=youtu.be
groovy. so all the visuals are created in Processing, with MIDI hooks from Ableton? Do you fire off a succession of sketches, or just have a mega-sketch with switch-cases or something? will you be doing this kinda stuff for MediaLive this year?