Late last year, as described in this previous post, I decided it was time to rebuild the VS3 live visual synthesis tool in a modern development environment. After a considerable amount of fits and starts, an “alpha” VS4 is finally up and running, with all new C# code (implemented in Unity 5.6). Still the same basic paradigm, but with greatly expanded creative options.
(The video above shows some output from some of the “alpha” functionality as well as a glimpse of some limited bits of the preliminary “alpha” UI)
A number of decisions had to be made along the way. One consideration has to do with aspect ratios (as in, the format of source material, the dimensions of the virtual compositing environment, as well as output). Obviously one wants to optimize as much as possible, but while also maintaining maximum flexibility. The current thinking is to be as “aspect ratio agnostic” as possible and to keep everything 1×1, internally (though output can be cropped and formatted as desired) – taking “refuge in the square” (to borrow Malevich’s phrase): avoiding unnecessary assumptions about composition as much as possible during the process, and shaping the imagery as desired at the final stage … (square is also, of course, a preferred aspect ratio for performance with projection and/or LED walls, so in many cases we might just stay 1×1 throughout).
VS3 was primarily a 2D real-time compositing system (with some limited 3D functionality); VS4 will do the same sorts of 2D compositing, but since the engine is always running in 3D space, we can move beyond 2D – or, say, switch between perspective and orthographic rendering – at any time we please. So you might say we’re “taking refuge in the cube,” rather than the square.
Now that we’re working in 3D space, full-time, some interesting historical references come to mind: In the early 20th century, artists such as Nicolas Schöffer and Bauhaus professor László Moholy-Nagy created motorized sculptures designed to shape space and modulate light and shadow. Moholy-Nagy’s Light Prop for an Electric Stage or “Light Space Modulator” is still operational at the Busch-Reisinger Museum at Harvard. Inspired by these works and processes, the VS4 environment could be thought of as a digital “electric stage” with a variety of virtual “props” – one in which we’re actively exploiting the possibilities of dynamic lighting, shadows, reflection, transparency, and so on.
There’s still a great deal of work to do before this is a usable beta instrument, nonetheless the basic architecture and UX is in place and feeling pretty solid. A couple more months to go: SPLICE 2018 in May in London is the target for a first major demonstration of the new tool …