Interactive Video

MERRY CHRISTMAS!

(to those who celebrated)

&

HAPPY NEW YEAR!

(forthcoming)

Well, finals have come and gone, another semester at a close.  This one, a far more comfortable (though no less rigorous) one than the last - not to say that the prior semester was without success, but I certainly made my way through this one with a far higher level of confidence resulting from a much stronger conceptual framework to undergird an equally stronger body of work (at least I think).  

In my last post I talked a bit about the most recent QR code mandala drawings that I included in my final crit, and so I wanted to take this opportunity to talk (at what regrettably seems to be enormous length...) about the other half of my final crit, which consisted of my first experiment with interactive video! [cue trumpet fanfare]

The Backstory:

It occurred to me that interactivity and bodily connection to the digital would become a crucial part of my artistic quest for some sort of postmodern cyber-mysticism.  For one, all spirituality is participatory, not merely observational.  Secondly, there's something sacred, as Marcel Mauss points out, in the bodily techne, or techniques (equated to technology [also techne]) that allow us to engage in unifying the mind and body in mystical contemplation: a theology of the body as a technological mediator for the divine - a techno-theo-logy of the body, if you will.  In light of that, there's something powerful in the ability of art to make people move.

So, I began constructing a stained glass/mandala video piece that would sense the viewer's presence and respond according to his/her movements.  The main component is an arched stained glass window (a combination of a mandala temple form at the top and a cruciform cathedral layout at the bottom).  Given that the semiotic cosmology of such an image is essentially vertical (heaven = up; earth = down) and therefore Platonic (upper focal point of mandala = good/ideal; lower outskirts = bad/derived), the viewer's presence would trigger crossfades in the video that would literally flip that on its head (and in many other directions).  Essentially, I want the viewer to experience a 'beyond' that is not above or elsewhere, but a 'beyond' that is within and throughout.  

The Experience:

So the viewer starts with this:

When they approach the video, they see themselves silhouetted within the window.

When their head enters the space within center of the mandala form at the top, it triggers a cruciform inversion of the imagery

If the viewer raises his/her arms to accord with the cross shape, on the way up (at 45º from the bottom) they trigger a diagonal spread of the image

Upon spreading his/her arms out 90º it triggers two floating, rotating mandala forms (the red you see was chromakeyed out to make a transparent background)

The Catch:

Of course, one of the problems with this sort of project, as I pointed out in my rant on ethics, is it's tendency toward a 'master-slave' dynamic; a "look what I can make it do" trick-pony sort of relationship of viewer to viewed.  Theologically and ethically there are a lot of issues here (if cyberspace is truly a viable venue for divinity then it can't be entirely controlled by the viewer; a true participation has to be a back and forth, not a one-way interaction; etc).  

So I built in a fail-safe mechanism.  Should the viewer decide to 'play' too much, the build up of triggered changes eventually amounts to a progressive dimming of the screen until it goes entirely black and refuses the viewer any more interaction.  Only if the viewer stands still for 15 seconds does the imagery restore itself to its former functionality!  

Here's a video of it all in action during a test run...

Then I just had to build a rear projection screen for it, which would serve as a false wall (hiding all the equipment behind it)...  I chose PVC piping for its lightness and modularity:

And then stretched satin over it (which entailed many trips to fabric stores, experiments with swatches, sewing inadequately sized pieces together, and a great deal of frustration...)

And finally put it all in place behind some of the school's moveable walls:

Behind these...

...was this!

Voila!

How it Actually Works:

Ok, so here's the nitty gritty, which quite honestly might be even more boring than all the stuff prior, but it may prove worthwhile for some cooperative Platonic reader out there...

All semester I'd taken up learning a programming software: Cycling 74's Max 6 with MSP and Jitter.  Essentially, its object-based visual interface allows you to create your own programs (called patchers) without needing to be familiar with traditional text-based programming code language.  Each "object" you create is given a function.  Information is then passed through one object to another, via patch cables, being transformed in some way by each object's designated function.  Max objects deal primarily with mathematical data, MSP with sound, and Jitter with video.  By stringing these objects together you can essentially make something that does just about whatever you want, to just about any type of input, ranging from something as simple as an alarm clock to an entire DJ's Mixer.

Here's the basic layout of my patcher:

At the top you see a "loadbang" object that basically sets up all the pieces that need to be loaded upon opening the patcher.  Next to it is a trigger button that sends a "bang" message (a "do something" signal - or a "1" in binary) to essentially turn on the machine, which then essentially has 4 parts:

Part 1 (top left corner of patcher)

I hacked a PS3 Eye camera to work with my computer as a webcam (easy to do; and the PS3 eye out-performs most other webcams that cost twice to even 4 times as much!  Check this site for more info). 

After tampering with the levels and turning of the auto-exposure, the camera - aimed at the viewer - takes a black and white, high contrast, live feed video.

pretend there's somebody standing in the frame...

Ordinarily this would feed directly into Max, but I had some issues with lag (note: Max prefers firewire cameras, not USB) so I built a work-around, opening the video in the macam driver app's window and taking a video screen grab of that window in Max.  Max then uses that silhouetted video of the viewer in the final mix and also sends that video information to be disected by part 2

Part 2 (colored boxes in patcher)

Each video file that I wanted to crossfade in based on the viewer's motion is represented in one of the colored boxes above.  Each box contains sub-patchers that do two things.  First (shown below in the left hand sub-patcher window) it isolates a pixel in the original black and white video feed of the viewer.  This pixel represents a location in the space occupied by the viewer.  It is by default a white pixel because of the high contrast.  When part of the viewer's body obstructs this pixel it turns black.  

Whenever that pixel changes color the second sub-patcher (on the right) uses that to trigger a crossfade with the given video.  Each video is assigned a different pixel location, so depending on where the viewer moves, different videos will crossfade in and out.

Part 3 

Each of these crossfades are then woven together and overlaid with the viewer's silhouette video to produce the video that will ultimately be output to the projector

Part 4

On its way out to the projector the video passes through one final sub-patcher, the fail-safe dimmer.

Besides triggering a crossfade, each bang message produced by the change of the isolated pixels' color also gets fed into this sub-patcher.  It ignores the first 40 bangs (allowing the viewer to mess with it for a time without immediately getting confused and frustrated by a constantly dimming screen).  After that 40th bang however, every subsequent bang amounts to a 10% decrease in brightness in the final image until the brightness is reduced all the way to 0.

At that point a timer is started.  When the timer counts up to 15 seconds (20 in the above image, before I adjusted it) then the brightness is restored to 100% and the bang count reset to 0.  However, until it reaches 15, every bang resets the timer, so that it will never reach 15 unless no bangs are sent; i.e. 15 seconds of complete stillness.

And once again, all this is then rear projected on the screen in front of the viewer so that they watch the results of their actions/interactions play out in front of them!


The Aftermath

With all said and done, I have to say I'm quite pleased with the outcome for a first try at such a project.  However, I've learned a LOT of things that might be better done differently in the future.

For one, the isolated pixel technology banks on a high contrast video.  In order for the viewer to stand out against the background, the wall behind him/her has to be well lit, but the viewer left in shadow.  Though functional, this does have the corollary effect of providing enough light to wash out the projection screen to some degree.  I know there are other ways to track motion with this sort of technology, and I'll be looking into those (still getting the silhouette imagery of the viewer however is another story...)

Secondly, I have much to learn about preserving the frame rate/quality of the videos within Max.  I managed to avoid some lag by loading all the videos to RAM upon startup, but there is still a bit of lag and degradation of quality.

Third, the screen itself, would of course be entirely unacceptable for anything other than a school final with a ridiculous 30 minute install time limit (yep.).  Ideally I'd love to find properly sized material with a tad more clarity.  But I'm content with what I had for what it was.

Finally, there is always much to be excavated in the way of concept and formalism of the imagery itself, but that I can save for other projects.

I'd love to tweak some of this stuff before settling on this as a finished piece, but it proved to be a promising adventure into the world of interactive video!


Ok, I'm done talking.  I swear.

The Next Meditations

It's been an eventful last few weeks with the semester finishing up.  I left off on my QR code mandala series (as seen here and at the end of this post) with the completion of Meditation 1.1 (Thusness, Elseness; Omnipresent), which to my delight will be traversing the atlantic for a works on paper show at Cardiff School of Art and Design in Whales in late January!

In continuing this series, I decided to move on to Meditation 1.2 (Thusness, Elseness; Intertwined).  By the way, I've created a naming system that adopts a digital numbering system that correlates with both the mandala format and the QR code function.  So 1.1 refers to the first mandala format I used (Yamantaka, or cosmological arguement) and the first QR function (random destination).  This is reiterated parenthetically with "thusness, elseness" referring to the paradoxical cosmology of thingness/nothingness I'm building, and "omnipresent" referring to the QR destination as anyplace.

So Meditation 1.2 (Thusness, Elseness; Intertwined) takes the cosmological mandala form, this time stripped down to its most essential elements...

...with a QR code that refers one piece to another piece.  It therefore manifested as a dyptic of inverses, in part inspired by my Hyalo project.

Considering the Mandala temple structure is essentially a Platonic hierarchy with "truth" at the center, I wondered what might happen when that center is actually created by a number of these mandalas overlapping, each with its own now de-centered center.  so I arrived at this image, consisting of 4 overlapping smaller mandalas encircled by one larger, rotated mandala.

I then decided that the two would be created in exactly opposite processes.  For the first I borrowed some charcoal erasure techniques I created in some of my earlier experimentations. 

step 1. The above mandala outline would be traced over top of the paper 4 times, leaving a colorless imprint of the 4 smaller mandalas

step 2. the paper was then blacked out in charcoal, leaving a white trace of the 4 small mandalas where the charcoal doesn't fill the imprinted grooves.

step 3. the final larger mandala was erased out of the charcoal.

step 4. where the lines intersected to create a central square, the QR code was drawn

For the second piece:

step 1: using carbon paper, trace the mandala form to graphite transfer it onto the white paper

step 2: leave the paper white (that was the hard step)

step 3: draw on the final large mandala

step 4: where each of the original center squares land a QR code is drawn

The code from one then takes to to an image of the other, digitally erased and laid onto a neutral gray background.

Finally, they were displayed next to each other.

Next I want to try creating a mandala form out of the code itself, before moving on to manipulating the form more, making it less an appropriation and more of my own invention.

I thought I checked the box for "Artist" not "Postmodern Theophilosophical Ethicist..."

It's been a while since I blathered on about the theophilosophical things on my mind, and, frankly, I'm feeling rather theophilosophical today.

I've been thinking lately about the ethics postmodern mysticism.  Ivan Illich's lecture Life as Idol has me thinking about the coercive potentials of theological rhetoric, in what Foucault terms "Biopower."  To grossly summarize Illich's main point, he says that the term "life" (according to the 25 criteria of Uwe Poerkson) has become a "plastic word" or an "amoeba word" - that is, a common word that has been imbued with so many specialized definitions that, once spat back out into colloquium, it essentially has lost all concrete meaning whatsoever to the lay person.  The trouble is, though it carries no concrete meaning, it still carries powerful connotation.  Thus, to say "Pro-Life" says little about what concretely one is to be pro- about, but uses the power of its ambiguity to strike an emotional chord (who can honestly claim to be "Anti-Life" and still sleep at night?).  Illich says that most "scientific" arguments for Life are both entirely unscientific and actually say nothing about "life" as a qualitative thing.  Rather, they borrow the connotative power of the word "life" to make a compelling ethical argument for some biological change.  "Life begins at conception," we often hear.  Really, that is to say that at conception there is some distinct biological difference, on a cellular level, that did not exist prior to that moment of conception.  But what earns that singular difference the title "life?" Aren't all cells, fertilized or not, alive?  We think of trees as alive, but do they have life?  Are they "a life?"  Does "life" then equal "soul?" What then, scientifically is "soul?"

You can see where the issue gets sticky, all polemics aside.  Regardless of your position on the example Illich uses, it seems that the term "life" is really only selected because it feels to be the right word, not because it actually is the right word.  "Life" as a substantive word (as in "to be [a] life" rather than "to have the quality of life"), Illich argues, is in fact historically traceable to none other than Jesus, who is the first person in western history to claim to be Life (John 11:25).  Therefore, every subsequent use of "life" in the same sense, apart from referring to Jesus, is actually heretical!  It is this that leads Illich to confront a forum of clergy, opening with the mystically oppositional curse, "To Hell with Life!"  They naturally took well to that...

The goal of the mystic and the postmodern philosopher both is to nip these semantics in the bud; to reach beyond the confining and coercing potentials of these semantics; to call a spade a spade (and not pretend that the spade is actually some Platonic sword against which no one can argue), in order to get at the heart of a limitless, undefinable God, bigger than what that spade could ever hold.

This is the battle I take on every time I co-opt a religious symbol or reference a specific tradition in my artwork.  The spades are all lost in a flurry of swords swinging violently and haphazardly in front of the viewer.

My mentor Iain brings up a related challenge in our digital culture's preoccupation with "interactivity."  Who really knows what that means anymore?  Is "interactive" too plastic a word?  We kid ourselves into thinking interactivity is the same as user-definability.  To make interactive art, a camera might sense the viewer's presence so that their movements trigger change in the piece.  But wasn't art always interactive?  To view a piece - even to think about a piece - is to interact.  Really, what we mean by "interactivity" is a sort of "master-slave" relationship; a sort of "look what I can make it do" effect.  It echoes Lev Manovich's assessment of the seeming media-induced inversion of "strategies" and "tactics" in his The Practice of Everyday (Media) Life (a riff on Michel De Certeau's The Practice of Everyday Life in which he asserts the hierarchical societal roles of those in power who create "strategies" - systems within which society functions - and those below who create "tactics" - ways in which to navigate those systems).  User-definability gives the appearance that the common man can now move from his role of tactician to strategist, and that the power institutions are therefore rendered reactive, more like tacticians than the strategists they once were.  Of course this is all a ruse, since the user can only manipulate the system to an extent within the parameters set up by the strategist in the first place.  You can customize your home page all you want, but it will never become Photoshop.  Thus it merely serves to further suppress and placate the lowly tactician.  The master-slave dichotomy of interactive art toes this very ambiguous line, I think.

I've decided to combat this hierarchy (or at least call attention to it) in my latest "interactive" endeavor.  The viewer's motion will indeed trigger events in the video they are watching, but each subsequent effect triggered will also amount to a progressive dimming of the video until, at some point, the video will shut off completely, refusing the viewer's coercion.  Only after a set time period of complete stillness will the video restore itself.  This way the more the viewer tries to become the video's master, the more he will realize he is actually at the video's mercy after all.

Finally, on the subject of Ethics, Iain asserted that (citing a source I can't remember now) every ethical problem is at its most basic level an aesthetic one.  How you treat a person, for instance, is an ethical issue.  However, in order to even get to that point, one has to sense that there is a person, and make a value judgement based on their sensory perception.  Perception is the territory of aesthetics; that you even recognize a mass of colors as a person in the first place already pre-forms the ethical framework within which you will deal with said person. (If you decided what you saw amounted to a tree, you would probably not decide to tease it based on it's skin color, etc).

I'm not sure what to do with this yet, but it seems of utmost importance for all artists, really.  Until I reach some conclusion, I suppose I'll keep boring you with stuff like this (afterwards I'll bore you with some really inspired stuff like this).

Whew, is that a big theophilosophical weight off my shoulders!  Ok, back to my to-do list...

Hyalo

Over the past semester here at Montclair State University, I've been experimenting with optical negation by means of subtractive and additive color mixing; i.e. mixing projected light and painted pigment to get things to look like they're disappearing.  This was the impetus for my latest projection installation Hyalo, which actually turned out to be something quite different! (originally titled Of Unseeing: Hyalo, but changed exactly for these unexpected effects, namely that it was very un-unseen)

I began with this:  I wanted to merge the mandala form I've been using (see my previous posts, and some hence-coming...) with a rose window stained glass form (both of which originate from similar formal and theological concerns, historically).  The term hyalo - my title - is a greek word referring to glass mosaics, or, more specifically, prototypical stained glass windows.  The first projectors to use photographic positive and negative transparencies, created by two french daguerrotypists in 1848, were in fact called hyalotypes, giving a nod to stained glass as the first form of projector.  So I set out to paint a rose window that would be cancelled out by digital projection.  Upon blocking the projector, would reveal the painting underneath with their shadow.

I suppose this all is a continuation of my preoccupation with Heidegger's notion of Aletheia - truth as both revealing and concealing - and this mystically paradoxical idea of negative theology.

The window...

was spliced into to inverse fragments, that would be painted on opposing walls:

 This way, you could never get the whole vision, because your back would always face one component of the whole.  To confound this further, you can only see those pieces by concealing them in the first place.  I designed the following "room" setup, with two opposing projectors, aimed across one another, so that there was virtually no way to enter the room without obstructing one or both of them.

Come time to exhibit, my first task was to build the room itself.  I claim no style points, but given the one day I had to build this thing (and a shortage of movable walls) I'm quite pleased with my ingenuity.

moveable wall 1, with an extended support beam for the "corner" of the room

my best ceiling mount yet, complete with faux-ceiling tile, hung low so that the viewer will obstruct it.

beginning to install the tar paper walls...

a doorway in the center of the long wall, with supports mounted into the ceiling.

the finished room, dubbed "the chapel" by our critic in residence Matthew Nichols.

my two opposing projectors

moveable wall #2

projecting the image to be painted

painting it on

the effect of projecting the image on top of the painted image (shown in shadow)

working out the colors for projection in photoshop (projected onto the painted image on the wall)

Here's where things got interesting...  The image only really disappeared from the angle you see above (and ONLY that angle).  Once I stood up, I found that the "matte" acrylic paints I used actually reflected quite a bit of the projected light, resulting in an oddly ethereal, luminescent, metallic sheen.  Deciding to embrace this inherently stained glass like effect, the project therefore became something other than what I intended.  Additionally, as far as I can figure, a slight shift in ambient light also changed the colors projected in the final project.  This was the result:

It did still retain its "reveal" moment when blocked by shadow

another odd optical effect with the lights on...

...which heightened the reveal, but lessened the sublime effect of the image itself.

So I decided to play up that original incidental optical effect, masked off the edges, and produced the following!  This is ultimately what was shown in my show with Jeremy Bell, Pneuma (Breath and Soul).

Essentially, this project marks the point of divergence for two subsequent projects.  One in which I continue to try to solve the problem of reflection and get it to disappear, and another in which I continue to play up this new stained glass effect.  I'll be procuring a colorimeter from the photo lab to try to get better readings on the mixture of light and paint, and I'll try other paints as well (note the wall paint is not reflecting the same way)  It will be interesting to see what emerges!