Hacking it as an Artist

Coming up in October, Newark, NJ will be hosting its annual Open Doors art walk, which turns out to be the perfect coincidental kickoff for Aferro Gallery's Activate Market Street initiative. As a part of this, I'm collaborating with Marc D'Agusto (fellow MSU alum and the exec. director of Gravity Arts Initiatives) on an interactive projection installation for Aferro, one of Newark's foremost galleries.

And so, I get to play with some new toys! If the timeline accommodates a bit of a learning curve, this will be my first foray into the artistic universe of a hacked Xbox Kinect! Armed with borrowed hardware and one of the greatest introductory computer vision textbooks I've ever seen, I've begun digging in. (Thank you, Eric Bowers!)

I've had some nominal experience with Processing (the open source programming software developed in part by Casey Reas, who, coincidentally, has a solo show opening at Bitforms next week), but this is a whole new beast. As you may well know, Kinect uses infrared projection to map out not only the pixelated image of what it sees before it, but to map out the depth (accurate to the millimeter) of each of those "pixels" away from the camera. With the right driver and programming library, Processing can access all of that information and do some massively cool stuff with it.

on the right you see the standard color image, and the left image is the depth image, accessed from the Kinect by a Processing code

Essentially, our proposal as it stands now deals with inverting silhouettes of live passersby and inserting them into a projected scene. The exact details are in the works, but we're looking to combine some of Marc's interest in relational aesthetics and the bodily metaphors of architecture in a state of decay/revitalization, with my experience with interactive projection installations and tendencies towards a mystical theology of perception and disorientation of spacial relationships.

an image out of our original proposal

There's a ton of work to be done still, but if I can get on the Kinect train before it leaves me behind at the station, I expect the project to really take on a new life. However, I've got some back up plans up my sleeve, including good ol' Max/MSP and a standard webcam, which I've used quite a bit in past installations. Just in case, I've been researching quite a bit on simple background subtraction and frame differencing patchers that can help achieve the silhouetting effect and facilitate some rudimentary image tracking.

One of 3 useful patchers I came across that may serve as a starting point for what I would build. This one displays pixels in white only when there is a change in them, effectively removing the static background and isolating the moving person in the foreground.

Mainly, I just always enjoy an excuse to learn something new for a project. Having fun hacking away!


Click here for the next post about this Project >>