Final Performance & Hand-in
Posted by Jake | Filed under Project Two
The final documentation post for project 2.
Project Overview
Project Title: Three
The project aims to create a journey that can be experienced as if one was there; choosing views, times and moods/emotions. A journey that can be both experienced and controlled by the three parties partaking in the performance;
- The Live Cinematographer
- The DJ
- The Audience
The piece is intended be an interesting and unique performance that changes each time it is experience, and provide a realisation of how differently places can be experienced due to time of day, your current feelings and what you are looking for.
Making the Patch
Building the patch in Max/MSP/Jitter was an on-going process, with constant tweaking throughout the project. As content was created, controller decisions were made and the amount of load that the Mac could take became clear, the patch evolved into its final state.
Throughout the project I tried to keep the patch as tidy and organised as possible; something I learned during the first Quartz Composer project. With a myriad of objects and hundreds of patch cords connecting them, keeping things tidy by aligning cords and using the send and receive objects, along with creating colour-coded areas for certain functions of the patch made error finding and debugging much simpler, and navigating the patch a breeze.
The patch makes use of several sub-patches and externals, from the resources folder and created by others. These are:
- Arduino2Max
- Smoothie
- analyzer~ by Tristan Lehan
The patch and all of its related files (excluding video and audio content) are in this .Zip file, and a screen-cap of the patch is below (right-click, view image to view full size)
Making the Controller
The cinematographer for this performance controls the content/locale of the performance, through an Arduino based controller with 3 potentiometers and 3 switches. Firstly, the patch is started and stopped with a toggle switch, the 2 locations can be jumped between using a side to side switch, and the time of day is cross-faded with a potentiometer. A momentary switch is used to trigger freeze-frames of key sights any time along the way of each location that the cinematographer chooses. In order to encourage crowd participation, the web-cam input or its pixel grid can be faded in and out of the visuals as required, using 2 further potentiometers.
The controller housing has been constructed from laser-cut acrylic, as it’s easy to do, relatively strong and has a good aesthetic. Exact holes were able to be made, making mounting the various components easy, and the plastic case will help avoid any accidental short circuits when mounting the Arduino PCB and other circuits. White spray-paint was used on the case markings to help them stand out in the dark, as well as 3mm LEDs to illuminate key points.The front and bottom of the case are latched on, and therefore easily removed if repairs are needed and to swap out the Arduino, and a small breadboard was mounted inside to make building and debugging the required circuits a simple task.
The Macromedia Freehand file for the case is here, and some images of it during construction are below.
Laser-cut top with switches mounted
Back with breadboard mounted, and Arduino mount markings
Switches, potentiometers & LEDs being connected
All wiring complete
The complete controller
Crowd Interaction
The audience’s control comes via computer vision; where a camera input is brought in to Max and converted to a monochrome grid of 9×12 ‘pixels’. The grid is then split into 3 columns, each column’s pixel brightness averaged, and then run into a series of ‘if’ statements, which determine which column and therefore which third of the audience is brightest, and uses this output to select which of the three views to use, left, center or right.
For the performance the glow-sticks are given out to the audience, to give them something to hold up and wave around to get the video to react. I had originally planned to put the sticks into a beach ball or balloons to further increase crowd participation, but the type I was able to get weren’t bright enough to be effective through the rubber. With more time and a higher budget for the project, I would have looked at something like LED throwies in balls, which could provide more light and be more effective.
This system works very well, as even with a camera that uses automatic exposure control because the three columns are averaged and compared with each other, it doesn’t matter how bright the room is overall. For the performance however the theatre will be as dark as possible, with the only sources of illumination being the screen and the audience’s control mechanism.
The Audio Control
The audio for the project was intended to be performed by a third ‘member’ of the performance; a DJ, but I was unable to procure one for the event (I don’t know any here in Wellington and I’m just a poor student). However, I did manage to get something as close as possible; two tracks created and made available for free download by an online friend from Canada a few years ago, mixed together, as if they were being performed.
The two tracks have two distinct “feels” to them, the first more upbeat and jovial, with the second a lower tempo with a moody element. This fits well with my performance, as over the course of the piece day slowly fades to night, and the city becomes a much more lonely place.
Soundtrack (.WAV, 54M) (Relaxxx & Found by Rephlexion)
The soundtrack controls the emotion of the piece; it is first analysed by the aptly named “analyzer” external object, and those values used to add effects to the visuals.
The loudness of the track alters the the brightness and saturation of the visuals; with low sound translating to a dim, gray-scale image and a high sound level translated to a bright, over-saturated intense visual.
High-pitch peaks and attacks are used to trigger a pixelated effect that quickly fades out, enhancing strong points in the audio and adding visual interest that relates to the grid overlay.
Finally, the sounds’ “brightness”, or difference in volume between the high and low frequencies, adjusts a motion blur effect on the visuals, which is targeted at being nearly zero at most times, and is only particularly noticeable during the song transition and at the end of the soundtrack.
Creating Content
The content for my performance ended up being rather difficult to capture on a budget. Firstly three of the Digital Media DV Cameras needed to be hired, which required three people. Next, they were mounted on a rig constructed from MDF that kept them pointing in three directions; the centre camera looking straight ahead and the cameras either side angled at approximately 45 degrees. This rig was then attached to the armrests of a hired wheelchair, to try and smooth the ride (although Wellington streets are terribly uneven). Unfortunately I forgot to take a photo of the rig, but it looked quite comical and I received many a funny look as I strolled through town.
Care needed to be taken to make sure the lengths of each section of filming were as close as possible. I used a sharp movement of the chair to enable me to line up the cuts in each of the three cameras when editing, and tryed to stick as close as possible to a specific route when filming the day and night sections. Once all the filming was done, for both of the locations at both times of day, I scaled the videos to be the same length, and they aligned very well. As the Cuba St. Section took longer to film, the video needed to be scaled more, which results in what appears to be a much faster paced journey. This fits quite well, with the fast paced inner city being contrasted with the slower paced waterfront stroll.
In the end, due to the low processing power of the Mac computers in use at the university, I was forced to heavily compress the video content; to 640×480 medium-quality photo-jpeg encoded .MOVs, at 12 frames per second. This managed to get a reasonable framerate of 11-12fps, which should look ok on a standard resolution projector. Some sample content is below; the three night views from the waterfront.
Left view
Middle view
Right view
The video content for the project is too large to be made available for download, as it is nearly 2gig and my bandwidth would be used much to quickly. However some of it is seen in the final video at the end of this post.
Coming Together
Below is a short video showing how the multiple elements of the project interact.
Final documentation video
The Performance
Overall the final performance went well, although there was a slight issue in that a light (I think an exit sign) caused the visuals to look left for most of the time. Other than that I feel it was a success and went off without a hitch. Below is a recording of a repeat performance’s visuals, albeit without an audience, although their interaction was simulated with a cellphone light in front of a webcam.
Performance video
Final Presentation
Posted by Jake | Filed under Progress, Project Two
- as opposed to Final Performance..
This went reasonably well, with some good feedback. The main points I took from the day have been covered below:
To make sure the piece is performed, rather than shown, which has a lot to do with my controller. Although it is too late to really change my controller, they way in which it is used will have significant bearing on the final visual result. For instance, a gradual progression from day-to night over the 5 minutes, and adjusting the trigger time based on the music. Also, the grid overlay will be used to add interest to the less-interesting parts of the loops, and the crowd overlay only used when the camera view has remained stationary for a significant period of time.
The audio for the patch, which is still quite undecided, could possibly be a sound scape of the locations, and is something I will look in to. However, it wouldn’t quite fit in with my concept of a DJ as one of the three participants of the performance, and I’m still leaning towards more of a musical soundtrack.
The audience lighting mechanism needs to be finalised. The suggestion of a large, inflated glowing ball came up, but the only financially viable way I can come up with to do this would be to put a bunch of glow-sticks into a transparent beach ball. Otherwise giving the audience glow-sticks to use or asking them to use their cellphones could be an option.
Here are some files from the presentation;
The Max patch as it was for the presentation,
The powerpoint presentation (wasn’t shown), along with some audio samples, 2 that I have quickly composed and one non-commercial song [link] [link] [link]
And finally the freehand file for the controller I plan to laser-cut
Max Patch, Content, Arduinos
Posted by Jake | Filed under Progress, Project Two
Woops,
Long time no blog, but hopefully this will make up for it as I’ve done a lot since my last entry. My Max/Msp patch is nearly complete, and seems to be working quite well so far. All that remains in it is setting up the audio and visual effects; as the video switching, syncing and cross-fading all seem to be in order, as does the computer vision. The patch (in text form) can be downloaded HERE. A preview of it can be viewed below (right click, view image for full size). Note that there are some placeholder items and the videos are currently disconnected from the cross-fader.
I have filmed my content and am currently in the process of compressing it to photo-jpeg format for use with the patch. With an hour of content looking in three directions this is proving rather time consuming! Once it’s finished I may post it to Vimeo if I have time tonight.
Myself and Pieter’s Arduinos seem to be lost in transit from Australia, and there has been no response from the company after several emails. Very hopeful that I will have a mock-up controller for Thursday however.
Filming Snag
Posted by Jake | Filed under Progress, Project Two
I had hoped to get the vast majority of my content on film this weekend; I’ve got a wheelchair and a rig for it; but the uni shop has now told me I cannot get 3 cameras unless I have 3 people in Digital Media each get one. My friends can’t make it into uni before the shop closes so it looks like I’ll have to wait until Monday. Yet again (this seems to happen often for me) the university staff have made life difficult by simply having no trust and a complete lack of motivation to help students..
Looks like I’ll be working on my MAX/MSP with no content over the weekend instead..
Digital Turntables?
Posted by Jake | Filed under Inspiration
I just ran across this video, the song is pretty horrible, but those digital turntables are pretty sweet, and I can think of a number of ways they could be further evolved.. Super cool.
