It has been a long time since I updated my journal with documentation regarding Air-Mashup, the new DJ tool I designed as part of NIME. The development process for this project was challenging, sometimes frustrating, and ultimately very fulfilling. It was by far the most complex physical computing device I have ever built. It forced me to stretch my abilities in realms of physical prototyping and software development.

Today I will focus on the iterative prototyping process I used to develop this project. Between the early October and mid-December I developed 5 different sets of physical prototypes and too many iterations of the code to count. Here is an overview of all my prototypes along with an outline of the software development challenges I encountered.

Prototype V0.1
The first prototype focused on testing the design of the structure that holds the proximity sensor and was constructed out of hand-cut pieces of foam core. It featured the proximity sensor and a laser light. In this phase of the design process I was able to identify the appropriate height for the “barrier” that ensures one’s hands do not come to close to the proximity sensor. This is important because the sensors effective range starts a few inches away from the sensor itself.

Prototype V0.2
The second prototype focused on testing the design of the control panel that enables users to control clips, loops, headphone monitoring, and filter sends. This prototype was developed using laser-cut pieces of foam core. This phase of the design process enabled me to identify several issues with the placement of the buttons. It also led me to re-evaluate the overall design of the piece, since I was unhappy with the large and clunky feel of this box.

Prototype V0.3
The third prototype was an utter failure. My focus at this stage was on testing a new approach for the design of control panel and the encasing for the proximity sensor. I decided to create this prototype using wood since I had a plank of basswood lying around. This was a mistake, I realized that it is always better to continue prototyping using easier to handle materials until the design is more developed. This piece went straight from the laser cut machine to the garbage.

Prototype V0.4
With the fourth prototype I was able to get back on track. My focus at this stage was still on testing a new approach for the design of the control panel and the encasing for the proximity sensor. This prototype was constructed out of laser cut watercolor board. This material is made of cardboard-like paper that is thicker and harder then foam core. This build helped me identify the final tweaks I needed to incorporate into the design of the control panel. It also led me to the inspiration to merge all of the control panels but to separate out components that hold each individual the proximity sensors.

Prototype V1.0
The final prototype is by far more complex than any of my previous builds. I went from working on a single control panel/proximity sensor combo to building out four integrated control panel/proximity sensor components. At this point in time I needed to finalize the physical prototype so that I could start preparing for the NIME performance. It took me several weeks to build this prototype: I had to order over 100 components and solder them (and often re-solder them) together to build the circuits; then I had to laser cut over 50 pieces of wood and plexi to construct the boxes. As can be expected, it took me much longer to build this prototype than any of the others. Here is a slideshow features pictures of the final prototypes and the build process.

Over the past two weeks I have made a lot progress on the project Air Mashup. First off, the project name has evolved (or devolved) into AM-UP – ok, this is really minor news. The real progress has been made in testing and setting-up the proximity sensors, which will make up a core part of the interaction experience.

In this journal entry I will first discuss the process I used to select and test different types of sensors. Then I will write a post about the development of my first and second generation prototypes, and discuss some of the main challenges I encountered.

Selecting the Right Sensor
I found three different types of range finders that I looked into. The main considerations that I had in mind with these sensors were: (1) resolution and range; (2) beam width and interference; and (3) price and value.

The first sensors to be considered were the Maxbotix Ultrasonic Range Finders LV and XL range. Though these sensors tend to be very reliable and easy to use, they did not offer the right resolution and value. The LV range has a resolution of 1-inch, which is just not enough, while the XL range has a resolution that is twice that but they cost $60 a piece. I also had some concerns regarding interference between range finders, since I am planning to space them out by 10- to 12-inches only.

The next sensors on my list were from Sharp’s Family of Infrared Proximity Sensors. These sensors feature the shortest range in the bunch but they also provide the best resolution. I looked at two models: the long range model that can measure from 15 to 150 cm, and the mid range model that can measure between 8 to 80 cm. These sensors also provide a narrow beam width and they offer a good price to value ratio.

The last sensor that I looked into was the Parallax Ping Range Finders. These range finders are much more complex to use than the previous two. They require that the sensor send out a signal and then listen for a “ping” to come back in order to measure distances. These range finders are similar in price and technology to the Maxboxtix. Therefore, they suffer from the same value and beam width issues.

After running these initial tests I decided that Sharp’s proximity sensors were the best solution for AM-UP. I went ahead and purchased seven of these sensors and shifted my efforts to developing prototypes.

Prototype One: Testing Dynamic Range

My first prototype was little more than a wooden board with three separate proximity sensors attached a varying distances. The purpose of this first test was to identify interference issues and to out the dynamic range of the sensor. Here are some of the key findings from this initial prototype:

1. the sensor’s beam width is very narrow. This means that each sensor only needs to be separated by about 11-inches to avoid interference. This also means that there is a small sweet spot where a user’s hand must be placed for the sensor to work.
2. the dynamic range of the sensor is sufficient but there are some important characteristics that need to be addressed in the code. First, if the user’s hand gets too close to the sensor (within 15cm) then the readings become unstable and unreliable. Second, noise occasionally creeps into the data and needs to be filtered out.

Over the past week I have started prototyping my Air Mashup (AM-UP) instrument. My first step was to purchase a few different varieties of infrared proximity sensors to identify the type that will work best. Based on my research I ruled out ultrasonic rangefinders because their beam is too wide, and would suffer from interference in the set-up that I envision.

I tested the sensors to understand two important characteristics of how they function: (1) how far apart do I need to place these sensors in order to avoid interference – this will determine how close I can place the sensors within my sensor array; (2) what is the sweet spot from a range perspective for each sensor model – this will determine the distance from the sensor that my gestures need to be confined to.

So here’s what I found out: I identified the best infrared sensor model for my instruments and I discovered that the distance between each sensor in the array needs to be roughly 12-inches and the sweet spot for each sensor is between 5 – 45 inches.

Here is an overview of the next steps I need to take:

1. write arduino code that is able to recognize different gestures that I can use to control the sound. This will determine whether I need to keep my gestures to simple up and down motions, or whether I can use more complex movements.
2. determine what additional buttons I need to add to the instrument for control of things such as clips selection, and filter/effects routing. I am currently considering using a monome to control the clip selection.
3. design and build the case/stand for the physical device.
4. write a max patch (or a live for max patch) that enables me control the clips using the data that is sent from the sensors which are hooked up to the arduino.
5. prepare the music that I want to use for my performance by creating clips and loops for use in ableton. Then play around with the clips to determine how I want to compose my performance (essentially, I need to plan my composition).

At this point most of my research will be accomplished by creating prototypes that are increasingly more refined. I will need to polish my max and ableton skills in order to get my so of the programming work completed. Lastly, I need to listen to the music I will be using as a source to identify the loops I want to create.

Last week I presented the concept for my instrument, dubbed “Air Mash-up”, to my New Instruments for Musical Expressions (NIME) class at ITP. This concept was inspired by my desire to create an interface that enables DJs to perform using more impactful gestures. It harks back to the Theremin, one of the first ever electronic instruments.

Above is a brief video of the presentation I put together, which features an overview of my inspiration along with a description of the design of the instrument. I have already begun to put together an initial prototype of this instrument, which I will share on my journal in the next couple of days.