Archive for October, 2010

Patterns in Language

Friday, October 29th, 2010

Treemap of My Language from Last Week

For the period of one week I used this keylogging tool to capture all of the language I used while on my laptop. This investigation was an attempt to insights regarding my thought and moods – it was inspired by our focus on language in the class Rest of You. Once the data had been captured I cleaned up the resulting text file to removed some of the most common words that I knew would skew the analysis (e.g. “the”, “that”, “this”, etc).

As luck would have it, during this same week we reviewed a treemap visualization library in Expression Frameworks.  This treemapping library for Processing was created by Martin Wattenberg and Ben Bederson’s and adapted by Ben Fry. It provides a simple (ok, maybe not so simple) framework for developing treemap visualizations from hierarchical data. In the case of my data, the hierarchy was created by the frequency of each word.

After some time struggling to understand how the library worked, I was able to generate some visualizations that looked interesting. I am still a long way from truly understanding the nuances available in this library but I have succeeded in tweaking it to add dynamic coloring and to limiting the words that are displayed. Here is a link to my code on github.

From this treemap, you can tell that last week I was working a lot on stuff related to my csa, data and sensors, heart rates, and time. Two interesting directions that I plan to explore in future key logging efforts include looking at the moods of my words, as was done by Patricia Adler and Patrick Hebron, and exploring the process of writing on a computer, as Josh Clayton did in a brilliant manner.


AMUP original [Proximity Sensor Test]

Wednesday, October 27th, 2010

Over the past week I have started prototyping my Air Mashup (AM-UP) instrument. My first step was to purchase a few different varieties of infrared proximity sensors to identify the type that will work best. Based on my research I ruled out ultrasonic rangefinders because their beam is too wide, and would suffer from interference in the set-up that I envision.

I tested the sensors to understand two important characteristics of how they function: (1) how far apart do I need to place these sensors in order to avoid interference – this will determine how close I can place the sensors within my sensor array; (2) what is the sweet spot from a range perspective for each sensor model – this will determine the distance from the sensor that my gestures need to be confined to.

So here’s what I found out: I identified the best infrared sensor model for my instruments and I discovered that the distance between each sensor in the array needs to be roughly 12-inches and the sweet spot for each sensor is between 5 – 45 inches.

Here is an overview of the next steps I need to take:

  1. write arduino code that is able to recognize different gestures that I can use to control the sound. This will determine whether I need to keep my gestures to simple up and down motions, or whether I can use more complex movements.
  2. determine what additional buttons I need to add to the instrument for control of things such as clips selection, and filter/effects routing. I am currently considering using a monome to control the clip selection.
  3. design and build the case/stand for the physical device.
  4. write a max patch (or a live for max patch) that enables me control the clips using the data that is sent from the sensors which are hooked up to the arduino.
  5. prepare the music that I want to use for my performance by creating clips and loops for use in ableton. Then play around with the clips to determine how I want to compose my performance (essentially, I need to plan my composition).

At this point most of my research will be accomplished by creating prototypes that are increasingly more refined. I will need to polish my max and ableton skills in order to get my so of the programming work completed. Lastly, I need to listen to the music I will be using as a source to identify the loops I want to create.


Excited, Emotional and Moody

Tuesday, October 26th, 2010

Since the beginning of the semester I’ve had a one track mind in regards to the Rest of You class. I’ve wanted to find a way to track my level of excitement, my emotions and moods so that I could gain insights about what makes me happy (or better yet, how I can lead a fulfilling life). This direction stems from my long-term interest to understand myself better and specifically to understand my relationship to the world around me. I strongly believe that I am responsible for my emotions; this is not to say that I can control the way my body responds to the world.

Emotions encompass more than just the physiological responses of my body, they also entail the conscious process the classify our physiological responses into emotion categories. I strongly believe that I have the power to be the conscious writer of the stories that my brain creates and that define my conscious experience of the world, and my emotional response to these experiences.

With all of this in mind, to succeed in my project I felt that it was important for me to find ways to capture three different types of data: 1) information about my physiological level of excitement; 2) contextual data about what I was up throughout this research (e.g. my activities and locations); and, 3) personal thoughts and reactions related to what’s going on in my life. From conversations with others during the last couple of days I am also considering adding a fourth data dimension to this project: capturing input from people who interact with me regarding my current mood.

So here is how I plan to capture all of this data:

  1. Physiological data: I’ve built a small portable biometric sensing device (pictured above) that will capture my heart rate and galvanic skin response. This little device will travel with me 10 – 12 hours a day. Whenever my vital signs go above a pre-defined threshold the light on this device will flash to request for me to categorize my excitement is either a positive or negative experience.
  2. Contextual data: I will use two separate processes to capture contextual data. First, the portable device that I have created will feature a GPS component that will enable tracking of my location and time of day. Second, every night I will update a calendar with detailed information about my activities for that day.
  3. Personal perspectives: to capture my feelings, emotions and moods I will use two main processes. First, the buttons on the portable biometric device will capture high-level positive/negative data regarding my emotions. The more informative means of capturing data will be via twitter feeds. Every time that I make a positive/negative input on the biometric device I will also send a tweet with a brief note regarding my current mood.
  4. Perspectives from others: to capture data from others regarding my emotional states and mood I will request that people send tweets to a specific hash tag with their input (or I may set-up a text message inbox – any suggestions of a good service where I can set this up, please let me know).

One last potential data set that is ripe for analysis is my emails and blog posts. I have not yet decided whether or not I will add this lens to the project. The good thing about this data is that I don’t have to proactively capture it, I can decide to analyze it on the back end.

As I capture data over the next month I will work to develop visualizations that will help me gain insights from all of this information. There are a couple of perspectives I know I want to explore: 1) my mood based activity type (particular classes at ITP, time spent with specific people, time spent doing yoga, time eating or drinking – better yet drunk); 2) my mood based on time of day or week (when am I most happy); 3) my mood graphed geographically (what places excite me?); 4) my perspective versus the perspective of those around me.

Here are some relevant links to pictures and videos related to this project:

1) geo-based visualization prototype of my GSR readings during a 2-hour bike ride
2) treemap visualization based on key-logging done over an entire week
3) pictures of the prototypes of my portable bio-metric device


Designing the Structure of Data Objects

Sunday, October 24th, 2010

The more programming I do, the more I understand the importance of the structure (or information architecture) defined in my code [not to mention, the nerdier the titles of my post begins to sound]. I have never doubted that this was true, especially after working for many years doing marketing for technology companies such as IBM, Microsoft, and HP. That said, I have only recently gotten a first hand understanding of how structure can limit or liberate the potential of my sketches (and also drive me crazy).

Recently, while working on multiple data visualization projects I’ve had to start thinking about the structure for capturing, managing and analyzing data. I’m not talking here about databases structures (these are much more complex); I am referring to the structure of objects that hold data. There are two main approaches that I have been using in my sketches, each has its own benefits and draw backs.

Let’s begin by envisioning a data table. Each column on this table holds a data field, while each row holds related entries in each field. Now let’s add one additional bit of complexity, most data visualization projects require that we read, manage, and integrate data from multiple different sources.

So here are the approaches that I’ve used so far (I’m sure there are many others that I haven’t explored): (1) entry-based objects that hold all data related to a given entry; (2) field-based objects that hold all entries related to a given field.

So which approach is better? Neither, it all depends on what you are trying to do. The first approach is often more effective for complex data sets that include different types of information, such as integers, floats, strings, and links. A benefit of this approach is that it enables the integration of data from multiple sources into the same object. The downside is that to use this approach you need to know the structure of the data.

The second approach provides a bit more flexibility because you can create a generic “field” data type that can be used to read an undefined number of fields (as long as they have the same datatype). If you want to create one application that reads different sets of data with similar data types this approach may be best.

Another benefit of this approach is that it makes it easier to compare multiple different values within a given field, since they are all contained within the same object.


Geo Mood Map: Data Visualization Prototype

Saturday, October 23rd, 2010

I have long been interested in the idea of doing a mood mapping projects for personal understanding and growth, quality of life improvements and a search for fulfillment. These are lofty goals and big expectations, I know. I am aware that my project probably won’t amount to much in these areas but in the least I expect to be able to glean some simple insights regarding what makes me happy.

Since the beginning of this semester my focus at ITP has shifted towards the exploration of personal wellness. For a long time I have been interested in helping myself and others create possibilities and fulfillment in their lives. Since last year many seeds have been planted that are now enabling me to make this shift. Here is a brief overview of the various “fertilizers” that have helped create this budding possibility:

  • Over the last year I have had a personal focus on my physical and mental health. Health was actually my theme for the year. As part of this initiative I am practicing yoga and meditation. I have also overcome an addiction that was acting as impediment to my personal growth.
  • During this time I have also developed a new relationship with food. This was largely driven by the need to start cooking to save money, but has now blossomed into my involvement in local food communities such as my neighborhood CSA.
  • During the past year I’ve also had the opportunity to listen to and participate in many inspiring conversations. Two, amongst the many, people who have contributed to my life in this area are my mom (aka my yogi) and Linda Stone (who has some very interesting notions about post-productivity computing – check it out here).

One thing that has become clear to me over the past several years is that life cannot be reduced to language. I know this sounds like an obvious claim. However, living in our mind-obsessed society I have noticed that I have become accustomed to focusing (better yet, limiting) my experience of the world around to what can be grasped by my mind and described in language. I know that my mind does not have the bandwidth to enable me to experience the true richness of the world all by itself. The models of the world that reside in my mind and the language which acts as a filter on my experience arbitrarily cuts up the real world into small dead symbols that lack the beauty of the reality to which they point.

Ok, now I am getting preachy and rather esoteric. To bring it back down to earth, I just want to say that it is only through our body and its intelligence that we can more fully experience life. So now bringing it back to my current pursuits, I want to work on developing technologies that are designed for human beings, not human minds.

I’ll keep today’s journal entry focused on the big picture. I’ll share more about the specifics of how I plan to achieve this in the next couple of days. In the meantime, you can download the code for this patch from github. I’ve already created a few prototypes so progress is happening at a steady clip.


Finding the Right Gas Sensor

Friday, October 22nd, 2010

Over the past couple of weeks we have been doing a lot of research regarding sensors that can detect the level of various gases present in our environment. Our main focus has been on finding detection of Carbon Dioxide, Carbon Monoxide, Oxygen, and Various VOC vapors (e.g. ammonia, benzene, formaldehyde, etc). After some research, a colleague of ours found a company called Figaro that offers sensors that can detect most of the gases we want to monitor.

We have come to realize that the sensors available that work with the Arduino (which we will use for prototyping) are not able to provide data regarding individual gases. They only provide data regarding the presence of a variety of gases. For example, Figaro’s TGS822 sensor detects a whole family of VOCs: Ethanol, Tulene, Acetone, Benzene & Xylene. In order to sense gases individually we would need to use high-end solutions that are outside of our current budget range.

During our research we came across some interesting information about the higher-end gas sensors available on the market (thanks to the help of Eric Rosenthal). Here is an overview of what we discovered:

NASA is doing some interesting work related to creating sensor arrays for sensing gasses. The applications of this research is related to leak detection, fire detection, and to create an “electronic nose”. These individual projects are aiming to create system that achieves these goals, using a package that is about the size of a postage stamp. Unfortunately, most of NASA’s work is associated to creating sensors that can used in engines and other high-stress environments.

In an article from the magazine Science Daily we learned about laser technology that has been recently developed in the European Union that enables the use of laser to sense a wide array of gasses. This type of sensor does provide the resolution we ideally wanted – enabling the detection of individual gases. Now if we can only get ITP to give us several thousand dollars to support this project then we can look into one of these.

Now that we have found most of the sensors we need, we have begun to focus on creating the circuits and writing the code to collect data and publish it to Pachube. Next up I will post some information about this step.


AMUP original [NIME Proposal]

Thursday, October 21st, 2010

Last week I presented the concept for my instrument, dubbed “Air Mash-up”, to my New Instruments for Musical Expressions (NIME) class at ITP. This concept was inspired by my desire to create an interface that enables DJs to perform using more impactful gestures. It harks back to the Theremin, one of the first ever electronic instruments.

Above is a brief video of the presentation I put together, which features an overview of my inspiration along with a description of the design of the instrument. I have already begun to put together an initial prototype of this instrument, which I will share on my journal in the next couple of days.


Developing Framework for Visualizing Biometric Data Across Time.

Wednesday, October 20th, 2010

I know the title of this journal is as long as it is boring – I am not feeling too creative today. Nonetheless, I am quite happy that I have recently made progress on a bunch of data viz projects that I have been playing with during the past couple of weeks. Let’s take it one step at a time, so today I will focus my efforts on sharing a video of a simple visualization that I created to map data from a data collection session I did a few weeks back.

Here is a link to one of my previous journal entry where I briefly talked about this endeavor.

My efforts during this session focused on tracking my galvanic skin response, the light level in the room where I sat, and the proximity of others. I also took notes regarding my activities during this two-hour period. Unfortunately, I have not been able to integrate this information into my visualization yet.  As you can see, my visualization is essentially comprised of three bar charts coupled with a timeline running and a navigation running along the top of the screen.

Over the coming weeks I will integrate information about my activities in the area between the navigation bar and the timeline. This information will contain brief descriptions and images of the content I was consuming (and the people with whom I was interacting). This part of the project is very important because over the next several months I hope to keep a detailed track of my biometrics data and activities with the goal of creating a personal mood map.

The ultimate goal of this projects is to help me to lead a more fulfilling life by making me more aware of things that make me happy, sad, annoyed, elated, frustrated, and so on. At this point I am trying to put together some of the scaffolding to make this a reality. I have made good progress on the biometric circuits and I am continuing to improve the data visualization elements of this project. However, I still need to work on creating a process that will enable me to effectively capture data regarding my activities and moods.

Here is a link to the code for this visualization (it is available on GitHub). You can download the code directly from this link: https://julioterra@github.com/julioterra/Time_Bio_Mapping.git - Stay tunned, more on all of these threads will be posted soon…


Arduino Meet Android, Thanks to Amarino

Tuesday, October 12th, 2010

Ok, so I like to give credit where credit is due. With that in mind I want to give a shout out to the Bluetooth Mate from Sparkfun and to Amarino. If it were not for these two dear friends I would probably be pulling my hair out right now, while stressing out because of all the other projects that would be feeling neglected and begging for my attention.

Over the coming week I still need to edit both my Arduino and Android sketches to make sure that these devices are able to send and receive the appropriate data. I also plan on using the Android phone to do some GPS logging. Now let me tell you a bit more about the Bluetooth Mate and Amarino:

The Bluetooth Mate, which I received last week, had been unfairly judged as a snobby high-priced component that was actually making me feel stupid for having spent so much money when I could have gone with the $20 option. However, what I soon found out was that this component makes up for its high price by being willing to play nice with other devices such as computers and cell phones. This meant that I was able to avoid hours and hours of stress that often associated to configuring bluetooth transmitters.

Now let me shower some praise on the Amarino. Amarino is comprised of two libraries, one for the Arduino and the other for the Android. Together these two resources make it much easier for you to establish communications between the Arduino and Android. That is not to say that this process is easy – some understanding of how to code for Android is still required to customize the code samples available on the Amarino site. You also have to know a little bit about how to use the terminal so that you can install the Amarino application on your Android phone. Below I’ve included some important tricks that I learned.

In order to install Amarino on your Android phone you need to add the path to thr tools folder from the Android SDK to your PATH. Supposedly this can be done by creating a file called .bash_profile that includes this path, and then saving this file to your home directory. However, I had to take a different approach for this to work on my computer – I had to edit the file called paths directly from using the Terminal application. This is how this is done:

  1. Launch the Terminal application
  2. Open the paths file using the following command: “sudo pico /etc/paths”
  3. Add the path to this file (do not change any of the other paths)
  4. Press Ctrl-X to exit the text editor
  5. Confirm that you want to save the file by pressing “y”
  6. Press return one last time to exit the editor (I know it’s repetitive)
  7. Exit the Terminal program
  8. Open the Terminal application
  9. Type the following command: “echo $PATH”
  10. Confirm that the appropriate path has been added

My (Sort of) Wearable

Tuesday, October 12th, 2010

Last week I focused my efforts on creating a wearable sensor circuit for logging my biometric information. After hours of soldering and de-soldering various components to more than one breadboard (and in the process, ruining several of these components), I was finally able to put together a bracelet that featured a working accelerometer and galvanic skin response sensor. A dead microphone and heart rate monitor were given a brief honorary moment of silence for their valiant service (or attempt at service).

For the GSR sensor I discovered that the electrodes definitely work best. The two little rings that you see in the pictures above did work but nowhere near as well as the two electrodes I borrowed from Mustafa. Here is a link to the Arduino code I have been using (please note that it is still a work in progress).

After all of this work my small battery (90mAh) was only enough to keep the circuit running for a short time and my openLog was not working properly. To top it all off, it took me several more days to get the bluetooth connection working so I was not able to log any data. In another post I’ll share more about getting my arduino’s bluetooth connection working and connected to an Android phone via the Amarino.

Despite all of these difficulties I am excited about the propects of being able to create a wearable biometric circuit. I was happy to play around with the Lilypad, as I’ve wanted to work with one of these for a long time. I have already began to envision a cleaner version of my wearable sensor circuit, which I plan to create over the next couple of weeks. Here is a brief overview of some of cool stuff that I am considering to include in my design:

Fabrick.it wearable components such as the snap bricks and the ribbon. These nifty components were designed by a fellow ITP professor (Despina). Now I just need help trying to sew this shit together.

Textronics “energy activated fabrics” are also pretty cool. Not sure what they mean by energy activated but they sell some conductive fabrics that have sensors and transmitters that measure heart rate and can communicate with the Polar sensors. Thanks to Mustafa for introducing Tamar and I to this stuff – now if only the lady from this company would get back to us so that we could put in the order.