Archive for July, 2010

Seesaws and Puredata

Wednesday, July 21st, 2010

Seesaw Application Screenshot

During the past several weeks I’ve been working with a New York-based interactive media artist named GH Hovagyam. I’ve been helping out GH with the programming for his latest installation, which he has been building at the Lower Manhattan Community Council artist studio in Governor’s Island.

The piece is titled Seesaw. It is an interactive installation that features a seesaw in a room next to two video projections that play clips from the movie “Two for the Seesaw”. People’s interactions with the seesaw control the video projections.

There will be an open studio weekend from this Friday through Sunday. If you are in New York stop by to check it out (I will probably be there on Friday only). I will make sure to capture some video to share here – it will actually be my first time playing on this seesaw.

Pure Data to the Rescue
I got involved in this project through Hans-Christoph Steiner, the instructor from my Introduction to Dataflow Audio Programming class at ITP. Hans referred GH to me, after I told him that I wanted to continue building my Puredata skills.

When I got started, GH had already build an initial prototype that worked for about 30 minutes at a time. My task was to create another version of this patch that would work without crashing for extended periods of time, and that was more streamlined. This task was easier said then done – especially considering that I had no previous experience using GEM (PD’s graphics engine).

GH chose to use Puredata because it is a powerful open source dataflow programming language. Though PD’s video engine is a bit less stable than Max/Jitter, it can be used without any concerns regarding licensing.

The Patch Architecture
I divided the application into a few core components. Some of these components were created as sub-patches, others were created as abstractions:

  • Application controller (main patch): Launches the application window and starts the video. It also houses all of the abstractions and sub-patches listed below.
  • Midi reader (sub-patch): Reads incoming MIDI messages and processes these messages to determine whether to turn or off video projection one and two.
  • Clip selector (abstraction): Selects a new clip and outputs (using the send object) the path and file names for the new video and audio clips.
  • Clip controller (abstraction): Loads and plays new clips based on messages from the clip selector and turns on and off video and based on messages from Midi reader.

Running the Patch
In order to get this patch running on your computer there are a few settings that you need customize first. Notice: If you don’t customize these settings the patch may crash your computer.

First, open the pd window_settings subpatch that is located in the Create & Destroy Video section of the main patch. In this patch, update the window dimension message that is highlighted in red.

Next, you need to update the path of the video and audio files and the file name database in the clip-selector patch. I recommend that you open this patch directly to make these changes. First update the path for video and audio clips. Make sure to append “%s.mov” to the end of the movie file path, and “%s.aif” to the end of the audio file path (as this is what makes sure the file names are inserted appropriately).

Last, update the database that is stored in the “coll” object. For the patch to work properly the video and audio clips need to share the same file names. Update the data in this file but make sure to keep to the proper format (“index number, path;”). To finish up, update the random object to make sure it is set to the number of items that exist in the database.


The Activities and Thread Classes – Learning Java for Android

Monday, July 12th, 2010

It has been a long time since I last posted to my journal. I’ve had a few hectic weeks as I’ve been working to finish the Android games course, I’ve started working with GH Hovagymian on his seesaw project, and had to prepare to share my second semester project from ITP at the Sony Wonderlab.

Today I am going to provide additional information regarding the structure of Android applications. A few weeks ago I wrote a post about the Java View class, which provides the basic building blocks for creating user interfaces. Today I will focus on providing an overview of the Java Activity and Thread classes.

Activities
An activity is a focused thing that an application can do. Almost all activities enable some type of interaction with the user. It is important to note that an activity is not an application. Applications are referred to a as a task in Android. Tasks usually contains multiple activities and often other components such as views, services, etc.

All activities live inside a window on the screen. Activities can be presented to the user as full-screen windows, as floating windows or embedded inside of another activity. There are two main methods that almost all  Activity subclasses will implement. These methods play crucial roles in the processes of setting up and leaving activities:

  • onCreate(Bundle) is where you initialize your activity. Most importantly, here you will usually call setContentView(int) with a layout resource defining your UI, and using findViewById(int) to retrieve the widgets in that UI that you need to interact with programmatically.
  • onPause() is where you deal with the user leaving your activity. Most importantly, any changes made by the user should at this point be committed (usually to the ContentProvider holding the data).

Activities are managed via an activity stack. The most recent activity is always added to the top of the stack. The previous activities remain on the stack and can be reactivated simply by removing the new activity from the stack.

The startActivity(Intent) method starts new activities by placing them at the top of the activity stack. It takes a single argument, an Intent, which describes the activity to be executed.

Activities ultimately have three main different states: foreground activity, visible activity, background activity. Foreground activities live on top of the activity stack. Users are only able to interact directly with activities that are in this state.

Visible activities can be viewed by the user but they are not at the top of the stack. The user usually cannot interact directly with these activities until those that are located higher in the stack are removed from the stack.

Background activities are lower down in the stack and are not visible to the user. These are the first types of activities to be killed when an Android device needs additional space.

To switch between these various states seven activity methods can be implemented in an Activity subclass (two of which I already discussed above). Here is the full list:

  • protected void onCreate(Bundle savedInstanceState);
  • protected void onStart();
  • protected void onRestart();
  • protected void onResume();
  • protected void onPause();
  • protected void onStop();
  • protected void onDestroy();

Here is a graphical representation of the Android Activity lifecyle, from the Android Developer website.
Android Activity Lifecyle

Activities can be killed by the OS at any time if they are not in the foreground state. The following methods can be called when an activities location in the stack is changed: onPause(), onStop(), and onDestroy() – however, only the onPause() is guaranteed to be called before a process is killed. Therefore, it is recommended that you should use onPause() to store any data when you are removing an activity from the foreground.

Threads
Threads enable android applications to run more than one activity at a time. Threads can be created in two ways: by extending the thread class and overriding its run() method; or creating a class the implements the runnable interface. In both cases, the start() method is called within an activity to actually execute the new Thread.

For my game I leveraged the code example provided by Matt. He created an extension of the Thread class called GameThread that includes the required methods (run, start, and stop). It also features a few functions that are meant to be overwritten by the main activity when it creates the thread during runtime. I will post the current code and screenshots of my game by early next week.