Category Archives: icm

🎵🔴🔶🔵 lego sound sequencer 🔵🔶🔴🎵

the winter show

Here’s the project setup, an hour before the show (and me, nervous and excited.)



things I learned:

it’s fun, it’s funny

Once people got it, they really loved it! People described it as “fun,” “addicting,” that they could “just keep going.” People smiled, laughed, got silly with the sounds: “ba-ba-ba-ba…. aaaaah….. hellooooooo?”


I would feel a little nervous if I saw them put on headphones without reacting… then immediately relax if they started smiling. “That’s so funny,” they’d say about the sounds. “Like a bunch of animals,” or “a load of babies.”




I feel good that I built up enough of a system that people started imagining other applications for it…

What if you could use it as a gameboard base, and create other blocks, puzzles, and drawings to play on top?

What if you could build upwards? Or save your song?


fewer bricks is better

It was often more effective to play with a few bricks at a time. More than roughly 2 of each color made everything a very noisy wash. From an interaction standpoint, it felt important to be able to see the causality of what you were building.


kids just wanna play

Kids just want to play with legos. They’re the best testers in that way, regardless of your project’s audience, because they’re not going to listen to you… they’re not going to read… they’re just going to explore hands first.

Here are some nice things that thoughtful little people made.

IMG_20171217_174200Like some of the adults, too, the kids were driven by what looked cool, not what sounded best. In fact, the temptation to make something with Legos was so strong that even people who were just loitering would just drop some bricks on the canvas for fun.


go with the grain

In some ways, I made a project which is against the grain of legos. You can’t push things together very hard, and you can’t stack things to any novel result.

In some ways, it was much better for adults, because it required deliberate care and attention. One parent said of their kid, “She doesn’t really know what to listen for.”
Adults might love the concept behind it, or start dreaming about the business prospects…
Legos shouldn’t required deliberate care and attention. In this sense, my project opposed Lego-ness.

Screen Shot 2017-12-22 at 1.36.28 PM.png

My project also opposed the grain of public interactive showcases: it was very sensitive to changes in noise and light. The tech wasn’t too forgiving– errors in image-processing sometimes felt jarring and obvious.

Finally, as more visitors came and played with it, it would often behave more and more “broken” until I restarted it. Though, I think that’s probably common for many projects. Still, my dream for it was for recorded voices to accumulate, erase, and become more interesting over time, capturing the vibe of the last musician.



pcom: adventures in fabrication

Spent this weekend at home recuperating from the week.
It was a rainy few days where I only left home once per day.

I was so tired. I moved in slow leaps and small bounds on fabrication… something that is very new and daunting to me, but felt very good after I reflect on my work Sunday.

Also, at this point, p5.js finally works!

and fabrication is coming along… (cardboard mockup)




trying different interfaces for record dashboard

Screen Shot 2017-12-04 at 2.25.42 AM.png

workspace circa 5pm today


to-do list, circa 4pm


icm final ✨✨

The last two weeks have been tough for me on many levels. I decided to pivot my final project from sandwiches. I got some great feedback on that project, but just not enough time to flesh it out the way I want.

For this week (our last week), I instead built out a virtual version of my pcomp project, which is a lego music sequencer. Some of the functional code I had already written last week.

Here’s the working prototype (which is sure to keep changing!)

Screen Shot 2017-11-28 at 11.20.37 AM.png

I was at first avoiding merging ICM + pcomp for a long time, because I don’t feel like this project, as a browser game, totally stands on its own as a satisfying interaction. Still, this was actually one of the most challenging things I’ve coded so far in ICM. I feel pretty proud, and I learned so much from the experience.

I was up for at least 8 hours in Bobst, where it’s climate-controlled to keep you AWAKE. M2M chocolate and tea kept me going through the wee hours…

The process really pushed me to be very deliberate, detailed, and creative with my debugging. I was working with several layers of storing information…

  • built-in 1D pixel color array ([r,g,b,a])
  • analyzing a 2D brickArray (based on x,y coordinates, incremented out per brick width and height)
  • asking the 2D brickArray if each index item was more red, green, yellow, or blue based on a ratio of [r] vs [g] vs [b]
  • based on color, storing values “R” “Y” “B” “G” as strings into a 2D “songArray”
  • interpreting strings and playing corresponding sounds
  • stepping through a column of sound at a time, based on frameCount% intervals

Some stills from that late night:

Screen Shot 2017-11-27 at 7.58.12 PM.png

Setting up an array of brick colors… why is it not working?? Ah, because I used “push()” incorrectly.

Screen Shot 2017-11-27 at 8.32.18 PM.png

Finally, console is telling me where each brick is on the grid. Took a while to make sure “x,” “y,” columns and rows were aligned and not reflecting on the wrong axis.


If the sound was successfully stored, it would tell me in the console. (see: “BLUE, beatbox @ x2, y6”)

Screen Shot 2017-11-27 at 8.32.20 PM.png

When experimenting with p5 pixels, I wrote a function to draw a rectangle for each unit to make sure the computer was seeing the image.


Here I was experimenting with pixels, to see if p5 could read the blue lego image as “on top of” the white grid image.”

When we tested it in class, my tester got the premise of it pretty easily. People liked the idea. Here were the questions I posed:

  • What do you think a physical experience offers that this one doesn’t? and vice versa?
  • Imagine you can record your own sounds. How long do you think you would play with this before getting bored?
  • What kind of sounds would you like to hear?

The “found sounds” aspect really appealed to people. Mimi asked me to consider if “playing with legos” is the main interaction, or “building music.” Someone suggested screaming lego people…


sandwich side note:

Here’s some of the feedback I got from user-testing my sandwich prototype. I would love to continue this sometime, maybe during J-term? I have too many ideas!

  • “professionalism” section really made people think about their personal definitions
  • people chuckled when they were defining “sandwiches” 
  • introduction felt a little long/text-heavy (for some, not all)
  • because it’s modeled after a work test, you want to just get it over with (for some)
  • the ending was confusing. needed resolution. more frustrating than communicating a lack of agency (“I would think your code was broken”)
  • user-testing prototype worked really well!


sandwich storyboard

ICM final project boss battle
week 1 —-> week 2 👾

is a storyboard a sandwich?

This week I wrote copy text and gathered imgs for my sandwich game.

J gave me the idea to model it after CAPTCHA image testing, which distinguishes bots from humans.


As I started putting it together, the copy text naturally materialized as some sort of weird AI job application procedure….

Here are some simple gifs of “gameplay.”


Applicants are first questioned on their sandwich identification skills…


…then proceeds onto bigger questions. What is professionalism? How do we define it? (It can be a euphemism sometimes.)

Though as a simple AI perhaps you are just looking for straightforward cultural symbols of what it is. Parsing for white men at computers…

Screen Shot 2017-11-15 at 2.51.50 PM.png

By part 3 in the game, the user doesn’t have any control over which images are chosen.

The user might click on a mother nursing her kid as a “leader,” but all of their choices become ultimately zapped…

Screen Shot 2017-11-15 at 2.52.24 PM.png

The whole world turns to sandwiches.


icm final: work + art + sandwiches 🍞


I want to design a p5.js game where you are in charge of deciding what a sandwich is… and eventually you are in charge of larger definitions too, like what art is, and what work is.

work? art?

I’ve raved a lot about this Ursula Le Guin piece by now…. which has been sitting in my mind and igniting all sorts of things for the past year.

It’s such a good read.

Lately, I have been thinking a lot about words. What they mean, how they define and restrict human activity, how they create dominant culture, how they exclude groups from power

I focus on this quote from Le Guin specifically, since it sums up well what I want to illustrate.

Screen Shot 2017-11-08 at 3.27.30 PM

So what even is work? How are we parameterizing these words—words which are NOT objective whatsoever but often designed like gated gardens?

Screen Shot 2017-11-08 at 3.27.16 PM


why sandwiches?

Before sandwiches, there was soup (probably, this is broadly true.)

I played this game last week called Something Something Soup Something, where you play an alien entity tasked with serving humans soup. Only, you don’t know what humans think is soup!

You have the very grand responsibility of defining what soup-ness is.

Screen Shot 2017-11-08 at 3.27.18 PM

As I was playing this game, I thought of how people on the internet have been debating what sandwiches are.

Is a hot dog a sandwich?

Screen Shot 2017-11-08 at 3.28.35 PM.png

The discussion has become so intense (lol) that people have created charts like this to navigate your sandwich philosophy personality type.

Screen Shot 2017-11-08 at 3.28.37 PM.png

Is a sandwich defined just as substrates A, between substrates B?
If no, then what?

the segue

Somehow (somehow), I’m going to transition from asking people about whether something is a sandwich… to asking if a sandwich qualifies as art.

Screen Shot 2017-11-08 at 3.27.41 PM

And what about making sandwiches? Is it work?Screen Shot 2017-11-08 at 3.27.48 PM

I want to escalate to increasingly domestic and “non-work” tasks until the player might even think it’s absurd.

I’m reminded of an exhibit at the Whitney called, The History of Protest Art. I haven’t seen, but I would love to.

As one protest, an “artist” invited people to do an activity for an hour, like walking, surfing the internet, eating a snack, and call it “ART.”

So, it ought to be as mundane and as everyday as those tasks. But I would like there to be a tilt towards economics, gender… economics of gender.

Custodians, lunch ladies, construction workers as artists, for instance.

stop & frisk data, 2016

This week’s homework was pretty challenging for me. I spent most time reviewing Dan Shiffman’s videos to understand how strings and APIs work. (It did help this week, to do part of the quiz *first*, instead of in retrospect.)

The other hard part was figuring out what kind of data set to work with. I spent a good couple hours getting distracted between all these API sites…

These are the ones I narrowed down to, not even the full list!

Screen Shot 2017-10-23 at 12.33.07 AM.png

I’ve always wanted to do something rooted in my surroundings; especially after civic hacking stuff back home in Boston, it lends this sense of investment to do stuff in your own city.

Screen Shot 2017-10-23 at 12.36.28 AM.png

There is so much just on NYC open data alone. (Not to mention, multiple rat sighting maps…)

Screen Shot 2017-10-23 at 12.38.00 AM.png

Here is the CSV for NYC Stop and Frisk data, in the year 2016, taken from the NYPD site.

Stop-and-Frisk is a NYC-specific program aiming to lower crime by detaining and searching people “at random,” for dangerous assets, drugs, contraband. It’s known for being incredibly racially biased, and having pretty much no effect on crime.

I wanted to highlight with visuals how biased and ineffective Stop and Frisk is in general, by using a data set to show amount of people stopped last year.

Screen Shot 2017-10-23 at 12.49.29 AM.png

I debugged by using console.log() to continuously print out strings. Later on, I wrote functions to display only text for specific races.

My dataset is currently incomplete, since I never did figure out what “Q” was an abbreviation for.

Overall, this feels like an unfinished experiment to me, and I would definitely like to do more! The aesthetics are kind of rough and arbitrary. Technically, I would like to push myself by adding a layer of data— e.g. showing how the MAJORITY of people who were frisked did not have any contraband, any arms, any weapons whatsoever.


As someone who can be impatient with direction-following, I definitely reached my limit this week of “figuring stuff out on my own.” In other words— I couldn’t just subsist on skimming libraries and glancing back at in-class exercises!