My 28 months (the residency was originally set for four months) at the New York Times was transformational in many, many ways. Cascade, which I initiated with Mark Hansen as a conceptual prototype, became a full-fledged project supported by an entire team of designers, developers and engineers. Along with Jake Porway, Brian House, and Matt Boggie, we built OpenPaths, which continues to be an exciting model for personal engagement with data. Mark and I, working with Alexis Lloyd, also made Memory Maps, a prototype for archive exploration, in which news stories are interwoven with the personal history of the user.

These successful projects were of course accompanied by unfinished sketches, necessary failures and inevitable dead ends. I built a visualization tool for household power usage that went nowhere, a few failed archive exploration tools, and one particularly bad interface for visualizing personal connections on Twitter. The R&D group, conceived and led by Michael Zimbalist, is very much a place that encourages real exploration – and the inevitable failures that result. This freedom to explore and to push boundaries is what has made, and will continue to make NYTLabs fertile ground for ideas and innovation.

Which brings me back to The Office for Creative Research, the new company I’ve founded with Mark Hansen and Ben Rubin. OCR is a multidisciplinary research group focusing on new modes of engagement with data. We’re looking to partner with companies, institutions, scientists, museums – any individual, group or organization who is facing novel problems with data. A browse through our collective portfolio will show our range of approach, from visualization to algorithm design to performance and installation. Our unique range of skills, drawing from both the arts and sciences, give us the ability to tackle almost any problem, from the laboratory to the gallery, and everywhere in between.

We’ve outlined the mission of The Office for Creative research in this memorandum, released today, and you can see more of our work on OCR’s freshly-launched website. While we already have a set of fascinating projects on the go for 2013, we are looking for innovative new partners. Please get in touch if you’d like to explore the possibility of working with OCR. Also, we’ll be looking to hire talented people in the spring, so if you’d like to work in New York City, exploring the borders between data, technology & culture, send us a message. 

It’s going to be an exciting year. We’ll be running a series of workshops at OCR starting next month, and we’ll be publishing a journal at the end of 2013 documenting the progress of our research. For regular news and data-related commentary, you can follow The Office For Creative Research on Twitter – @The_O_C_R.

I’d be remiss not to end this post with a thank-you to the many talented people at the New York Times who made my time there so tremendously enjoyable. It’s a world-class organization, filled with world-class human beings, and I’ll always be grateful for having had the chance to spend time there.

Happy New Year,

-Jer

When the dust settles on the 21st century, and all of the GIFs have finished animating, the most important cultural artifacts left from the digital age may very well be databases.

How will the societies of the future read these colossal stores of information?

Consider the eBay databases, which contain information for every transaction that happens and has happened on the world’s biggest marketplace. $2,094 worth of goods are sold on eBay every second. The records kept about this buying and selling go far beyond dollars and cents. Time, location and identity come together with text and images to leave a record that documents both individual events, as well as collective trends across history and geography.

This summer, Mark Hansen and I created an artwork, installed at the eBay headquarters in San Jose, which investigates this idea of the eBay database as a cultural artifact. Working in cooperation with eBay, Inc., and the ZERO1 Biennial, the piece was installed outside of the eBay headquarters and ran dusk to midnight from September 11th to October 12th.

As a conceptual foundation for the piece, we chose a much more traditional creative form than the database: the novel. Each movement begins with a selection of text. The first one every day was a stage direction from the beginning of Death of a Salesman which reads:

A melody is heard, played upon a flute. It is small and fine, telling of grass and trees and the horizon. The curtain rises.

Before us is the Salesman’s house. We are aware of towering, angular shapes behind it, surrounding it on all sides. Only the blue light of the sky falls upon the house and forestage; the surrounding area shows an angry glow of orange. As more light appears, we see a solid vault of apartment houses around the small, fragile-seeming home. An air of the dream dings to the place, a dream rising out of reality. The kitchen at center seems actual enough, for there is a kitchen table with three chairs, and a refrigerator. But no other fixtures are seen. At the back of the kitchen there is a draped entrance, which leads to the living room. To the right of the kitchen, on a level raised two feet, is a bedroom furnished only with a brass bedstead and a straight chair. On a shelf over the bed a silver athletic trophy stands. A window opens onto the apartment house at the side.

From this text, we begin by extracting items¹ that might be bought on eBay:

Flute, grass, trees, curtain, table, chairs, refrigerator. This list serves now as a kind of inventory, each explored in a small set of data sketches which examine distribution: Where are these objects being sold right now? How much are they being sold for? What does the aggregate of all of the refrigerators sold in the USA look like?

From this map of objects for sale, the program selects one at random to act as a seed. For example, a refrigerator being sold for $695 in Milford, New Hampshire, will switch the focus of the piece to this town of fifteen thousand on the Souhegan river. The residents of Milford have sold many things on eBay over the years – but what about books? Using historical data, we investigate the flow of books into the town, both sold and bought by residents.

Finally, the program selects a book from this list² and re-starts the cycle, this time with a new extracted passage, new objects, new locations, and new stories. Over the course of an evening, about a hundred cycles are completed, visualizing thousands of current and historic exchanges of objects.

Ultimately, the size of a database like eBay’s makes a complete, close reading impossible – at least for humans. Rather than an exhaustive tour of the data, then, our piece can be thought of as a distant reading³, a kind of a fly-over of this rich data landscape. It is  an aerial view of the cultural artifact that is eBay.

A motion sample of three movements from the piece can be seen in this video.

Before Us is the Salesman’s House was projected on a 30′ x 20′ semi-transparent screen, suspended in the entry way to the main building (I’m afraid lighting conditions were far from ideal for photography). It was built using Processing 2.0, MongoDB & Python. Special thanks to Jaime Austin, Phoram Meta, Jagdish Rishayur, David Szlasa and Sean Riley.

Since early in the year, I have been working with my mother Diane Thorp to produce hand-woven textiles that contain non-repeating patterns. Weaving Information Files (WIFs) are produced via a custom-written software tool, and are then woven on a 16-harness floor loom equipped with an AVL Compu-Dobby interface.

Here’s a zoomable view of six metres (almost 20 feet) of the handwoven result:

You really (really) want to hit the fullscreen button and zoom in – it’s a 95 megapixel (25,283 x 3,738) image. Alternately, you can go here to see it in a bigger window.

You can read more about the project here, and you can see a set of images documenting the project here. The next step is to weave a much longer section – we are aiming for something above 100′.

If you’re in New York, you can see the 6 metre long section from Infinite Weft, on exhibit at Bridge Gallery in NYC, until October 10th.

Bridge Gallery
98 Orchard Street
New York, NY 10002
Subway F, J, M, Z Delancey/Essex

So far, the discussion has been mostly around the characters of the Avengers, at a collective level. Lots of data is available about each individual character, as well – for example we can look at any Avenger and see every appearance they’ve made over the last 50 years. Here’s Captain America’s record number of appearances:

And Iron Man, who’s not too far behind:

An my personal favourite, Hawkeye:

In each of these graphics, I’ve marked the issues where the character has returned after a significant absence. We also, of course, see their first appearances (Hawkeye’s being in issue #16th, ‘The Old Order Changeth‘). You can see a pile of other Avengers’ ‘appearance maps’ in this Flickr Set – if there’s another character you’d like to see, let me know.

For the first time here we can see that we can gets some information about the individual issues past the issue number. We can look at the title, the characters who appeared in the issue, the geographic locations involved in the issue (from Alaska to the Kree homeworld), and more (the Comic Vine API offers the possibility of concepts to be linked with individual issues as well, but this information hasn’t been well-populated in the wiki).

One thing that you might have noticed from the graphics so far is that there are a lot of spikes – issues in which a lot of Avengers characters are present. The most spectacular example of these ‘party issues’ is Volume 3, #10, ‘Pomp & Pageantry”, in which a whopping 119 Avengers appeared! Here are all of these party issues since 1963:

We can see that these heaps-of-heroes issues are a pretty new phenomenon – and also that the current Avengers writer, Brian Michael Bendis, LOVES a party. He’s written lots of issues with more than 30 avengers, and even a couple with more than 50.

Which brings us nicely into a discussion about creators. So far we’ve been focused mainly on fictional characters – what about the real people that made these comic books? Like, for example, Sam Rosen, and Artie Simek:

Mr. Rosen and Mr. Simek hand-lettered all of the dialogue, and drew all of the word balloons for most of the first 50 issues of the Avengers, most often alternating back and forth, issue to issue. They’re part of a group of about 7 letterers who have been responsible for most of the Avengers typography:

Similarly, we can see that there are about 10 people who have been editors on the series for long stretches of time:

You don’t see nearly this kind of consistency with pencillers:

Or inkers:

I wondered after getting a look at how these creators were involved in the history of the series, if perhaps they (particularly the writers & editors) might be responsible for some of the content decisions that I examined in the last. For example, are there certain editors or writers who included more female characters in their books?

I overlaid a heat map onto the creator maps just saw above, with red stripes to indicate a high number of female characters and blue/green stripes to indicate the boys-club issues. Here are all of the editors again:

And all of the writers:

While it probably begs for some statistical analysis, it does seem that the gender balance gets a boost when Jim Shooter takes up the series in the early 70s. Indeed, he’s in charge during the high water mark of Avengers feminism in 1983-1984, a level which the series never gets back to.

We can see similar correlations for the numbers of gods/eternals per issue:

Or robots/synthezoids/androids per issue:

From these we can see that while Brian Michael Bendis DOES like to party, he DOESN’T particularly like robots, and definitely isn’t a big fan of the gods.

Besides that, what have we learned from this two-part data-exploration of the Avengers? You’ve probably learned that I have too much time on my hands. I’ve learned that I really need to get my old collection out of storage and revisit some of these excellent stories. I’ve also learned that, if there’s one form of punctuation that defined the silver age of comics… it’s the ellipsis. So, to finish us off, here’s a medley of the 53 ellipsified issues in the history of Earth’s Mightiest Heroes:

(You might see a blank box here, in which case you might want to try viewing the page in Chrome).

Last week, I was pointed by Santiago Ortiz to this excellent chart made by Theo Zaballos, in which he plots the relative interestingness in Avengers characters from the animated series, over time. It’s a fantastic example of the power of visualization to help us understand things – or, put another way, the power of building systems to think about systems. It’s also a reminder that visualization doesn’t always need to be pitted against huge, world-changing tasks – it can be useful in exploring small, fun, even seemingly frivolous things.

I started reading comics in 1985 (coincidentally, when I was 10). For years, I’d visit the comic shop every Wednesday, and pick up a stack of titles – and The Avengers was a real mainstay on my list. I was always more of a reader than a collector; my longboxes were full of dog-eared issues from incomplete series, which I revisited over and over again until the stories imprinted themselves in my brain.

There’s a huge storehouse of mythology, cultural touchstones, and real historical events contained in the pages of the 570 issues of the Avengers.

Inspired by Theo, and using comicvine.com’s API, I’ve put together a few datasets and some tools that I can use to visually explore some of this leotarded history.

The Avengers has been published pretty much continuously since 1963. Here are the covers of all 570 issues:

Now, you might be aware of a little, low budget art-house movie that’s being released tomorrow about this particular group of costumed heroes. That movie features 5 avengers – Captain America, Iron Man, Thor, Hawkeye, and Black Widow. But did you know there were 127 more Avengers? You may know that the Avengers were created by Stan Lee and Jack Kirby, but you might be surprised to hear that there were 184 other people who invented Avengers characters. In total, there have been 581 men and women who written, edited, pencilled, inked, colored, lettered, and otherwise created at least one issue of the Avengers.

Let’s start with a look at those characters. My first thought was to use images of the characters in my visualizations, but while the Comic Vine API provides images in all kinds of sizes, the styles of drawing are so varied that it ended up not holding together. Instead, then, I built a small tool that let me go through those characters and pick three colours that I thought represented them the best (everybody gets a shield!). Here are all of the Avengers in an overlapped plot that doesn’t really tell us much, but gives you an idea of what these icons look like:

These character icons can be drawn at any size, and give us a nice way to plot the characters that isn’t just dots or boxes. Here’s all of the Avengers again, this time plotted by their number of appearances:

Below Captain America is a cluster of the most consistent Avengers – Iron Man, Vision, Scarlet Witch, Thor, Hawkeye, Wasp, and Henry Pym (aka Ant Man). That blue and grey dot trailing just behind is Jarvis, the Avengers’ butler – who also happens to be an honorary member of the team.

Using those same shield icons, but sorting by issue so that characters in an issue together form a radial line, here is every appearance of ever Avengers character in every issue:

It’s not too helpful, but we can use this same system, and filter it by any number of criteria. For example, let’s look at just first appearances of Avengers:

You can see the same graphic in a timeline form here:

I built a little tool to let me assign three colours to each Avenger, so they’re all represented by small spheres (now would be a good time to look at the full resolution version of that image – a good strategy for everything I’m going to put in this post) We can see a big cluster of major Avengers appearing in the first few episodes, with some other big names coming in the next few years (Vision, the Avenger with the 3rd most appearances in issues, doesn’t come along until #57). While there are a couple of major additions along the way (She-Hulk & Photon in 1982), we can see that the cast of characters for the team is defined pretty early.

One of the first things that I was interested in was the gender balance in the Avengers over time. While there have been women on the team since the beginning (Janet Van Dyne, aka The Wasp, appears in issue #1), has this changed or increased over the 50 year span of the series?

Let’s have a look:

You’ll notice that the Wasp (in yellow), and the Scarlet Witch (in red), pretty much hold the fort for the female Avengers until the late 70s, at which time variety and frequency of female characters increases. There are some dips – 1990 to 1992, and 2005 to 2007, and overall the female ratio at the Avengers mansion peaks in the 1980s and never quite gets back up to that level again.

Of course, there are many other categorizations of comic characters that we can make aside from gender. I mentioned Vision before, who is a robot (OK, OK, he’s a synthezoid). How have superheroes of the electronic persuasion fared amongst Earth’s Mightiest Heroes?

I’m glad you asked:

Here we see some much more interesting patterns. Robots are big from the late 60s to the early 1990s, after which they disappear. There’s a robot renaissance of sorts from 1990 to 2005, but again they lose density (see what I did there, Avengers fans?).

We can do the same thing with Gods, and Eternals (if you don’t know the difference, ask your local comic shop clerk):

Again, there is some real patchiness here. Now, the clever ones among you might be wondering if these patterns are tied to historical periods, or if they are linked to the preferences of specific writers, editors, or artists. Is that crowded patch of Gods in 1985 due to a cultural fascination with myth? Or do Mark Gruenwald & Jim shooter just really, really like Thor? Great questions, and ones that I’ll take a look at Part 2 of this post.

This week-long dig through Avengers data has been fascinating. Even as an Avengers fan, it’s been surprising to see the depth and richness of content that finds its way into the pages of every issue and volume. As I’ve been working, I’ve also been reading a lot about the various people – inkers, letterers, writers, who have built the Avengers story over time. It has been a good reminder, particularly in the wake of a blockbuster film, that myths are rarely formed by individuals.

Finally, it should give anyone fearing a shortage of Avengers storylines and characters for possible sequels some reassurance – 5 down, 127 to go. (Mr. Whedon, if you’re looking for a researcher, you know where to find me.)

‘Nuff said. (For now.)

Most of us probably know that the loom played a part in the early days of computing – the Jacquard loom was the first machine to use punch cards, and its workings were very influential in the early design of programmable machines (In my 1980s basement this history was actually physically embodied; sitting about 10 feet away from my mother’s two floor looms, on an Ikea bookself, sat a box of IBM punch cards that we mostly used to make paper airplanes out of). But how many of us know how a loom actually works? Though I have watched my mother weave many times, it didn’t take long at the start of this project to realize that I had no real idea how the binary weaving patterns called ‘drawdowns‘ ended up making a pattern in a textile.

To teach myself how this process actually happened, I built a functional software loom, where I could see the pattern manifest itself in the warp and weft (if you have Chrome you can see it in action here – better documentation is coming soon). This gave me a kind of sandbox which let me see how typical weaving patterns were constructed, and what kind of problems I could expect when I started to write my own. And run into problems, I did. My first attempts at generating patterns were sloppy and boring (at best) and the generative methods I was applying weren’t very successful. Enter Ralph E. Griswold.

Ralph Griswold was a pioneering computer scientist, best known for developing the string programming language SNOBOL. He spent a decade at Bell Labs, studying non-numerical computation, and went on to become Regents’ Professor at the University of Arizona. After this illustrious career in computing, he shifted his attention to the mathematics of weaving. Mr. Griswold died in 2006, but he left behind a huge archive of resources for weavers and curious learners, including academic papers on pattern generation using cellular automata.

The first successful pattern possibilities for Infinite Weft came from applying and modifying the techniques in the paper. I built a simple interface in which I could advance the automata generation by generation, and switch between a set of very simple rules (courtesy of John von Neumann). Here’s what a sample pattern generated from these von Neumann automata looks like on the software loom:

And here’s a sample, woven on a table loom with black & white yarn to make the pattern clear:

While these techniques produce fairly satisfactory results, the automata themselves tended to repeat after not too many generations – while you can alternate between rules, and start with different ‘seed’ patterns, and adjust the threading of the loom to get a variety of finished patterns, the systems themselves would inevitably repeat. What about a truly infinite weft?

As it turns out, there are some cellular automata that are non-repeating. Given any generation N, the result of the next generation, N+1, can’t be predicted from the outcomes that have happened before. Could I apply such an automata to generate an infinite ‘pattern’? Well, I gave it a try, and the results look promising. Here is a look at a pattern generated using Wolfram’s Rule 30, a (quite possibly) universal cellular automaton:

And a similar pattern, hand-woven by my mother:

Now we get into some pretty interesting conceptual territory. In theory, a long enough stretch of this woven textile would be Turing-complete – a computable fabric. Embedded somewhere in the pattern would be instructions to solve any conceivable problem. Past the math, this system also lets us challenge what we think of as a pattern, in a fabric context (after all, this pattern has really no pattern at all).

This project is still very much a work in progress – this blog post is a peak into the process and chance to get some of my thoughts into writing. The next obvious step is to finalize work on the pattern generation, and get some large-scale textile woven from my mother’s ‘real loom’, which is a 16-harness floor loom (for this we’re going to need a computerized dobby head, which is a bit of an investment). I would also love to see other weavers outputting sections of this ‘infinite’ weft – please get in touch if you have a loom and would like to try weaving a section.

Source code for Infinite Weft is available in a public GitHub repository here.

And, as always, please don’t hesitate to leave a comment if you have any questions or suggestions.

Last year, I released a video visualization of the 1236 exoplanets identified by the NASA’s Kepler mission. Since then, there have been another 1091 candidates identified, and I thought it’d be a good time to update my visualization – and release the source code.

So, here it it: http://github.com/blprnt/Kepler-Visualization

I’ve tried to comment the code as well as possible – and the sketch overall is fairly simple. You will, of course, need Processing to get it running, as well as Karsten Schmidt’s esssential toxiclibs.

I won’t give the whole talk away, since you can watch it for yourself below. As always, I’d love to hear feedback/questions/ridicule – you can leave a comment below, or join in on this already feisty thread on Google+.

Oh, and Batman. And Wayne Gretzky.

You can watch the video below (it’s about 50 minutes). As always, questions, feedback and bizarre non-sequiturs are welcome.

These days my spare time is typically measured in seconds – so I took advantage of a rare free morning today to put together a very simple list of the some of the work that I have built since the beginning of 2009. You can find a link to this list at the top of the site – or you can just click here. As always, comments and suggestions are encouraged and appreciated.