7 Days of Source Day #2: NYTimes 365/360

NYTimes: 365/360 - 2009 (in color)

Project: NYTimes 365/360
Date: February, 2009
Language: Processing
Key Concepts: Data Visualization, NYTimes Article Search API, HashMaps & ArrayLists

Overview:

Many have you have already seen the series of visualizations that I created early in the year using the newly-released New York Times APIs. The most complex of these were in the 365/360 series in which I tried to distill an entire year of news stories into a single graphic. The resulting visualizations (2009 is picture above) capture the complex relationships – and somewhat tangled mess – that is a year in the news.

This release is a single sketch. I’ll be releasing the Article Search API Processing code as a library later in the week, but I wanted to show this project as it sits, with all of the code intact. The output from this sketch is a set of .PDFs which are suitable for print. Someday I’d like to show the entire series of these as a set of 6′ x 6′ prints – of course, someday I’d also like a solid-gold skateboard and a castle made of cheese.

That said, really nice, archival quality prints from this project (and the one I’ll be releasing tomorrow) are for sale in my Etsy shop. I realize that you’ll all be able to make your own prints now (and you are certainly welcome to do so) – but if you really enjoy the work and want to have a signed print to hang on your wall, you know who to talk to.

Getting Started:

Put the folder ‘NYT_365_360′ into your Processing sketch folder. Open Processing and open the sketch from the File > Sketchbook menu. You’ll find detailed instructions in the header of the main tab (the NYT_365_360.pde file).

Thanks:

Most of the credit for this sketch goes to the clever kids at the NYT who made the amazing Article Search API. This is the gold standard of APIs, and really is a dream to use. As you’ll see if you dig into the code, each of these complicated graphics is made with just 21 calls to the API. I can’t imagine the amount of blood, sweat, and tears that would go into making a graphic like this the old-fashioned way.

Speaking of gold standards, Robert Hodgin got me pointed to ArrayLists in the first place, and has been helpful many times over the last few years as I’ve tried to solve a series of ridiculously simple problems in Processing. Thanks, Robert!

Download: NYT365.zip (140k)


CC-GNU GPL

This software is licensed under the CC-GNU GPL version 2.0 or later.

27 thoughts on “7 Days of Source Day #2: NYTimes 365/360”

  1. I think you forgot to add the link to the source code! Thanks for posting this! I'm trying to learn to program and seeing code really helps.

  2. I think I am having the same problem as Thomas with the zip file. It may or may not be due to the colon placed in the file name, which I think is generally not allowed in Windows. The unzipping utilities will not process anything with an invalid file name – Thomas how did you get around this?

  3. Ok, so I get the sketch running but hit the wall…
    font = createFont("Meta-Normal", 100);

    It says it cannot use that font with PDF Export. It is in the data folder. I try to change simply to Helvetica, and still it doesn't work.

      1. I found a solution. If you go to processing tools > create font you see a list of fonts. Pick one of the adobe____ fonts and that should work?

  4. hi , I get the same font error here as well, the following lines

    Use PGraphicsPDF.listFonts() to get a list of available fonts.
    processing.app.debug.RunnerException: RuntimeException: The font “Meta-Normal” cannot be used with PDF Export.

  5. hi I get a similar font problem…..here is the error
    many thanks

    Use PGraphicsPDF.listFonts() to get a list of available fonts.
    processing.app.debug.RunnerException: RuntimeException: The font “Meta-Normal” cannot be used with PDF Export.

  6. Just wanted to post a comment so that others might know for future use. While poking around with this project I tried running the code and received notification from NYT that I've exceeded their queries per minute – will be throttling the requests in the code; let me know if you would like the update to the code with the throttling and I'll pass it along. Also I'm considering setting up a quick proxy for dev so I'm not always hitting their api. I can post a quick php file for that as well if anyone else is interested.

    Thanks,
    Heidi

  7. Help?

    java.io.IOException: Server returned HTTP response code: 403 for URL: http://api.nytimes.com/svc/search/v1/article?quer
    at sun.net .www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at java.net.URL.openStream(URL.java:1010)
    at processing.core.PApplet.createInputRaw(PApplet.java:4532)
    at processing.core.PApplet.createInput(PApplet.java:4500)
    at processing.core.PApplet.loadStrings(PApplet.java:4785)
    at NYT_365_360$TimesArticleSearch.doSearch(NYT_365_360.java:482)
    at NYT_365_360.connectYear(NYT_365_360.java:142)
    at NYT_365_360.setup(NYT_365_360.java:116)
    at processing.core.PApplet.handleDraw(PApplet.java:1583)
    at processing.core.PApplet.run(PApplet.java:1503)
    at java.lang.Thread.run(Thread.java:680)
    The file "http://api.nytimes.com/svc/search/v1/article?query=%20publication_year:1984%20desk_facet:Foreign+Desk&fields=+&facets=per_facet,org_facet&api-key=e46*****************************" is missing or inaccessible, make sure the URL is valid or that the file has been added to your sketch and is readable.
    processing.app.debug.RunnerException: NullPointerException
    at processing.app.Sketch.placeException(Sketch.java:1543)
    at processing.app.debug.Runner.findException(Runner.java:583)
    at processing.app.debug.Runner.reportException(Runner.java:558)
    at processing.app.debug.Runner.exception(Runner.java:498)
    at processing.app.debug.EventThread.exceptionEvent(EventThread.java:367)
    at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
    at processing.app.debug.EventThread.run(EventThread.java:89)
    Exception in thread "Animation Thread" java.lang.NullPointerException
    at processing.core.PApplet.join(PApplet.java:5841)
    at NYT_365_360$TimesArticleSearch.doSearch(NYT_365_360.java:482)
    at NYT_365_360.connectYear(NYT_365_360.java:142)
    at NYT_365_360.setup(NYT_365_360.java:116)
    at processing.core.PApplet.handleDraw(PApplet.java:1583)
    at processing.core.PApplet.run(PApplet.java:1503)
    at java.lang.Thread.run(Thread.java:680)

    1. I think that is a problem with the API. You are hitting the allowed requests per second, so after that the resource is unaccessible.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>