I’m a voracious reader, of analog books, magazines and hypermedia. When I want to focus on a particular topic and shut out the world, there’s nothing like a long form paper book: no links, no ads, no comments, no feeling that I’m missing better content somewhere else, no quick check of Twitter or to see what the weather will be tomorrow. I have exactly no desire to get an e-book reader and mess with this balance – the gravitational pull of hypermedia is too strong for me.

My aversion to e-books aside, I’ve recently added a digital element to my reading. I’m nearly through Gettysburg – A Testing of Courage, and during the course of reading it have constructed a full mental image of the battle. The landscape and geography in and around Gettysburg, of course, is a major character in the narrative. After reading about Culp’s and Cemetery Hills, Big and Little Round Top, I just had to see them. Some searching, viewing of maps and in particular a browsing around of the flickr tag “gettysburg” and I had some amazing ‘real’ images to supplement those of my mind’s eye.

Have you been supplementing your reading, particularly of non-fiction, with the web? I’m not sure why it took me so long to bring these two [favorite] activities together, perhaps it’s been my strong separation of analog and digital reading. I think I’ll soften that stance a bit.

Note: the above image is of Little Round Top, one of the most important places in America’s history. Thanks to Candice for giving me permission to use this image.

It took me almost a year, and a week off of work, to catch up on reading and finally get to Vanity Fair’s piece: How the Web Was Won: An Oral History of the Internet. And while I’m sure your reading list is as backed-up as mine is (well, was!), if you’re in this industry you really should read it. It’s a first-person account by many of the major players of the invention of the Internet and the World Wide Web.

But in case you don’t, here’s a few choice quotes:

Paul Baran – inventor of packet switching (breaking up data into packets to reassemble on the other side):

I get credit for a lot of things I didn’t do. I just did a little piece on packet switching and I get blamed for the whole goddamned Internet, you know? Technology reaches a certain ripeness and the pieces are available and the need is there and the economics look good – it’s going to get invented by somebody.

On the decision by the Government to give the building of the hardware Interface Message Processors to Bolt, Beranek & Newman:

In a congratulatory telegram to the company, Senator Edward M. Kennedy referred to I.M.P.’s as “interfaith” message processors.

Leonard Kleinrock – member of the ARPANET Team, on the Internet coming on-line:

September 2, 1969, is when the first I.M.P. was connected to the first host, and that happened at U.C.L.A. We didn’t even have a camera or a tape recorder or a written record of that event. I mean, who noticed? Nobody did. Nineteen sixty-nine was quite a year. Man on the moon. Woodstock. Mets won the World Series. Charles Manson starts killing these people here in Los Angeles. And the Internet was born. Well, the first four everybody knew about. Nobody knew about the Internet.

Vint Cerf – co-designers of the TCP protocol on the invention of @ to identify people on a network:

A guy named Ray Tomlinson, at Bolt, Beranek & Newman, figured out a way to cause a file to be transferred from one machine through the Net to another machine and left in a particular location for someone to pick up. He said, I need some symbol that separates the name of the recipient from the machine that the guy’s files are on. And so he looked around for what symbols on the keyboard were not already in use, and found the “@” sign. It was a tremendous invention.

Marc Andreessen on creating the first browser:

It sounds obvious in retrospect, but at the time, that was an original idea. When we were working on Mosaic during Christmas break between 1992 and 1993, I went out at like four in the morning to a 7-Eleven to get something to eat, and there was the first issue of Wired on the shelf. I bought it. In it there’s all this science-fiction stuff. The Internet’s not mentioned. Even in Wired.

Jeff Bezos on early Amazon:

When we launched, we launched with over a million titles. There were countless snags. One of my friends figured out that you could order a negative quantity of books. And we would credit your credit card and then, I guess, wait for you to deliver the books to us. We fixed that one very quickly.

…and so on – the Morris Worm, The Browser Wars, the first e-mail used in a legal case (Jon Poindexter in Iran Contra) – read it. This history is compelling and important.

Rich and I gave a talk two weeks back at the IA Summit about how the ever-increasing amount of data we’re creating makes possible a new type of science: one that looks for patterns and correlation in data and can provide the fuel for the next round of scientific discovery.

This week I open the new Wired Magazine and there’s a perfectly relevant and current example of this trend – the Allen Institute of Brain Science and their quest to map out the genetic make-up of the brain. I wish I had come across his a week earlier – it’s complete reinforcement of our thesis:

To achieve this, the Allen Institute reimagined the scientific process. There was no grand hypothesis, or even a semblance of theory. The researchers just wanted the data, and, given the amount needed, it quickly became apparent that the work couldn’t be done by hand. So, shortly after the institute was founded in 2003, Jones and his team started thinking about how to industrialize the experimental process. While modern science remains, for the most part, a field of artisans – scientists performing their own experiments at their own benches – the atlas required a high-throughput model, in which everything would be done on an efficient assembly line. Thanks to a team of new laboratory robots, what would have taken a thousand technicians several years can now be accomplished in less than 20 months. The institute can produce more than a terabyte of data per day. (In comparison, the 3 billion base pairs in the human genome can fit in a text file that’s only 3 gigabytes.) And the project is just getting started.

The scientists at the Allen Institute are producing all of this data to enable easy access to the structural brain, specifically to catalog which genes are expressed in which of the brain’s regions. But in a larger sense, they’re producing the data first without a specific purpose in mind – thinking that by making this data available and accessible, they are opening the possibility for future discovery:

They remain excited by the idea of working on the frontier of science, by the possibility that their maps will allow others to make sense of this still inscrutable landscape. In other words, they are waiting for the future, for some scientist to invent an elegant theory that explains their enigmatic data.

One of the focus points of our talk was Tim Berners-Lee’s TED talk, where he expresses his frustration with the current Web – a delivery platform for human-readable documents – and implores all of us to make our raw data available (now!). He wants us all to stop hugging and beautifying our data and rather make it available for others to gain value from now.

Sounds a lot like Jonah Lehrer describing the scientists working at the Allen Institute, when he says “we don’t even know what we don’t know.”

Last week, both Sam Ruby and Bill de hÓra cited Mark Pilgrim’s post Dive into history, 2009 edition, about the history of his book Dive Into Python and its future – now that his publisher has commissioned Dive Into Python 3, it’s a series.

It’s a very insightful post about how the web has evolved over the past ten years, and about XML and HTML. But I was intrigued by the fact that both of these thoughtful technologists cited the same line:

“HTML is not an output format. HTML is The Format.”

de hÓra included one more sentence: “Not The Eternal Format, but damn if it isn’t The Format Of The Now.” and he suggested in the comments: “The Format Of The Long Now.”

This is great stuff, and it spurred a thought of my own: if HTML is The Format Of The Now, or even The Long Now, then perhaps HTTP, its counterpart, is The Protocol Of The Now. Maybe even The Long Now.

This thought is intriguing to me, because I’m in the business of building HTTP applications. And I believe that it’s important that people using HTTP understand its underlying architectural style, REST. I find REST and the debate and discussion around it fascinating. To see people grappling with its ideas, and trying to integrate it into their own mental model of how software works, is really quite… fascinating.

I think HTTP really might be The Protocol of the Long Now, and I look forward to seeing whether things actually turn out that way.

How long will it be until we have hyper-social behavior listed as a human disorder? I suppose it might be defined as ‘one who manages their social network identity, often via frequent updating of status, at the expense of real, human relationships’.

On the topic of [electronic] social behavior, an interesting discussion making the rounds tonight after the BBC’s Darren Waters’ article Social networks ‘are new e-mail’. We kicked around the very same topic on this blog two months ago. Dare Obasanjo comments:

These days I’m more likely to post an interesting link by sharing it on Twitter and have it filter out to my social networks on Facebook and Windows Live than I am to share it via email and risk spamming a bunch of my friends and coworkers. As more people embrace social networking, the trend of using email for certain types of sharing will likely decline.

Also coincidental that this topic comes up on the same day as the Times Magazine piece Growing Up on Facebook, where Peggy Orenstein ponders the social changes that might come from living one’s youth out in the open. She comments:

College was my big chance to doff the roles in my family and community that I had outgrown, to reinvent myself, to get busy with the embarrassing, exciting, muddy, wonderful work of creating an adult identity.

I was also caught by her usage of the phrase “growth through loneliness” while lamenting this generation’s lost ability to break away from the past. I can very much relate to that, having gone to college half-way across the country (FB/Twitter/and e-mail free) and having travelled for a month alone through Europe (OK, that came later and I did enjoy the occasional e-mail from an Internet Cafe).

And just while we’re trying to digest and understand the impact of these new social behaviors on the human brain, new ones are popping up and becoming adopted at ever-increasing speeds (due, of course, to the fact that the existing social networks are leveraged to propagate the newcomers). Let’s just hope we know what we’re doing, else we’ll have to add more than just hyper-social behavior to that list.

This post is a part of a series about mining everyday data, based on a talk Rich Ziade and I are giving at the IA Summit in Memphis later this month.

I recently read Stephen Baker’s Numerati which details how the math elite are increasing their influence over our lives. It was a decent read, albeit one that might have been better as a magazine article. That said, one quote from the book has stuck with me and greatly influenced my upcoming talk at the IA Summit:

Jack Einhorn, the chief scientist at a New York media start-up called Inform Technologies, predicts that the great discoveries of the twenty-first century will come from finding patterns in vast archives of data. "The next Jonas Salk will be a mathematician," he says. "Not a doctor."

Continue reading»

If you haven’t seen it yet, The Reader is a stunning adaptation of Bernhard Schlink’s novel. It tells the story of Michael Berg, a young German boy (David Kross) who meets Hanna Schmitz, a woman twice his age (Kate Winslett), with whom he begins an affair during the summer of 1958.

As their month-long romance develops, Michael discovers that Hanna loves being read to. Hanna is spellbound as Michael reads to her from, amongst other titles, The Odyssey, The Adventures of Huckleberry Finn, and her favorite, The Lady with the Little Dog.

“Read to me first, kid. Then we make love,” Hanna tells her young lover as she sets the rules for the relationship. To meet her demands, Michael reads to her in bed, in the bathtub, and around the rest of Hanna’s tiny apartment. Kross is seen opening the books, turning their pages and even acting out scenes from the novels. Winslett, who won a Best Actress Oscar for the role, hangs on to each and every word that Michael reads off of the pages.

After seeing the seductive portrayal of books in The Reader, I couldn’t help but to think of Kindle, Amazon’s unromantic wireless reading device.

Imagine if the books in The Reader were replaced by a Kindle. Would Michael use it in the tub? It’s not waterproof, so there’s a high probability of damaging it. Would Hanna throw it to the floor in a rage of passion, right before another lovemaking encounter? Highly unlikely! The breakable Kindle, as we all know, comes at a hefty retail price. Hanna wouldn’t ask Michael where he got a particular book; the answer would always be Amazon.com.

When Hanna is serving her life term in a German prison, she checks out The Lady with the Little Dog from the prison. Although it is uncertain if, and when, libraries (including those in prison) will have Kindles, it’s currently impossible to download free ebooks from the New York Public Library to the Kindle.

Some readers prefer to start a new book with a pen in hand. Sorry Hanna-you cannot write in the Kindle! If you love to jot down notes along the margins or underline key sections of the text, this gadget isn’t for you.

The Kindle alters the reading experience for a single user and also has implications for the reading community. It is another electronic toy and most people are not willing to lend it out. You might let your friends borrow a book for a long airplane ride, but is anybody willing to part with a $359 device?

Needless to say, I will not be getting a Kindle since I’m a fan of hardcovers and paperbacks. I enjoy reading as an experience. I love the smell of used books and how the pages feel between my fingers, features which are unavailable on the Kindle. I often use concert ticket stubs as bookmarks. On rainy days, I enjoy sitting on my sofa and admiring my bookshelves, which are alphabetized and organized according to genre and region.

Whether I’m at home or on the road, bookshops are always a part of the itinerary. Some of my favorites include the Strand in New York City, Powell’s Books in Portland, Beckham Books in New Orleans and Foyles in London. I’m also guilty of overpaying for titles from the United Kingdom’s The Man Booker Prize list before they’re released to the American public.

When I was in elementary school, my parents refused to buy a video game console and I spent many afternoons reading from The Babysitter’s Club and Sweet Valley High series. I have spent a large part of my life curled up with a good book. You can call me traditional but I can’t imagine changing how I read.

It was a bit disheartening to learn that Toni Morrison, one of my favorite American authors, is a fan of the Kindle. However, it won’t stop me from showing off my autographed copy of her novel, Love, to guests.

This begs the question: Will people ever say that they have an autographed Kindle book from a Nobel Prize winning author?

Too good – on the difference between information and meaning:

Victor Hugo, the French writer, went on vacation after publishing Les Miserables. After a week or two, he was curious about how the book was doing, so he sent his publisher in Paris a simple message: “?”  The response: “!”

[This post is a part of a series about mining everyday data, based on a talk Rich and I are giving at the IA Summit in Memphis later this month.]

Data is pervasive these days. Much of our lives are being tracked, both directly online (search / purchasing / media / social & communication) and by real-world entities (schools / retail / work). This is coupled with a new trend: organizations are making their data increasingly available to outside parties providing a boon to professional and amateur researchers.

One example of the trend of making formerly private data public is LendingClub – a social lending network that pairs lenders and borrowers for personal loans.  It’s an interesting business, but what is really compelling is that they expose of nearly all of their data publicly through endingclubstats.com. In a recent Wired article about transparency in the financial industry author Daniel Roth discussed how Kevin Bartz, a statistician doing work for a mortgage lender, used the publicly available LendingClub data to search for correlations and trends that could help the lender avoid bad loans. What Bartz found in the LendingClub data was the most common words on the written portion of loan applications where the borrower later defaulted on that loan:

…among the red flags: ‘need’, ‘bills’, and ‘business’. “Those were all words that reflected that the borrower might be in financial difficulty at the moment,” Bartz says. Another one was ‘also’, which Bartz theorizes meant that the loan was being used for more than one purpose.

The mortgage lender could use this finding along with other indicators of potential risk in order to make underwriting decisions. LendingClub is participating here for their own benefit as well, as they have ‘open-sourced’ the algorithm that determines interest rates borrowers pay on their site and are looking to the extended community for help in perfecting their pricing model.

This is similar to the Netfix Prize – where Netflix has opened up their extensive dataset of recommendation information to the public and will award $1 million to the team that can improve their recommendations by 10%. Researchers get a wealth of invaluable real-world data and Netflix gets loads of good-will and an improved ability to service their customers.

I think you’ll see this trend continue – organizations will enter into a mutually beneficial relationship with the extended community by exposing their previously closely-guarded data in hopes of tapping the collective intelligence that is now available to them. Here’s two examples of what we might see from this trend: A sports franchise directly tapping into the amateur statisticians who are constantly analyzing performance data (sure, the Red Sox hired Bill James, but why not tap directly into a community of Bill James?)An insurance company exposing their portfolio of risks to the community looking for new ways to identify potentially bad policies.

Can you think of other possibilities?