Beautiful Data: The Stories Behind Elegant Data Solutions

As Google Director of Research Peter Norvig says in his essay on data for a natural language corpus, “Most of this book deals with data that is beautiful in the sense of Baudelaire: ‘All which is beautiful and noble is the result of reason and calculation.’”  Admittedly, Beautiful Data isn’t the book I expected, but it was captivating nonetheless.

Based mainly on its title and cover image, I anticipated Beautiful Data would be something more like Jonathan Harris meets a college math book.  However, the book is neither filled with visual representations of information, nor is it heavy on the equations. Rather, it’s a collection of individual essays all loosely tied to the topic of data usage.

Don’t look for how-tos, lectures, or any mechanics.  This book is the liberal arts version of computer science.  Thirty-nine individuals, each with varying degrees of experience and all covering an array of industries, present stories on how they use data and how data have influenced their work.  Their case studies and examples sometimes touch on the philosophy of data and analysis and occasionally are intimate portrayals of the marvels and shortcomings experienced when dealing with information.

Offering alluring glimpses into the projects and data centers of organizations like Facebook and Google, graduate student research, and the making of a Radiohead music video, these are stories of data experiences.  Beautiful Data feels a bit like bedtime reading for the data scientist, statistician, or programmer.  When you’re done going through the stacks, logs, and code of the day, choose a story and learn how other people are capturing, moving, and understanding bits of information.

I began with the essay on applying aspects of user experience to data collection, “Beautiful People: Keeping Users in Mind When Designing Data Collection Methods.”  The story starts with the notion that researchers can often get better data from users who don’t specifically know that they are being surveyed.  For those times when a researcher must explicitly ask users to complete a survey in order to gather valuable data, the authors go through a case study on designing a demographic survey to collect perceptions of luxury products.

I enjoyed the essay from Jeff Hammerbacher, the once data manager at Facebook and now founder of Cloudera.  More than just an inside look at the struggles Facebook encountered as its databases expanded during the site’s rise to popularity, the essay, “Information Platforms and the Rise of the Scientist,” is the story of how Hammerbacher went from a 17 year old hiding out in his local library to a research scientist at Facebook, despite his “potentially suboptimal background.”

Returning to Peter Norvig, his piece “Natural Language Corpus Data” has a mundane title but is an essay I wish I could have read in college while I was deep in linguistic corpora and word frequencies.  In amazingly straightforward terms, Norvig breaks down what a linguistic data corpus is and what kind of information the Google n-gram corpus holds.  More technical than many of the essays, Norvig does include the Python and calculations specific to his discussions, but non-programmers should be able to skip through the more technical details and still come away with a strong understanding of the ideas.  As Norvig explains of the data points in the Google corpus, it isn’t merely the collection and aggregation of data that makes it beautiful; “The data is beautiful because it represents much of what is worth saying.”

Beautiful Data: The Stories Behind Elegant Data Solutions is published by O’Reilly Media.  You can purchase a copy from O’Reilly, Amazon, or other book sellers.  I received a free download of this book from O’Reilly to write this review but I chose the book based on my own interests in data and analysis.

I love real books, and although some people have speculated that the age of the paperbound book has come to an end, I don’t believe physical books will become extinct anytime soon. However, I’m excited and optimistic about all the ereaders that could be hitting the market soon. Amazon’s Kindle surprised me when it was released, but it appeared too expensive and clunky for me to rush out and buy one. What got my attention was the growing collection of Amazon ebooks.

With shelves now stocked with thousands of popular books in digitalized form, suddenly with just one click I can get instant access to most of the books I browse through. The creation of Kindle ereader software for iPod Touches and iPhones means that I can start reading the book on my screen in a matter of seconds. I now constantly struggle to not buy a version of each book I add to my wishlist.

That’s why I can’t wait for the development of a quality ebook library or subscription service. One of the problems with owning so many books is that they’re extremely difficult (heavy) to move. Books take up space. They also cost money. Sometimes a lot of money. I’m sure that’s why libraries were invented. However, where I live the library is rarely open during the hours when I could get to it, and many times they don’t have the books I’m looking for. I also just find it incredibly convenient to push a button and get what I want instantly. (Others must also find instant access convenient as well, or services like Hulu and streaming Netflix wouldn’t have done so well).

I am convinced that having instant access to thousands (hundreds of thousands) of digital books would be beneficial to my life. I would happily pay a (reasonable) fee to become an ebook library member or to join an ebook subscription service. Over the summer, I tried a Netflix-style book rental service, but nothing beats a two-week wait like an instant download (and the book rental subsequently went out of business).

Ebook Libraries Now

Libraries like the New York City public library, the Chicago public library, and many others across the nation already offer ebook lending services, but you have to be a card holder. This typically isn’t possible unless you live in that state or county. There are several global ebook lending services out there, but they’re not at all what I’d consider an “ideal” alternative. At this point in time, the companies who are best equipped to start a lending service are the companies who have already have relationships with publishers and who have already amassed catalogs of digital material – the likes of Amazon, Barnes & Nobles, and Google.

The influx of ereaders and tablets set to hit the market this year will be nothing without content. Amazon and Barnes & Noble have stocked their virtual shelves with the digital version of the latest top sellers, and they’re adding new ebooks each week to satisfy Kindle and Nook users. Initially, I believe things will be shaky as tablets and ereaders quickly become more common and users, retailers, and publishers alike struggle to deal with issues like digital rights and cross-compatibility.

Simple and Functional

Ultimately, in order to remain competitive and to truly reach the masses, retailers will have to deliver their digital media in a format that works on all devices. Trying to determine the differences between mobi and epub and pdf is too confusing for the mass population. Either the digital book will work with your device, or it won’t. If it won’t, the distributor is losing out on market share.

Along with the greater adoption of ereaders will come a uptick in digital media sales, some of which might help floundering old media publications, but only if they embrace the technology creatively and with open arms.

A good ebook library or lending service would need to function like a cross between Netflix and Amazon. With ebook revenue projected to top $500 million in 2010, now is an excellent time to get such a service going. The current services leave a lot of room to improve the usefullness and functionality of ebook lending.

An Open Market for Ebook Lending

Ebook readers want access to anything, anywhere. Project Gutenberg won’t ever fill the desire for current popular fiction and nonfiction. The service should offer a see-inside-the-book option, much like Amazon currently allows you to download a sample to read from your Kindle or Kindle-device. This is a great way to get someone hooked, or to convince him that this isn’t the book he was looking for. Readers also want simple and easy access; they locate a book, click download, and in seconds they can begin to read. The best services will allow users to read books on any of their devices: smart phones, ereaders, and computer. A bonus service would also allow readers to synchronize their places and bookmarks across devices, similar to the Kindle Whisper.

What would really distinguish a library or lending service is a strong set of community features. Users should be able to create Amazon-style wishlists and recommendation lists, there should be a robust community of users writing reviews, and they should be encouraged to share in book clubs or discussion groups. The library should show smart reading recommendations based on the user’s favorites and rental history, à la Netflix. Without these services in place, many will abandon for another site like Amazon.

Navigation could be a potential issue. As the Apple App store has shown, you can still be a successful company and do a terrible job of making your products known. With potentially hundreds of thousands of books, catalog navigation and search need to be topnotch.

Students Could Be The Earliest Adopters

Students could make the ereader market, but currently they can’t get what they need to read. Right now in the etextbook market, publishers are tentative. While ebooks could mean that authors and publishers end up with a higher percentage of the textbook revenue, no mass wave of publishers have begun digitalizing their material. Coursesmart offers a textbook subscription service to students, but it’s only available for iPhones. I have used my phone to read small amounts of material, I can’t imagine ploughing through an entire semester’s courseload on such a tiny screen. The proprietary format and DRM currently used in etexts makes them incompatible with textbook-optimized ereaders.

I think the final service that could heighten the usefullness of an ebook library or lending service, is the ability to take notes. While Kindle owners can currently highlight passages, bookmark pages, lookup words, and take notes, it seems likely that features such as these wouldn’t survive a lending service. But they should. Ideally, users would be able to take notes, export the notes from their device, maintain whatever they’ve notated even after they’ve “returned” the lent book, and synchronize the notes with text if they were to check it out again at a later date. These features would truly put a world of information at the fingertips of many.

Illustration by kandinski.

(Photo by Motleypixel)

Seth Godin on Why You Should Become an Artist

“The lizard hates it when you read books like this one.”

While I hope no one finishes Seth Godin’s Linchpin: Are You Indispensable? thinking she has the cortex of a gilla monster floating around inside her skull, Mr Godin wants you to gain the courage to overcome the “lizard brain” within.

The primitive brain in reference is the part of us that controls our basic involuntary behaviors, our rage, and our survival skills.  Obeying the needs of the lizard brain is what keeps so many workers in mindless assembly line-style jobs.  Even if your occupation has never required you to put parts and pieces together on a routine basis, chances are great that you’ve held a position where you clocked in each morning, clocked out each night, and spent your day pushing papers, crunching numbers, and waiting for instructions from your boss – all in anticipation of your next pay check.

But you could be a linchpin.  A linchpin doesn’t wait for orders, she makes things happen on her own.  She is a leader, an organizer, and a doer.  Most importantly, a linchpin is an artist.

Linchpins are indispensable in the workplace.  They are so deeply ingrained in the success of your company that your boss is afraid of what might happen to the business if the linchpins leave.  A cog, on the other hand, can be replaced. In fact, a cog can probably be replaced by another cog willing to work for less.

In the current economic situation, it’s good to be a linchpin.  “I couldn’t have written this book ten years ago, because ten years ago, our economy wanted you to fit in, it paid you well to fit in, and it took care of you if you fit in,” Mr Godin writes.  Now is the perfect opportunity to begin sculpting businesses based on the passion of linchpins.  The age of outfitting your company with minimally skilled, minimally paid, easily replaceable workers is over.  While successful essential companies ride the roughest economic waves and come out ahead through the merits of their linchpin employees, businesses that hirer cheap easily replaceable labor race to the bottom of the competitive market as others easily copy their method.

But this book isn’t about failing businesses.  It’s about encouraging every reader – who currently sits complacently at his desk, doing just what’s necessary to keep from being fired – to speak up, step up, and give more.  It’s about ignoring your urge to just survive and embracing your ability to create art.  Artists create something extra, something that doesn’t easily fit within the mold of “a day’s work for a day’s pay.”  Artists have passion, they have ideas, and they do more with their skills.

Even if you’ve never painted a picture in your entire life, you hold the abilities to become a great artist.  There is something that you’re great at, and it’s something that you love doing.  Mr Godin says that there are plenty of people who can play the flute just as well as you can or program in Python even better than you.  The linchpin though, is the skilled person who is able to process and synthesize many different ideas and elements, putting together unique combinations and drawing connections where other people couldn’t.

And linchpins have passion.  Their passion isn’t project-specific, it’s people-specific.  He suggests that if the internet wasn’t around for a linchpin like Jeff Bezos to start a company like Amazon, you wouldn’t find him just sitting around being a nothing, a nonpassionate lump.  His passion would compel him to create greatness elsewhere.

Linchpin is a book filled with microparables and mini case studies for the modern worker.  I suspect that any random person could flip to any random page, grab a paragraph, and glean some daily inspiration.  However, just to see what kind of poignant wisdom I could grab through chance, I shuffled through the book, stopped on a page, and plopped my finger down.  Here is that passage:

Fear of Art
How powerful is the art you are able to create? Do genes and upbringing and cultural imperatives force you to surrender in your quest to deliver art that matters?
Was Harper Lee born to write To Kill a Mockingbird?  Is there some combination of genetic gifts and parental nudging that created the perfect opportunity for her to generate such a monumental piece of art?
Let’s go back to the beginning of this book.
Everyone, every single person, has been a genius at least once.  Everyone has winged it, invented, and created their way out of a jam at least once.
If you can do it once, you can do it again.

We are all artists.

Like a preacher for the modern work-weary, Seth Godin is rousing and is here to coax greatness from his followers.  He wants each and every person to become a linchpin in his own position.  However, if you’re looking for Linchpin: The How-To Manual, this isn’t it.  Linchpins don’t use maps; Mr Godin doesn’t offer step-by-step instructions for becoming a superstar at work.  Yet you can expect to find inspiration, rationale, and gentle directions leading you to the path of artistry.

What may not have been stressed enough in this book is that a portion of the workforce already acts with a linchpin mentality, and they’re growing every day.  While you were sitting around contemplating how your life might be different if you were more assertive in boardroom meetings, a linchpin just took your job.

The factory mentality that’s developed over the years is falling, but it’s falling faster within a young generation of workers who have yet to spend half of their adult lives in a cubicle building up their 401k.  Upon graduating from college, many within the most recent wave of workers weren’t handed that just-comfortable-enough job.  So they got creative.  They don’t have a retirement fund to protect, and they don’t have a tenure around the corner.  Many don’t understand how we arrived at a place where there is just enough work for everyone to spend 40 hours in the office each week.  How did we get to a place where my time is worth exactly $10 an hour, $20 an hour, or $30 an hour?

Linchpins aren’t afraid to assert their ideas and take action.  If the worst were to happen, they are confident they can find another job.  Many may not be as motivated by money as your typical cubicle cog, but most will work for meaning.

I think that Linchpin compliments Tim Ferriss’ The Four Hour Work Week and Gary Vaynerchuck’s Crush It! though the three authors have different styles and are approaching slightly different aspects of work.  However, the three books together will encourage you toward success by driving you away from the cubicle and onward to greater passions.

For linchpins, this means giving away what you do best.  If what you create is an art, what you gain from sharing your art (what you know and do best) is a more fulfilling and passionate career.  “Everyday, successful people race to give away their expertise and to spread their ideas.”  These people are linchpins.

Mr Godin’s Linchpin is appropriate for people of all types and in all stages of their careers.  No doubt, fans of his previous books on marketing, work, and life will be the first to pick it up, but I think it will best speak to those workers who have already started questioning their current positions and who have already begun asserting themselves.  After all, it’s easier to make the jump if you’re standing on a diving board.  Linchpin might just be the push they need.