Wednesday, June 29, 2011

Why Applications Are Like Fish And Data Is Like Wine

This is not an original title, nor even an original thought.

It was unabashedly lifted wholesale from James Governor's Monkchips website.  And this is not the first time I've done this.

The most recent time, a quick interaction resulted in a philosophical question that still rattles around in my head today: should information be on the balance sheet?  Now that we're better understanding big data IT models, the answer is more decidedly yes.

But this post isn't about big data -- it's about IT philosophy in general, and how it's changing.

To Be Explicit Fish is usually best served fresh and simple: from the boat to your plate with as few intervening steps and wall-clock time as possible.  Best of all when you go to a top-shelf fish restaurant and there are many varieties to choose from that indulge individual palette preferences.

A steady diet of cod, for example, isn't something to look forward to -- even if it's fresh and no matter how many different sauces you put on it.


Trust me on this, I know.

And when it comes to wine, sure -- there are many interesting "young" wines -- but a well-made wine only grows in value over time.  If you're into wine collecting, you'll also appreciate the appeal of vertical flights that illuminates how wines (and winemakers) evolve over time.

Indeed, interesting wines can be much more valuable if they're part of a time series -- and kept a while.  Even if you're not 100% sure how it will turn out.

So, what do these seemingly random observations this have to do with the more serious business of applications and data?

Not to oversimplify, but it's an important discussion.  Most interesting business models seem to be converging to "smart people using data through applications".

So a bit of philosohical meandering might be worthwhile :)

Old School Vs. New School Views On Applications
Big disclaimer: I do not currently live in the end-user application development world.  It's been *decades* since I've written anything meaningful that a user would see.  So the observations here are strictly indirect ones gathered from people who *are* in this world.
Here's what they tell me is starting to matter in their world:
  • Shortening the elapsed time between "there's a need" and "here it is, try it".  If they can take it from months to weeks to days to even hours in some cases, the business value is exponentially increased.  Fresher is better.
  • More modest apps focused on doing a few important things very well vs. lots of things very poorly ("there's an app for that"). No fish stew using leftovers.
  • Thinking in terms of iterating over a wide variety of focused use cases vs. more generic ones.  Variety is good.
Sounds like fish to me.  Freshly served from source to consumption -- end-user apps don't age well in these times.

Give people exactly what they want, when they want it, and don't slather it with a lot of unwanted extras.

Miss the window, and the fish isn't all that appealing anymore.

Old School vs. New School Views On Data
Many, many years ago, I was rather surprised to learn that re-processing historical data with new tools was a big deal in the energy exploration business.  I kind of naively assumed that -- once they'd extracted the value from a data set -- well, that was that.  Sure, they'd keep a copy around, but ...
My brother the geophysicist set me straight: there was a continual progression in new algorithms, more processing power, more recent surveys that complement historical work, new prices for oil, etc. -- all made for a compelling case to "never throw anything away".

Even if you couldn't see the obvious value today.

And, since then, I've often thought that could be the case in many, many situations.  The problem is -- it's hard to look forward and figure out what might be valuable at some future data in some future context.  I often get asked "how do you know what to keep and what to toss?"

I feel rather helpless here -- it depends.

Clearly, anything to do with transactions or events is an obvious choice to keep around.  But what about something ginormous like email with all the embedded docs we send each other?  I think of corporate email as a giant DVR recording the mental processes of a large corporation.
Somewhere in all that corporate jibber-jabber I would think there are potentially useful nuggets worth mining -- someday.  But making tradeoffs against finite resources and real-time requirements is difficult at best.

I started enjoying wine at a relatively early age.  Since then, I've kept a very modest wine cellar for quite a while.  All it takes is a bit of discipline to not drink your entire collection on some random night.

I'm hanging on to certain wines, but not quite sure when I'm going to be drinking them.  I do know that -- as a collection -- they're becoming more enjoyable and valuable -- up to a point, that is.
When I occasionally buy more wine, I face hard choices between what stays in the cellar and what goes up to the kitchen for immediate consumption.
Decisions, decisions :)

The Opposite Of Today's Reality?
In many IT shops I work with, the exact opposite tends to happen.  Valuable data gets routinely tossed as "stale", and lumbering applications are kept around way past their sell-by date.
I know, I know -- there's money involved to do so effectively.  I get that part.

But, I'd argue, there's not much value in stale fish.

Or fresh grape juice, for that matter :)

By: Chuck Hollis