Monday, April 29, 2013

IT is a fashion industry

You know when people laugh at the fashion industry for saying that 'blue is the new black' and because of its ridiculous amount of fawning over models, designers and the like?  Is that really different to IT?  We've got our fashion houses - Google, Facebook, Apple.  We've got the big bulk conglomerates IBM, Oracle, SAP, Microsoft and oh hell the fawning that goes around...

I'd say the comparison goes even deeper however.  EAI, Web Services, REST... what are these?  They are all integration approaches.  EAI was going to save the enterprise and create a well managed estate that the business could use and could be changed easily and enable integration with external companies... Web Services were going to save the enterprise by standardising the interfaces enabling a well managed estate the business could use and could be easily... REST was going to save all of IT by enabling interfaces that could be dynamically changed and enable integration...

The point is that the long term challenge is the same, system to system integration, yet we have fad based approaches to solve that challenge.  Its like the fashion industry and dress lengths, it goes up and down, but its still a dress.  The real difference in IT however is that the fashion industry does this better, sure they change the hem, but it still works as a dress.  In IT we concentrate so much on the hem length that we don't even bother with the fact that system to system integration appears to be as hard in 2013 as it was in 1999.  We even know why, the Silver Bullet tells us that technology won't solve the problem on its own.  But do we listen?  No because we are followers of fashion.

This analogy to fashion applies to the age discrimination in IT, we love the young and shiny, and age discrimination against new entrants is wonderfully not present.  However the flip side of that is there is an over emphasis on the new in IT, so we prefer doing things in 'new' ways rather than in 'working' ways, and unlike in the fashion industry we don't actually learn from sales what is successful.  If we've got a fad (hello REST) that works in some places but not in others we'll keep on pushing that fashion even as it fails to set the world on fire.  Its the emperor's new clothes effect, and in IT we do the equivalent of the beauty industry.  In the beauty industry you'll see adverts for 'age defying creams' advertised by 16 year old models.  In IT you'll see enterprise solutions pushed by using Google as an example.  We love the new, we love the young, and we really rather hate facing up to the fact that IT is quite an old industry now and 90%+ of the stuff out there is a long way from new and shiny.

The analysts and vendors are the Vogue and Fashion Houses in this world, the pushing of the new as the 'must have' technology and dire warnings if you dare to actually make the old stuff work.  The concentration on the outfit (the technology product) and little about how it actually works in the real-world (operations).  You know when you see outfits at London, New York, Milan or Paris fashion weeks been shown on the news under the 'what madness do designers think we will wear next' section.  Is that so different from an analyst or vendor pushing a new technology without explaining at all how it will fit into the operations of your current business?  We will see things like people declaring the end to SQL... and then a few years later those same people championing SQL as the approach, now that they've realised people can operate their technology if they do that.

The final place I'll talk about IT and fashion is in the 'rebadging' that we see.  In the fashion industry you see old ideas rehashed and pushed down the catwalk as being 'retro'.  There is at least some honesty in the fashion industry as they talk about it being inspired by an era, when we all know what they mean is 'I didn't have an original idea, so I copied one that was old enough that people will think its original to copy'.

In IT we don't even have the honesty of the fashion industry, what we do is see a new trend and claim that old technologies are actually part of that new trend.  We'll take an old EAI tool and slap on an SOA logo, we'll take a hub and spoke broker and call it an ESB.  This re-badging of technology goes on and on, sometimes you'll be in a meeting and suddenly realise 'hang on, I used that 12 years ago... how the hell is it now new?'. This would be fine if the focus was on building a robust product, but too often its just about how to get on an RFP and shift a few more units with actual investment in new approaches being few and far between.

IT and the fashion industry are miles apart in many ways, but the faddish nature of our industries make us very similar.  The problem is that fashion is allowed to be faddish, its not expected that a business will rely on something made 20 years ago but with IT this faddish behaviour is a big problem.  We are meant to be constructing systems on which a business can rely, not just today but 5 years from now and still be leveraging in 10, 20 or even 30 years time if its well constructed and does the job well.  There are mainframe systems out there doing exactly that, and why haven't they been replaced?  Because the new stuff didn't do the job.

IT needs to stop being like the fashion industry and more like the aircraft manufacturing industry, sure they have 'fads' like an all composite aircraft, but that is based on sound data as well as strategic vision. Its not just based on it being what the cool kids do.   We can do the cool stuff, we can do the new stuff, but we need to recognise that there is lots of stuff out there that needs to change, we can't just use Google or Facebook as references or examples, that would be like Boeing selling a plane technology based on what people did in movies, its just too far removed from the enterprise reality.  We need to stop blindly following IT fashion and start critically appraising it and shouting 'emperor's new clothes' when its bullshit.  Most of all we need to look at enterprise technologies based on how they improve the 'now' not based on 'if only we could replace everything', evolution is the revolution in IT.

Its time for IT to grow up and take responsibility for the mess we've created.

Thursday, April 25, 2013

The Hadoop hump - why enterprises struggle to move from Proof of Concept to Enterprise deployment

At the recent Hadoop Summit in Amsterdam I noticed something that has been bothering me for a while.  Lots of companies have done some great Proof of Concepts with Hadoop but they are rarely turning those into fully blown operational solutions.  Being clear I'm not talking about the shiny, shiny web companies where the business is technology and the people who develop are the people who support, I'm talking about those dull companies that make up the 99.9% of businesses out there where IT is part of the organisation and support is normally done by separate teams.

There are three key reasons for this Hadoop hump

  1. Hadoop is addressing problems in the BI space, but is a custom build technology
  2. Hadoop has been created for developers not support
  3. BI budgets are used to vertically scaled hardware
These reasons are about people not technologies.  Hadoop might save you money on hardware and software licenses but if you are moving from report developers in the BI space to Map Reduce/R people in Hadoop and most critically requiring those same high value people in support its the people costs that prevent Hadoop being scaled.  The last one is a mental leap that I've seen BI folks struggle to make, they are used to going 'big box' and talking about horizontal scalability and HDFS really doesn't fit with their mindset.  

These are the challenges that companies like Cloudera, Pivotal and Hortonworks are going to have to address to make Hadoop really scale in the enterprise.  Its not about technical scale, its about the cost of people.

Wednesday, April 24, 2013

Big Data, Fast Data, Orange Data, Blue Data - its decisions that count not data

Oh the chanting is out, Big Data, Fast Data, the three 'V's and of course the ubiquitous elephant are roaming across the IT landscape as the next great hype monster.  Its going the same way as pretty much every IT hype exercise.  Yes this links to the hype cycle but the way it happens is sadly predictable in IT.

  1. Company has a business problem they think about it in an innovative way
  2. Company needs to build a new piece of technology to help with the process, or chooses to just because they can
  3. Company announces the business results they have
  4. IT industry focuses on the technology
  5. IT industry creates a sticker - 'Big Data'
  6. IT vendors produce versions of that technology or start claiming their technology does that anyway
  7. IT vendors ramp up the marketing spend around the technology
  8. Everyone forgets why this all started
Why is it that Google and Yahoo created and use Hadoop?  Well its because they had a completely different scale of data problem and needed to do challenging analytics in a different way.  Whether it be ad serving or search itself the point was that the aim was functional the data was required to deliver on that functional goal.

When people rave about Social Media, Open Data and various Big Data sources and talk about Hadoop in glowing terms that is all fine and good, but the real point here is about decisions and making them better.  Its about what you can improve in your business not about having to store every piece of data out there.  Concentrating on HDFS as being the important thing in Big Data is looking at flooring being the only important piece of a house.

The hype is on with Big Data, the hype is on with Fast Data but the reality is back to why people started looking at these volumes of information and critically about what it actually means when you start considering massive scale information in terms of governance, management and analytics and that all comes back to a simple set of questions.

What decisions do I need to make better?  What information do I need to make those decisions better? How do I get the right analysis of that information delivered to the point where the decision is made?

We need to stop focusing on Hadoop, R, HANA and all of the technical pieces and start looking at the real trend here, and that is the integrating of analytics into operational processes, the old world of transactional data v analytical data has gone and that is a massive change for IT, not simply in technology but more critically in mindset.

The real shift is not 'Big Data' its the end to post-transactional reporting, its about a single IT infrastructure that mixes real-time information with historical analytics to support better operational decision making.

As with SOA the real shift is a mental one not a technical one and its the mental shift that is hardest to achieve.  Can traditional IT departments move away from the separation of BI from Operational systems?  Clearly the business is going to do that so the only question is whether the IT department is along for the ride or is just the kindergarden where people play with toys.

Monday, April 22, 2013

The single eye of enterprise architecture

There is a famous phrase
In the kingdom of the blind, the one eyed man is king
IT has always had a problem communicating with the business, and the business communicating with IT.  To fix this IT created something called Enterprise Architecture which aimed to provide a framework around the internal IT estate in a manner that helped that conversation. We can argue how successful that was but what is clear is that with the rising consumerisation of IT and the massive increase in technical literacy amongst business people that this boundary approach to IT with EA as the gatekeeper isn't really going to work.  Historically the EA function was that one-eyed man, it had enough depth in IT to provide some control and enough engagement with the business to act as the leader through the challenges of IT.

The business isn't blind to IT anymore.

So what has been the reaction?  Unfortunately it appears to be in some cases to take a single-eyed view of the problem, a single eye looking through a single lens.  That single eye recognises the increase in sophistication of the business and sees the rise of Value Networks and Business Architecture and thinks 'Oooooh, we have to do the business architecture stuff as well' and makes that a sub-section in the overall encompassing enterprise architecture.

This is missing an opportunity.  A huge opportunity.  EA was created in an era where IT organisations were internal only and where external integration was extremely rare.  In the world of SaaS, Cloud, Social and OpenData however this is no longer the case, outside-in is the norm.

Sure we need groups to looking at integration standards and help with IT operations, but the communication with the business doesn't need an interpreter in the same way anymore. Business Architecture is a rightly growing area but its not a subset of EA, or even an extension of EA its a discipline in its own right, one that owes more to business schools than to IT departments.  So what does IT need to do?  Well it needs to be able to translate that architecture into solutions much more quickly.

IT needs to look at the business with two-eyes again, not try and extend and old concept through its own view of the world.