Assembling Data into Information

For the independent record labels I considered the 80′s to be that combination of the golden age and end of an era. The rising economics made it impossible for them to retain their artists and produce records without partnering and eventually being acquired by the larger label. What made these independents different; closer to the grassroots scene, the willingness to take on the progressive and innovative, lacking the generic feel one got from the likes of EMI or Sony.

A reflection of the past decade and how the theory of “Creative Destruction” has shaped a climate more suited for innovation today; reveals a shiner outlook then bestowed the 80′s independent labels. Joseph Shumpeter; an early 20th century economist; popularized and further developed the theory where the the process of intnnovation and progress destroyed and replaced the old with the new.

From an perspective of measured and record economic indicators these past ten years have been described as the “lost decade” rightly so. Just look at the employment indicator in 1999 estimated 108 million employed (private services) to 2010 where yes its at the same 108 million; due in part to the combined loss of net worth caused by both the dot-com and financial bubble bursts estimated between 14 – 16 trillion; though there are positives to show in between the final numbers are what matters. From an innovation perspective though, we learned and created a collection of more leaner, agile, and sustainable environments keeping future in mind this time; providing a little light in the tunnel.

The Beginning

At the start of the decade the internet had become THE platform and business model; it had disrupted and changed the way people were now searching, doing business, and of course shopping; there was no question on the longevity of this boom. E-commerce was now the buzzword; Amazon and eBay were changing how we buy and extend our outreach to sell; and Paypal was disrupting how we were performing the simple task of commerce on the internet. Even as the Webvans and boo.coms; who collectively took in 400 MM in capital investment in order to innovate; would start to show cracks; there was no looking back or stopping the progress in motion.

These collection of websites were being designed and rolled out on best of breed software platforms from ATG, Vignette, and Broadvision; which were created to ease the development and management efforts required. Weblogic coins the phrase “Application Server”; building a common API layer that allows communication between the database and the application; disrupting not only application deployments on the web; but more so putting the concept of client-server to sleep for good. This progression was all in part to influence of Java in the development space; making the hardware layer transparent at execution; allowing for fewer implementations and displacing and disrupting C/C++ in future application development.

Along with this rise interest and activity of the internet dramatic growth in data generated occurred and the need to act on it in real-time became a nescessity. Much of the efforts to date to analyze this semi-structured data would be considered primitive, time consuming, and no where close to real-time. Personify; a platform I had experience with; would be a pioneer on how we would interact with this deluge of data, relate with our online shopper to understand their behavior, and further insight into why Ryan just left his shopping cart in the middle of the digital aisle and walked out the door. Much of what they did gave way to real-time analytics.

The B2B landscape was partaking in this change as well; the web was becoming the presentation layer of choice, and gave way to efficiencies in business processes. Ariba; innovated and transformed how we did procurement; providing a transparent landscape to work with vendors; wanting to differentiate themselves from ERP and SCM; created the term Operating Resource management (ORM); and started to carve out a market from the leaders; Oracle and SAP. Integration had a new buzzword; EAI; which provided organizations the capability of sharing information from the newly acquired and legacy applications; with minimal customization; from the likes of Tibco, Webmethods, and Seebeyond.

Reality Returns

So what went wrong; a topic that has had its share of talk time; bottom line this innovation came at a cost. The simple economics demonstrated that the costs associated to building out and maintaining this infrastructure (software, hardware, hosting) was not supported by the revenue these start-ups were able to generate. Though the wheel of progression was slowed; it had not stopped; a number of start-ups would cease to exist either from the lack of viability of their business model or a victim of the fallout, investors were left to lick their wounds, but through it all this didn’t deter those to continue forward. Creative destruction; thought to be stalled; was already paving under an industry barely out of infancy; the future outcome being the scrutinizing of costs and validity of business models. The sun would rise the next day; and the future was beginning to look okay.

The Looming Clouds (of change)

Emerging from the rubble; Google was beginning to evolve and disrupt; as described to me as not just a search engine; but as a incubator of people and information; which to date Google has been quite successful at accomplishing, Apple would deliver an iPod and iTunes redefining the user experience, Amazon survived to become the A to Z, a platform for third-party sellers, release the kindle, and provide the technology for the next cog in the wheel of progress; referred to as the cloud. The internet itself was starting a new phase in progression; the rise of social started to take place; with MySpace, YouTube, blogging sites, Facebook, and Twitter. This shift brought with it a more diversified use of the web; a place for community interaction, personal productivity, entertainment, and cost-effective environments where organizations could run their business operations.

Disruption would come in the form of economics; how do we get costs down so we can realize the value; Linux would give us an operating system to work on commodity hardware, MySQL would provide us with a database that was free, minimal footprint, and fast enough on the read, Apache the platform to run the application, and PHP the ability to script the code. Each of these developed separately; bound together to form what would be called LAMP stack. This would make the ability to create applications economically manageable, simple and quick.

The barriers present to cost of entry had started to come down; as did initial investments made to these new breed of startups, all which made for a leaner approach of doing business. With this evolution in technology and the agile approaches progressing with it; one would believe the methods organizations implemented to examine their operational data, better understanding of their customer’s behaviors, manage risk, and measure financial performance, would have been part of this forward progress made. But upon further inspection of how these organizations were using business intelligence to support decision-making activities; the same problems were still being discussed and encountered in 2010 as if it were 2001. The functionality being added by the Business Intelligence vendors during this period of time was considered more style over substance and was not solving the ease-of-use or providing data transparency. To further suppress any possibility of innovation; in the past decade; the market leaders would be digested and integrated into the software portfolios of the Oracles, IBMs, and SAPs.

The perception moving forward was that business intelligence would regress to the point that it would be nothing more than a collection of empty buildings collecting maintenance; couldn’t have been further from the truth. Reminiscent of the same cultivation that propelled change in the earlier part of the decade; newcomers have started to emerge fresh with innovative ideas disrupting how we think and can perform business intelligence activities in these shaky economic conditions.

Employing a proven and now mainstream combination of a SaaS and scalable-cloud model that had been vetted over the past few years by the likes of SFDC, Workday, and Netsuite; these SaaS BI vendors have witnessed a swift acceptance to business intelligence space. These emerging vendors look to distance themselves as replacement for Business Objects, Hyperion, and Cognos by causing disruption in terms of economics and functional use. A current roll call of vendors in the SaaS BI space includes; Birst, Good Data, Indicee, Pivotlink; and while there has been recent fallout of earlier pioneers LucidEra and Blink logic; these existing vendors have used this to their advantage by adapting business models and still delivering innovation. Providing users with a platform that offers a higher level of user experience, the beginnings of a collaborative workspace , ease-of-use, and more user driven self-service options; while lowering the costs, resource requirements, implementation time frame that have plagued the on-premise tools.

The adoption of SaaS as a deployment platform has extended beyond BI to the analytical functions that are aligned closer to the business operations business from sales and supply chain optimization, forecasting, predictive analytics and performance management. Offerings from Right90, Steelwedge, Aha, eVia, Xactly, Adapative Planning, Appian, and Host Analytics are the beginnings of a unified approach to monitor the enterprise with the same advantages of the SaaS BI vendors. What can the combination of these analytical functions in a single ecosystem do in terms of disruption? To paraphrase Ray Wang; an industry analyst; “The biggest benefit of SaaS is not the software; but the collective intelligence of the network. SaaS moves to information brokering” together with a recent blog post from Boris Evelson; an analyst with Forrester Research; discussing the estimated cost of producing a business intelligence report; plenty.

Those vendors who design and deploy from the beginning on cloud scaling architecture and a common platform (PaaS) that permits for open and efficient data integration; in turn change the economics of scales associated with the costs development and support and the on boarding at the same time creating an ecosystem that opens new channels for revenue that traditional business intelligence solutions will most likely never have access to. When aligned; all this could produce most significant disruptor for business intelligence; the introduction of context to the data being analyzed; leaving behind the days of a generated report open for interpretation.

As with any emerging technology; its not without challenges in the adoption into the mainstream; and as bright as the future is for SaaS BI will be; it will come at a price. Technically; there are concerns related to platform stability and security that need to be addressed, operationally the cost of sale needs to be monitored closely along with the pricing strategy, and the concept of Freemium needs to become either a limited or fading trend for these vendors to survive. There most likely will be further fallout as these vendors with similar functionality start to differentiate themselves, those that will survive will need to develop strong ISV relationships to extend value and revenue models, and acquisition looms as the on-premise vendors want to establish themselves in the SaaS space. However; the true measure of success will be how much further these vendors can penetrate the 25% user adoption rate in the existing BI solutions used by organizations today.

Even with these challenges; the climate is much better for these vendors; compared to the dot-com startups of a decade ago; based on the economies of scale alone. The upfront costs in the development together with the ongoing management are much lower for these vendors; allowing for further innovation; which should result in increased value for their users and a revenue channel with further reaches. These factors should translate into a platform that will allow organizations to ask more questions about their data and closing the gap between the event and the actionable decision. Those ripples will be disruptive.

As for my I.R.S records t-shirt or pets.com hand puppet; reminders of what were; instead see these vendors leading the way in realizing the goals and intentions written by H.R. Luhn in 1957 on using business intelligence to support decision making activities