Playbook for Innovation

There are numerous definitions; perspectives and understanding exist for innovation in market place. It is educational to listen, analyze and understand various school of thoughts on the subject and most of it is useful. My definition on innovation for a profit organization is:“Innovation is a better or new method to bring efficiency or generate revenue”. As always, since Stone Age, innovation is the back bone for future & future economy and this message was echoed by The President of United States, The Prime Minster of India, major management consultants and chief executives of corporate world. In the recent survey conducted by McKinsey, 84% of executives say innovation is extremely important to their companies’ growth strategy. Strong message and emphasis on innovation from senior political leaders, management consultants and top executives motivates citizen of a nation and members of a corporate world to think and work on innovation. But the real challenge being faced by corporate world is lack of executional leadership capacity and refined steps to cultivate innovation.

In absence of executional leadership capacity,a structure for innovation within a corporate world, the members who would like to invest their time to be innovative, go down on a path which does not provide fruitful result. Innovation initiatives in an organization without a framework nor a structure is similar to the people who tirelessly worked hard, creative, extremely smart who were passionate to develop a flying machine by watching the behavior of birds. They were successfully able to fell down with wings in terms of flying.

In my own experience, I have seen in organizations where innovation program is established by placing suggestion boxes, launching bright idea database and introducing contemporary furnished conference rooms. When an organization is placing suggestion boxes for innovative ideas, the organization culture is too far behind in general communication. The immediate goal and focus of that organization should be to work on basic general organization communication.

“The real challenge being faced by corporate world is lack of executional leadership capacity and refined steps to cultivate innovation”

By just having a bright idea database, the employee who would like to take the organization imperatives and be part of it would come up with ideas which are impossible in reality due legal, regulatory, and compliance reasons. For instance; for an auto finance industry, a bright idea from an employee is to enter mortgage business segment. It is an idea, may be a bright idea but the company may not have license to be in that segment, nor capital to get into that market. Without this key information, employees are going to work very hard and think about the new ideas which are not practically possible to implement.

Playbook for Innovation:

  1. Establish an innovation program office
  2. Develop an innovation framework
  3. Communicate innovation framework to the organization
  4. Manage innovation
  5. Measure innovation
  6. Report innovation

1. Establish an innovation program office:

Make it as one of the performance measure of a strategic objective of a strategy map (strategy). Assign this task to an executive leader who has visionary ideas with executional insights – I called it as “executional leadership capacity”. It is challenging to find an executive leader in an organization with this trait.

2. Develop an innovation framework:

Let the program office develop this framework. The framework is a tool helps the organization to think outside the box within a business context boundary. There are five components to the framework. They are a) Organization change management: Partner with human resource department. Bring necessary training and coaching to the organization that helps members of the organization to think outside the box. Instill during the training that organization is willing to face both positive and negative consequence of each individual who are thinking outside the box. b) Business-IT alignment: Strong partnership with business team is critical for the innovation program office’s success. To accomplish it, identify partner relationship manager or IT ambassadors for each business unit and develop a sustainable bi-directional communication plan to enable fluid ideas flowing between all teams. c) Industry insights: Partner with the business strategy or business development team. Provide a periodic economical and industry data pertains to the business unit to entire organization. The organization must be aware of whom they are competing in the market, what is the market volume, market segment, how the distribution are spread out, what are the growth opportunities in the competitive landscape and etc. d) Business process competencies: Partner with business process management team or business process operation team or the team who manages the business process for the entire organization. This component of the framework should help the reader of the framework to understand how organization makes money.  e) Technology competencies: Identify technologically savvy and curious members in the organization and ask them to study game changing technology trends which are in the pipeline. At this time the game changing technology trends are: big data, mobile computing, social computing and cloud computing. The members must not be nominated by managers, the members of the team must be volunteered who wants to contribute in this domain.

3. Communicate innovation framework:

The framework is a document that contains all the above components. Make the framework available to the entire organization in all possible media and channels. If the organization management training and coaching technique is effective, organization will seek for the framework and keep it for their reference. It is program office responsibilities to keep the framework up to date and make it available to organization. The framework should also be made available as part of orientation training for new hire for both employee and contractor/consultants.

4. Manage Innovation:

It is the program office responsibility to guide organization to differentiate disruptive & sustained innovation combining with traditional and non-traditional approaches.

5. Measure Innovation:

It is the program office responsibility to measure how program office is performing by measuring number of disruptive & sustained innovative ideas submitted, reviewed, rejected, approved, funded, implemented, benefit realized and etc.

6. Report Innovation:

It is the program office responsibility to report all program office performance metrics to IT balanced scorecard to provide a holistic view on the organization performance.

Traditional EDW vs Big Data

Big data is the newest buzz word in the industry. Executives and information technology experts are all dropped off from cloud computing buzz and hopped into the big data band wagon. Generally, the excitement and buzz in market leads into a misconception of a new idea and takes few iterations before the key concept of new idea is widely understood.

Is Big Data a new concept? – No. The concept has been there for four decades and it has been named as enterprise data warehouse (EDW) and the focus of EDW is primarily on the internal structured data.

The objective of this blog is to bring the key concept of big data by comparing it with enterprise data warehouse.

The simpliest view of a data warehouse is to take all the operational data to one place as single point of truth for the organization and all the combination of analytical reports are generated out of it. A typical enterprise data warehouse data flow is given in the figure above. If EDW is already in existence, what is big data and why this big data, big data di? (I mean: now?)

What is it? – To go back to my last article on Money ball architect, big data is a collection of internal and external information that required for Money Ball architects. Based on my definition, a Money Ball architect (otherwise called data architect or data scientist) shall work to identify a set of differentiating data from a massive data set. Differentiating data will be modeled and derived when the product, service, consumer & partner trends are studied and understood. The consumer, partner, product and economical data is unstructured in uncharted territory. A massive data set in uncharted territory includes both internal, external structured and unstructured data. The massive data set is called big data.

Why is it now? –  A need arose for big data with emergence of social media and other unstructured data widely used both internally and externally in an organization. The unstructured data includes the customer status update in facebook, twitter, youtube video upload, picture upload from a smart phone and voice assistance like Siri. The behavior of consumer, end user actual experience, product acceptance & adoption are viral, unstructured and paradoxical.  With rapid adoption and growth in mobile technology- the consumer interaction, purchasing habits, product reviews are done viral. Simplified approach for the consumer to engage in an experience increased the complexity of analysis from a service provider perspective.

“The behavior of consumer, end user actual experience, product acceptance & adoption are viral, unstructured and paradoxical”

An unsatisfied customer does not call “1-800-sup-port” number any more to file a compliant. They tweet, or update in their facebook status about their experience. The companies trying to measure the customer satisfication by analysing the internal customer compliant database sure will miss the reality. Traditional and trivial data analytics are not good enough anymore. Availability of technologies like Hadoop, HDFS, Avro, MapReduce, Zoo Keeper, Pig, Chukwa, Hive, HBase,R Programming make the big data concept practical.  Emergence of massive unstructured data through social media , utilization of it for daily activities and availability of technologies led into the bigdata now.

All of the core technologies for Bigdata are open source tools. With minimum hiccups during the Easter weekend, Hadoop, MapReduce was successfully installed, configured and functional in Ubuntu Linux runing on Virtual Box on the host OS Windows 7.

There are lots of commercialized version and open source tool available to run an enterprise big data infrastructure. I will write a big data technology landscape as my next topic related to big data.

MoneyBall Architect

Yesterday, I had a coffee talk with one of my external mentee (outside the organization) and he is joining a new employer next week as a data architect. He asked my advice. I started with a disclaimer; my views are not just for a data architect. I expect any architect who joins new organization to do the following. It can also be generalized as a mentoring advice for who joins new organization. The following were my spontaneous response to him.

1. Understand the core business of the organization. If it is a profit organization, understand, how the company is making money? Translate the business model into cash flow diagram in a highest level. Do not make assumption based on the generalized business practice or models. For instance, increasing the customer traffic may increase sales and profit in retail sector but it may not be the case for boutique luxury product or service offering organization. In the boutique luxury product or service organization, the focus may be to retain existing customer. Not to increase the customer base since the supply is very limited and unable to even meet current demand.
2. Understand the culture of the organization. Is the company culture is innovative, fast followers, conservative, aggressive risk takers, collaborative, bureaucratic, autocratic, open, hierarchical (control) and etc.
3. Do due-diligence, investigate, communicate, communicate and communicate with all the key stakeholders in the organization to accomplish 1 and 2.

“It is easy to complicate a thing but it is damn hard to simplify it”

After the short 30 minutes meeting, while driving to work and rushing to take my 8.30 am call in my car, I was thinking the following.

There are terminologies like canonical data model, Meta model, master data management, enterprise data flow, enterprise data bus, enterprise service bus, big data and etc in the realm of data architecture. Quite often, I hear from a passionate data architect about these terminologies in a way, I struggle to understand the tangible benefit. For instance, I hear the definition for enterprise data flow as, enterprise data flow is a structured method that record analyze summarize organize explain the key information which are illustrative to bottom line core business process with inbound outbound flow that indented for the understanding enrichment enhancement and education of key decision maker to make right business decision at the right time to improve overall objective of the business. I didn’t hear the above exact definition but I exaggerated a bit to make my point using Raju Hirani’s idea. Main goal of enterprise data flow is to show critical information to improve ultimate business purpose (like profit). I see architects engage in a prolonged discussion to define taxonomy, framework, methodology, process, tools, governance, stewardship, data quality, reference model and etc. All are great topics and leads into an intellectual discussion, but, sometimes, I noticed the discussion missed to address the ultimate purpose.

It is easy to complicate a thing but it is damn hard to simplify it. My expectation from an architect, including data architect, is to work really hard to simplify the architectural work.

I visualize a data architect as a money-ball architect. For those who have not seen the movie money-ball, the movie is about real life experience in a base-ball team Oakland Athletics where the coach hired Yale graduated economics student who was so passionate about the game and league. He studies the league rules, player profile and creates near optimal data model and analytics to run a successful professional baseball team in the league with lowest investment.

Any successful data architects are money-ball architects. Money-ball architect follows the rule, break the rule, create a new rule and break it until money-ball is identified in the massive multi-dimensional data domain, model the money-ball sub-domain data, identify the key business differentiator from the sub-domain and use it to improve ultimate business purpose.

Money-ball architect will start using canonical data model, Meta model, master data management, enterprise data flow, enterprise data bus, enterprise service bus, big data, taxonomy, framework, methodology, process, tools, governance, reference model (follow the rule). Identify the areas which are not directly contributing to identify the money-ball (break the rule) and drop those areas. Introduce a new concept which directly contributes more to identify the money ball (create new rules) and repeat it until the money ball is identified, modeled and used to improve ultimate business purpose.

To become a successful data architect, create a path for yourself to become a money-ball architect for your organization.