DevSecOps – Two Decades ago

Initiated my career by starting a company, immediately, the next day after my 4 years of under graduation in computer engineering. Almost three decades ago, the startup mind set for a young computer engineer in Southern India was a novel concept while most of the talented computer engineers headed to a safe corporate job or to the higher education to increase their changes to get into a safe corporate job. Initial capital investment for the startup company was provided by my father which was used to buy desktop computers. Ideas, confidence, enthusiasm and hard work were the capital that I brought to start this startup venture. With heavy capital investment, startup was bloomed, and all the work was done in two desktop computers.

Company was a massive idea army with two people. During those days, in day time, I primarily focused on sales and marketing for our startup. After dinner time, soaked my soul into conceptualizing potential product definition based on requirements that I heard during my cold sales call & less than a minute marketing pitch to a small to medium size manufacturing and service companies. Like the most startup, we didn’t have a product nor customer and were desperate to find both until our initial operating cost runs out.  In spite of having lots of ideas in our human memory bank, we prioritized the ideas which fetches both product or solution and customer in a shortest time horizon.  It was a self-funded startup and was not started with a product in mind but started to solve significant industrial problems using software and consultancy.

In a retrospect, we followed product & agile mind set, developed a minimum viable product to attain the business goals in an efficient manner. We were able to implement these constructs, even though, we never heard of these terms in technical journals (like Dr. Dobbs), text books, collaborations with other software engineering experts, industry and clients. We were intuitively following these constructs based out of necessity.

After couple of years of startup journey, like most of fellow computer engineers during those yesteryears, I ventured into a safe technology corporate job. I gained completely a different set of experience working for corporate giants. Customer already committed to a project, sales and marketing pitches were already done with the customer, a contract was signed, expected outcome and time line were defined, team was formed, roles and responsibilities within the team were defined, a methodology was chosen, a formal meetings were scheduled and it was so organized.  I was so excited that all basics were taken care in that environment and even more excited that I was able to focus on solving challenging technical problems in the areas like lexical analyzer, device drivers for tablet, build process, automatic testing of products in multiple operating system, source code control systems and etc. I implemented an end to end automation of the build process, functional testing, system testing, integration testing, packaging (software was released in tape medium to the client who run in Silicon Graphics, Sun Solaris operating systems and in CD for Windows 3.11 and WinNT operating system)  installation scripts ( used to install the software in the client environment), installation testing and etc. The whole process was implemented and maintained by one person – me and along with other responsibilities (implementing device drivers, kernel programming, lexical analyzer) that I had at during the project. In a retrospect, I implemented continuous integration and continuous delivery for a one of the most complex technical projects that I worked in my career. As an industry, we didn’t use terms like CI/CD or DevOps but that was the implementation of CI/CD – Devops constructs that automated the end-to-end process from code to deploy (package that can be shipped to customer). The developers across the oceans were able to develop a new feature in the product for the next release or fix a bug (identified by the client or through internal quality assurance team)  by checking in the code in the code repository, merge the code for release, automatically test with no manual intervention and early morning report was automatically sent to all developers with any compilation or code quality issues.  This level of automation was implemented more than two decades ago and the driving factors for this level of automation was the complexity involved in the product. The product had a single copy of source code that run on multiple version of multiple operating system (HP-UX, Sun Solaris, Silicon Graphics, Window) and product features were generally available in all operating systems.  Even though, we were proud of our accomplishments during those days, we never met our end customers. We had multiple layers and teams in-between end customer and the development team. We worked on the assumption that all the features we developed were useful to the end customers but never met a customer or customer group.

After gaining years of invaluable experience in the technology-based corporations, I ventured into automobile, auto financing, bank, financial services, distribution and medical device manufacturing companies and adding more experience to my elongated career in this type of company.  Information technology space has evolved in last two decades, but one fundamental function has not changed is, developing a technology solution to solve a business problem. Machines has not arrived yet to automatically solve business problems.  In today’s corporate world, almost everyone wants to develop technology solution to solve a business problem like a startup company that evolves their solution based on the customer’s demand and like a technology company which automates end-to-end IT processes and solely focus on the business outcome.

The key takeaways are a) Enterprise Information Technology organization within a corporation primarily provides business value to the bottom line by developing software solution. In recent years, we heard the phrase, software eating the world, which is true. The software solution is an output produced by an IT organization since the inception of IT and it grew into eating the world. The software solutions are not just an enabler of an established business but also it creates new channel, product, and customer. b) The terminology and definition may be relatively new to the IT industry, but these concepts are not. As I mentioned through my personal experience, these concepts were used in pieces even few decades ago. c) With integration of various tools, open source, processes and methodologies, the continuous integration, continuous deployment, concept has been made available for easy consumption. d) The software solution development methodology and approach are more critical than ever. To enhance the collaboration among the developers, end customer, business partners, marketing team, architects, shared service organization and other stakeholders, various concepts are in place like CI/CD, DevSecOps, Agile, Kanban and these concepts/solutions are more matured and proven. It is imperative for the organization to implement these concepts/solutions and realize the expected business outcome which is, to provide respective service or product to delight their customers.

Architect’s view on Compliance & Risk Management

If we study to get just good grades; we may or may not learn. However, if we study to learn; we will always get good grades. 

The mission of an information security in an organization must be to protect and safeguard the company’s assets like customer information or intellectual properties. Objective of a compliance and risk management in an organization is to measure the success rate of information security team’s mission. If the mission of an information security team becomes to be compliant with regulation and other internal & external governance bodies; the company’s asset may or may not be protected.

Have we ever thought about why we go fast (relatively speaking) in a car? Because, we could and the cars are capable of going in high speed; but if you think deep; the reason why we go fast in car is because we have breaks.

When we go fast and do not have systematic brakes; then it is called extreme sports. We don’t want to run an established organization as an extreme sport. As an organization; we need to go in high speed but we need to have systematic way of controlling the speed with a proven brake system.

Information security, compliance, risk management teams exist in an organization to execute projects & programs faster.

Playbook for Innovation

There are numerous definitions; perspectives and understanding exist for innovation in market place. It is educational to listen, analyze and understand various school of thoughts on the subject and most of it is useful. My definition on innovation for a profit organization is:“Innovation is a better or new method to bring efficiency or generate revenue”. As always, since Stone Age, innovation is the back bone for future & future economy and this message was echoed by The President of United States, The Prime Minster of India, major management consultants and chief executives of corporate world. In the recent survey conducted by McKinsey, 84% of executives say innovation is extremely important to their companies’ growth strategy. Strong message and emphasis on innovation from senior political leaders, management consultants and top executives motivates citizen of a nation and members of a corporate world to think and work on innovation. But the real challenge being faced by corporate world is lack of executional leadership capacity and refined steps to cultivate innovation.

In absence of executional leadership capacity,a structure for innovation within a corporate world, the members who would like to invest their time to be innovative, go down on a path which does not provide fruitful result. Innovation initiatives in an organization without a framework nor a structure is similar to the people who tirelessly worked hard, creative, extremely smart who were passionate to develop a flying machine by watching the behavior of birds. They were successfully able to fell down with wings in terms of flying.

In my own experience, I have seen in organizations where innovation program is established by placing suggestion boxes, launching bright idea database and introducing contemporary furnished conference rooms. When an organization is placing suggestion boxes for innovative ideas, the organization culture is too far behind in general communication. The immediate goal and focus of that organization should be to work on basic general organization communication.

“The real challenge being faced by corporate world is lack of executional leadership capacity and refined steps to cultivate innovation”

By just having a bright idea database, the employee who would like to take the organization imperatives and be part of it would come up with ideas which are impossible in reality due legal, regulatory, and compliance reasons. For instance; for an auto finance industry, a bright idea from an employee is to enter mortgage business segment. It is an idea, may be a bright idea but the company may not have license to be in that segment, nor capital to get into that market. Without this key information, employees are going to work very hard and think about the new ideas which are not practically possible to implement.

Playbook for Innovation:

  1. Establish an innovation program office
  2. Develop an innovation framework
  3. Communicate innovation framework to the organization
  4. Manage innovation
  5. Measure innovation
  6. Report innovation

1. Establish an innovation program office:

Make it as one of the performance measure of a strategic objective of a strategy map (strategy). Assign this task to an executive leader who has visionary ideas with executional insights – I called it as “executional leadership capacity”. It is challenging to find an executive leader in an organization with this trait.

2. Develop an innovation framework:

Let the program office develop this framework. The framework is a tool helps the organization to think outside the box within a business context boundary. There are five components to the framework. They are a) Organization change management: Partner with human resource department. Bring necessary training and coaching to the organization that helps members of the organization to think outside the box. Instill during the training that organization is willing to face both positive and negative consequence of each individual who are thinking outside the box. b) Business-IT alignment: Strong partnership with business team is critical for the innovation program office’s success. To accomplish it, identify partner relationship manager or IT ambassadors for each business unit and develop a sustainable bi-directional communication plan to enable fluid ideas flowing between all teams. c) Industry insights: Partner with the business strategy or business development team. Provide a periodic economical and industry data pertains to the business unit to entire organization. The organization must be aware of whom they are competing in the market, what is the market volume, market segment, how the distribution are spread out, what are the growth opportunities in the competitive landscape and etc. d) Business process competencies: Partner with business process management team or business process operation team or the team who manages the business process for the entire organization. This component of the framework should help the reader of the framework to understand how organization makes money.  e) Technology competencies: Identify technologically savvy and curious members in the organization and ask them to study game changing technology trends which are in the pipeline. At this time the game changing technology trends are: big data, mobile computing, social computing and cloud computing. The members must not be nominated by managers, the members of the team must be volunteered who wants to contribute in this domain.

3. Communicate innovation framework:

The framework is a document that contains all the above components. Make the framework available to the entire organization in all possible media and channels. If the organization management training and coaching technique is effective, organization will seek for the framework and keep it for their reference. It is program office responsibilities to keep the framework up to date and make it available to organization. The framework should also be made available as part of orientation training for new hire for both employee and contractor/consultants.

4. Manage Innovation:

It is the program office responsibility to guide organization to differentiate disruptive & sustained innovation combining with traditional and non-traditional approaches.

5. Measure Innovation:

It is the program office responsibility to measure how program office is performing by measuring number of disruptive & sustained innovative ideas submitted, reviewed, rejected, approved, funded, implemented, benefit realized and etc.

6. Report Innovation:

It is the program office responsibility to report all program office performance metrics to IT balanced scorecard to provide a holistic view on the organization performance.

Traditional EDW vs Big Data

Big data is the newest buzz word in the industry. Executives and information technology experts are all dropped off from cloud computing buzz and hopped into the big data band wagon. Generally, the excitement and buzz in market leads into a misconception of a new idea and takes few iterations before the key concept of new idea is widely understood.

Is Big Data a new concept? – No. The concept has been there for four decades and it has been named as enterprise data warehouse (EDW) and the focus of EDW is primarily on the internal structured data.

The objective of this blog is to bring the key concept of big data by comparing it with enterprise data warehouse.

The simpliest view of a data warehouse is to take all the operational data to one place as single point of truth for the organization and all the combination of analytical reports are generated out of it. A typical enterprise data warehouse data flow is given in the figure above. If EDW is already in existence, what is big data and why this big data, big data di? (I mean: now?)

What is it? – To go back to my last article on Money ball architect, big data is a collection of internal and external information that required for Money Ball architects. Based on my definition, a Money Ball architect (otherwise called data architect or data scientist) shall work to identify a set of differentiating data from a massive data set. Differentiating data will be modeled and derived when the product, service, consumer & partner trends are studied and understood. The consumer, partner, product and economical data is unstructured in uncharted territory. A massive data set in uncharted territory includes both internal, external structured and unstructured data. The massive data set is called big data.

Why is it now? –  A need arose for big data with emergence of social media and other unstructured data widely used both internally and externally in an organization. The unstructured data includes the customer status update in facebook, twitter, youtube video upload, picture upload from a smart phone and voice assistance like Siri. The behavior of consumer, end user actual experience, product acceptance & adoption are viral, unstructured and paradoxical.  With rapid adoption and growth in mobile technology- the consumer interaction, purchasing habits, product reviews are done viral. Simplified approach for the consumer to engage in an experience increased the complexity of analysis from a service provider perspective.

“The behavior of consumer, end user actual experience, product acceptance & adoption are viral, unstructured and paradoxical”

An unsatisfied customer does not call “1-800-sup-port” number any more to file a compliant. They tweet, or update in their facebook status about their experience. The companies trying to measure the customer satisfication by analysing the internal customer compliant database sure will miss the reality. Traditional and trivial data analytics are not good enough anymore. Availability of technologies like Hadoop, HDFS, Avro, MapReduce, Zoo Keeper, Pig, Chukwa, Hive, HBase,R Programming make the big data concept practical.  Emergence of massive unstructured data through social media , utilization of it for daily activities and availability of technologies led into the bigdata now.

All of the core technologies for Bigdata are open source tools. With minimum hiccups during the Easter weekend, Hadoop, MapReduce was successfully installed, configured and functional in Ubuntu Linux runing on Virtual Box on the host OS Windows 7.

There are lots of commercialized version and open source tool available to run an enterprise big data infrastructure. I will write a big data technology landscape as my next topic related to big data.

MoneyBall Architect

Yesterday, I had a coffee talk with one of my external mentee (outside the organization) and he is joining a new employer next week as a data architect. He asked my advice. I started with a disclaimer; my views are not just for a data architect. I expect any architect who joins new organization to do the following. It can also be generalized as a mentoring advice for who joins new organization. The following were my spontaneous response to him.

1. Understand the core business of the organization. If it is a profit organization, understand, how the company is making money? Translate the business model into cash flow diagram in a highest level. Do not make assumption based on the generalized business practice or models. For instance, increasing the customer traffic may increase sales and profit in retail sector but it may not be the case for boutique luxury product or service offering organization. In the boutique luxury product or service organization, the focus may be to retain existing customer. Not to increase the customer base since the supply is very limited and unable to even meet current demand.
2. Understand the culture of the organization. Is the company culture is innovative, fast followers, conservative, aggressive risk takers, collaborative, bureaucratic, autocratic, open, hierarchical (control) and etc.
3. Do due-diligence, investigate, communicate, communicate and communicate with all the key stakeholders in the organization to accomplish 1 and 2.

“It is easy to complicate a thing but it is damn hard to simplify it”

After the short 30 minutes meeting, while driving to work and rushing to take my 8.30 am call in my car, I was thinking the following.

There are terminologies like canonical data model, Meta model, master data management, enterprise data flow, enterprise data bus, enterprise service bus, big data and etc in the realm of data architecture. Quite often, I hear from a passionate data architect about these terminologies in a way, I struggle to understand the tangible benefit. For instance, I hear the definition for enterprise data flow as, enterprise data flow is a structured method that record analyze summarize organize explain the key information which are illustrative to bottom line core business process with inbound outbound flow that indented for the understanding enrichment enhancement and education of key decision maker to make right business decision at the right time to improve overall objective of the business. I didn’t hear the above exact definition but I exaggerated a bit to make my point using Raju Hirani’s idea. Main goal of enterprise data flow is to show critical information to improve ultimate business purpose (like profit). I see architects engage in a prolonged discussion to define taxonomy, framework, methodology, process, tools, governance, stewardship, data quality, reference model and etc. All are great topics and leads into an intellectual discussion, but, sometimes, I noticed the discussion missed to address the ultimate purpose.

It is easy to complicate a thing but it is damn hard to simplify it. My expectation from an architect, including data architect, is to work really hard to simplify the architectural work.

I visualize a data architect as a money-ball architect. For those who have not seen the movie money-ball, the movie is about real life experience in a base-ball team Oakland Athletics where the coach hired Yale graduated economics student who was so passionate about the game and league. He studies the league rules, player profile and creates near optimal data model and analytics to run a successful professional baseball team in the league with lowest investment.

Any successful data architects are money-ball architects. Money-ball architect follows the rule, break the rule, create a new rule and break it until money-ball is identified in the massive multi-dimensional data domain, model the money-ball sub-domain data, identify the key business differentiator from the sub-domain and use it to improve ultimate business purpose.

Money-ball architect will start using canonical data model, Meta model, master data management, enterprise data flow, enterprise data bus, enterprise service bus, big data, taxonomy, framework, methodology, process, tools, governance, reference model (follow the rule). Identify the areas which are not directly contributing to identify the money-ball (break the rule) and drop those areas. Introduce a new concept which directly contributes more to identify the money ball (create new rules) and repeat it until the money ball is identified, modeled and used to improve ultimate business purpose.

To become a successful data architect, create a path for yourself to become a money-ball architect for your organization.

Future of analytics

Analytics, simply defined, a discipline of analysis has been in use for centuries. I was invited to IT leadership symposium organized by secure24, a hosting provider vendor in Michigan, USA. The event was choreographed by Thornton A. May.   It was attended by selective senior executive IT leadership team from various industries in the region.  Event was kicked off with a great opening presentation by Mr. May. The presentation was basically a story telling on how IT industry evolved to add tangible business benefit with simplified historical and anthropology examples. I really enjoyed it.

The second part (part II) of the event was panel discussion. The panel members provided very intriguing ideas, message and concept. I learnt few new things from the panel discussion.

Third part of the event; each table was given a topic and asked to discuss about the topic and present it to everyone.  My table got the topic: “Future of analytics” and I was nominated unanimously to represent our table.  My contribution to the table topic was that the future will depend on the social media and social networks. Other’s contribution was mainly on geo-fencing and its role in the analytics.  Since I was nominated to represent the table, I was structuring my thoughts on how to represent our views while listening to other table topics. Mr. May run out of time and our table was omitted and our views on the topic was not heard. I decided to display my structured thought in my blog. This is how I would have presented.

The panel discussion provided great insights and I learned few new things from the panel members. Top two things I fully agree with panel members are:

1.  Successes of IT organization are measured by its capability and capacity to execute and innovate.
2. Most critical differentiating factor of a successful IT organization is not adapting the latest technology trends like: cloud computing, mobile computing, service oriented architecture, integrated identity management. Most critical differentiating factor of a successful IT organization; “right people at the right job”. People are the one who make things happen in an organization.

“Vision without a plan is a dream and a plan without a vision is run around”

I would like to augment my view to the above points; vision without a plan is a dream and plan without a vision is a run around. People are the one who make things happen, not technology.

To be futurist and strategist, these are the few concepts to keep in mind (a repeat from my last blog post)
1. Timing is everything
2. Learn history and study current (identify the driving factors)
3. Unleash the core and its dependencies – Understand what really matters and its dependencies.
4. Connect the dots

Timing is everything:
Imagine that if  iPhone was launched 12 years ago. We would have connected to the apple store via dial-up (AOL) to down load angry bird. Data plan would have cost us $500 a month and connection speed would have frustated a lot. All other external factors would have made iPhone under PALM category.
Learn history and study current:
As far we know, when Grog in 5000BC used two sticks and rocks to graph the upward trend in sales of his new invention, the wheel, the concept “analytics” was born. Almost took seven thousand years to make a leap in this area.  What did we learn from the history? Analytics played a significant role for a mega success like Romans, Henry Ford and others. Those who understand the deep meaning of analytics made everlasting impact. Current expectation is, let the system make decisions  and receive confirmation from end user to execute the plan.  Trust over the system and acceptance of system generated decision have been increasing. Adoption to this model is accelerating.  Navigation system in the car is a prime example for the current state.
Unleash the core and its dependencies:
People are the one who make things happen and in the most of the profit and non-profit organization, people are direct or indirect consumer of goods and service. It is extremely essential to understand people to define or approximate the future. How people view the world? People define the world based on what they see and hear. World is blue when they view through blue glass and it is red when they view through red glass. Deeper view: for decades, the world has been defined numerous times every day by each individual through the social network. Historically, the social network was through snail mail, family gathering, corporate functions, bars and other occasional events. With the advancement of the technologies like smart phones, global networks, wireless networks, software tools, the social networking happens instantaneously. We are defining, redefining our world based on the instantaneous connection through invisible cosmic social network fiber. It becomes an addiction because we wanted to know what is happening around the world we defined. So facebook is addictive.
Connect the dots:
Advancement of wireless network, mobile platform, social media, and end user computing devices led to higher sophistication and at the same rate people’s mechanical monotonic life style has raised up by few notches.  They don’t have interest or time to view the product or service’s sales offering when they don’t need or not in the mood. At the same, they want to make an instantaneous execution of a decision when a systematic analysis was already performed and a decision is presented with highest level of confidence.
Conclusion:
The future of analytics will be presenting decision to you and by click of a button (or slide of a screen) you can execute the decision. For example, based on the social interaction,system will identify a consumer interest and capacity and capability;skiing during January time frame, received  hefty bonus during x-mas time and carry over vacation from the previous year should be taken before first quarter. System will be presented an offer; 5 day ski trip to the best place with lowest possible rate, with best possible quality. Once you confirm the acceptance of the recommendation/decision, everything will be taken care by the system. Once the consumer enter Denver for skiing, geo-fencing will kick in and based on the interest, pattern, spend behavior the most suitable offers applicable at Denver during that vacation time will be presented by consumer’s car while the consumer driving from Denver airport to Breckenridge ski resort.