Digital transformation dominates most CIO priority lists pertaining to questions such as:  How will digital transformation affect IT infrastructure? Will technology live on-premise or in the cloud? Depending on where that data lives, an organization requires different skill sets. If you’re building these resources in-house, then you need an infrastructure as well as people to build it, manage it, and run it.

As you consider implementing a digital transformation strategy, it is helpful to understand and adopt an event-driven data approach as a part of the cultural and technical foundation of an organisation. One definition of event-driven data architecture describes it as one that supports an organisation’s ability to quickly respond to events and capitalise on business moments. The shift to digital business is also a shift from hierarchical, enterprise-centric transaction processing to more agile, elastic, and open ecosystem event processing.

Nearly all business-relevant data is produced as continuous streams of events. These events include mobile application interactions, website clicks, database or application modifications, machine logs and stock trades for example. Many organisations have adopted an event-centric data strategy to capitalise on data at the moment it’s generated. Some examples include King, the creators of the mobile game Candy Crush Saga that uses stream processing and Apache Flink to run matchmaking in multi-player experiences for some of the world’s largest mobile games. Also, Netflix runs its real-time recommendations by streaming ETL using Apache Flink and event stream processing. And when advertising technology company, Criteo needed real-time data to be able to detect and solve critical incidents faster, they adopted stream processing and introduced an Apache Flink pipeline in their production environment.

So should we all adopt a stream-first mindset? Maybe, but it’s not as simple as that.

There are a number of considerations to take into account when transitioning to real-time data processing – anything from the purely technical to organisational requirements. Developers need to be prepared to support and build upon a faster, more distributed architecture designed to deliver continuous value to its users. In addition, a solid data strategy, clear vision and adequate training are required.

So what differences can we highlight between a traditional and an event-centric data strategy? What should CIOs and IT leaders keep in mind while going through such a transition? Let’s take a closer look…

There are new responsibilities for the IT department
When you change to event stream processing, this affects how your business perceives IT and data systems. Your IT department will take on additional responsibilities. Your infrastructure will enable multiple tiers of the organisation to access and interpret both real time and historical data independent of heavy, centralised processes. Making the most of this approach requires stricter control over how data is processed and applied to avoid people getting stranded with piles of meaningless information.

Your SSOT (single source of truth) is recalibrated
Your data strategy will ultimately impact the outlook of data authority as well as the level of chaos within your organization stemming from increased data creation. From the single-point data store in a monolithic data architecture, your focus will change to a stream processor, making data and event-driven decisions as you react to events in real time or using sensor data to find the cause of a system failure that might impact the operation of your business.

Data is constantly on the move
In monolithic architectures, data is at rest. But in event stream processing, data is “in flight” as it moves continuously through your infrastructure, producing valuable outcomes when data is most valuable: as soon as it is generated. You need to reimagine your systems and infrastructure to handle large volumes of continuous streams of data and make appropriate data transformations in real time.

Your focus is reacting to data
Your data infrastructure opens a different perspective, moving from a “preserving-my-data” to a “reacting-to-my-data” state of mind. Stream processing enables your digital business to act upon events immediately as data is generated, providing an intuitive means of deriving real-time business intelligence insights, analytics, and product or service customisations that will help differentiate your company from its competition. Therefore, your system needs to focus on endorsing this continuous flow while minimising the tradeoffs required to process it.

Figure 1: data at rest – focus on preserving the data

Figure 2: data “in-flight”- focus on reacting to my data in real time

A change in culture is needed
Adopting an event-driven architecture requires careful planning and groundwork in order to drive a successful transition. For a successful transition, both cultural and technical considerations should be taken into account. It expands way beyond the data infrastructure teams and requires the early involvement of multiple departments within the organisation. A ‘new’ data approach requires CIOs to align with their IT and data leaders on a shared vision. This is very important whilst the enterprise evolves from a passive request/response way of gathering data insights to an active, real-time data-driven way of operating.

Stream processing with Apache Flink enables the modern enterprise to capitalise an event-centric data architecture, and leverage the value of stream processing: understanding the world as it manifests in real time through powerful, distributed and scalable data processing.

If you want to learn more about the latest developments in the stream processing space, the upcoming Flink Forward conference in San Francisco is a great source of thought leadership and inspiration about how to use stream processing to power a real time business of tomorrow.

This is Part 2 of Data Strategy series discussing “How To” following Part 1 of Data Strategy that dealt with “5 ‘W’s of Data Strategy”. 

Strategy is about doing the right things and tactics is about doing things right. Data Strategy is about doing the right things to distill Data into insights for the organization.

As I mentioned in Part 1, a good enterprise Data Strategy should be actionable to your specific organization and evolutionary to adjust to disruptive forces. Now let’s discuss the process to guide your organization to lay out your organization-specific Data Strategy. It is a 4 step process that I’ve used often and this high-level framework for Data Strategy addresses the key elements of People, Processes, Technology, and Data.

Step 1: Planning And Discovery

This step encompasses identifying business objectives & needs, enlisting sponsors & stakeholders, defining scope & schedule, and discovering technology & data assets that have a role in the Data Strategy. I’ll dig little more into business objectives and stakeholders points which I believe are very crucial.

Identify Business Objectives and Problems that need to be solved with data

Data Strategy should align to business objectives and address key business problems / needs as the primary purpose of Data Strategy is to unlock business value leveraging data. One way to accomplish this is to align with corporate strategic planning process as most organizations have a strategic planning process anyway. Some of the examples for business objectives / business needs: Drive customer insights, Improve product and services efficiently, Lower business risks, Drive revenue growth and/or profitability, Regulatory compliance

Identify Key stakeholders, team members and sponsors

In my opinion, there are 3 types of people you need to take into account.

(a) Executive sponsor(s) : I can’t under estimate the importance of finding and aligning with executive sponsor (s) that’ll support you through the ups and downs of formulating the Data Strategy and implementing it.

(b) Right talent on your team: Make sure to influence and evangelize to the people with right skills / talent to be on your team. Explore both internal talent as well as external consultants.

(c) Potential trouble makers: Every project / initiative will have some ‘stakeholders’ who either deliberately or unintentionally are opposed to change. Knowing who they are and their motivations upfront will help you later in the process.

Step 2: Current State Assessment

In this step, focus primarily on current business processes, data sources, data assets, technology assets, capabilities, and policies. The purpose of this exercise is to help with gap analysis of existing state and the desired future state. As an example, if the scope of the data strategy is to get a 360 view of customers and potential customers, the current state assessment would include any business process, data assets including architecture, capabilities (business & IT), and departmental policies that touch customers. Current state assessment is typically conducted with a series of interviews with employees involved in customer acquisition, retention, and processing.

One important observation I made during assessment is that you’ll come across people in the organization that are natural data evangelists. These people truly believe in the power of data in making decisions and may already be using the data and analytics in a powerful way. Make note of these people and make sure to take their help in later phases to drive a ‘data-driven’ culture in the organization.

Step 3: Analysis, Prioritization, & Roadmap

This phase is probably the most intense and contentious phase and without a doubt will account for majority of the time in formulating data strategy. With Big Data and Cloud computing, the analysis has gotten even more complicated than in the past. With the desired future state in mind, analysis should focus on identifying gaps in data architecture, technology & tools, processes and of course people (skills, training etc.). Big Data brings new data sources into the mix and Cloud computing enables new options for data integration and data storage.

The gap analysis will present multiple strategic options for initiatives and the next task is to prioritize these options with business objectives / needs as the primary criteria. The sponsors and stakeholders will have a key role to play in prioritizing these initiatives. The end result of this phase is a roadmap to roll out the prioritized data initiatives. Without going into too many details, some of these data initiatives could be Data Governance, Data Quality, and Master Data Management (MDM).

Step 4: Change Management

Some people would argue that Change Management is not a distinct step but my past experience has shown that the best of Data strategies have faced an untimely death precisely because of lack of focus on change management. Change management should encompass organizational change, cultural change, technology change, and changes in business processes. Data Governance, which deals with overall management of availability, usability, integrity, and security of data becomes a crucial component of change management. Appropriate incentives and ongoing metrics should be key part of any change management program.

Data Strategy Components

As a bonus, I’ve decided to include a section on various components in the final Data Strategy output. I believe that Data Strategy document should include all or at least some of these components:

Background / Context: This section should articulate background that necessitated the Data Strategy in the first place. Examples could be: Corporate strategic direction, Digital Transformation initiative, or mergers & acquisition related context etc.

Business case: The sole purpose of Data Strategy is to unlock business value and this section should articulate the value being unlocked both quantitatively and qualitatively. The business case is probably the toughest one but a necessary one.

Goals: This section identifies specific Data Strategy related goals and ideally in a SMART fashion (Specific, Measurable, Agreed upon, Realistic, Time-based)

Implementation roadmap: This section connects the strategy with tactics with a roadmap on how the strategy will be implemented over a period of time.

Risks and Success factors: Strategy should directly address various risk factors and success enablers (or accelerators). Time and time again, change management is either a major risk or success enabler if not thought through in a detailed fashion so make sure to address it head on in this section.

Budget estimates: What good is a strategy if it doesn’t have budget estimates. My advice is to be realistic and as comprehensive as possible. If you take short cuts to get a strategy approved, it’s just a matter of time before it comes back to bite you.

Key Performance Indicators (KPIs) and Metrics: To ensure that the strategy is either on track or needs to be adjusted, identify KPIs that need to be tracked on a short term and long term basis.

Like this article? Subscribe to our weekly newsletter to never miss out!

Follow @DataconomyMedia

I am sure you’ve come across many 2016 statistics on Data and Analytics as I have. I’d like to use couple of statistics from IDG’s Enterprise 2016 Data & Analytics Research to start this article. As per their research, 78% of enterprises agree that data strategy, collection & analysis of big data has the potential to fundamentally change the way their company does business over the next 1 to 3 years.

On the other hand, Bernard Marr of Forbes in his Sept 2015 article ’20 mind boggling facts everyone must read’ mentioned that “Less than 0.5% of all data is ever analyzed and used, just imagine the potential here.”

So how do you tap into this huge potential of data collection and analysis in enterprises? I believe that a comprehensive enterprise-wide Data Strategy can give significant competitive advantage in the marketplace. So how does an enterprise get on with such a strategy if it’s not already there? That’s exactly what we shall do in this series of articles on data strategy starting with some basics.

WHAT is Data Strategy?

WHY do we need a Data Strategy?

WHEN should I start or have a Data Strategy?

WHO in our organization should drive this Data Strategy?

WHERE do we start with Data Strategy?

HOW should we go about with Data Strategy?

I’ll address the 5 ‘W’s in this article and address the ‘How’ part separately as it deserves much more detailed attention.

What Is Data Strategy?

I define Data Strategy as a strategy that lays out a comprehensive vision across the enterprise and sets a foundation for the company to employ data-related or data-dependent capability. We can argue about the exact semantics of this definition but the key points are the bold words.

My broad guidance as you lay out this Data Strategy is to make it actionable for your specific organization and industry and somewhat evolutionary to adjust to disruptive market forces. Make the Data Strategy incorporate some guiding principles to accomplish the data-driven vision, direct your company to select specific business goals, and be a starting point for data-driven planning across the company.

Why Do You Need a Data Strategy?

This question may be asked in multiple ways but the essence of this question is: Is it worth it?

Not sure if you belong to the camp that believes data is an asset to the organization but if you do, you must believe that data that your organization owns is a resource that has economic value and you expect it to provide future benefit just like any other asset. I am sure you consider your employees as assets and you have employee-focused strategy (attracting and retaining etc.). If so, shouldn’t you treat data the same way? The preeminent market research firm Gartner states that “Information is an under-managed, under-utilized asset because it’s not a balance sheet asset.” It’s about time we gave data some respect and have a data-focused strategy.

According to the IDG research I mentioned above, the top 3 problems that organizations expect data to solve problems are as follows.

(1) Finding correlations across multiple disparate data sources

(2) Predicting customer behavior

(3) Predicting product or service sales

Without a comprehensive enterprise-wide Data Strategy, how can you expect to solve any of those problems mentioned above? Either you won’t have answers to these questions or you will be answering them very inefficiently consuming lot of resources.

Without a Data Strategy, the organization will be forced to deal with myriad data-related initiatives that most likely are in progress by various business groups / departments. Believe me, different business units within the company are not sitting around waiting for an enterprise Data Strategy if you don’t have one. These initiatives may be some kind of Data Analytics, Business Intelligence, Master Data Management, Data Governance, Data Quality program, or Data warehouse. And worse yet, these businesses may be dealing with inaccurate, incomplete, or inconsistent data leading them to wrong business decisions.

When Do You Need Data Strategy?

I am sure you are expecting an answer like ‘NOW’. Even though that is probably the simple, easy and the correct answer, I’d advise you to tie your Data Strategy to some major corporate initiative or business planning cycle. It could be some kind of Digital Transformation initiative, Business Reengineering, or even annual strategic planning. It could even be tied to a merger and acquisition event. Why? It is practical to justify having a data strategy initiative if you don’t have one. It is a much steeper hill to climb to justify data strategy just on its own and out of the blue. I am not saying that it can’t be done but based on my prior experience, it is easier to get it off the ground tying to some other business event / initiative.

Who should drive it?

I know I’ll get lot of flak for saying this but I’ll say this anyway. Please don’t hand off your enterprise-wide strategy to your Chief Information Officer (CIO) and wash your hands off. I have nothing against CIOs but data is a corporate asset and not just an IT asset. In my opinion, enterprise Data Strategy belongs at Chief Operating Officer (COO) level. No, I am not being ambitious but being direct and blunt about it. If your organization has a Chief Data Officer (CDO), let him/her be the owner of this Data Strategy. I hope that CDO reports directly to CEO/COO.

Where do you start?

What I meant by this question is should a company start Data Strategy at the corporate level or at a business unit level? Based on the thesis of my article so far, the answer obviously is at the corporate level. But depending on the organization’s operating model i.e. how it is structured, in some cases it might be practical to start at some business unit.

I have seen instances where a business unit started Data Strategy which raised some uncomfortable questions for the corporate level entities. This in turn resulted in a corporate wide Data Strategy initiation. You might call it a sneaky way to force the organization to face the music but it works. It all depends on who is evangelizing the Data Strategy and how much political clout they have in the organization.

This part of this series addressed the 5 ‘W’s. Stay tuned for part II where I’ll tackle the ‘How’ part.

Like this article? Subscribe to our weekly newsletter to never miss out!

Follow @DataconomyMedia