I will go through the seven steps you will end up using in doing your analytics projects. These steps will help understand how to identify and prioritize research questions, convince collection of high priority doable hypotheses, and define and acquire right data.
Business opportunities abound as the speed of how the consumers act and react to the markets and how the markets acts and reacts to consumers’ behaviors feed each other. These opportunities have their own life cycles , of creation and destruction.
There are two major challenges organizations face, and analytics as a strategy comes to the rescue.
(1) The complexity of business processes are connected to the changing environments of how consumers act and react, and in turn contributing to the new life cycles of value generation (consumer dynamics), the creation of new generations of products arising out of the old ones (product dynamics), and together how organizations, governments, and consumers push and pull to receive their share of the value in the market place (market dynamics). Yet organizations have to abide by the ethical considerations of operations.
(2) While these complex interactions are going on in the market, the previous generations of concepts of organizational and market efficiencies are becoming standard best practices in all organizations, eliminating any differences in uniqueness of products and services.
Connecting the ideas of being relevant, timely, and personalized with the dynamics of business processes mean we collect right data, make it available for analysis, do pertinent analysis, and use the insights and implementation of analytical results in the right way.
From the point of view of an organization that is trying to be competitive using analytics as a key strategy, the patterns of activities on how these opportunities are identified and leveraged are explained as a seven step processes, listed as (1) spotting the opportunities, asking right research questions, (2) (4) defining and confirming the right measure to use, (3) listing all possible structural hypotheses, and identifying verifiable hypotheses, (4) defining and locating the data sources, (6) asking, getting, auditing data quality and preparing the data, (6) analyzing and building models, and (7) presenting for implementation.
In the following, these concepts are explained.
1. Identifying and articulating the business opportunities - What is the right research question and what is its measurement?
Each and every dynamic positioning among consumers and markets may look like a great opportunity, but not all opportunities are the same in terms of value it can bring to the organization and the ripeness of the market situations that will yield itself for value creation.
2. Defining a right measure to use
Questions need to be connected to measurable outcomes so that we can analyze, rank, and make conclusions. Typically organizations use return on investment to rank opportunities and among the highly ranked opportunities, select the one that is ripe for execution and implementation, popularly stated as picking low hanging fruits. It is also possible that there is enough previous research done on a select opportunity that is relevant for an organization. In any case fundamental to these activities or insights is that there is a specific measurement that one has to use as explaining the core problem/opportunity. Moneyball is a great example here. Oakland A's did not have enough budget to compete compared to the top teams and the immediate effect was that they were loosing their top players. Oakland A's was hit by double trouble. The question is how can I compete in the market place with a budget that is one third of the budget of the top teams and how can I retain the players who help achieve that. In Moneyball case, the market was ripe with data to show that OBP(on base percentage) based strict play has twice the likelihood of winning a game, but no team was willing to create the corporate change with that approach. Also, the players who can be supportive of OBP are inexpensive. In setting OBP as the strategic metric, Oakland A's addressed their challenge.
In terms of defining the right measure to use for analysis, it is tightly connected to the research question prioritized. If it is a simple classification problem where we hypothesize that high value customers are highly educated and making above average income, this measurement of interest is a conditional probability.
On the other hand, if the hypothesis is that those with two or more children of age less than 15 years old are more likely to be a loyal customers in the next 3 years, then the measurement is likelihood but it is more subtler than the previous example; it is a loyalty survival probability.
The way these two probabilities are defined, calculated, interpreted, implemented, the value they bring to the organizations, and how the consumers need to be engaged are all very different, depending on the context and the product/customer segments of an organization.
So pay close attention to the hypotheses and make them help identify and define the right measure.
3. Converting business/process related research questions leading to verifiable, computable, and attributable hypotheses, via the key measurement.
Continuing on the path of analytics as a key strategy, once we identify the right opportunity to pursue, we have to unpack this organizational opportunity into components of analytical steps. The first step is asking right research questions from different angles and spotting the most probable and most valuable question along with its measurement, that would help us seek valuable details that can be attributed to consumers’ behaviors so that consumers can be engaged in a right way through various campaigns. There is a systematic collection of thoughtful activities moving from business process questions to verifiable hypotheses. This is what MckInseyites call as "Structural Hypotheses" or MECE, mutually exclusive and completely exhaustive collection of hypotheses that explains the differences in the original measurement.
“Hypotheses are suppositions or proposed explanations made on the basis of limited evidence as a starting point for further investigation” (Google definition)
For example, if our limited historical observation points out, at organizational level, that the minimum monthly sale happens during the holiday season, a season where, at macro level, consumers spend the maximum amount of disposable income, it is a missed opportunity for the organization.
However, it could be because our products and services are not tuned to the holiday season (product dynamics), or because we are not having the right campaigns during those months (consumer dynamics), or because we are not matching the competition efforts (market dynamics).
This is a missed opportunity with maximum possible value to the organization because the whole economy is buzzing with almost one fifth of spending by consumers during the holiday season.
The testable hypotheses are,
– There are high value consumers who are looking for our products during holiday season
– Our products captures or represents holiday moods and sentiments
– Our average marketing efforts in terms of time and money, for high value customers are lower than our competition
– The average marketing reach out by channels of communication to our high value consumers need to be matching at least the levels of what the competition is spending.
Note that hypotheses are the links that explicitly connect the data to the research questions and hence to the opportunities.
The opportunities are realizable in terms of being relevant, timely, and personalized, if the measurement arising from important hypotheses are estimated for each and every one of the consumers, truly becoming a one to one marketing solution.
4. Defining data sources
5. Extracting, Transforming, and Loading for analysis
7. Stating conclusions including the levels of uncertainty in conclusions and caveats
Once we settle on our hypotheses, we are ready to define the right measure and the right data. Most of the times, the challenges of locating right data is not that much compared to not giving enough careful attention to defining the right measure. Continue in the next section, …