Data analysts and decision-makers together turn data into actions. In some organizations, this partnership could look like a tug-of-war, pitting the analysts’ facts against decision-makers’ gut instincts. In others, it could look like a relay race, with each partner running alone except for the brief moment of passing the data summary report. Or it could also look like a soccer match, with data and associated information strategically passed back and forth towards the goal. This last kind of scenario becomes possible where data analysts and decision-makers have a good understanding of each other’s work and have established effective communication and trust. Both will be seeking a clear understanding of the decision context as a means to ensure the analyst has the necessary information to choose among the myriad data sources, metrics, and algorithms. Together they will have established a common language aimed at avoiding common biases, clarifying assumptions, and being attentive for anomalous “weak signals” or other unexpected observations.
Data-Savvy Decision-Makers
An analyst brings you data with intent to inform you and also usually to persuade you to some point of view or action. As decision-maker, you need to understand and trust the data; not just the result presented, but also how the result and underlying data fit with all other available information. Then you can determine whether the data justify a confident or cautious action, or recommend no action at all.
Likely your analysts have followed procedures to select data, tidy data, explore data, analyze data, and finally present data with a concluding summary: “The data say…” At every point in that process, the data analyst made critical decisions. For example, what data to include versus exclude, what patterns represented signal versus noise, what algorithms to use, and what visual or table results to illustrate results. These choices are likely made with a mix of standard protocol, best professional judgement, and inevitable some other more personal biases.
There are many questions a data-savvy decision-maker could ask to better understand the data and allocate the appropriate level of trust to this data, relative to other information.
RAW DATA QUALITY
- Do you have any concerns about the quality of the underlying data?
- What percentage of data was missing?
- How were missing data handled?
DATA SIGNAL VERSUS NOISE
- If you removed outliers, what were the characteristics of those points?
- During analysis, did you get any results that contradicted these results?
DATA APPLICABILITY TO PROBLEM AT HAND
- How closely does the scope of the data match the scope of the problem we are addressing?
- What hypotheses (or other constraints) shaped your data selection choices?
FORWARD LOOKING
- What is the life span of results inferred from these data?
- If this analysis will be repeated AND both uncertainty and risk is high: Is there additional data that would improve the confidence in results?
- Are the underlying data or results relevant to other decisions or analyses?
Decision-Savvy Data Analysts
As an analyst, you turn data into information to guide organizational decisions. Your results offer a window to the state of the organization internally and relative to the world. Decision-makers might challenge you to tasks as diverse as identifying inefficiencies in operational processes, characterizing consumer trends, or developing metrics to monitor organizational health.
Understanding decision context and decision-maker needs can help you select data resources, metrics, and algorithms that provide the necessary information at the minimum effort and cost. Understanding the long term data vision and data applications enables you to weigh the alternative ways to structure and manage the data. Some tasks may be automated or built-in at the first pass through, saving much time on future iterations and translational applications. Information can continue to be distilled with judicious addition of more data – but more data usually costs more money and increased precision is not always necessary. Further, if there is healthy communication among data teams, the data analyst may have a unique perspective on how data needs for one project overlap with other projects.
Decision-savvy data analysts proactively seek to understand the decision-makers’ vision for how the data will ultimately serve the organization. With the responsibility to choose the most appropriate data strategy, questions such as those below can narrow the analysis scope and focus data product design and delivery.
OBJECTIVES
- Is the primary goal to support good decisions or exclude bad decisions?
- What are the decision-making criteria that the data must address?
DECISION CONTEXT and CRITERIA
- What outcome do you expect and hope to see?
- Is there any critical value or outcome that serves as a threshold for your decisions?
- Is the critical information an absolute or relative value?
- Are decisions dependent on data from multiple sources that could be merged?
DECISION FREQUENCY
- How much, and in what ways, does the decision context change through time?
- Is this a one-time or recurring decision?
NECESSARY PRECISION
- To justify a decision, how certain do you need to be of the data and results?
- Is qualitative data (e.g., rank) adequate to support the decision?
The goal of decision-makers’ questions is not to test the data analyst or cast suspicion on the results. Analysts should anticipate and be able to answer these sorts of questions fairly easily. Instead, the goal is to improve the value of the data to the decision-maker. Similarly, the goal of the data analysts’ questions is not to cast doubt on the decision-makers’ knowledge or leadership. Rather, the goal is to ensure that needs and assumptions that impact data analysis decisions are clearly articulated.
Asking such questions (1) promotes thoughtful data analytics and high quality results, (2) awareness of data strengths and weaknesses allows appropriate reliance on that information component among many, (3) ensures key assumptions are shared by analysts and decision-makers, (4) improves overall data literacy and communication, and (5) emphasizes the dynamic nature of data and decision-context within complex systems.
Director of Decision Analysis
Dr. C. Ashton Drew has over 20 years’ experience focused on the challenges of decision-making and planning within complicated and complex systems. Trained in ecological sciences and natural resource management, her work combines quantitative and qualitative data analytics from both natural and social sciences to address emergent challenges. As a collaborator with PlazaBridge Group, she assists organizations to evolve the data culture and awareness necessary to transition to data-driven, anticipatory practice. Read More
The post Organizational Data Culture appeared first on PlazaBridge Group, LLC.