Assessing Integrated Planning

It is not enough to suggest that one can “know it when you see it” in regard to integrated planning. We need to be able to articulate, measure, diagnose, and develop interventions (actions taken to improve a situation) to increase any organization’s ability to achieve highly developed integrated planning capacities.

I have a habit of getting a bit philosophical before diving into a discussion of measurement and assessment. Not the wayward questioning of the questions life, but epistemologically philosophical. Measurement is a way to generate knowledge about something specific yet unknown. Epistemology is the study of knowledge to include methods for generating knowledge and ways to understand validity, or how well knowledge and truth overlap. If we consider truth to be what is, knowledge is a human creation of the awareness of truth along the spectrum from verified fact to belief to opinion to utter misguided dogma. Our goal in measurement is to develop a system to uncover the truth and generate a way for us to collectively talk about it and use it purposefully.

Measurement. First I will assume that integrated planning is an organizational manifestation. For a discourse on this, see Integrated Planning as an organizational Manifestation by Brodnick and Norris. To understand the degree to which an organization is planning in an integrated manner, we must explore the components of integrated planning from within and around the organization. There are several ways by which we can do this. One way is evaluation by experts where an individual or team can experience an organization through observation, interviews, and other interactions over the course of time. This kind of evaluation typically uses a framework or lens for understanding integrated planning followed by a written summary or report. This is a common form of organizational assessment, most often qualitative in nature where validity is based on the expertise of the evaluators. As a construct such as integrated planning becomes more and more refined over time, its components become quantified and we can apply counting and measuring techniques, which allow for the use of statistics and comparative analyses. In that latter stages of measurement, we can begin to apply normative evaluation, broad comparative and predictive statistics using defined criteria, and eventually we develop predictive validity. Predictive validity would allow us to measure degrees of integrated planning within and across organizations and correlate and predict important performance indicators like profit and revenue generation, efficiency and effectiveness measures, and organization success in varying environmental landscapes. We are not there yet.

My belief is that integrated planning is evolving from evaluation by experts to early measurement techniques and I hope this article and resulting framework help support this evolution. To help this along, I have developed a rubric for the assessment of integrated planning. A rubric is a framework for assessment. It has three key features. First, a rubric has the core components or dimensions that define the concept being assessed. Second, a rubric has a scale or levels of development that describe the degree of evidence or proficiency for each component or dimension. Finally, a rubric has an integrated scoring system that allows for the dimensions to be quantified and sometimes combined to produce holistic scores. Rubrics emerged from the educational assessment field in the 1990s, but the concept had been part of the evaluation sciences for several preceding decades.

Dimensions for integrated planning. My proposed dimensions of the rubric for integrated planning are:

1. Strategic Thinking (Horizons, Scanning, Nimbleness)

2. Analyzing (Data availability and integrity, Technical capability, Historical, internal, external inventories)

3. Planning (Generate plans, Achieve alignment, Manage the process)

4. Engaging (Breadth of engagement, Depth of engagement, Ability to influence)

5. Communicating (Frequency and accuracy, Transparency, Shared awareness)

6. Executing (Pursue priorities, Alter and correct course, measurement, accountability, decision making)

7. Innovating (Generate ideas, Move to action, Create value)

8. Resourcing (Free up and reallocate existing resources, Generate new funds, Fund priorities)

9. Leading (Inspire vision, Manage people and relationships, Generate results)

Levels of organizational development in integrated planning. Organizations express varying levels of mastery of any of the 9 dimensions of integrated planning. When building a rubric, it is always helpful to explore and explain these levels of mastery and define them in such a way that multiple evaluators could observe an organization and generally agree how developed a specific dimension is after observation. This leads to reliable assessment and scoring. I recommend these four levels of mastery:

1. Emerging: capacity does not yet exist or is nascent, uneven awareness

2. Functional: capacity demonstrates basic functionality but needs development     

3. Highly Developed: capacity shows strong results and is both  effective and efficient

4. Exemplary: capacity is truly an example of the best in class among peers

Developing an instrument. A next step in the process of quantifying and assessing integrated planning will be to fully develop, test, and validate a rubric along with an instrument with multiple items. Once data collection progresses, we can test for validity and reliability, item and scale properties, and internal and external dimension relationships. The final stage will include publishing the instrument with a corresponding manual and building the normative database over time. I have an instrument in the works under copyright.