6 Ways to Improve Your Analytics Use
1/24/2022
Since we surveyed chief business officers on campus data use for the 2019 NACUBO Study of Analytics, the COVID-19 pandemic greatly impacted college and university operations. But did this impact institutional ability to leverage analytics? NACUBO convened six of the 334 survey respondents from 2019 for focus group discussions on analytics use and persistent challenges to greater adoption.
All six participants indicated that the pandemic affected the use of analytics at their institutions. The COVID-19 pandemic has demonstrated how critical having access to quality, just-in-time data is for decision-making. A CBO from a small private institution said, “I think the pandemic has sharpened the need for more and better data for decision-making.”
The CBOs agreed that higher education still has challenges to navigate before the sector can fully leverage analytics. The challenges discussed by the focus groups were not new; they reiterated major challenges identified in the 2019 NACUBO Study of Analytics.
In the following six sections, this brief summarizes these challenges and provides approaches to address them:
- Define analytics for your institution
- Build time for the strategic
- Invest in meaningful collaboration
- Set realistic and purposeful goals for data management and use
- Provide support and set expectations
- Creatively invest in staff
Define analytics for your institution
“I think maybe the problem with analytics in general is it's a little bit elusive,” explained a CBO at a community college in the West. “Everybody, including me, thinks, ‘Analytics: That sounds great!’ I'm bought into the idea that it's valuable and helpful and will give us insight and understanding into our data and maybe what we can do differently or how we can make better decisions. I get all that; I think most people do. What’s lacking is that impetus to ask, ‘What is the problem that we're trying to solve? What is analytics going to do? What does that actually look like?’ I think we've been a little bit fuzzy.”
In higher education each institution is unique, which makes having a shared vision for analytics across institutions challenging. Colleges and universities have diverse missions, serve different students in different regions, and have their own histories and cultures. Since analytics efforts should be guided by institutional priorities, there is no single template for how analytics initiatives should look. Analytics initiatives may have different champions, focus on different metrics, and demand different actions from one campus to another.
To address this challenge, a community college CBO explained, “We need a lightning rod or some sort of catalyst to articulate the problem and also solution.” The six focus group participants agreed that analytics should be defined as an institutional asset and, more specifically, posited that an analytics strategy should focus on institutional priorities. A CBO at a private four-year institution in the West phrased it this way: “What are the outcomes you want? I think it helps if you get everybody on board and say, ‘This is where we want to get.’ Then let's talk about how we get there and think about what systems or processes or things we have to do to make that a reality.” When viewed as a strategy rather than a tool or technology to implement, institutional leadership can define analytics as a process that facilitates data-informed decision-making around the institution’s priorities.
CBOs did not reach a consensus around which leader—the president, provost, CFO/CBO, chief information officer, senior institutional research or effectiveness officer, or other senior leader—was in the best position to articulate and lead analytics efforts. A CBO from a four-year public institution in the South explained, “In the perfect world, that might be the chancellor or the president, but they’re far busier than we are, so expecting them to take the lead on this is unrealistic.”
Who is best suited to lead these efforts may depend on the institution’s culture, current priorities, or other factors unique to the campus. One participant posited, “The CFO is really the only position that has both the standing and the understanding to bring this before the cabinet—that’s been my experience. There's no one else focused on this. It's difficult, but I don't know where else in the organizational structure you would look for a champion on this to a emerge.”
A few focus group participants talked about how their institutions were growing analytics efforts out of the business office. Although this approach won’t work at every institution, a CBO from a public university in the Northeast described some benefits of this approach. When she arrived at her current institution, she said it was difficult to explain how higher education’s funding model changed since the Great Recession—especially since some of her colleagues thought “there was a pot of gold, and we [the business officers] were just hiding it.” She explained, “We all grew up in a time of growth, so a lot of folks are used to add-ons, and the reality is that's not the world we live in today. We have to reallocate existing resources. We're not used that. We don't like it.” Now, through some organizational changes and investments in analytics, this institution has moved to be more data-informed. Defining analytics as a strategy to support the long-term health of their institution helped stakeholders understand how their institution planned to use data and what their individual roles were in leveraging analytics.
Analytics efforts won’t always grow out of the business office—at other colleges and universities it may be more appropriate, depending on the institution’s goals and needs, for initiatives to grow out of academic affairs, student affairs, or another unit.
Suggested Resources
- Joint Statement on Analytics
- Accelerating Analytics: Data Governance for Business Officers
- Analytics is a Skill, Not a Tool
Build time for the strategic
In the 2019 NACUBO Study of Analytics, 77.4 percent of CBOs identified lack of time or people being “too busy” to leverage analytics as a barrier to the use of analytics in higher education. During the COVID-19 pandemic, college and university leaders had many urgent needs to meet, making time an even more limited resource. A CBO from a public university in the South explained, “We are constantly dealing with the urgent rather than the strategic. And I would put analytics in that strategic category.”
Although focusing on urgent student and campus needs should be the priority, focus group participants highlighted that analytics can support decision-making during crises. For example, one CBO explained that data supported difficult staffing decisions that needed to be made, and other CBOs discussed using data to support decisions about course delivery modes and offerings, monitoring enrollment and financial impacts, and other critical areas impacted by the health crisis and transition to temporary remote learning. The challenge then, as a CBO from a small private college in the Northeast put it, is focusing on analytics: “You plan your day when you get in, and you get off track.”
The CBOs agreed there is no easy solution for finding more time to focus on the strategic, but that it was something that needed to be done so their institutions could thrive. A CBO at a public university in the South explained, “We struggle to get past the day-to-day to focus more on the long-term strategic things that we ought to be focused on—and that we’re increasingly going to have to be focused on.” Focus group participants did agree that collaborating with other members of the president’s cabinet around the use of analytics may help to identify ways for analytics to be leveraged as an institutional strategy.
Suggested Resources
Invest in meaningful collaboration
Nearly 62 percent of CBOs who participated in the 2019 NACUBO Study of Analytics said that a siloed campus or lack of collaboration was a barrier to the use of analytics their institutions. Focus group participants discussed three reasons why collaboration around data is challenging for colleges and universities: higher education’s siloed organization, individuals having different perspectives and backgrounds, and distrust around data, particularly data that is difficult to understand or is outside of one’s subject matter expertise. A focus group participant from a public university in the South explained, “I think we discuss collaboration. I think everyone is genuinely interested in seeing that move forward, but as soon as we break up and go back to our respective units, those silos are just so difficult to overcome. I think there's a variety of reasons for that.”
The phrase “silos of excellence” is often used to describe units within colleges and universities. A community college CBO from the West explained, “When you're entrenched in your assignments and responsibilities in either finance or student affairs or academic affairs or IR—wherever it is—you are focused on just getting your own thing done and doing it well.” However, addressing today’s complex issues may require collaboration and for data to be integrated from multiple sources.
Sometimes when different departments and units do collaborate, they struggle to find common ground and may even have distrust for one another. One CBO shared that his institution’s senior institutional effectiveness officer offered to collaborate with the business office on analyzing finance data. In the focus group, he shared, “What I don't say to them is: I wonder if you really have an appreciation for all the accounting and finance issues that we deal with so that you can interpret the data well enough to make decisions. And I guess the conclusion of that is: Well, I'm not sure if you really do, so then I tell institutional effectiveness, ‘Why don't you just let us figure that out and do the reporting and the analytics on our side, and you keep over there with students and faculty.’”
A CBO at a public university in the South said, “We just talk a different language.” He added, “And there is that the element of distrust, where we just have not had that dialogue with them that would help both of us.” He explained that addressing these challenges requires effort. He said, “I would be the first to say there are areas of the university that I don't have any background in. Reaching common understanding on a lot of these issues is hard work, and it requires someone to focus on it—not just assume it will happen in the background.”
Focus group participants agreed that the responsibility for creating a collaborative culture largely fell on senior institution leaders, including themselves. A CBO at a community college in the West explained, “I think it's incumbent on us at the cabinet level to try to build those collaboration efforts. It's just that I don't think any of us have done a very good job of that.”
Creating this culture will require business officers not only to think about institutional policies and practices that may improve collaboration among units, but also to reflect on their own roles as collaborators. The CBO who expressed reluctance to collaborate with his IE counterpart explained, “Thinking of myself at my own situation, maybe I need to reach out a little bit more to my counterpart in institutional effectiveness and say, ‘I want to help facilitate that and make it more of a part of our culture. That’s going to take a while, but maybe you and I working together can bring this to the cabinet and have it be more of a comprehensive discussion.’ And maybe we just need to each do a better job of reaching out and forming those partnerships with academic affairs or IR directly, or maybe the student side of the house.”
Suggested Resources
Set realistic and purposeful goals for data management and use
In the 2019 NACUBO Study of Analytics, 71.5 percent of CBOs indicated poor data quality was a barrier to leveraging analytics at their institutions. Focus group participants also raised this issue. A CBO at a community college in the West expressed skepticism about his institution’s data accuracy. He explained that when viewing reports, he often thinks, “Hmm… I'm not sure if I really think that's right, so I'm just going to go with my gut… I want to go this way instead of what the data is telling me because I'm not sure if I completely think that you've done an excellent job of cleaning up that data. If it's not perfect, then how can I really rely on it?”
Mismanagement of data and errors in analyses exacerbate concerns about data quality. A CBO at a private university in the Midwest shared, “We did have a challenge with our institutional research data last year. They were working directly with faculty, who found an error, so now, based on that, there's this mistrust of the data. You just can’t overemphasize the importance of accuracy enough. Once you find out that one thing you did wrong, everything comes into question.”
Even when data and analysis are accurate, participants indicated that institutional culture can still make leveraging analytics a challenge. A CBO at a private university in the Midwest explained, “I think that goes back to culture—and even beyond that, to human nature. We look for data to confirm what we already believe to be true. I think that's a big barrier to overcome, the idea that ‘If the data confirms what we already think, that’s great and we’ll go with it, but if it doesn’t, it must not be good data.”
To address these challenges, focus group participants indicated the importance of investing in tools, policies, and processes that help institutions manage and use their data. Some commented on the value of investing in a data warehouse, while others talked about working with enterprise resource planning (ERP) vendors to improve data accuracy and usability. Focus group participants also stressed the importance of having a data dictionary and a strong data governance plan.
Regardless of the tools and resources, participants indicated a need to create a culture where individuals recognize and fulfill responsibilities with both data management and use. A CBO at a small private college explained, “We're not afraid to invest money into these things, but there is a culture to some extent where I think [staff, faculty, and leadership] believe that the systems produced the data and everything was going to be right—but the individual departments have to devote the energy to ensure the accuracy, and that’s the biggest challenge that I have at my institution. I keep saying that it's really our data. Individual departments have to ensure of their accuracy; the systems don't do it by themselves. And that's the biggest cultural piece that I'm working on.”
As business officers work with their colleagues to set goals and expectations for data management and use, they may experience challenges. As they do so, they should remember the cliché, “Don’t let perfect be the enemy of good.” Despite best efforts, institutions will still have imperfect data. Leaders need to simultaneously focus on continued improvement of data management while holding decision-makers accountable for making data-informed choices.
Suggested Resource
- Accelerating Analytics: From Data to Decisions
- Accelerating Analytics: Data Governance for Business Officers
Provide support and set expectations
Setting goals and establishing expectations for data management and use helps stakeholders see their role in a data-informed institution. Yet, ensuring stakeholders are aware of their responsibilities does not necessarily ensure that they have the data literacy skills to translate data into action. This challenge, end users having access to analytics but not knowing how to translate it into actionable information, was cited by 80.9 percent of participants in the 2019 NACUBO Study of Analytics.
Focus group participants also discussed this issue. A CBO at a private university in the Midwest explained that stakeholders at her institution talk about the importance of being data-informed, but when they make requests for additional data for discussions, she finds herself wondering, “But what will you do with it?” She added that translating data to action is the “tricky part.”
Much of the discussion about translating data for action focused on challenges stakeholders have with understanding and acting on financial data. A community college CBO explained, “I think there’s a technical aspect to what we do that a lot of people in the institution don't really understand completely—and don't really need to understand or want to understand. Accounting, finance, reporting, ratios, and trends mean a lot to people like us who have been trained in that field. There's so much detail to understand; it's hard to explain that to someone who hasn't been trained in that area.”
This means that, “Taking the time to make sure that people understand what the data is telling them is just super key,” explained a CBO from a private university in the Midwest. Focus group participants discussed two ways that CBOs could help improve data literacy skills at their institutions: serve as educators and build a culture that values data for decision-making.
Although there are exceptions, most higher education business and finance professionals have more training, technical expertise, and contextual knowledge around business and finance data than most of their colleagues. A CBO from a private university in the West explained that the professional backgrounds of stakeholders, particularly faculty, influence their priorities as well as how they view and interpret data. He elaborated, “It's a challenge explaining the analysis of data and having them understand it and accept it as being real. Oftentimes, they have counter ideas of what the data is saying. We work through those things… but that does require us to do a lot of education.”
“One of the things that I found to be helpful in working with other cabinet members is that I try to put information in language that they can understand,” explained a CBO from a small private college. He added, “They see me as a teacher, and they very much appreciate it.” Another CBO explained that her institution recently appointed a new president, and the advice from her departing president was, “No matter who comes in, the best thing you can do is spend quality time with that person so that they understand what the data tell you.” She added that institutional data literacy “starts at the top.” She explained, “We think about trying to communicate with faculty and other staff members about what the data mean, but I would say it’s not limited to them. Even with the president and the president’s cabinet, there are varying degrees of understanding.”
In addition to educating institutional stakeholders, CBOs also stressed the importance of educating governing board members on how to read, interpret, and make decisions using data. “The pandemic has brought a greater focus by the board on us needing to provide them the right data for their decision-making,” explained a CBO from a small private college in the Northeast. He added, “The board is now getting more involved because they see how tenuous so many of our institutions are in terms of being able to survive into the future. They are starting to ask sharper questions, and I think the pandemic really did highlight trying to look five years down the road with good models.”
Focus group participants noted that educating governing board members may take some special considerations. A CBO from a private university in the Midwest explained, “They don't look at this data but four times a year, and they tend to forget what we told them the last time. Having consistency to your board materials is very helpful in establishing that understanding.” A CBO at a private university in the West suggested, “Something we do with our board is we have very consistent variables and goals that we track. We review those at every one of our meetings.” Providing consistent views of the same types of data helps teach stakeholders where to look for what kinds of information. Regularly discussing the metrics helps stakeholders learn how to interpret the data.
It also helps when business officers “put the information into context so individuals can make a judgment,” said a CBO from a small private college. He said that simply sharing charts and figures isn’t enough. He explained that sometimes it seems the reaction to data visualizations is, “Oh! It’s a chart, and it looks good and conveys information;” however that does not mean stakeholders are able to translate the information into actions or decisions. As he’s working on a board dashboard, he said one of his goals is for board members to be able to look at what he was presenting and know if something is “good or bad, as simple as that.” He said the data should be presented with visuals and contextual information to allow board members to make judgements about performance and decisions about future actions.
In addition to educating stakeholders, focus group participants said CBOs needed to work with other leaders to create a culture where using data is “safe.” A CBO from a private university in the West explained, “When we're sharing data, we try not to overreact to it. We want people to be able to have very open conversations and ask, ‘If we're not performing well, why is that? What happened? What can we do?’” This means there needs to be a safe space for questions and mistakes, without finger pointing or blaming.
There is a balance to create within a data-informed culture; focus group participants said accountability is also critical. A CBO at a private university in the West said that sometimes when stakeholders have data presented to them, their reaction is, “Well, I read it, but I disagree. I don't think we should be doing this. Stop the whole machine.” Data rejection can be just as unhealthy for an institution as data blaming; while data blaming builds fear of data, allowing stakeholders to reject the use of data does not advance quality decision-making efforts.
Suggested Resources
- Accelerating Analytics: From Data to Decisions
- Accelerating Analytics: Communicating Business and Financial Information
Creatively invest in staff
The 2019 NACUBO Study of Analytics showed that 78.9 percent of CBOs see workforce capacity challenges or not having enough skilled staff members as a barrier to the use of analytics, and focus group participants reiterated this issue. A CBO at a small private college explained that due to budget limitations, it was challenging to compete with other employers when searching for candidates with experience in data and analytics.
Focus group participants said this is an even bigger concern post-pandemic. “In this environment, I really worry that we’re going to lose a lot of our key people,” explained a CBO from a private university in the West, adding that IT and analytics skills are in demand elsewhere.
A CBO from a public university in the South described this as “the perfect storm associated with capacity and attracting and retaining talent.” He shared examples from his own institution, citing two recent IT staff losses. One left for a position outside of higher education, where the institution couldn’t compete on salary. The other left for a larger university that offered a pay increase and allowed for a remote work arrangement. The CBO added, “So, he’s staying here and working there, and that’s a new phenomenon for us.”
One approach for addressing this challenge is changing policies about work arrangements. A recent survey conducted by EDUCAUSE and CUPA-HR revealed that the vast majority of IT and HR professionals would prefer having a more flexible work environment (90 percent of IT professionals; 85 percent of HR professionals). One focus group participant explained that her institution was already working to move to more flexible options. She said, “We have moved—and particularly our staff that do our analytics moved—to a more flexible remote work policy. I'm in that camp to that says, ‘I don't want to lose my good employee to X institution because they're going to allow them to work remotely.’”
Instituting flexible work policies may not work at every college or university, and, even at institutions where policies do help address workforce challenges, it will likely need to be used in conjunction with other methods for attracting and retaining staff with data and analytics skills. “Training someone that wants to stay here, loves the environment, and wants to be part of the campus community for the long term—that's a great solution,” explained a CBO at a community college in the West. However, he pointed out a challenge with this approach: “Sometimes you train someone, and they get a new set of skills and suddenly they become quite marketable elsewhere.”
Focus group participants also discussed sharing services or outsourcing as options for investing in analytics capacity. A CBO at a public institution that is part of a state system explained that his system is researching the possibility of increasing shared services in particular areas. Other CBOs mentioned sharing services in IT, IR/IE, and/or other core analytics roles—either within a system or as part of a collaborative—as options. However, they also said that engaging in shared services opportunities might exacerbate collaboration challenges. For example, a CBO at an institution in a state system explained that his system was working to collaborate around data use, but that not all institutions were using similar data management and use practices (e.g., different data dictionaries, different reporting capabilities). Further, some didn’t want to lose their autonomy within their data efforts by assimilating to system expectations.
Outsourcing some analytics efforts was another approach suggested by focus group participants, but they acknowledged challenges with this approach as well. A CBO at a public university in the South explained, “There are downsides because those folks are managing our system at the same time they're managing several other institutions’. That means you can’t always be the priority that you would like to be, and when you controlled the resources, you could be. Now you're competing.”
A CBO from a public university in the Northeast shared some advice from her experience working with vendors and consultants. She said, “Sometimes you go out and buy a system before you’ve really thought through what you’re going to do, so you’re running but you haven’t even learned to walk yet.” Plan your journey first, she advised: Institutional leaders needed to clearly define expectations before making investments in analytics efforts.
