In the Spring 2021 edition of SchoolCEO, we showed you how engaging your community helps you create lasting change. What we’ve learned is that inclusive innovation stems from the different needs and experiences of your school community. Their combined ideas push you toward meaningful, long-term change.

As you work to involve your stakeholders, you might be tempted to issue a survey. You wouldn’t be alone. We’ve seen schools issue surveys on everything from curriculum decisions to strategic planning priorities to the use of a mobile app. It’s not just in education, either. As the private sector has moved toward being more customer-centric, businesses have begun to rely on surveys to assess their employee engagement, marketing impact, and customer satisfaction.

While surveys were once incredibly time-consuming  and expensive, tools like SurveyMonkey have made them cheap and seemingly easy. And when the global pandemic hit, this strategy only became more popular.

“We live in a survey-centric world, especially since COVID-19,” says Sandy Cokeley, APR. Cokeley, a former school communications professional, is the founder and CEO of SCoPE, an organization that helps districts issue communication surveys. As the pandemic made communication more difficult and created a multitude of new problems in education, “districts were surveying everybody about everything,” Cokeley tells us. “All of a sudden, surveys became very popular for schools.”

But just because you can issue a survey doesn’t always mean you should. While surveys have their uses, we’ve also seen schools stifle their innovation processes by relying on them too heavily. The purpose of innovation is to discover previously unknown solutions to the problems your district faces—but when you create a survey, you often define possible answers before they reach your families. This limits the degree to which your community can participate in the innovation process—and may even keep you from finding the best solutions.

Many of the issues with surveys start before you write a single question. Plenty of strategies can help you improve your survey design and distribution, but before you worry about all that, ask yourself: Should I run a survey in the first place?

Surveys are snapshots.

Look, we’re not saying that surveys are never useful. They can be incredibly helpful—but only in very specific circumstances. Unfortunately, most school districts issue surveys without first considering whether this approach will actually help them discover the data they want.

As Cokeley points out, surveys can only help you “know what people are currently thinking, feeling, saying, seeing, and doing.” They’re snapshots. But just like a photograph, a survey is a static glimpse of the past. This means that by the time you look at the data, it may already be outdated. What’s more, a single survey can’t tell you much without some context for comparison. After all, how can you measure whether or not your community’s feelings toward your district are improving if you don’t know how they felt a month or a year ago?

“One of the biggest challenges schools face with doing their own surveys is that when they get their results, they have no comparative data,” Cokeley explains. If you want to see how you’re getting better or getting worse, you can’t run just one survey; you need to run multiple.

Say you issue a survey to gauge your staff’s general feelings on the school district. If 70% of teachers  are “satisfied or very satisfied,” is that good? Are staff members happier now than they were when you began your superintendency? Are your principals’ one-on-one meetings helping? Who knows? That single data point can’t tell you how your initiatives have impacted your staff’s perceptions; it only assesses how your team felt on the specific day they filled out the survey.

Cokeley learned this lesson firsthand at New York’s Pearl River School District, where she served as Director of Community Relations and Quality for more than 20 years. “When I did my own homegrown survey, I would share the results with my superintendent—like, 87% of our parents say we do a really good job of communicating during a crisis,” Cokeley says. “But when he asked if that was good, I had to say that I honestly didn’t know. I thought it was pretty good, but I really didn’t know.” As Cokeley learned, in order for any survey to be useful, you need an established baseline within the population you are surveying so that you can compare results over time.

To build an effective survey strategy, you should view each individual survey not as a singular snapshot but as frames in a film reel, where still images are strung together to show complex scenes. One still gives you a good image, but you can only see the story once the images are played in order. To tell if your initiatives are working, you need to regularly check your results against relevant surveys you’ve run before. That way, you can see not just how your community members feel today, but also the impact your choices are having.

Surveys take time—and money.

Before you get started, you have to decide whether or not you’re willing to invest the time and money it takes to run several surveys over an extended period of time. After your baseline survey, you’ll need multiple iterations of data to accurately assess the impact you’re having. This will get time-consuming and expensive, especially if you’ve hired an organization to help.

With the availability of free or cheap tools, which make the process fast and easy, you may think you can avoid expenses and save a little time. However, even if you build your survey without spending a dime, you’re nowhere close to done. You’ll still need to promote it to ensure you’re getting the reach you need.

This promotion can get expensive. Whether it’s gift card raffles, advertisements, email campaigns, or simply the cost of your time, you’ll need to devote adequate resources to getting your survey out to families. After all, surveys are only as good as their responses, so you’ll often need to incentivize participation to get good data. All those costs add up quickly.

Surveys are based on assumptions.

Even if you’re willing to commit the time and energy needed to effectively administer multiple surveys, your commitment isn’t over. Before you run your baseline survey, you first need to ensure that your questions are designed to give you the data you need. Why bother to run surveys if you can’t count on your questions to give you accurate answers?

Before you get started, you have to recognize that your questions are based on assumptions and biases. Take, for example, this true/false statement from a school climate survey run by CU Boulder’s Center for the Study and Prevention of Violence (CSPV): “My child’s teachers have made efforts to establish a relationship with me regarding my child’s academic progress.”

CSPV thoroughly tested this question for clarity and based it on the best available research on school climate. But even this question reinforces certain assumptions. Is academic progress the only reason a teacher should establish a relationship with a parent? What about students who are cared for by someone other than a parent? Even using the phrase “academic progress” assumes that there should be some defined progression of learning.

Another question in the CSPV survey asks for a true or false answer to the following statement: “Adults in the school make sure that all students follow the rules against insults, teasing, harassment, or other verbal abuse.” If you’re trying to innovate how your school handles bullying, this question could certainly help you chart how your faculty and staff are improving. But because the question places responsibility only on the adults in the school, it’s unlikely to help you consider how to get students involved in bullying prevention.

You won’t ever be able to remove all biases and assumptions from your surveys, and that’s okay; many of them are harmless. But any assumption is a potential limit on your innovation. To get the full story of what your community is feeling, you need to understand the inherent biases in your questions and use other methods of evaluation to account for them.

Surveys can’t predict the future.

It’s extremely common for schools to run a survey to find out if families will support a new initiative or program. However, this misguided strategy is actually one of the biggest obstacles to true innovation.

Because a survey is a snapshot of one moment in time, it can only describe what people were thinking or feeling at that particular moment. It cannot tell you how they will behave or react in the future—because they themselves don’t know. Here’s the problem: Humans are really bad at knowing what they want before they experience it. We are all bound by the contexts in which we exist. But if you’re running a survey, you’re likely trying to innovate in some way—which means bringing your community something they’ve never experienced before.

The private sector learned this lesson in the grocery store aisle. While there are now seemingly endless varieties of spaghetti sauce on the shelves, that wasn’t always the case. In the early 1980s, Prego was struggling to compete against Ragu, so they brought in market researcher Howard Moskowitz to help. To figure out what the industry was missing, Moskowitz created 45 different sauces and conducted extensive taste testing with the public. He found that Americans were evenly divided among three preferences: plain, spicy, and extra chunky.

But until Moskowitz conducted this research for Prego, no company had offered chunky spaghetti sauce; through years of consumer research, customers had never asked for it. They didn’t know they wanted it until it was right in front of them. Prego soon launched a line of extra chunky spaghetti sauce that took over the industry, making the company $600 million over the next 10 years.

When Prego simply asked customers what sauces they preferred, respondents only thought to mention the types they’d already tasted before. They didn’t have a frame of reference for anything else. It wasn’t until Moskowitz put a new sauce in front of them that they knew what they liked. In the same way, if you ask your community what they want to see from your schools in the future, they will only be able to answer through the lens of what they’ve experienced in the past.

Imagine you’re considering a mobile app for your district. You might be tempted to ask families if they would use such an app. But how could they know? Perhaps they’ve had a bad user experience with another school’s mobile app, coloring their views on the idea. Or maybe they have no experience with school mobile apps at all—meaning they don’t know what to expect. In either case, your community members can’t know for sure how they’d react to an innovation until you actually roll it out.

Remember, innovation is about solving problems with previously unknown solutions. While it’s tempting to simply “give the people what they want,” if you let a survey prescribe a solution, your innovations will be restricted by the questions you’ve asked and your community’s past experiences.

Surveys tell you what—not why or how.

For innovation to be lasting and impactful, you have to get your community involved in discovering the solutions. But even Cokeley understands the limit to what surveys can contribute to the innovation process. “It’s hard to do this work without surveys, but it’s not all you should do,” she says. “It’s only one component of your research.” While they can’t effectively prescribe solutions to problems, they can help you discover issues that need to be addressed and places where you should innovate.

Survey data is actually a lot like student assessment data: It’s just one of the many indicators of how your district is doing. Assessment data might tell you that a student struggled on a reading exam, but it will never tell you why they had trouble or how to help. However, that data does give you a place to start. It shows you where to dig in, to go beyond the hard numbers and discover the root of the problem—and its solution.

The same is true for survey data. While it won’t tell you what to do, it will tell you where you need to dive deeper and investigate. Once you’ve determined which issues need more attention, approaches like empathy interviews, focus groups, and other in-depth interactions can help you get to the why and how.

Surveys are crucial to innovation.

We’ve gone through some of the flaws in the ways schools often approach surveys, but make no mistake—they are still critical to the innovation process. While they cannot tell you why your problems exist or prescribe solutions, they can still show you how your innovations are impacting your district. “That’s why surveys are so essential in communications planning,” Cokeley says. “They help you identify your starting point and determine whether you’ve reached your goal. I can’t imagine doing this work without them.”

Let’s go back to the app example from earlier. Surveys won’t tell you how to successfully implement an app for your families, but they can help you evaluate its success once you’ve rolled it out. If you aren’t satisfied with the number of downloads you’re getting, you can ask families if they’ve even heard about the app. If they haven’t, you’ll know that your problem is a lack of adequate marketing. Of course, that poll can’t tell you how to promote your app better, but it can tell you if you need to do so.

As we move beyond the pandemic, schools across the country are trying to figure out how to better meet the needs of their families and students. It has never been more important to include your families in this innovation process. But just because you want or need to get your community involved doesn’t mean a survey is the answer.

Surveys are just one tool, and like any tool, they have specific uses. If you want data that will help your innovation process, you need to first genuinely ask yourself: Can a survey tell me what I need to know in this situation? If the answer is No, then that poll will only waste your school’s time and money while producing useless information.

The hard truth is that engaging your community in innovation is messy. Other investigative strategies, like empathy interviews and focus groups, take a lot more time, effort, and planning than a simple survey. But in the end, it’s the combination of both approaches—well-timed surveys and more personal ways of collecting data—that will help you discover a more nuanced picture of your community’s needs and produce lasting innovation.

SchoolCEO is free for K-12 school leaders. Subscribe below to stay connected with us!