Diversity Climate Surveys: Worth the Effort

By Pat Disterhoft, associate professor of education; Debbie Giunta, director, Center for Cultural Fluency; and Arianne Walker, director,
Office of Institutional Research and Assessment, all of Mount St. Mary’s College, Los Angeles, California.

	Mount St. Mary’s College

Aimed at fostering an environment that values and embraces all forms of diversity, Mount St. Mary’s College, a Catholic liberal arts college in Los Angeles, wanted to take a snapshot of the campus climate for diversity to inform ongoing and future efforts. We decided to conduct a comprehensive assessment of the campus diversity climate for students, faculty, staff, and administrators. Prior to embarking on this ambitious project, we had only limited data from previous student surveys. The survey project discussed in this article was intended as a baseline—to help us see what we are already doing well and where we need to make changes. Though painful at times, our evaluation efforts were well worth the effort.

Survey Development and Rollout

Developing the survey took a long time. In fact, the coordinator of the survey effort once remarked in frustration, “I might not have embarked on this project if I had known how long it would take.” A committee of four that included the coordinator of the CDI grant, the assistant provost, an external evaluation consultant, and a representative from institutional research and assessment began by exploring climate surveys from other institutions. Using these as models, each member developed questions to be asked. The most difficult part was paring down the number of questions. We spent hours, nursed headaches, and argued during this process. Eventually, we used the theoretical framework developed by Hurtado and her colleagues (1999) to consolidate the questions and make sure we addressed three of Hurtado’s dimensions: structural diversity, psychological climate, and behavioral/intergroup campus climate. We maintained regular communication with the college administration even at this early stage because we understood how integral their buy-in and support would be.

	Buy-in from campus administration is critical to being able to ask the questions that need to be asked and to mobilize the campus to think about changes based on the survey results.

Almost a year into the process we had surveys to pilot. We were aware of the importance of piloting and made some changes based on the feedback we received. In retrospect, one of our errors was not listening carefully enough to the feedback from the pilots. Practically every suggestion that we did not heed resurfaced in the final survey results. For example, in an effort to be sure our results were valid, we asked variations of questions multiple times. However, the resulting length of the survey frustrated respondents.

A newly formed diversity committee proved to be an asset in the development and rollout of the survey. Several members of the committee were involved in piloting the survey, and others translated the survey into Spanish for staff. Committee members also helped by generating interest in the survey and assisted with advertising.

Communicating Results

Because they were fairly representative of our various college constituencies, we were satisfied with the survey response rates: 22 percent from students, 30 percent from faculty, and 43 percent from staff/administration,. The external evaluator and the director of institutional research worked together to make sense of the massive amount of data. As an analytical team, their collaboration enabled them to catch small errors, talk through ambiguities, and conduct in-depth analyses. A report of the results was written for each of the primary college constituents: students, faculty, and staff. These summaries were powerful in part because they focused only on statistically significant differences between sub-groups of the campus population. Each report, no longer than six pages, began with highlights and challenges in bulleted format and included comprehensible graphs and charts of findings.

The reports and their PowerPoint summaries were presented for review and refinement to the CDI steering committee and the President’s Cabinet. Next, the results were disseminated to the entire campus community through presentations, an email summary, the campus Web site, and articles in various campus publications. We learned that by highlighting our successes, people were prepared to accept our challenges.

Outcomes

• Increased attention to diversity issues: In addition to more informal conversations about diversity, various campus groups, including Human Resources, Residence Life and Psychological Services, have already developed programs to address issues identified in the surveys as needing attention, such as cross-cultural communication and sexual orientation issues. In the year ahead, we anticipate that more events and workshops will be planned to respond to challenges identified by the survey.
• Empowered voices: Those who are committed to diversity issues, including such issues as fairness and representation, can speak with authority now that survey results confirm the desire and need for change.
• Cultural change: There is the perception that the college already has become more open and communicative. We will be able to measure whether this perception persists with the next administration of the survey.
• Appreciation for surveys: People have gained a new respect for what can be achieved with surveys and with having hard data.

Recommendations 

• It’s worth the effort: data are compelling.
• Buy-in from campus administration is critical to being able to ask the questions that need to be asked and to mobilize the campus to think about changes based on the survey results.
• A theoretical framework is useful for survey construction.
• Pay attention to all feedback from the pilot survey.
• Disseminating survey results to the campus community should be timely and done in a variety of attractive and accessible ways. Making detailed reports available is essential, but so are briefer summary e-mails and articles.

Resources

For copies of the Diversity Climate Surveys, please contact Arianne Walker, Ph. D., Director of Institutional Research and Assessment, awalker@msmc.la.edu.
For a model of how to write up survey results attractively or view our survey results, visit www.msmc.la.edu/ NewsFacts/Diverity/ diversity_climate_survey.htm.

Sources

Hurtado, Sylvia, Jeffrey Milem, Alma Clayton-Pedersen and Walter Allen. 2000. Enacting diverse learning environments: Improving the climate for racial/ethnic diversity in higher education. San Francisco: Jossey-Bass.