Response to "A National Evaluation of Social Norms Marketing Interventions"
Alan Berkowitz, Ph.D.
and Reality: A National Evaluation of Social Norms Marketing Interventions
to Reduce College Student's Heavy Alcohol Use by Henry Wechsler, Toben
Nelson, Jae Eun Lee, Mark Seibring, Catherine Lewis and Richard Keeling.
(Journal of Studies on Alcohol, 2003, 64:484-494) evaluates the effectiveness
of social norms marketing campaigns by comparing schools in the College
Alcohol Survey (CAS) that met the authors' criteria for a social norms
marketing campaign with schools that were determined to not have a
campaign. When data was analyzed from the 1997, 1999 and 2001 CAS
surveys, these two groups of schools did not differ from each other
on seven measures of alcohol use. As a result, the authors concluded
that the study "does not provide evidence to support the effectiveness
of social norms marketing programs, as currently utilizedů" and they
urged "college administrators and health educators to base their prevention
programs on scientific evidence instead of the perception of promise."
This is an important study that merits attention and careful scrutiny.
Its announcement was planned with the assistance of a public relations
firm and was preceded by great secrecy in the form of an embargoed
press release that when released was reported in almost of the major
national media outlets.
results of the study hinge on the authors' definition of a social
norms marketing program. These were identified in two ways: administrators
were asked if their school had "ever conducted a social-norms campaign
to decrease alcohol use and related-problems," and an index of student
exposure to these campaigns was created based on students' reporting
that their school provided information on student drinking rates,
and/or that they saw posters or signs, read announcements or articles
in the student newspaper, or received mailings and handouts.
number of experts, including Michael Haines, social norms co-founder
H. Wesley Perkins and William DeJong have questioned the methodology
of the study. They point out that the definition used for a social
norms campaign is not strict enough to ensure that a valid campaign
was conducted. In fact, with 50% of CAS study administrators reporting
that they had such a campaign, it is entirely likely that the measure
is too broad and overestimates the number of accurate, well-designed
campaigns. In addition, the measures that were used to identify student
exposure to social norms marketing do not guarantee in any way that
what the students saw meet the minimum requirements of a bona-fide
social norms effort.
authors make a number of criticisms of the social norms literature
that would also apply to their own study. For example, they mention
the low response rate in some social norms studies, yet the CAS response
rates for the 120 schools ranged from 22-86%. With only 215 surveys
distributed on each campus this computes to an "N" of 47-185 per campus,
independent of campus size. This is hardly an adequate sample size,
especially in schools with large enrollments that were found to be
most typical of schools reporting social norms campaigns. The authors
also present an incomplete review of the literature, draw incorrect
conclusions from the studies cited, and make unsupported criticisms
of social norms theory.
provide an example of how a school with a weak campaign could have
been included in the study, consider a hypothetical case of a campus
with a first-year orientation presentation that includes a discussion
of campus drinking norms. The campus also has posters or signs, announcements
in the student newspaper, and mailings or handouts on alcohol prevention
with unspecified content. Although this campus does not have anything
else relating to social norms other than the orientation workshop,
it would qualify as having a social norms campaign in Wechsler's study
even though the program is too limited in scope to have an impact
on overall drinking rates. In another example provided by William
DeJong in this issue, a campaign that violated the basic premises
of social norms media construction would also have been included in
that instead of asking about social norms campaigns the study had
asked administrators if they "have policies and disciplinary approaches
to decreasing alcohol use and related problems on campus" and then
asked students if their school had policies, if they were enforced,
and if they had received brochures or information about them. These
schools would constitute the "policy group" in this imaginary study
and would have been compared with the schools that did not report
having policies. Determining if schools have policies and if their
students know about them is in no way a valid measure of the effectiveness
of policy enforcement and other strategies recommended by Wechsler
and his colleagues. Yet this was the methodology chosen to evaluate
the social norms approach.
more valid measure of the effectiveness of social norms marketing
campaigns could have been conducted with the CAS data set. The study's
authors could have asked a panel of social norms experts to rate the
quality of the campaigns on those campuses that reported having social
norms programs. Then, those campuses rated as having excellent programs
could have been compared with other campuses in the study. This would
have created a more valid evaluation of the effectiveness of social
norms marketing campaigns, properly executed.
summary, it is necessary for social norms practitioners to be familiar
with this study and understand its limitations. This is especially
important given the large amount of uncritical attention that it has
received. Our job is not only to design, implement and evaluate social
norms interventions that are theoretically sound and faithful to the
principles of social norms, but also to educate key stakeholders and
audiences on our campuses to understand criticisms and answer questions.
In this regard, Henry Wechsler, Richard Keeling, and their colleagues
have once again provided us with an excellent opportunity to do our
D. Berkowitz, Ph.D.
The Report on Social Norms