The Department of Homeland Security has partnered with with researchers Argonne National Laboratory’s Decision and Information Sciences (a lab I noted in my post on research centers of excellence) and University of Illinois at Chicago to study individuals’ confidence in the ability of federal, state, and local governments, and their public safety agencies to prevent acts of terrorism after an attack has occurred. The report is titled, “Understanding Public Confidence in Government to Prevent Terrorist Attacks“, and as the authors’ state, “The question we ultimately want to answer is$mdash;what value do individuals place on their confidence in government and law enforcement agencies to prevent terrorism? Knowing how much citizens value their confidence in government and law enforcement agencies will help allocate resources to prevent terrorist attacks and to protect critical infrastructure.”
To study this, the authors administered a questionnaire to small groups of people shown videos of various attacks types of attacks (see chart above). These groups were then reexamined over lengthening intervals of time to also measure how temporal distance from an attack affected confidence. The authors report the following findings:
(a) although the aggregate confidence level is low, there are optimists and pessimists; (b) the subjects are discriminating in interpreting the nature of a terrorist attack, the time horizon, and its impact; (c) confidence recovery after a terrorist event has an incubation period; and (d) the patterns of recovery of confidence of the optimists and the pessimists are different.
These findings are interesting, although they seem to follow closely what we might expect at the outset (particularly a and c). As the final goal of this research seems to be policy prescriptive, there are several methodological concerns that warrant discussion. First, the sizes of the test groups are far too small (seven individuals each) for any statistical generalizations to be made from the data. The number of individuals in each group should be at least tripled in order to increase confidence in these findings. Second, the study contains no control group, which raises suspicion about exogenous factors that may be contributing to survey responses. For example, a world event may have occurred during the test period that could be skewing their results on cyber attacks (Georgia/Russian conflict perhaps). Finally, the sample of test individuals does not appear to be random, with groups containing several people self-reporting employment in security related fields. Clearly, this raises concerns about endogeneity (i.e., the bane of social science), as opinions about confidence in government may be self-enforcing for individuals in security positions. Having worked on various survey studies within the government, I am sympathetic to the difficulties associated with getting participant variation, however, is this case the problem does considerable damage the analytical result.
I look forward to reading follow up analysis by they authors, and hope they will be able to expand the scope of their next study to account for some of these methodological shortcomings.
Special thanks to co-author Thomas E. Baldwin for providing me a copy of the complete report.
Photo: Journal of Homeland Security and Emergency Management
Automatically Generated Related posts:
Interesting indeed. What I find especially interesting is this comment in the study “The question we ultimately want to answer is – what value do individuals place on their confidence in government and law enforcement agencies to prevent terrorism? Knowing how much citizens value their confidence in government and law enforcement agencies will help allocate resources to prevent terrorist attacks and to protect critical infrastructure.”
Now admittedly I understand that the scope of the study is constrained to “what value do individuals place on their confidence in government and law enforcement agencies to prevent terrorism?” However, the second part of the quote is what intrigues me. How would the perceptions of the public help to allocate resources to prevent terrorist attacks and protect critical infrastructure? Does this mean to say that the results of the study should be used to satisfy the public so that they believe that the areas perceived as weak are well protected? While I see how allocating resources to the perceived “soft spots” would possibly increase the public’s trust in the government to protect them, would it really increase the ability of the government to protect the public? I say no, and could go on forever on the lesser known “soft spots” (ie the power grid) but I’m done rambling.
Obviously I went on a tangent here…but one I felt had to be pointed out. Please tell me I missed something…be kind this is my first post.
[Reply]
Dave, I think you raise the most concerning aspect about this study, which is the policy prescriptive intent that seems to be driving it. The authors are a bit unclear regarding your question about exactly how the result will be used; however, at least part of the research is clearly going to be evaluated by DHS policy makers.
See paragraph 3, page 3: In recognition of its importance, the Department of Homeland Security includes public confidence as one of the principal metrics used to assess the consequences of terrorist attacks…Therefore, in order to establish a meaningful metric, it is necessary to understand how public confidence varies among individuals, is affected by different types of terrorist events, and changes as a function of time.
This is precisely why more care should have been taken in the research’s design and methodology.
[Reply]