Transcript:
Today we are going to discuss a sensitive topic in American politics: gun violence. This topic has many dimensions to it, and I want to make clear that the objective here is not to determine which side is correct. Instead, we will focus our energies on one specific question: are gun laws effective? As we'll soon see, the answer is not a simple yes or no. The first task is determining what we mean by "effective."
The most common metric used to justify the effectiveness of gun laws is the firearm mortality rate. Using the Gifford Law Center's rating of state gun laws in conjunction with data from the CDC, we see that as scores increase, firearm mortality rates decrease. The problem here is that it tells us nothing about other modes of mortality. If firearm homicides go down, but other types of homicide go up, has anything really been accomplished? Not really. Based on this, we can conclude that firearm mortality is an insufficient measure of gun law effectiveness.
For this reason, we are going to use the overall homicide rate as our primary metric, with suicide rates and mass shootings as secondary metrics. One thing I do want to point out here we define homicide rates as deaths by assault, excluding incidents of legal intervention. In other words, homicide rates do not include incidents of self-defense. And rather than rely on an opaque grading scale, we want to track changes in specific features of gun laws. To do this, we leverage a database from Everytown Research that tracks changes in state gun laws since 1991.
We will start by conducting a brief overview of the information and structure of the database itself. We'll then move onto a discussion about the impact of weak laws in neighboring states. With these items in place, we can zero in on the features of gun laws that result in the most significant reduction in homicide rates. Following this is an assessment of the regulatory impacts on mass shootings. Finally, we will close out with some recommendations and next steps.
The Everytown Research database frames features of gun laws into a collection of 85 of yes/no questions. Each question falls into one of eight categories, including mental illness, domestic violence, criminal history, etcetera. To give you a sense of the general types of questions included, let's look at a few.
For criminality: are felons prohibited from having firearms? Are people with domestic violence restraining orders against their significant others allowed to have guns? What about individuals committed to a psychiatric hospital for emergency care? Or those who have renounced their US citizenship? As you can see, these questions leave little room interpretation, but we need to be aware of the fact that not all responses are universally applicable.
For some states, the answer may be yes for possession, but not for purchases, or vice versa. Type of gun matters, as well. If state law prohibits domestic abusers from purchasing handguns, that doesn't necessarily prevent them from buying a rifle. Nor does it mean they necessarily have to relinquish any guns they already have.
So, for each question, there are, in reality, four dimensions. And we treat each dimension with equal weight. If a state responds to a question with, "Yes, but only for handgun purchases," we give a score of one out of four, or 0.25. An answer of "yes to all" would yield a score of one. The closer the score is to one, the stricter the laws in that state.
We also want to consider the strictness of neighbor state laws. To illustrate why, we examine the relationship of Giffords Law rankings with homicide and suicide rates. On the right, we see a clear correlation between suicide rates and gun laws: as laws get more stringent, suicide rates drop. But when looking at homicide rates on the left, the relationship isn't as clear. Though there is a clear downward trend, there are an awful lot of exceptions. We quantify this relationship through the use of the r-squared value: 0.067, which indicates that the grade explains only 6.7% of the variance in homicide rates. Why might that be?
Some posit that neighbor states with weak laws undermine the efforts of states with strong laws. Let's test this theory by looking at the average grade of neighboring states. For now, we will look only at homicide rates.
The graph on the right now shows neighbor state GPA on the x-axis and homicide rates in the target state on the y-axis. Here again, we see that the graph on the right exhibits a more explicit relationship, which can be verified by looking at the r-squared score: 0.164. Neighbor state laws have more explanatory power than in-state laws. Let's put this in more realistic terms by considering a state with strong gun laws: California. The implication here is that the laws of Nevada, Oregon, and Arizona are better predictors of the homicide rate in California than California law itself!
Now, we don't want to rely on Gifford's grading scale, but we don't want to ignore the potential impact of this phenomenon either. So, for each question in the Everytown Research database, we'll factor in the average response of neighboring states in an attempt to predict homicide rates. We'll also see if we can identify a small subset of questions that lead to sufficient accuracy results. Starting with backward elimination in a multilinear regression, we isolate the top 20 features of gun laws.
This is a correlation heatmap that shows how each variable moves with one another. The bottom row represents feature correlations with homicide rates. Blue values indicate that affirmative responses to questions are associated with lower homicide rates. Red shows the opposite effect: an affirmative answer would increase homicide rates. Because we are searching only for laws that reduce homicide rates, we don't have any red boxes in the bottom row.
We won't look at the questions themselves just yet, but I want to draw your attention to the suffixes of each variable. In 13 of the 20 features, neighbor state responses are more important than target state responses, which supports our earlier observations using Giffords Scorecard. Unfortunately, the multilinear regression model isn't a particularly good predictor of homicide rates. So, let's plug this data into a few machine learning models and see how well they can predict homicide rates.
Five traditional machine learning models, as well as one deep learning model, were engineered using a grid search. Each model was trained using 5-fold cross-validation with 80% of the data, attempting to maximize the explained variance score. This score tells us how much of the variance in homicide rates can be explained by responses to these 20 questions about regional gun laws. Let's find out how well each model performs when testing it on the remaining 20% of data it has never seen before.
Here, we can see that XGBoost outperformed every model, including Keras Regression, which was the deep learning model. By examining the answers to 20 questions related to regional gun laws, it was able to achieve an explained variance score of 91.23% when looking at new data. Let's plot the predicted values against the actual values to help visualize the results.
In this chart, red dots represent a random selection of actual homicide rates since 1991. The blue line represents homicide rate predictions of our top model. As you can see, the actual values are narrowly scattered and evenly distributed around the prediction line. It's hard to believe the model can make such accurate predictions by simply asking 20 questions about laws in the local region. But what we don't yet know is how much of an impact differences in these laws can make.
We can estimate the potential reduction in homicide rates by comparing model predictions when all responses are "yes" to when all are "no." When doing this, we see that the potential reduction in homicide rates is 54.2%. To put this number in context, firearm homicides have historically accounted for 67% of all homicides. The results imply an approximate 80% reduction in firearm homicide rates.
So, which features of gun laws are the biggest drivers of this reduction? Ultimately, we can boil it down to 7 primary questions, some of which pertain to the target state, and others that are more relevant for the neighbors.
Let's start right at the top with the first question. "Does law enforcement have the authority to prohibit people from carrying a concealed gun on the basis that they pose a danger or threaten public safety?" The response to this question by both the target state and its neighbors is the single, most important factor in reducing firearm homicides. One thing you may be asking yourself is, "Who determines if someone poses a danger?" The answer, unfortunately, is a bit vague: local authorities. So, what this alludes to is police discretion. We see this topic come up more explicitly in question 5. "Can police exercise discretion in prohibiting individuals from getting concealed carry permits?" Underlying these two questions is one underlying theme. Do citizens need to justify their desire to obtain a concealed carry permit, or do local authorities need to justify their refusal to issue one? Regardless of your stance on the issue, the data tells us that when police have more discretion, homicide rates are lower.
We see two other questions related to concealed carry: one focuses on firearm training, and the other on criminal history. These fit squarely into the debate of whether concealed carry is a privilege to be earned or a right to be taken away. It's also worth pointing out that three of these four features carry significance in the context of neighbor state laws. Only two are significant for in-state responses.
We also see that two of the top three features pertain to those with domestic violence restraining orders. And we have one question that asks whether or not state law prohibits fugitives from having firearms. It was a bit surprising to find that any state would not have these types of restrictions in place.
But as we can see here, fewer than half of the states have any prohibitions on fugitive firearm possession. And in 19 states, individuals prohibited from having firearms due to domestic violence restraining orders are not required to turn-in their weapons. These counts are as of this year: 2019.
So, there's a lot to chew on here, and I'll let you sit with it for the time being. In the meantime, let's shift gears a bit and talk about mass shootings.
Despite best efforts, we were unable to establish a clear link between state gun laws and the occurrence of mass shootings. The initial results were not particularly surprising since these types of events are quite rare, statistically speaking. So, rather than look at state laws, we will look at federal regulation; specifically, the Public Safety and Recreational Firearms Use Protection Act. Otherwise known as the Assault Weapons Ban, this law was in effect from September 1994 through September 2004. What kind of impact, if any, did the ban have on mass shooting?
Here, we are going to consider data from 1982-2016. And we are going to look at three target variables: number of incidents, total fatalities, and fatalities per incident on an annualized basis. For each target variable, we want to know if there is a significant difference in averages in years with the ban in effect. Let's start with the annual number of incidents.
Here, the red curve represents years with the ban in place, and the blue curve represents years without a ban. The dashed, vertical lines represent the respective means of each curve. Since the blue line is further to the right of the red line, this is an indication that the average number of incidents per year was higher without the ban. Specifically, there were 3.1 incidents per year without the ban, but only 1.67 in years with the ban.
When looking at fatalities per incident, we see similar results. With the ban, there were 5.4 fatalities per event, but that number jumps to above eight in years without the ban. Putting these together, we can see that there were 20 fewer fatalities per year when the ban was in place. The results seem to suggest that the assault weapons ban was quite effective at reducing the frequency of and the number of fatalities from mass shootings. But there's one problem.
Have a look at the changes in these results over time. When looking at the frequency and total deaths, we see the numbers are significantly higher in the years after the ban than before the ban. The spike in 1999 due to Columbine and other copycats even seems to be part of a longer trend. So, in terms of total fatalities and frequency of incidents, we cannot conclude that the ban had any significant impact.
Fatalities per incident, however, is another story. We see a clear drop around 1994 and a subsequent increase after 2005. The difference in means is significant at the 95% level. So, though the ban may not reduce the frequency of these events, it can improve the survivability of them.
And even if we consider the possibility that would-be shooters will turn to other methods of mass murder, we can see that the difference is even more pronounced. There is an average of 8.4 fatalities per mass shooting and 2.4 with alternative methods.
So, with all of this in mind, how should state legislatures proceed if they want to curb gun violence? First, they need to start with the basics. Prohibiting fugitives and domestic violence offenders from having firearms is not likely to cause much controversy. Though these are the most significant features, we can treat them as a stand-in for a host of legal gaps. These include, but are not limited to: individuals found not-guilty by way of insanity or incompetent to stand trial, those have renounced their US citizenship, and those who were dishonorably discharged. A surprising number of states lack restrictions in these areas.
The data suggests the next thing would be to give law enforcement more discretion when it comes to issuing concealed carry permits. However, a substantial percentage of the population may be unwilling to yield this much power to law enforcement. As an alternative, immediate family members might be empowered to petition a court to rescind permits or have firearms removed altogether when they identify red flags. Otherwise known as "extreme risk protection orders" or "red flag laws," such an approach could potentially yield similar results without giving too much power to enforcement agencies. In 2015, only two states had such laws in place, but that number has since grown to 17. It is still too early to gauge the effectiveness of these laws, so an incremental approach is warranted. As a next step here, it would be a good idea to research how various law enforcement agencies identify red flags. This could help establish the proper framework for courts to make decisions.
And finally, states need to collaborate with their neighbors. States like California, for example, are likely to realize more significant gains if they work with Nevada and Arizona. It doesn't mean they need to take the same, strict stance as California. As mentioned before: they simply need to get the basics right. The unfortunate reality is that, while many of these features are already in place at the federal level, enforcement capabilities are limited. And the Supreme Court has ruled that the federal government cannot compel states to enforce federal law. Without analogous state laws, federal statutes are toothless.
Ultimately, the goal is to reduce the rate of homicides and suicides, not to disarm law-abiding citizens. Making a concerted effort across the country to enact real common-sense reform and start enforcing federal laws already on the books is the best way to start making progress.
References
A technical blog with more detail on the code and methodology behind this analysis can be found on Towards Data Science.
The repository for this project with all relevant Jupyter Notebooks can be found on Github.