By Drew Conway, on January 19th, 2010
I promise, this will be my last AnalyticsX related posting for awhile. If you will allow me to indulge just one more I will return to more regularly scheduled ZIA posting soon.
As my previous posts have noted, one of our main goals in competing in the AXP is to show the value of spatial analysis techniques in predicting homicide as a social process. We have seen that historical data shows that there is significant positive spatial autocorrelation for homicide. The next step; therefore, is to gather crime data and specify a model that attempts to correct for this spatial autocorrelation.
Continue reading Heatmap of Predicted Probability of Homicide in Philadelphia
By Drew Conway, on January 18th, 2010
Last week I discussed conceptualizing the homicide in Philadelphia as a networked process, and noted that by observing the changes in probabilities for adjacent zip codes it appeared that this social phenomenon seemed to move between adjacent locations over time. There are; however, several statistical tests that can confirm spatial autocorrelation, e.g., the probability of a homicide in one zip code is not independent of where it exists relative to other zip codes. These test are somewhat superior to simple visual abstractions, as they provide rigorous evidence of the this phenomenon.
From the visualizations presented last week; however, our expectation is that there will be significant autocorrelation, particularly for those zip codes that form the core chain of observed homicides. There are two tests I will use, Moran’s I and Geary’s C, both of which test got this type of autocorrelation. To do so, I will use the R package spdep, which has a robust set of features for performing spatial analysis.
Continue reading Spatial Autocorrelation of Homicide in Philadelphia
By Drew Conway, on January 15th, 2010
The NYC Analytics X Prize team have been working for less than a week, and I am already very impressed with what we have been able to pull together. We have an impressive interdisciplinary team, and as I have said before, other teams should fear us. All of our work is completely open source; however, so feel free to browse the repository, and ask questions or make contributions.
One of the methods our team is interested in applying to the problem of analyzing and predicting homicide as a social and geographic (zip codes) problem is spatial regression. The primary motivation for this method is to correct for the correlation of errors between predictions for geographic proximal geographic units. It is only useful; however, it is can be shown that spatial relations effect independence of observations. In terms of the Analytics X Prize, the units are zip codes, and proximity is defined as sharing a border, i.e., zip code adjacency.
To begin to address this question we can use a convenient alternative conceptualization of the data; whereby, rather than using a map partitioned by zip codes the city of Philadelphia can be reconstructed as a network of zip codes connected by physical adjacency. That is, if two zip codes share a border than there are connected. By adding the homicide data to this network it is possible to observe what—if any—spillover or transference of this social phenomenon is occurring over time. After the jump are three network visualizations using this method, where each node is a Philadelphia zip code, and nodes are sized by the proportion of homicides reported in that zip code for 2007, 2008 and 2009.
Continue reading Homicide in Philadelphia as a Networked Process
Popular Posts