Yesterday the SocNet listserv sprung to life when one user asked for citations to critiques of network analysis. This discussion was later picked up by the Net-Heads at orgtheory.net, where a brief but interesting commentary ensued. I think Fabio’s point at orgtheory is an excellent one with regard to the fallacy of the functionalism argument, he writes:
I don’t quite buy this criticism. Isn’t it reasonable to ask two simultaneous questions? We can ask how individuals affect networks (e.g., tie formation) and we can ask how networks reflect patterns of behavior, which can be seen as resources (e.g., vacancy chains). Sociology isn’t the idea that all that matters is structure, it’s about discovering the interplay between action and structure.
I am not a Sociologist (enough to make some people disregard my opinion straight away), however, I have put a lot of effort into thinking about networks and how to analyze them. In that vein, I have noted several problems with the field in my time that, given the sudden interest in highlighting the disutility of this science, are worth noting. Also, since I am partial to ordering things in list, here are my top five problems (in no order) with the science and community of network analysis.
- The Myth of Predictability – At some point, long before I became enamored with the power of networks, someone declared that network analysis could predict the dynamics of groups. I do not know who it was, but I have seen the evidence of this declaration many times. The problem with this idea is that we simply do not have the requisite mathematics to support such a claim. I was having this discussion with fellow Net-Head Ivan Labra the other day, my point was this: networks are matrix objects, and we have been doing algebra on matricies for hundreds of years; however, we do not know how to do calculus on a matrix. Algebra can inform us about the current state of an object (represented as a function) but calculus is what gets you to a mathematical representation of dynamics, i.e. trajectory. Someone (orders of magnitude smarter than me) needs to dedicate their career to developing these mathematics, because without this foundation the idea that network analysis can predict will remain a myth.
- Constant “Reinvention of the Wheel” – The strength of network analysis is that it is fundamentally interdisciplinary, however, this can also be a great weakness. As concepts about network analysis moved from Sociology to other social science disciplines, and then again from the social sciences to the hard sciences there has been a great deal of duplication and down right theft of ideas. This is a topic that I have covered in the past, but it is worth noting here because it serves only to stymie progress in this field. If you are working with networks, and you see something you think is novel, or come up with a new metric, please do at least a minimal literature review to be sure someone else has not already had the same observation. It will prevent confusion, promote learning, and assist in genuine invention.
- Link Chart ≠ Network Analysis – This may be more of a personal pet peeve borne out of my own development as a network analyst inside the military community, but if you are going to stand up in front of a audience with a link chart (see above) and call it network analysis please expect to see my hand launch into the sky. Simply because your data links people and you can visualize that, it does not mean you have performed network analysis. This is akin to displaying a line plot of some stock’s price over a quarter and claiming you have performed statistical analysis—no—you have reported data. As with all other statistical processes, network analysis is meant to draw meaning and inference from the structure, which requires an understanding of these methodologies, their strengths and limitations.
- Lack of a Unified Definition – The previous problem is really a derivative of this problem, that is, as with many other sub-disciplines the boundaries of what network analysis actually is has not been well-defined. For example, I have written in the past that simple dyad comparison, which makes up a large partition of what is claimed to be network analysis in contemporary social science publications, are repackaged binary dependent variable models. To me (other likely will disagree) network analysis is about structure, therefore, in order to perform it the analysis must at least appreciate all of it and how it affects dyad relations. It is hard, however, to take issue with researchers making the claim that this is network analysis. Unlike the link charters, these models are actual analysis, but they lack in their treatment of the network. The community needs to consider the value in slowing the push of networks to every research area, and rather consider the boundaries.
- It’s Not for Everyone/Everything – When a new science struggles to gain traction and respect and then suddenly becomes accepted on a larger scale (for networks the catalyst was 9/11), pioneers and practitioners can sometimes become imperialistic about its value. This idea has been discussed here with regard to quantitative methods in political science, and in some ways network analysis also suffers from this attitude. In many discussions I have had with network analysts a problem will arise that could be examined with network analysis, but upon further reflection would be better addressed with a different set of tools. This does not, however, prevent the inevitable squabble about why network analysis must be used and the enumeration of its many accomplishments and references in major news publications. Furthermore, not every audience is keen to hearing about how structure tells the whole story. There is a time and place for network analysis, and it is OK if that is not every time and place.
I look forward to reading where you think I got it wrong, as I am sure there are many other problems with the science that I have missed.
Photo: Able Danger Blog
Automatically Generated Related posts:
Right on. I once tried explaining #3 to a colleague, and basically got yelled out of the room. Bloody hell. Mind you, same guy also yelled me out of the room when I tried to point out that there’s distinction between causation and correlation. Sigh… good times.
[Reply]
You were obviously dealing with one of my old clients, we need to share war stories!
[Reply]
[...] . Counter to CLRN’s reviews of supplemental electronic learning resources, textbook reviews On the Disutility of Network Analysis and the Top 5 Problems – drewconway.com 06/19/2009 Yesterday the SocNet listserv sprung to life when one user asked [...]
From number one: “but calculus is what gets you to a mathematical representation of dynamics, i.e. trajectory.”
I don’t know calculus, but I understand it was created to handle the problem with infinity, so it must be, as you used the word “trajectory”, like targeting towards a point whose distance is impossible to find, but location is doable by moving off the vector path and finding the arc of trajectory towards the point, the endpoint somewhere on the line 90 degrees tangent to the trajectory? Sounds very complex if that is sort of how it works.
From number four: “To me (other likely will disagree) network analysis is about structure”. And, “The community needs to consider the value in slowing the push of networks to every research area, and rather consider the boundaries.”
It sounds to me like you are saying that the math needed for network analysis is for the trajectory of the structure (where it is going), which seems reasonable (reasonable in that ‘structure’ is the result of perpendicular forces at work in the network itself, which may be predictable).
But then, there are really two structures, the outside structure, which gives it boundaries (not the same boundaries that I referenced) or a sort of surface tension or form, and an inside structure that is built from the function. I am guessing, it is the outside structure that people want to predict—to learn where the network is going, or its dynamics.
The inside structure gives the outside structure a level of predictability, but, if I understand the question of math (or lack-of) correctly, it appears there is no math to target where the outside structure will be by looking at the function of the inside structure.
When the Department of Homeland Security asks the engineer what are the unintended consequences of your application to our network, the engineer has to answer, “I don’t know until I install it” is this one of the ways that you mean by unpredictability? I thought the reasons the engineer could not predict the outcome was because of complexity, transparency (I assume no one wants to be totally transparent), natural resiliency (some things keep working no matter how incompetent a person is), or some other dynamics.
To think that predictability is not possible because no one has the math is something I never thought of, or at least I thought math was a given, or as you say the function of the network had something to do with the predictability. Although I enjoy your web site, I can see network analysis is something I want to steer clear away from. Lots of luck in the future.
[Reply]
Point 3 is right on. My personal pet peeve with networks research is similar: people who expect papers on Facebook/Twitter to qualify for SNA conferences. Maybe a corollary to that is people who write actual SNA papers, but then go on to confuse social networks with social networking sites in their commentary (I’ve seen this in publications generating out of engineering disciplines).
But as far as point 1 — which seems to be the most significant to me — can you please elaborate on why you think calculus is necessary for making predictions? Can’t statistical models used to make predictions, too? What about agent-based models based on metrics determined by SNA? (Since network phenomena are essentially chaotic, I am not sure I would expect any prediction about them to be anything but probabilistic, anyway.)
I also agree with pt 5, though I want to say that we haven’t yet discovered everything networks can do. It’s still a fairly young approach, and I think it will only get more and more powerful for the time being.
p.s. if I was going to add my pt 6, it would definitely be the difficulty of teasing out causation from networks models.
[Reply]
Larry: my point about calculus is simply to highlight my thought that prediction in networks can only come when we have the requisite mathematics. To your point about inner and outer structure, I see them as one and the same. I think those that want prediction want it for the whole network structure, which as your comment underscores, is something to approaches an understanding of infinity. And again, as you note, why I think something like calculus will get us closer.
Typo: a fair point, and I should be more specific. I think statistical models can help researchers understand the effect of nodal level variables on the current state of the network, but not where it is going. Agent-based models do a better job, but they are limited by the assumption of the designer.
[Reply]
Drew: What would the alternative look like, had we had the requisite mathematics?
[Reply]
We need a way of representing the data of a network as a continuous entity. With a differentiable function, we can manipulate the parameters of it to understand the first, second etc. order conditions to find local and global maximums and minimums. This tells us how the function is changing and moving over time, allowing for a two-dimensional understanding of dynamics.
We need to consider how the binary (or weighted) data of a network can be represented as a continuous object–that is the alternative that I imagine.
[Reply]
Drew: Interesting! Would the continuity be between different configurations of the network, e.g. the addition/removal of nodes and edges — and the maximums and minimums would correspond to specific configurations of the network? Is there an alternative where this calculus is performed not on the sociomatrix itself but on scalar metrics of the sociomatrix (e.g. census of triads, centralization, etc.), and then these are used to infer states of the network?
[Reply]
Typo: excellent questions, and probably better addressed by someone much smarter than me, but I’ll try. Both of your points are methods and processes I have thought about in the context of this problem. I do think the local max/min would correspond to certain configurations of the network, but their meaning would be very difficult to parse out. Is a max some kind of saturation level? Is a min some primitive skeletal configuration? I do not know, and I doubt the answer to either question is yes; hard thinking problems.
To your second point, I think using scalar metrics as a means to get to continuous models is a good first step, however, these metrics are still only pictures of a network. Any inference drawn from them, therefore, will be biased by this static foundation.
[Reply]
“To your point about inner and outer structure, I see them as one and the same.”
Do you mean to say that, if you are looking at the network from an outside advantage point, you see the inner structure as being the same as the outer structure?
Or do you mean, because of complexity and possibly corruption (lack of transparency) you are simply unable to distinguish the difference between inner and outer structure, as if there was a difference, and I am saying that I think there is a difference?
It could be your use of the term “edges” in this or previous postings, that is the cause of this miss-understanding, if there is any. To me, an edge is looking from inside the network outward, and I am trying to look at the network from the outside inward.
Of course we are looking at the same edges, if I am actually able to see them, and I am not, it is simply from what advantage point (orientation) a person is coming from. My advantage point is not the same, apparently, as yours, because I am coming from a point of ignorance, and your point is well defined.
Of course the advantage of looking from the outside inward is the ability to identify the network quickly. This (edges) is a concept that I am giving much thought to and find it interesting, if I understand it correctly.
[Reply]
As analyst we are always looking at a network from the outside, though perspective has nothing to do with the problems with network analysis as a science of community. Fortunately, there are few (if any) out there that dispute perspective.
The term edge is how we describe connectivity; it comes from graph theory. For a detailed discussion of the mathematical representation of graphs see here.
[Reply]
1) Yes, as evidenced by attempts, e.g. Liben-Nowell(2003), but not for a lack of calculus. Matrix calculus does exist (try wikipedia), but it’s hard. The bigger problem is just that in the end, just the network doesn’t give you much, we’re not great at networks+attributes, we’re not great at over time, and networks are sparse -> false positives.
2) Indeed, I see this every other week.
3) Network Analyst anybody?
4) I’m not so sure about this. A better understanding of the external correlates of dyadic connections is pretty fundamental to a better understanding of networks, especially prediction.
5) Absolutely. Just because OLS regression doesn’t work for networks doesn’t mean we should abandon statistical inference. I do think this is a bit related to what you define as network science. Sometimes understanding the structure isn’t necessary, but it’s usually interesting.
[Reply]