Tuesday, March 26, 2013

The Deluge: Twitter and Hurricane Sandy

Information is often valuable, but it's crucial in crisis situations. In a broad and basic attempt to move toward harvesting social media to determine a population's mood, my group looked at the use of language on Twitter as Hurricane Sandy bore down on the east coast of America during late 2012. What I show here is a basic visualization of tweets by time, location, and valence (here calculated by a hilariously rough positivity/negativity measure).

The tweets were scraped by team member Jacek Radzikowski (who is also available on Twitter here) focusing on the terms "Sandy", "hurricane", and "frankenstorm". The scraped tweets consist of much more than a 140 character string and a timestamp: if the user has included a description of their location in their profile or enabled twitter to use their phone's GPS, the tweet can contain some very specific locational information. For reasons that will become apparent below, we wanted to try to gain access to the user-provided location information rather than relying exclusively on GPS-derived geotagged tweets.

In this post, I distinguish between what I call "geotagged" tweets (tweets with associated coordinates) and "geocoded" tweets (tweets with locational information that was run through the Yahoo PlaceFinder to produce a set of coordinates).

We collected 1060915 tweets in all, of which 692376 have some geographic designator and 10237 have GPS-derived coordinates. For those of us who understand better in scientific notation, that's
  • 1.1x10^6 total
  • 6.9x10^5 with some info (~65%)
  • 1.0x10^4 with coordinates (~1%)
(...so our motivation for investigating the geocoder is pretty obvious, right?)

To derive some basic measure of how positively people were talking about their impending doom, we took a hand-coded set of words (AFINN, available here). For each tweet, the text is stripped of punctuation, converted to lower case, and broken into individual words. The valence is set equal to the sum of the valence values of each component word found in the AFINN wordset, normalized by the number of valence-having words in the tweet. In the following images, the tweets are colored by this normalized quantity, with more positive tweets in green and more negative tweets in red.

A composite image of Sandy-related geotagged tweets in the New York area
A composite 50% sample of Sandy-related geocoded tweets in the New York area
A composite image of Sandy-related geotagged tweets in the USA
A composite 25% sample of Sandy-related geocoded tweets in the USA

Finally, the following video displays the valence and location of tweets as they were updated over the course of the storm's approach and impact.

 This work is still very much in its infancy, but we found it very interesting and hope to do more with it in the future. Stay tuned!

Thursday, March 7, 2013

Sustainable Agriculture through Modeling

An acequia-irrigated field in Santa Fe, NM
A map of land use around Taos, NM
Irrigating agricultural land has been a major concern throughout history. When Spanish explorers began to settle in what would become New Mexico, they brought with them a system of communal irrigation management that they had learned in their turn from the Moors. This system, consisting of both the physical network of ditches and the social structure associated with their maintenance and utilization, persists to this day under the name "acequias". As water management issues emerge as an increasingly serious topic in the US Southwest, how sustainable are traditional, acequia-dependent forms of agriculture?

To investigate, Andrew Crooks and I developed a spatially explicit agent-based model of the area around the town of Taos, New Mexico. The model was coded in Java utilizing the MASON simulation toolkit and its GIS add-on, GeoMASON. The focus of the model was the farmer agents, who made choices about whether to participate in traditional agriculture (maintaining the acequia ditches and harvesting crops) or to sell their land (resulting in it permanently transitioning to residential use). Farming agents were also influenced by both the physical environment and social factors, including the selling habits of their neighbors and their own personal valuation of the traditional lifestyle. We track the overall conversion of land from farmland to non-agricultural land.

The simulation's interface

A paper documenting our findings was published in Computers, Environment and Urban Systems and is available here, and a video with a bit more detail about the setup and execution of the simulation is included below. A very attractive locally-produced '90s video with more information about acequias is available here!

A special thanks to Michael Cox and John Paul Gonzales for making this project possible!

Full Reference:
Wise, S. and Crooks, A. T. (2012), Agent Based Modelling and GIS for Community Resource Management: Acequia-based Agriculture, Computers, Environment and Urban Systems. Doi http://dx.doi.org/10.1016/j.compenvurbsys.2012.08.004