SETOSim - an ABM of Evacuation on the coast of the Seto Inland Sea

A few colleagues and I had a paper published last year! I wanted to talk briefly about some work we did in creating the paper, which was entitled "Simulating emergencies with transport outcomes Sim (SETOSim): Application of an agent-based decision support tool to community evacuation planning". Rather than rehashing the paper, I'd like to talk a little about the context in which the work came about.

The research was originally conceived by Dr Hitomi Nakanishi of the University of Canberra, Australia. With the support and guidance of Professor Yoshihiro Suenaga of Kagawa University, Japan, we developed a simulation of evacuation behaviour situated within the city of Takamatsu, Japan.

Dr Hitomi Nakanishi and Professor Yoshihiro Suenaga on a research boat trip in 2018. Professor Suenaga was showing us how the coastline at different points in the Seto Inland Sea gave rise to weather hazards. He often conducts research scuba dives to gain a better understanding of the sea floor.

Takamatsu is Hitomi's home town; it is also the home of Kagawa University. It is, by Japanese standards, a "relatively small" town - meaning it is home to a mere half a million people and has professional sports teams and its own light rail system. It sits on the coast of the Seto Inland Sea, which exposes it to storm surges and various other alarming manifestations of weather.

The port of Takamatsu as seen from a nearby viewpoint We cleverly visited on a day when there were no typhoons, and were therefore not blown away.

In light of this, we built an agent-based model (ABM) of households making decisions about how, when, and where to evacuate during storm surge events. The ABM was able to make use of Professor Suenaga's storm surge models to explore evacuation events for notional future weather events - an exciting combination of different models!

An example of one of Professor Suenaga's tidal current simulations in the Takamatsu port area. See the paper linked at the beginning of the post for more information.

In 2018 Professor Suenaga came up with funding which allowed us to travel to Takamatsu and hold a symposium. We invited local people to come and hear about the evacuation model we had built. They were asked to fill out a survey about their own experiences evacuating during storm surge events. We later expanded  that survey to make it available to an even larger population, allowing us to gain a better understanding of local experiences and perceptions.

A local newspaper reporting on our symposium!

Based on these surveys, we expanded our work. Hitomi and I returned in the summer of 2019 to work further on the model and to share it with representatives of local government and community groups. Given that my Japanese is...shall we say, limited, my role in these meetings was to run situations and to help visualise different scenarios. I was also able to answer technical questions about how simulation could be used and what kinds of maps or metrics could be drawn from our work.

Professor Suenaga (back left) and me (back center) sharing the simulation results with the Ritsurin Town Community Group.

I found it hugely interesting to see how people responded to the simulation. Local government officials were keen to see the overall performance of the system, which is great and one of the ways that I truly think this kind of simulation can contribute to these kinds of planning efforts. Equally, local community members found it interesting to see their own communities contextualised in emergency events. This latter point was perhaps the most exciting thing for me. A lot of community members mentioned how difficult it was to decide to evacuate, and how abstract the whole thing felt; seeing a simulation of the journey to shelters, and the actions of their neighbours, made the situation more clear to them. They began to discuss reaching out to other neighbourhood groups to better coordinate evacuation efforts in the future, making sure that their plans synched.

This has been one of my favourite projects, and it's one that I think has the real potential to be of use. I hope to return to it - and to Takamatsu - in the not too distant future!

Hitomi leaving なタ書, a bookstore in Takamatsu.

Freight Traffic Control 2050

I wanted to write a bit about a project which has just wrapped up: Freight Traffic Control 2050, an EPSRC-funded grant and originally a collaboration among five universities.

The focus of the work was on trying to understand the last mile of freight delivery. Freight is something I've often excluded from my research into road usage and traffic because the data is usually proprietary. I primarily work with open data, so it has proved difficult to gain access to this kind of information for my own analysis. Unfortunately for me, while freight vehicles represent only a small percentage of vehicles on the road, their characteristic driving patterns mean that they have a disproportionate impact on congestion, emissions, and road safety.

Through the partnerships FTC 2050 had developed, I had access to a wealth of data which had been collected by team members and shared by our project partnerships. Being able to combine this kind of information with agent-based modelling techniques was a great opportunity, and I'm so grateful to the project team members and partners for making it possible!

The rest of the team did a bunch of great analysis of the data, while I developed a simulation framework to track the effectiveness of different delivery strategies together with Kostas Cheliotis. We were able to incorporate their findings into the behavioural framework of the drivers, which is something we'll hopefully be publishing soon.

In the interim, we've finally made our simulation framework public on GitHub. There's a short wiki, which we intend to develop further.

This work ended up garnering quite a bit of interest; we were interviewed for an article in the New Scientist (behind a paywall here) and later for BBC Radio 4 (both live on the radio program You & Yours and in a longer segment on the podcast Smart Consumer - please enjoy my now weirdly trans-Atlantic accent!).

The project was a great experience and I'm sad to see it come to an end. We've got some follow-on funding to bring the simulation to festivals throughout the UK, so if you pass by the Science Tent at a big festival, keep an eye out for FTC 2050 work!

In the meantime, my thanks to the project members: Tom Cherrett and Fraser McLeod of the University of Southampton; Tolga Bektas, now of the University of Liverpool; Maja Piecyk, Julian Allen, and Marzena Piotrowska of the University of Westminster; Adrian Friday, Oliver Bates, and Carolynne Lord of lovely Lancaster University; and of course Kostas Cheliotis, formerly of CASA but now happily repatriated!

Walk This Way - Pedestrian Modelling informed by Video Data

In exciting news, a project for which I developed the software has been published in the ISPRS International Journal of Geo-Information. Our paper is entitled "Walk this Way: Improving Pedestrian Agent-Based Models through Scene Activity Analysis", and explores how data can help inform pedestrian agent-based models. Pedestrian modelling traditionally been challenged by the need to collect data to calibrate and validate models of pedestrian movement. In our model, we use real video data to explore how different navigation metrics incorporate features of the built environment into movement patterns.

The following video shows an example of a model run, and the code to support the work is available here.

 

More information is available here!

Crooks, A.T., Croitoru, A., Lu, X., Wise, S., Irvine, J. and Stefanidis, A. (2015), Walk this Way: Improving Pedestrian Agent-Based Models through Scene Activity Analysis, ISPRS International Journal of Geo-Information, 4(3): 1627-1656.

Thanks to the great team I worked with on this!

AAG 2015

Hi friends! If anyone will be attending AAG 2015, please consider checking out Andrew Crooks and my work on crowdsourced data and agent-based modelling! Andrew will be talking as part of a panel called "Geosimulation and Big Data: A Marriage made in Heaven or Hell?" about some of the work we've done together on ABM and big data.

The talk will include videos highlighting my thesis work, which focused on how social media and mobile phones can influence the spread of crisis-related information through a community, prompting higher-level trends. Check out the video below to see a simulation of the Colorado Springs community reacting to the wildfire of 2012!


Have fun in Chicago!

International Map Year

According to both the International Cartographic Association (ICA) and the United Nations, it's apparently International Map Year! A very happy International Map Year to everyone.

In the past year, I've been enjoying attending mapping parties thrown by the Humanitarian OSM group and using their Tasking Manager to map areas where geographic data is useful but lacking. The team has done a really impressive job of developing both a friendly, useable tool and a community to use it.

In the coming year, I'm especially excited to see what comes of the Missing Maps Project, an effort to map places where humanitarians anticipate crises but lack the spatial data to respond to them. This is similar to the kind of work that amazing groups like MapAction and CrisisMappers end up doing in the heat of the moment. It'll be interesting to see how the availability of good data during the first few hours and days of a crisis impacts the situation.

Happy new year!

New Publication - Triangulating Social Multimedia Content!

In further social media news, my group at GMU has published a paper on how social media can be used to detect the location of events! Our paper, titled "Triangulating Social Multimedia Content for Event Localization using Flickr and Twitter", has just been published in Transactions in GIS.

A three-dimensional perspective of an example of wildfire location assessment.

For more information, please check out my advisor Andrew Crooks' blog post about the paper here or the paper itself below! Thanks, everyone!

Panteras, G., Wise, S., Lu, X., Croitoru, A., Crooks, A.T. and Stefanidis, A. (2014), Triangulating Social Multimedia Content for Event Localization using Flickr and Twitter, Transactions in GIS. (pdf)

MSF Scientific Day: Understanding the Spread of Disease

I wrote earlier about having had the pleasure of working with Ivan Gayton of Medecins Sans Frontieres (MSF) on a project which was shown at MSF's Scientific Day. The goal was to explore how agent-based modelling can help researchers understand how disease spreads in space and time.

ABM is a powerful tool in this context because of its ability to fuse a number of important features which influence the spread of disease. In particular, it can bring together aspects of disease which elude other methodologies:

• ABM can incorporate spatiality in a way that, for example, SIR models can't, capturing the patterns that John Snow highlighted in his early work in GIS
• because ABM can simulate individuals with heterogeneous ages, healths, access to resources, and so forth, it allows for richer representation of disease transmission (i.e. the unusual tendency of Swine Flu to infect the young more than the elderly). System dynamics models of any kind have difficulty doing the same
• human behaviours such as seeking treatment or moving around the environment rather than staying home obviously influence the spread of disease; these dynamic processes can't be addressed by standard GIS techniques
• all of these apply not only to the susceptible individuals, but to the disease and its vectors as well: simulating how tsetse flies move relative to a river or how long a reservoir of bacteria will live on a water pump allows us to project the interactions among these things

Putting my money where my mouth is, Ivan and I have been developing a simulation - or rather, a library to support simulations - which can simply and quickly generate patterns of the spread of disease. The model Ivan presented at MSF Scientific Day highlighted how human-to-human infections such as influenza can spread through an environment, in contrast with water-borne or insect-borne diseases.

A sample visualisation of human-to-human infections is shown in the following video, which reflects a very simple model of individuals moving around the environment throughout the incubation period of the disease, only choosing to seek treatment or stay home during the second phase of the disease. Individuals have homes, and may choose to visit random "friends" based on how distant they are: they therefore frequently make short trips and occasionally travel much longer distances.



This is a very simple case, with very simple behaviours and transmission mechanisms; regardless, it tells the story of how individuals bring diseases home from the larger city to their small towns. It suggests how an epidemic might develop in space and time, and it represents a first step toward understanding how we can respond to and limit the outbreak of disease.

I'm very interested in this project, and while I don't have much time to devote to it, I hope to be writing about it here in the future. In the meantime, I have some (shamefully underdocumented) code available on GitHub here, so do watch both these spaces for updates.

GISRUK 2014 - Crowdsourced Data and Agent-based Modelling

Quite a while after the event, I should say that I enjoyed the conference at GISRUK this April, and I particularly enjoyed talking to some really great researchers about their current projects. It's been rare that I've attended a conference in which so many sessions were so applicable to my research; many thanks to the organisers and the amazing job they did! Also, whoever suggested the ceilidh is great and should feel really good about him/herself.

As for my presentation, I was interested in how rapidly the datasets generated in response to crisis scenarios efforts become usable in a modelling context. Essentially, I wanted to address the question: at what point can we use crowdsourced data in a crisis/crowdsourcing scenario?


The context

A while back, Andrew Crooks and I built a basic model of aid distribution in Haiti after the 2010 earthquake (documented in this paper). The model made use of a number of different data sources, many of which had to be crowdsourced because of the paucity of good information about Haiti before the earthquake.

Figure from the paper. Data on the devastation focused on Port-au-Prince. Model inputs include: geo-referenced information about the (A) amount of damage to buildings, (B) the population, and (C) roads and aid points.

Based on the underlying data about where people were, how damaged their houses were likely to be, and the state of the road network, we simulated how different aid centre setups might be utilised by the population, trying to gain a sense of the relative quality of the various setups. The setups were analysed based on how much aid was successfully distributed and how energetic/healthy the population was at the end of the simulation.

By design, agent movement was influenced by the road setup. In the original paper, the road network was drawn from OpenStreetMap a few months after the earthquake; as a result, the model was run on a dataset which volunteers had had time to construct and clean. And they did a tremendous amount of work, as shown here:


OpenStreetMap - Project Haiti from ItoWorld on Vimeo.

Given that the amount of data available to responders changed rapidly over the first few days of the crisis, I wondered when the data could be pulled into an agent-based model and meaningfully used.


Testing the model

To that end, I decided to run the model with a particular aid centre setup on a variety of different extractions of the data, essentially running the model on the roadmap as it existed at different points in time. Six different datasets are drawn from the extractions of OpenStreetMap by Geofabrik. The extractions I utilised were taken from the GeoCommons repository. To give a sense of the development of the road network over time, the following image compares the change in road mapping within Port au Prince during the second week of mapping:

Map showing OpenStreetMap data for Port au Prince as of different days of January, 2010. Newer roads are in yellow.

So how did the model fare on these different datasets?

As in the original paper, I compare the results along a number of different metrics, specifically the cumulative amount of energy left among all of the individuals at the end of the simulation and the amount of aid which goes unconsumed in each case. In the following table, I show the resulting statistics from 100 runs each of the various extracted datasets where 80 units of aid are initially available.



Dataset	Energy	Aid Leftover
GeoFabrik (Jan 18)	1732063215 (554143)	15.2 (2.2)
GeoFabrik (Jan 19)	1732124066 (689490)	14.7 (2.8)
Ouest (Jan 27)	1732148510 (552321)	20.1 (1.6)
Ouest (Jan 29)	1732659245 (594081)	20.5 (1.3)
Ouest (Feb 9)	1732432082 (634083)	20.4 (1.5)
Ouest (Feb 16)	1731946456 (676668)	20.2 (1.4)

The average value (standard deviation) of each metric for the given dataset.

Analysing the results

Looking at the results, it seems clear that there exists some variation depending on the data. Individuals make choices about whether to seek out aid depending on their perceived costs associated with getting to it, and the results reflect this decision-making. There's a clear break between the scenarios using data generated after January 27 versus by January 19 when it comes to the amount of aid left over. In terms of energy levels, the difference is far less pronounced. If more individuals seek out aid, there is more crowding around the centres, which can be costly in terms of energy and can limit aid distribution.

So what does this suggest for future work about crowdsourcing data?

Firstly, the fact that the same aid setup scenario produces different results depending on the dataset suggests that the datasets we use are in fact important, and that we haven't been worrying about this in vain all this time (a comfort!).

Secondly, the results shown here suggest that despite the changing quality of the road network, the results of the model are usable fairly early in the game; the food utilisation changes, but not radically, and the overall energy levels remain fairly constant. Differences certainly exist, but they won't prevent the results from being usable.

Future steps

Based on this work, I remain enthusiastic about the potential of agent-based models to contribute in this kind of context. I made some bold assertions in the previous paragraph, and I think they bear further exploration:

• exactly how do the results change over time, in terms of the experiences of individuals or specific populations?
• can we project which areas will be most influenced by the data and either forecast or direct data-gathering accordingly?
• what steps can modellers take to build uncertainty and assumptions of uncleaned data into these models so that they are useable in these contexts?
• where are the tipping points in the dataset? How do behaviours interact with the data we utilise?

Obviously these are bigger than a single blogpost, but I'll be thinking about them and posting about them in the future!

Reboot!

After a long silence, I can report that I've ~FINISHED MY THESIS~ and am reemerging into the world of humans. To that end, I'll attempt to be actually blogging in the near future, god help us all.

Since last I posted, I've had a few big-ticket items I wanted to mention:

1. I successfully wrote and defended my thesis; as of May 17, I am now Dr. Wise (definitely either a superhero or supervillain name, it remains to be seen which)


2. CASA was generous enough to award me the best paper award at GISRUK 2014, for research I'll be posting about here in the coming weeks

   Congrats-Sarah Wise, winner of the CASA prize for best paper, awarded in memory of Sinesio Alves Junior #gisruk2014 pic.twitter.com/nGZ8gPRjH0
   — CASA (@CASAUCL) April 18, 2014



3. friend and collaborator Ivan Gayton of Medecins Sans Frontieres (MSF) presented some work we've been doing simulating the spread of disease at MSF Scientific Day on May 30. I'll talk about that work a bit more as it develops further

   At #MSFsci, watching @ivangayton present our model! pic.twitter.com/0Xl1wdSqCc
   — Sarah Wise (@ComplexityWise) May 23, 2014

That's about the shape of things at the moment. I'll be writing here every week or so about what I'm doing, so please do watch this space!

Trawling Social Media Part 2: Flickr

My adventures in Instagram came to a sudden and tragic halt once I encountered the bugginess of Instagram's media/search function. Other people have been documenting a decline in the number of responses they get to this API call for a few months now (see, for example, here) and at this point I get nothing by Error Code 400 in response to every request. Instagram's locations/search function does work correctly, but only returns 33 Colorado Springs-geocoded images over an eight-day period encompassing the evacuation of more than 20,000 people from that city. Disappointing!

Bereft of data, I've been investigating Flickr today. I found this tutorial helpful in getting started. However, for roughly the same period of time I'm seeing only 96 images being shot within a 5km radius of the center of Colorado Springs. I've posted the bulk of the code here, in case anyone else wants to give it a try.

A sample of the images drawn from my Flickr data pull

If anyone has had different experiences with this, I'd be thrilled to hear it, but I'm willing to tentatively say that if you're going back more than a few months and trying to extract data, you should expect to encounter a lot of bad behavior from APIs. This makes a lot of sense given the focus of these companies, but the takeaway: caveat emptor.

Trawling Social Media

Lately, I've been digging into using the APIs of various social media platforms as tools to help explore the spread of information, sentiment, and so forth, specifically focusing on Twitter and Instagram. It's been interesting and sometimes challenging, as I'm going over old tricks (PHP) and learning new ones (authorization and so forth). There have been a number of resources I've found helpful, and I thought I'd post something about my experiences here as a guide to others who are also just getting started with using social media APIs for research.

OAuth

In order to access the APIs, it's necessary to authenticate with the API itself. Both Twitter and Instagram use the OAuth protocol to provide users with access to their data. This involves having an account, registering an application, and generating and providing access codes in the appropriate places. I found the following to be very helpful in understanding and interacting with OAuth:

• 140 Dev Twitter OAuth Programming - a tutorial on using OAuth in the context of Twitter applications. You have to sign up as a member to get access to the text, but I highly recommend it.
• tmhOAuth - An OAuth library used in the 140 Dev tutorial, which with minimal modification can be used to access Instagram data as well (specifically, by changing 'api.twitter.com' on line 40 to 'api.instagram.com').

The App Itself

I've been using PHP to do my data extraction, but other sites discuss using javascript, etc, to do something similar. If you're comfortable with the command line, using PHP is incredibly simple and I highly recommend it. I've included a simple script which, if you edit it to include your target username, will pull the last 100 tweets from a user. To use it, you can drop the tmhOAuth.php and cacert.pem files into a directory with a copy of your application tokens, put a simple PHP script in that same directory, and type

commandline> php myScript.php > outputfile.txt

and voila! You're done. Until you run into 429 Errors, aka rate limiting.

Rate Limiting

Rate limiting will probably make you want to tear out your hair. Twitter and Instagram have different limits on how many times you can query the API within however much time. Twitter even breaks rate limiting down by the kind of query you send - for example, under the current REST 1.1 API policies, a user can submit only 15 "GET lists" queries relative to 180 "GET statuses/user_timeline" queries within a 15 minute period. So be aware of this, and factor it into your applications.

Anyway, hope this is helpful to someone!

Sample PHP files

AAG 2013

I had the opportunity to attend and present some of my work at AAG 2013 this past weekend. I had a great time and met a bunch of great researchers. Thanks, everyone!

An animated wordcloud of Colorado wildfire-related tweets, normalized in size relative to the search term "fire"

My presentation was about simulating the wildfires that swept through Colorado last summer. I thought I'd post my slides here to give a sense of what I've been up to these past few weeks. You can also check out the interactive mapping of tweets I show in the presentation here!

The Deluge: Twitter and Hurricane Sandy

Information is often valuable, but it's crucial in crisis situations. In a broad and basic attempt to move toward harvesting social media to determine a population's mood, my group looked at the use of language on Twitter as Hurricane Sandy bore down on the east coast of America during late 2012. What I show here is a basic visualization of tweets by time, location, and valence (here calculated by a hilariously rough positivity/negativity measure).

The tweets were scraped by team member Jacek Radzikowski (who is also available on Twitter here) focusing on the terms "Sandy", "hurricane", and "frankenstorm". The scraped tweets consist of much more than a 140 character string and a timestamp: if the user has included a description of their location in their profile or enabled twitter to use their phone's GPS, the tweet can contain some very specific locational information. For reasons that will become apparent below, we wanted to try to gain access to the user-provided location information rather than relying exclusively on GPS-derived geotagged tweets.

In this post, I distinguish between what I call "geotagged" tweets (tweets with associated coordinates) and "geocoded" tweets (tweets with locational information that was run through the Yahoo PlaceFinder to produce a set of coordinates).

We collected 1060915 tweets in all, of which 692376 have some geographic designator and 10237 have GPS-derived coordinates. For those of us who understand better in scientific notation, that's

• 1.1x10^6 total
• 6.9x10^5 with some info (~65%)
• 1.0x10^4 with coordinates (~1%)

(...so our motivation for investigating the geocoder is pretty obvious, right?)

To derive some basic measure of how positively people were talking about their impending doom, we took a hand-coded set of words (AFINN, available here). For each tweet, the text is stripped of punctuation, converted to lower case, and broken into individual words. The valence is set equal to the sum of the valence values of each component word found in the AFINN wordset, normalized by the number of valence-having words in the tweet. In the following images, the tweets are colored by this normalized quantity, with more positive tweets in green and more negative tweets in red.

A composite image of Sandy-related geotagged tweets in the New York area

A composite 50% sample of Sandy-related geocoded tweets in the New York area

 

A composite image of Sandy-related geotagged tweets in the USA

A composite 25% sample of Sandy-related geocoded tweets in the USA

Finally, the following video displays the valence and location of tweets as they were updated over the course of the storm's approach and impact.

 This work is still very much in its infancy, but we found it very interesting and hope to do more with it in the future. Stay tuned!

Sustainable Agriculture through Modeling

An acequia-irrigated field in Santa Fe, NM

 

A map of land use around Taos, NM

Irrigating agricultural land has been a major concern throughout history. When Spanish explorers began to settle in what would become New Mexico, they brought with them a system of communal irrigation management that they had learned in their turn from the Moors. This system, consisting of both the physical network of ditches and the social structure associated with their maintenance and utilization, persists to this day under the name "acequias". As water management issues emerge as an increasingly serious topic in the US Southwest, how sustainable are traditional, acequia-dependent forms of agriculture?

To investigate, Andrew Crooks and I developed a spatially explicit agent-based model of the area around the town of Taos, New Mexico. The model was coded in Java utilizing the MASON simulation toolkit and its GIS add-on, GeoMASON. The focus of the model was the farmer agents, who made choices about whether to participate in traditional agriculture (maintaining the acequia ditches and harvesting crops) or to sell their land (resulting in it permanently transitioning to residential use). Farming agents were also influenced by both the physical environment and social factors, including the selling habits of their neighbors and their own personal valuation of the traditional lifestyle. We track the overall conversion of land from farmland to non-agricultural land.

The simulation's interface

A paper documenting our findings was published in Computers, Environment and Urban Systems and is available here, and a video with a bit more detail about the setup and execution of the simulation is included below. A very attractive locally-produced '90s video with more information about acequias is available here!

A special thanks to Michael Cox and John Paul Gonzales for making this project possible!

Full Reference:

Wise, S. and Crooks, A. T. (2012), Agent Based Modelling and GIS for Community Resource Management: Acequia-based Agriculture, Computers, Environment and Urban Systems. Doi http://dx.doi.org/10.1016/j.compenvurbsys.2012.08.004.