ESARS: A Situation-Aware Multi-Agent System for Real-Time Emergency Response Management

This paper describes ESARS, a real-time situation-aware social media- enabled emergency situation alert and reporting system, as a decision support system built on multi-agent software design architecture for emergency situation management. The impact of an incident or disruption due to the incident could be minimized by implementing real-time intervention strategies that involve event monitoring, detection and situation identification via classification and prediction, notification, visualization and reporting that culminate in providing emergency support within time. The nature of agent behavior, which is autonomous, proactive and cooperative, makes them a suitable method for the design and deployment of a dynamic system of this nature. The system relies on historical and streamed real-time geolocation-enabled Twitter data stream for the target emergency events to provide decision-makers with dynamic, comprehensive, and timely information specific to the emergency situation.

Introduction

Incidents are among the most unfavourable events that significantly affect society (and its components) and decrease the reliability of the system. The ubiquity of the Internet and social media platforms and the increasingly widespread access to smartphones present means to collect and harness developing information and information that contributes to situation awareness for management and response teams during an emergency [1], [2]. Messages and posts published on microblogs have been known to contain relevant and valuable information that is timely, often geographically tagged, and describes ongoing events [3]. As stated in [4], this information can increase awareness of an emergency situation as situational awareness is a vital concept for emergency management, where timely and informed decision-making is important for efficient operation and management of the situation [3], [5], [6].

Emergency Situation Management (ESM) is a complex multi-dimensional process that involves a large number of interoperating entities [7] consisting of teams of humans and systems. Emergency situation is a dynamic environment where events are always changing with a lot of uncertainties on what is/will be happening. The situation is unpredictable and, at the same time, requires an urgent and rapid response in order to save lives and reduce further damage to properties [8]. Emergency events such as floods, oil spills, earthquakes, hurricanes, communal conflict/war, riots, shootings or (terrorist) attacks, etc., may cause emergency situations and bring unique challenges. Particularly, sudden-onset emergency situations demand officials to make fast decisions based on minimum information available to deploy rapid emergency response. However, information scarcity during time-critical situations hinders decision-making processes and delays response efforts [9]–[11].

Multi-agent environment comprises of multiple software agents working together to accomplish specific objective. An agent is a software program that acts flexibly on behalf of its owner to achieve particular objectives and must be autonomous, reactive and proactive with the quality of good listener, analyzer and cooperative in nature as well as good coordination, collaboration, communication, forming good negotiation and coalition with other agents [12]. This proactive behavioral nature of agents makes them a suitable method for designing and deployment of a complex and dynamic system with ever changing situation [13]. Multi-agent systems (MAS) had great success in distributed, dynamic and complex problem-solving environments, and using autonomous agents provides intelligent decision-making capabilities [14]. The intelligent attributes of the multi-agent will ensure that the complex tasks in emergency situation management can be implemented in the most efficient way [15]. The multi-agent plays a very vital role, especially in collecting information, predicting events and notifying relevant agencies and people in emergency management and generating reports, among others, while interacting and communicating with each other to handle the emergency situation efficiently in the process [16].

In this work, we present the Emergency Situation Alert and Reporting System (ESARS), a social media-enabled situation-aware multi-agent-based system for real-time emergency situations management. The paper describes how the key processes of autonomous emergency event-related data monitoring and extraction, event detection and situation identification via classification and prediction, geocoding, notification, visualization and reporting are built on MAS architecture and capabilities suitable to the characteristics of small and large-scale emergency events response. The system relies on historical and real-time dynamic streaming geolocated information from Twitter to continuously monitor and detect anomalies and predict emergency events and their inferred locations. It allows for the extraction and classification of incident-related, geographically bounded information from Twitter messages in real-time in order to provide for timely coordination and response. Interaction with the system is via a web-based interface.

The paper is presented as follows: We review some related works and tools that have been developed to aid emergency responders during emergencies based on multi-agent systems approach. We describe the detailed architecture and multi-agent design and organisation of the ESARS system, followed by the implementation of the design. Finally, we conclude with final remarks in the last section.

Related Work

A number of emergency response tools and systems have been developed in order to enhance the power of emergency response and aid emergency responders during emergencies based on multi-agent systems approach. Some examples of such systems include EVResponse [17], ALADDIN [18], SAIDA [19], WIPER [20], AID [21], Disasters 2.0 [22], DrillSim [23] and more are being developed.

EVResponse, as described in [18], is a special GIS-based response management system that combines GIS capabilities with web-based voice translation technologies such as VoiceXML to coordinate emergency management activities, using web services to provide real-time reporting capabilities.

Molina et al. [19] presented the development of the SAIDA system, a multi-agent organization of a computer system designed to operate on data recorded by sensors (rainfall, water levels, flows, etc.) in real-time to assist operators during river flood emergency situations in the area of decision making. It applies multi-agent techniques to interpret the data, predict future behavior and recommend control actions. The WIPER system in [20] is intended to be an integrated system that will help detect possible emergencies and suggest and evaluate possible courses of action to deal with the emergency. The system is designed as a multi-agent system using web services, and the service-oriented architecture relies on the interactive use of partial aggregate and detailed real-time cell phone network data to continuously update the system. AID in [21] is an agents-based user interface for disaster management that combines superior human intelligence with the efficiency of multi-agent systems to enhance resource mobilization, like ambulances, fire brigades, etc., to reduce life and property loss.

Disasters 2.0, as presented in [22], is a social portal approach for disaster management. It presents an approach for human and agent collaboration during disaster management by providing facilities for integrating and sharing user-generated information about disasters. Ku-Mahamud et al. [15] proposed a software design architecture taking advantage of the intelligent agent technology to facilitate autonomous notification and auto-generate situation reports for flood disaster management, which helped to produce scheduled reports with a standardized format that can reduce duplications and redundancies of information.

Proposed System

Overall Architecture

In this work, we propose a real-time situation-aware and social media-enabled emergency situation alert and reporting system (ESARS) built on multi-agent software design architecture for emergency situation management. Fig. 1 shows the system architecture of ESARS, which includes the organization, interaction and communication among the different components of the system. The different major components of the system include Tweet Crawling and Capture, Tweet Components Extraction, Data Preprocessing, Geocoding, Classification and Prediction and Decision Support.

Fig. 1. Architecture of the proposed ESARS system.

Tweet Crawling and Capture component handles the filtering of the message stream (raw tweet data stream) from the Twitter Streaming API (data source) by identifying domain terms (and hashtags) defined in a set of domain dictionaries where only messages with these domain terms will be extracted for subsequent analysis [3]. The domain dictionaries include a categorized list of terms for target emergency situations. Table I shows some examples of tweets which contribute to situation awareness (SA) and non-SA tweets.

Table I. Examples of SA Tweets that Contribute to Situation Awareness and Non-SA Tweets [26], [27]

SA tweets
#Floods continue to wreck havoc in #Nigeria. Patients of our Saviour’s Hospital in Bamadi, Delta, evacuated to nearby hotel
State police are responding to a report of a shooting at an elementary school in newtown [url]
Non-SA tweets
The flood situation in rivers state is a rising cause for concern. #NigeriaInfoPh
@nrcs ng continues to respond to urgent needs of people affected by #foods in #Nigeria, includidng giving food and household items to 10,000 families

The system performs real-time analysis and classification of Twitter stream with the aid of Apache Kafka, a distributed streaming platform and processing pipeline that allows messages to be processed in separate threads. We create a decoupled Twitter stream where we essentially divide the stream into two different modules using Apache Kafka as a message queue [24]. The “Producer” module collects the data from the Twitter stream and saves it as logs into the Kafka Cluster (queue) without doing any processing. The “Consumer” module reads the logs in the queue or cluster and processes the data separately [25]. This way, the system processes the raw data without worrying about the stream getting disconnected because the entire Twitter stream has been essentially decoupled by the system. The Tweet Components Extraction module handles the extraction of the different components of interest that are associated with a particular message where they exist (i.e., text and geo-location information). The preprocessing of the extracted tweet text, which is written in a very informal way and usually contains emoticons, special characters, stop words, user mentions, hashtags, etc., is handled by the Data Preprocessing component of the system.

For each tweet, multiple locations could be found inside its text and metadata: place mentioned within the tweet text, Geographical Positional System (GPS) information, location in a user profile, etc. The Geocoding component deals with the estimation of the inferred locations of events for mapping using available location information. The Text Classification and Prediction component analyzes incoming tweet data for anomalies. It is classified to determine whether perceived reported incidents represent potential emergency events, and the prediction of event type is made based on the outcome of the classification. The Decision Support component acts as a front end for the ESARS system. Interaction with the system and information dissemination from the system to emergency coordinators and responders by way of a notification, real-time map visualization of the incidents, generation of reports, etc., is done through a web-based interface.

For the purposes of training, validation and testing of the text classifier(s) for event prediction, extracted data stream (tweets) are stored on a database as historical data. Subsequently the messages’ text components are extracted. The system performs pre-processing of the tweet text to extract features that are used in the offline training, validation and testing of text classifier(s) for the online analysis and classification of tweet textual data in real time.

The use case diagram of the proposed system is presented in Fig. 2. The actors are the online monitoring system, emergency coordinators (including the administrator), the responders (public agencies) and Twitter data source that provides the affected person’s posts for evaluation for a potential emergency situation.

Fig. 2. Use case diagram of the proposed system.

The ESARS Multi-Agent System Design and Organization

The Multi-Agent System architecture presents emergency management operations and the main agents involved. This is presented in Fig. 3. The main agents are identified as Crawl and Capture, Tweet Component Extraction, Text Classifier, Geocode, Notification, Visualization and Report Generation.

Fig. 3. Multi-agent-based emergency situation reporting architecture.

Each agent is created based on the present situation of the event to model a self-organizing behaviour [13] of the system. This is presented in Fig. 4. For the purpose of working with a real-time stream of data and to evaluate how many incidents our system can detect, the crawl and capture software agent for tweet monitoring and capture is deployed. When the system is running, the agent crawls, pulls and filters the message stream from the Twitter Streaming API based on the domain terms (and hashtags) for target emergency situations.

Fig. 4. Self-organizing multi-agent in the ESARS.

A tweet may consist of text and image(s) and also comes bundled with a relatively rich set of metadata. The tweet might embrace multiple locations inside its text and metadata (spatiotemporal metadata). The tweet components extraction agent extracts the different components that are associated with a particular message where they exist (text and geo-location information). The tweet text is preprocessed by the text agent to extract features, converting the raw text data into a well-readable format to be used by the classifier to determine if they constitute an emergency situation. The agent removes all emoticons and non-ASCII characters, stops words in the text, normalizes all characters to their lower-cased forms, tokenizes the text, etc., [3].

Using multi-elemental location inference method, agent for geocoding predicts the inferred location of tweets by exploiting the geotagging and inherently attached data elements. For each tweet, multiple locations could be found inside its text and metadata: places mentioned within the tweet text, geographic positional system (GPS) information, user profile, etc. When choosing a location for mapping, the agent for geocoding estimates and gives the highest priority to the place where the tweet is posted (geographical coordinates), then the location mentioned in the tweet text, and then finally, locations related to the user profile. It also performs reverse geocoding, which allows a geolocation to be transformed into an address. The Text Classification and Prediction agent analyzes incoming tweet data for emergency situation. The data is classified to determine whether the perceived reported incident represents a potential emergency event, and the prediction of event type is made based on the outcome of the classification.

Upon detection of a potential emergency situation, Notification agent is activated. Using a database of system users’ and responders’ contact details, the agent will send an automated notification/alert SMS and/or email messages containing the inferred location (estimated by the geocoding agent) and details of the potential emerging emergency activity first to the user of the system. An alert will only require notification to be sent to the emergency coordinator and accessed via the interface. Upon assessment and approval, the automated message(s) will be sent to the relevant agencies for timely response.

The visualization agent will visualize information geographically. The location of each message collected, analyzed, and estimated (by the agent for geocoding) is used to produce a map layer on the base map to visualize possible locations of incidents where the messages are referring to or were published. Visualization and methods of displaying this information also help to increase situational awareness for users.

The Report Generation Agent gathers data from the database for report generation. During an emergency, the report-generating agent will begin its activity once event prediction is made and the notification agent is activated. This agent will gather information from the database and the current emergency situation and generate a report in a printer-friendly form. The operation of the agent is adaptive to the stage of the emergency to accommodate reports from the first responders, if there are any. The result of analyzing, classification, prediction, real-time map visualization, notification/alert, reporting, etc., of the incident-related events can be presented in the front end of the system for decision support.

There is a Control Agent, which is the facilitator or coordinator for the system. It selects agents and assigns responsibilities according to the situation. Geocoding, Notification, Visualization and Report Generation agents are created at runtime upon detection of an emergency situation. In normal circumstances, the control, tweet crawl and capture, tweet components extraction, text preprocessing and classification and prediction agents will be active.

The sequence diagram of the proposed system is shown in Fig. 5. It illustrates the interaction between the different agents in the system, the messages to be sent and the sequence of how the system produces notification sent to the right agency based on the type of emergency situation identified by the system.

Fig. 5. The sequence diagram of the proposed system.

Implementation

The different autonomous functions have been integrated into a web-based Social Media-enabled Emergency Situation Alert and Reporting System (ESARS) that has been developed using Python and Flask, among other tools and technologies. Fig. 6 is the screenshot of the main page. Fig. 7 shows a chart of collected tweets based on the domain terms of the target emergency situations and classified tweets (incident-related or non-incident tweets).

Fig. 6. Screen shot for web-based emergency situation alert and reporting system.

Fig. 7. Collected and classified tweets based on the domain terms of the target emergency situations.

Conclusion

In this paper, we describe ESARS, a social media-enabled situation-aware multi-agent-based decision support system for real-time emergency events detection and reporting. The paper describes how the key processes of autonomous emergency event-related data monitoring and extraction, event detection and situation identification via classification and prediction, geocoding, notification, reporting, web-based interaction and visualization are built on multi-agent system architecture and brought together to enhance the decision-making process during emergencies. The system relies on historical and real-time streaming information from Twitter to continuously monitor and detect reported events and predict emergency events and their inferred locations to provide decision-makers with dynamic, comprehensive and timely information specific to the target emergency situations.

The nature of agent behavior which is autonomous, cooperative and proactive makes them a suitable method for designing and deployment of a dynamic system. The Multi-Agent Systems (MAS) approach has proven to be an effective solution to solve the emergency situation management tasks due to the distributed organizational framework, the use of the behavioral principles of mobile intelligent agents, and a natural fit to model the teaming of both the human and system entities. The intelligent characteristics of the multi-agent ensure that the complex tasks in emergency situation management can be implemented in the most efficient way.