Comparing Different Metrics Quantifying Pedestrian Safety

The quantification of pedestrian safety is an important research topic. If reliable quantification is possible, it can be used to predict and prevent dangerous situations, such as the crowd crush at the 2010 Love Parade. To quantify safety, we can use several metrics like density, velocity, flow and pressure. Unfortunately, there are several methods to evaluate these metrics, which may give different results. This can lead to different interpretations of similar situations. Researchers compare these metrics visually or search for trends in fundamental diagrams. This is inherently subjective. We propose an objective methodology to compare these methods, where we emphasize the different quantifications of peak “dangerousness”. Furthermore, we refine existing methods to include the obstacles in environments by replacing the Euclidean distance with the geodesic distance. In our experimental analysis, we observe large differences between different methods for the same scenarios. We conclude that switching to a different method of analysing crowd safety can lead to different conclusions, which asks for standardisation in this research field. Since we are concerned with human safety, we prefer to err on the side of caution. Therefore, we advocate the use of our refined Gaussian-based method, which consistently reports higher levels of danger.


Introduction
At the 2010 Love Parade [6], 21 people were crushed to death and hundreds got injured. In 2006 and 2015 hundreds of pilgrims got trampled during the Hajj to Mecca due to dangerously overcrowded situations [7]. Motivated by such disasters, researchers study ways of preventing these from happening again. By studying metrics like density, velocity, flow and pressure, early warning signs for potentially dangerous situations can be found.
In this paper, we refer to density, velocity, flow and pressure as metrics. Different methods exist to evaluate them. For density, the best-known method is the grid-based method [5]. A regular grid is superimposed on the environment, dividing the environment into cells. Another way of determining density uses Gaussian distributions [7,12]. Such methods place two-dimensional Gaussian distributions on the pedestrian's locations. By adding the Gaussian distributions together, a density field is formed. A third method to determine local density uses Voronoi diagrams [13]. A Voronoi diagram is a division of the environment in cells, such that all points within a cell are closest to a single pedestrian. The density within each cell is 1 divided by the area of the cell. Details for these methods can be found in Section 2.1.
It is possible to determine the velocity by using density [15]. Using the density and velocity, both flow and pressure can be determined [7]. We describe their implementation in Section 2.2.
Fruin [5] computes the danger level based on the value of these metrics within a (small) region of the environment, which is mapped to six non-overlapping intervals. This method is called Level of Service (LoS). The different intervals encode situations from safe to dangerous. The exact boundaries of these intervals depend on different factors, such as the measurement location and even culture [1].
When using LoS, the outcome of the safety evaluation is dependent on the measurement location and on how obstacles are handled. Quantifying these different outcomes is no easy task. One way to compare different methods is by visually inspecting the results. An example of two density fields is given in Figs. 1(a) and 1(b). We can observe a difference between the two images, but this difference is hard to quantify. The absolute difference between the two fields ( Fig. 1(c)) only emphasizes that differences exist.
Duives et al. [4] compare density methods using fundamental diagrams. For different scenarios they formulate what trends they expect to be present in these diagrams, and look for them. One drawback of this method is that it requires expert knowledge about what trends are to be expected. Furthermore, it is inherently subjective due to the human classification step.
Duives et al. [4] also propose an objective measure of similarity between density methods called scatter. The scatter is the range of measured velocities for non-overlapping density intervals. One downside of this measure is the interdependency between these metrics. A small change in the density method can result in a large shift in the measured scatter due to the potential recategorization of measured velocities.

Our Contribution
In this paper, we propose an objective methodology for comparing methods that compute safety metrics. We also refine existing methods to consider obstacles in the environment. We achieve this by replacing the Euclidean distance by the geodesic distance [10]. The resulting differences are showcased in Fig. 1. Method 1 uses the Euclidean distance, whereas method 2 uses the geodesic distance.
We also performed experiments on environments to test if our methodology yielded new insights into the differences between methods. We conclude that the classification of a situation as being safe depends on the method that is used, and that our refined methods consistently classify situations as more dangerous.

Overview
In Section 2, we discuss the different methods used for evaluating the metrics. Here we also introduce our refinements of two methods. Next, in Section 3 we give details of the four measures used to quantify the differences between the methods. These measures are used in Sections 4 and 5 to evaluate the different methods on three basic environments and several scenarios. We end with a conclusion in Section 6.

Methods for Measuring Safety
As discussed in Section 1, different metrics exist. Furthermore, there are different methods for computing each metric. In Section 2.1, we discuss different density methods. The considered methods are either grid-, Voronoi-or Gaussian-based. How the resulting density fields can be used to determine velocity, flow and pressure fields will be discussed in Section 2.2.

Density methods
The grid-based method was first defined by Fruin [5]. Intuitively, this method counts the number of pedestrians in a cell , where the cells are defined by a regular grid which is placed over the environment. This grid does not consider the obstacles in the environment. Next, this number is divided by the area of the cell to get the local density. All the pedestrians are considered to be points, and can therefore only be in one cell at a time. The only parameter for this method is the width of a cell. A problem with this method is that obstacles, such as walls, have no effect on the measured density. This can be seen in Fig. 2(a). The large jumps in density when a pedestrian moves to the next cell cause another problem. This last issue can be partially alleviated by modelling pedestrians as discs and measuring the fraction of the disc in a cell.
Steffen and Seyfried [13] take another approach at minimizing these large jumps in density. Their Voronoi-based method describes the free space that is available to a pedestrian by using Voronoi diagrams. A Voronoi diagram of input points is the partitioning of the environment into cells, such that every position within that cell is closest to exactly one point. Steffen and Seyfried [13] use the locations of the pedestrians for calculating a Voronoi diagram as the input points. After obtaining this diagram, the density within each Voronoi cell is defined as 1/ . Here, is the area of Voronoi cell . Next, this Voronoi cell density is used to calculate the density within grid cells by using a weighted average of all the Voronoi cells that intersect that grid cell.
One drawback of this method is that the area of a Voronoi cell can be large, while it is locally very dense (e.g. pedestrians on the perimeter of a dense group). To this end, Steffen and Seyfried suggest a limit of 2 square meters on the area of a Voronoi cell. Furthermore, it is not mentioned how the obstacles should be handled. In our experiments with the Voronoi method, we will remove the area of a cell that is covered by obstacles. As a result, higher (more accurate) densities are reported.
However, only removing obstructed regions from a Voronoi cell can still give the illusion of too much free space. We show an example of this in Fig. 2 Here, a Voronoi cell is split into two disconnected pieces by an obstacle. To remedy this, we propose to use a geodesic Voronoi diagram [10], which accounts for the obstacles in an environment. This also changes the shape of the Voronoi cells. We exemplify this in Fig. 2(b) and (c). Some line segments are replaced by curves, because of the nearby obstacles.
Gaussian-based methods measure density for points instead of areas. Here, the contribution of a pedestrian to a point depends on the distance between the pedestrian and the point. This distance is used as input for a function that determines the contribution. More formally, the density at point is defined as Here, is the set of all pedestrians and ( , ) is the Euclidean distance between point and pedestrian .
One of the first methods that uses this is due to Helbing et al. [7]. They use the following function: This equation is a variation of a Gaussian distribution. Here, is the distance between a pedestrian and a point.
is the only parameter of this method. It influences the pedestrian's contribution to the perceived density. Duives et al. [4] point out that influences the contribution and should be picked carefully.
Plaue et al. [12] suggest a method that bypasses this issue altogether. Instead of having as a fixed parameter, it is dependent on the current locations of the pedestrians at time . With all locations given as ( ), the function used to determine for a pedestrian at location is In their experiments the parameter is set to 4. In addition to using a dynamic value for , Plaue et al. also take obstacles into account. They do this by setting ( ) = 0 for points that are inside obstacles. The volume of the Gaussian distribution that would normally fall inside the obstacles is redistributed amongst the obstacle-free points. As a result, the contribution to the density of a pedestrian in closed-off places increases, resulting in higher local densities. This becomes clear when we compare Figs. 2(d) and 2(e). However, this method still increases the density at the opposite side of an obstacle. In our example, we can see that there is a region with higher density close to walls because of the presence of pedestrians at both sides.
To further take the obstacles into account, we use the geodesic distance. When two pedestrians have the same Euclidean distance to a measurement point, they should not always contribute equally to the local density, because one or more obstacles may cause a detour for the pedestrian. The geodesic distance takes this detour into account. Such a geodesic Gaussian density field can be seen in Fig. 2(f).

Derived Metrics
Although density is an important metric for determining pedestrian safety, it is not the only one available. We mentioned velocity, flow and pressure in Section 1 and gave an intuitive definition. In this section, we will show how these metrics can be determined.
To compute (local) velocities, we use an adaptation of the method described in Helbing et al. [7,Equation 6]. In this method, the local velocity is defined as: Here, is a location, is the current time, ( ) is the set of locations of the pedestrians, ⃗ is the velocity of the pedestrian at location and ( , ) is a weighing factor. Helbing et al. use a Gaussian distance-dependent function for the weighing factor. This is the same function as given in Eq. 1.
Since we used different density methods, we will have ( , ) reflect this. That is, we use different definitions of ( , ) for the grid-, Voronoi-and Gaussian-based methods respectively. We do this to better reflect the underlying division of the environment in regions. These functions are given in Eqs. 4 through 6. In these equations, is the location of a pedestrian, is a cell used by the density method, a point and is the area for the Voronoi cell . Finally, the function ( , ) is the geodesic distance gd( , ) when evaluating our refined method, and the Euclidean distance ( , ) in the other situations.

Comparing different methods
From Fig. 2 it is clear that different methods can give different results, but are these differences significant? We will look at four measures for comparing these methods to try and answer that question.
When we analyse a method , we will look at a region of interest within the studied environment. This area is divided into a set of cells for 1 ≤ ≤ . These cells follow from the method we choose. The value for such a cell is given by ( , ).
First, we look at the maximal value for within , which enables us to compare measured peak densities, velocities, flows and pressures. We also look at the maximal difference between two methods. They are given in Eqs. (7) and (8).
These two measures do not offer more information than a visual inspection. The differences are accented, but other information is lost. For that reason, we introduce two new measures. For these measures it is important that is centred within the area we want to study. This, however, should not be a problem since we are interested in local values. We call the first one the quadratic score (qs). We define it as follows: Here, is the obstacle-free area of cell and = ∑ =1 is the obstacle-free area of . The resulting value is a number in the range of 0 to 1. A value of 1 denotes that all cells are at the maximal value. When gets closer to 0, it means that a large area of has low values. This function ensures that regions which are closer to high (i.e. dangerous) values are emphasized. Furthermore, this method evaluates to simple scalars. Therefore, it is possible to use existing statistical methods to determine if there is a statistically significant difference.
The last measure we discuss is a comparison based on how the industry often interprets the values from the metrics. Usually, a certain threshold value is used or categories are specified. An example is the LoS concept [5]. Eq. 10 calculates the difference in categorization between two different methods.
Here, ( , ) is a function that maps ( , ) to the category's number. For example, when a single threshold is given, a value ( , ) < maps to 0 and all other values to 1. A higher score means that there are many differently classified cells. The difference between the classifications is squared to emphasize larger differences.

Experimental Setup
We performed experiments to test whether there are statistical differences between the methods. To that end, we implemented all the methods and measures described in Sections 2 and 3 in the Utrecht University Crowd Simulation framework [14]. The different parameters we used for the methods are shown in Table 1. The Voronoi-based method is the one that Steffen and Seyfried [13] refer to as . We tested the methods on the three environments depicted in Fig. 3. These environments are building blocks for larger environments and are frequently used in studies [2,3]. For both the U-turn and corner environment, simulated pedestrians (agents) moved from line to line . For the T-junction environment, we tested two different variations. One with one entrance at line and two exits located at and (scenario 1), and one variation where the agents entered from and and moved towards (scenario 2). The agents were created at a random position behind the starting lines. The rate at which the agents were added was varied from 0.5/ to 2.5/ . The agents' preferred speed was set at 1.4 / .
We recorded the location and velocity of the pedestrians every tenth of a second for 10 simulated minutes, starting 2 minutes after the first pedestrian reached the exit. We used this data to calculate the different fields. We also calculated the time-average fields over a timespan of 1s, 10s and 60s.

Results
The analysis of the results is split into three parts. First, we look at how the size of the averaging window influences our analyses. Second, we will perform an in-depth analysis for the U-turn environment to show what information can be extracted using the discussed measures. Third and last, we make general observations for all the different environments tested. We only discuss the and measures. Other results are available on the author's website [9].

The Size of the Averaging Window
We analysed the effect of different averaging windows. The size of the averaging windows seems to be of little effect for the Gaussian-based methods when looking at the maximum values. Furthermore, the shape of the curves for the reported maxima stay the same. In case of the quadratic score, some details disappear when we increase the size of the averaging window. We found that averaging windows larger than 10s are not needed. Therefore, we will report all results for an averaging window of 10 .

In-depth Analysis of the U-turn Environment
We summarized the results in Figs. 4 and 5. At first glance, it seems that the results for for the Voronoi-based density methods give unexpected results for lower inflows: the entire environment is at the peak density. When the inflow is increased, it steadily declines. This is an artefact of how the Voronoi-based method is defined. Steffen and Seyfried [13] defined a minimal density within all Voronoi cells. When the cells are large enough, the reported local density value is only determined by this maximal area. When the inflow is increased, this setting becomes less and less influential on the maximal measured values.  Settings Grid-based [5] = 1 meter Gaussian-based [7] = 1 meter; Sampling distance = 0.1 meter Gaussian-based [12] = 0.7 meter; Sampling distance = 0.1 meter Gaussian-based (geodesic) = 0.7 meter; Sampling distance = 0.1 meter Voronoi-based [13] = 1 meter; Max. Voronoi cell area = 2 meter Voronoi-based (geodesic) = 1 meter; Max. Voronoi cell area = 2 meter Proceedings from the 9th International Conference on Pedestrian and Evacuation Dynamics (PED2018) Lund, Sweden -August 21-23, 2018 Another interesting observation is the ordering of the different Gaussian-based methods. For the density, flow and pressure, the score is always lower, but for velocity it is always higher. This is a result of the use of the geodesic distance. The Gaussian-curves are more localized around the locations of the pedestrians in our version. As a result, less of the curves are on the opposite side of the obstacle and opposing velocities do not cancel each other out near the obstacle. This means that the geodesic Gaussian does not influence the area on the opposite side of the walls.
We also determined the for density and velocity measurements. The bins that were used are shown in Table 2. In case of the Gaussian method, it is interesting to note that the differences according to the velocity measurements were much bigger. It is also interesting to see that the Voronoi-based methods also show differences, although the was similar for all different inflows. However, at what inflow these differences register differs greatly on what metric we use. Further research is needed to determine what metric is more suitable or if more metrics should be used in conjunction.

Analyses of All Environments
For all tests for statistical significance we used ANOVA with a significance level of 0.05. For each environment, we performed statistical analyses for and . This reported that there were differences between the different methods. Using Tukey-HSD post-hoc analyses, we found that at almost all flows, all methods were different from each other at almost all levels of inflow.
The situations where these differences were insignificant were at the lower inflows for the Voronoi-based methods. This is probably a result of the maximal Voronoi cell size, as discussed in Section 5.2. For the other environments, similar results were found. That is, the geodesic Gaussian consistently reports higher values than the other Gaussian-based methods. The two Voronoi-based methods seem to generate similar results (although the differences are still significant).
Therefore, we cannot simply use one cut-off point for determining if a situation is safe. This is already widely known when looking at different situations and cultures, but to the best of our knowledge it was not yet shown for different methods. This asks for standardisation in this research field.

Conclusion
In this paper, we have discussed different metrics for evaluating pedestrian safety. Each metric can be evaluated by several different methods. We described a refinement for existing methods, namely the usage of the geodesic instead of the Euclidean distance, which takes obstacles into account. We have shown experimentally that this change results in significantly higher densities, flows and pressures.
Furthermore, we discussed four measures for comparing different methods. The maximum ( ) and maximum difference ( ) are already used to show differences between two methods. We introduced the quadratic score ( ) and bin distance ( ) to better show the differences between methods. We analysed all methods by using these four measures and concluded that the differences between the methods are significant. Since we are concerned with human safety, we prefer to err on the side of caution. Therefore, we advocate the use of our method, which consistently reports higher levels of "danger".
One major selling point of analysing the differences between different methods using , , and is that it leaves no room for subjective interpretation of the results. As a result, any researcher performing a similar study should be able to end up with similar conclusions.

Future Work
Although the measures described in this paper show that there is a difference between different methods, it is still not easy to explain what causes them. Therefore, it stays important to look at renderings of the respective fields. It would be interesting to research if there is an automatic classification possible that captures what causes the differences. Furthermore, we only tested on smaller environments. We still need to determine if these measures are effective for larger environments, such as a building or a city.
It would also be interesting to see how the geodesic distance influences the measurements for the different metrics on multi-layered environments [8]. Previously, this was difficult because the Euclidean distance for pedestrians in multi-layered environments is not well defined, but the geodesic distance is. Therefore, it is possible to use the two methods described in this paper for multi-layered buildings.