Automatic Detection and Computer Vision Analysis of Flow Dynamics and Social Groups in Pedestrian Crowds

Computer vision played a vital role in the field of video surveillance. However, recent developed computer vision algorithms rarely solve the problems related to real time crowd management. The phenomena of crowd like sports, festivals, concerts, political gatherings etc, are mostly observed in urban areas, which attracts hundreds of thousands people. In this thesis, we have developed algorithms that overcome some of the challenges encountered in videos of crowded environments such as sporting events, religious festivals, parades, concerts, train stations, airports, and malls. The main theme of this thesis is two fold ,i.e, understanding crowd dynamics in videos of (i), high density crowds and (ii) low density crowds. Typical examples of high density crowds include marathons, religious festivals while malls, airports, subways etc covers low dense situations. In this thesis, we adopt different approaches in order to deal with different kinds of problems coming from these two categories of crowd. In particular, first part of the thesis, we adopt holistic approach to generate a global representation of the scene that captures both dynamics of the crowd and structure of the scene. This was achieved by extracting global features, i.e optical flow from the scene. For the crowd flow segmentation problem, the optical flows vectors are clustered by using K-means clustering followed by the blob absorption approach. Using the segmentation information, we continue to estimate the number of people in each segment by carrying out the blob analysis and blob size optimization approach. This approach however, provide useful information for understanding crowd dynamics yet it lacks significant information for understanding crowd behavior. Therefore, in this thesis, the current crowd flow segmentation and counting approach is further extended in order to coup the challenges of crowd behavior understanding. The extension adopts optical flow for the identification of pedestrian movements, and it considers the analyzed video as a set of sequences. The latter are analyzed separately, producing tracklets that are then clustered to produce global trajectories, defining both sources and sinks, but also characterizing the movement of pedestrians in the scene. In the second part of the thesis, We propose a novel approach for automatic detection of social groups of pedestrians in crowds by considering only start (source) and stop (sink) locations of pedestrian trajectories. We build an Association Matrix that captures the joint probability distribution of starts and stops locations of all pedestrian trajectories to all other pedestrian trajectories in the scene. Pedestrians exhibiting similar distribution are combining in a group, where as similarity among the distributions is measuread by KL Divergence We adopt bottom-up hierarchical clustering approach, which is three step processes. In first step, we treat all the individuals as independent clusters, In the second step, couples are detected and after pruning of bad couples, Adjacency matrix is generated. Later on, in step three, using the Adjacency Matrix, groups of couples, those have strong intergroup closeness (similarity) are merged into a larger group..