Predicting Crowd Agitation from Raw Video: A Lightweight Multimodal Regression Framework for Social Tension Estimation

Continuous, quantitative awareness of crowd tension – a perceptually grounded, continuous measure of crowd agitation, distinct from a binary violence label, an unnormalized abnormal-event score, or raw motion magnitude – is required for proportionate public-safety response; however, the dominant paradigm in computer vision approaches remains restricted to post-hoc binary violence detection. To overcome this limitation, an edge-centric multimodal regression framework is proposed that estimates normalized Social Tension Index (STI ∈ [0, 1]) directly from surveillance video without manual feature engineering. A frozen CLIP encoder (512-d) captures spatial semantics, while Farneb¨ack optical-flow descriptors (2-d) model short-range dynamics and audio RMS energy (1-d), where available, provides acoustic agitation; this 515-dimensional representation is mapped to a tension score by a gradient-boosted LightGBM regressor. A hybrid annotation protocol utilizing expert scoring together with motion and audio heuristics has high agreement between annotators (κ = 0.81), and a label-integrity study demonstrates no circular bias introduced by heuristic components (only a 0.031 R2 drop under purely human labels). Five-fold cross-validation on RLVS gives MAE = 0.0255 ± 0.0053 and R2 = 0.8732 ± 0.0631, with a binary accuracy of 93.75% downstream (precision 100%/recall 87.5% on High-tension alerts). Measured on a single CPU core, the edge decision head runs in 1.65 ms per clip (606 clips/s); the frozen-encoder design does not require any retraining on specific scenes, and this measured feasibility on resource-restricted hardware makes the framework a promising tool for graduated territorial risk monitoring, pending field validation.

@article{Ouza2026,
  title     = {Predicting Crowd Agitation from Raw Video: A Lightweight Multimodal Regression Framework for Social Tension Estimation},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {17},
  number    = {9},
  year      = {2026},
  publisher = {The Science and Information Organization},
  author    = {Abderrahim Ouza and Khalid Ounachad and Mohamed El Ghmary and Ali Choukri},
  doi       = {10.14569/IJACSA.2026.0170999},
  url       = {https://doi.org/10.14569/IJACSA.2026.0170999}
}