Ruthu

Master'sArtificial Intelligence

Bachelor'sArtificial Intelligence

Mapping tea plantations at scale to track human–elephant conflict as elephants migrate through farmers' tea plantations.

Migration

Conflict

What is a computer vision model?

A busy city street with motorcyclists, cars and apartment buildings

59	51	167	135	133	143
66	108	62	27	38	78
81	66	58	29	24	27
77	56	71	27	20	29
43	50	91	61	21	29
125	76	55	20	45	32

The red patch, as numbers (brightness 0–255). To a computer, an image is just numbers.

Photo from COCO val2017 (image 296649), via Flickr, CC BY-NC-ND 2.0. Shown unmodified.

f(x; θ)

0.70.21.2-2.01.2-1.3-1.31.2-2.02.1-2.0-2.00.20.20.70.7

-2.00.7-2.01.8-1.3-2.01.21.80.12.12.1-0.41.8-0.4-1.3-0.9

1.80.1-1.3-2.01.81.8-2.0-1.3-0.4-1.3-0.40.1-0.91.80.1-2.0

h₁ = ReLU(W₁x + b₁)
h₂ = ReLU(W₂h₁ + b₂)
⋮
words = decode(h_L)

θ = millions to billions of learned numbers

Output: text

People are walking and riding motorcycles on the street

One of five human-written captions for this photo.
The output could also be a number, a label, a box…

Collected from yesterday's talks and general listening in on social science conversations

Looking (gazing)
Pointing
Touching
Pose
Counting people

Whether someone intervened
Gender

Gaze-LLE demo: arrows from each person's head to where they look — **Gazing** · Gaze-LLEDemo from the project page

OpenPose demo: skeletons drawn on dancing people — **Pose** · OpenPoseDemo from the project page

DM-Count demo: a crowd photo turned into a density map and a people count — **Counting people** · DM-CountDemo from the project page

Real CCTV footage is not displayed

to maintain data privacy.

Representative CCTV-like videos from the ShanghaiTech Campus dataset are used as proxies throughout, with the same models run on them.

Person detection · Faster R-CNN (ResNet-50 FPN v2)
Pose · ViTPose
Gaze · Gaze-LLE
Perceived gender · Qwen3.8-27B

Inter-coder reliability

Human–human

84%

Human–AI

80%

example numbers

From a real study: Marie’s CCTV social-distancing paper (Krippendorff’s α)

	Human–human	Human–AI
Counting people	0.98	0.85
Spotting <1.5 m violations	0.93	0.85 (0.84 on new frames)

The first AI version scored only 0.52 on violations, and rose to 0.85 after the distance estimate was improved. Krippendorff: α above 0.80 is reliable, 0.67–0.80 tentative.

Bernasco, Hoeben, Koelma, Liebst, Thomas, Appelman, Snoek & Lindegaard (2023), “Promise Into Practice”, Sociological Methods & Research 52(3): 1239–1287. Human–human: 221 double-coded still frames.

Computer vision as a filter

Illustrative numbers: 1000 body-camera videos, but we only care about some of them.

1000 body-camera videos

340 remain

90 to review

More than one police officer

✗ 1 officer: out

✓ 2+ officers: kept

Crowds

✗ no crowd: out

✓ crowd: kept

Touch detection: asking Qwen where people touch

A man and a woman hugging in a restaurant — Photo: COCO val2017 #6763 via Flickr, CC BY-NC-SA 2.0 (unmodified; the box is drawn on top).

Qwen’s response (verbatim)

What is happening

A man and a woman are posing together for a photo, smiling at the camera. The man has his arm around the woman's shoulder, and they are both leaning in close to each other.

Where the touch is

The man's right hand is on the woman's left shoulder, and his left arm is wrapped around her back.

Box around the contact

[0.12, 0.32, 0.94, 0.61] as fractions of the image width and height; this is the highlighted area.

Teaching Qwen to find touch with GRPO

GRPO (Group Relative Policy Optimization): Qwen tries several answers, we score them against a human-marked touch area, and it learns from which answers beat the group.

Illustration of the idea: the boxes and scores are simulated, not our trained results. Photo: COCO val2017 #6763 via Flickr, CC BY-NC-SA 2.0.

Final output: a segmentation of the touch

The photo with Qwen's box around the contact area — **Before:** a box around the contact area

The photo with the two touching hands cut out as a segmentation mask — **Final:** a mask on the two hands that touch

After training, the answer is a mask instead of a box: the exact pixels of the two people’s hands in contact. Illustration: the masks were made with SAM prompted at the hands, not by a GRPO-trained model. Photo: COCO val2017 #6763 via Flickr, CC BY-NC-SA 2.0.

Where and how AI can fail

Trained on

mostly clear, well-lit photos and video

GazeGazeFollowVideoAttentionTargetGaze360

GazeFollow: head box + where annotators say they look

Pose (OpenPose)COCO KeypointsMPII Human Pose

Touch / contactHOT

Counting peopleShanghaiTechUCF-QNRFNWPU-Crowd

Real samples from the datasets, with their own annotations drawn on top.

Ability transfer

Applied to

CCTV-like proxy clip: small, low-resolution people — CCTV: small, low-res peopleShanghaiTech Campus, proxy for our footage

Real first-person helmet-camera clip — Egocentric (first-person)*helmet cam, CC BY 4.0, Wikimedia Commons*

Real handheld shaky-cam clip: a walk down a London street — Shaky-cam (handheld)*handheld walk, CC BY 3.0, Wikimedia Commons*

In our CCTV footage the median person is only about 60 × 140 px.

Unfortunately…

People don’t say: Let me do this crime — or this intervention — in bright lighting so it’s easier for a computer to see or train on.

Pose and gaze, run on CCTV-like footage

pose skeleton (ViTPose) head box gaze target (Gaze-LLE)

Proxy footage: these are not our CCTV clips.

What’s hard for AI to do?

Tensing and going limp
Subtle changes in muscle tone, with little visible motion
Speed of approaching someone
- Different frame rates: 24, 25, 30 or 60 fps, variable rates and dropped frames change how far someone moves between frames
- No depth from one camera: pixel motion is not metres per second, and the same speed looks faster up close than far away
- Moving cameras: body-cam and shaky-cam motion mixes with the person’s own motion
- Walking straight at the camera: it shows up only as growing size, which is the hardest motion to read
- Motion blur, rolling shutter and lens distortion: positions smear and bend
- Noisy tracking: small people, occlusions and identity switches make positions jitter, and speed amplifies the jitter
- Edited or sped-up video: timestamps, cuts and time-lapse can't be trusted

Sequentiality

Sequences of touch, sequences of other things: very interesting to explore.

Sequences of touch

Sequences of other things

looking

pointing

touching

intervening

Computer vision is moving towards this

EgoPolice — a dataset from Princeton’s computer vision team (Visual AI Lab) · released 7 July 2026

About interpersonal police–civilian interaction and violence: physical contact, weapon out, handcuffing, injury
180+ hours of real, first-person body-worn-camera footage, labelled second by second (9 action classes)
Rapid, irregular camera motion, dense interactions, rare high-stakes events

EgoPolice examples: easy (top) and hard (bottom) cases for four actions — Examples from EgoPolice: easy (top row) and hard (bottom row) cases. Figure from Saez-Diez et al., “EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage”, arXiv:2607.06468 (2026).

Datasets

Released publicly by the computer vision community, with social-science aspects: non-human primate behaviour and “violence”

XD-Violence

4,754videos

217hours

6violence classes

Wu et al., ECCV 2020. Movies and YouTube, with audio. Frames from the dataset paper.

PanAf-SBR

600videos

216kframes

283kdetections

Braszczok et al., arXiv:2607.17399 (July 2026). Wild chimpanzee camera traps; 7 social behaviours. Figure from the paper.

BossouChimpanzee

226videos

70hours

7.8Mframes

Schofield et al. incl. A. Zisserman, arXiv:2610.05293 (Oct 2026). Bossou, Guinea; 23 identified individuals; from a 1,200+ h archive. Clip from the project page.

Reach out!

Excited to see how we can collaborate and apply it together.

nl.linkedin.com/in/ruthu-rooparaghunath

r.h.rooparaghunath@uva.nl

https://hulikalruthu.github.io

Video AI for Public Safety