Multimodal Pain Classification (m-PainAttnNet)

Mon, 01 Apr 2024 00:00:00 +0000

Automated pain assessment using physiological signals offers an objective alternative to self-reporting. m-PainAttnNet extends to a multimodal setting, fusing ECG, EMG, and EDA signals through attention mechanisms to classify pain into 5 intensity levels. Evaluated on the BioVid Heat Pain Database (87 participants).

Key result: 98.8% accuracy on 5-class pain intensity classification, with cross-session robustness >75%.

Architecture

The model stacks three learned components:

Multi-Scale CNN (MSCN) — extracts temporal features at short, medium, and long scales from 1D physiological signals; dropout 0.4–0.5 for regularisation.
Squeeze-and-Excitation ResNet (SEResNet) — channel-wise attention recalibrates feature importance; residual connections maintain gradient flow; output dimensionality reduced to 30 channels.
Transformer Encoder — 2-layer encoder, 6 attention heads; captures long-range temporal dependencies with layer normalisation and feed-forward networks.

Multimodal Fusion Strategies

Three fusion points were evaluated:

Early fusion: concatenate modalities before MSCN
Late fusion: process each modality independently, combine at the final layer
Attention fusion: learn weighted modality combinations (best results)

Results

Ablation over architecture components

Architecture	Accuracy	F1-Score
MSCN only	70.2%	70.7%
MSCN + SEResNet	98.8%	98.8%
MSCN + Transformer	~93%	~93%
Full m-PainAttnNet	97.1%	97.1%

SEResNet’s channel attention provided the single largest accuracy jump (~28 pp). The full model trades a marginal accuracy drop for better generalisation.

Modality ablation (full model)

Modalities	Accuracy
ECG only	96.8%
ECG + EMG	97.2%
ECG + EMG + EDA	98.8%

Dataset

BioVid Heat Pain Database — 87 participants, 5 pain levels (0–4: none to very high), synchronised ECG, EMG (trapezius, corrugator, zygomaticus), and EDA recordings across multiple sessions.

Technical Stack

PyTorch · NumPy · SciPy · Scikit-learn · K-fold cross-validation · Google Colab (GPU)

Project Report

Course: CS6530 — Advanced Topics in AI, IIT Hyderabad (Jan–Apr 2024) · Supervised by Prof. Nagarajan Ganapati

Semantic Segmentation of Aerial Drone Images

Wed, 01 Nov 2023 00:00:00 +0000

A systematic benchmark of four segmentation architectures on the — 400 high-resolution aerial images (up to 4000×6000 px) with pixel-level labels across 23 urban classes. The goal: understand the accuracy–cost tradeoff across CNN and transformer approaches under real GPU memory constraints.

Best result: UNet at 86.4% accuracy, outperforming DeepLabV3+ (~80%), Swin Transformer (~82%), and SegNet (~75%).

Dataset

Semantic Drone Dataset — 400 images, 350 train / 50 test, 23 semantic classes:

Structures: building, wall, fence, bridge
Nature: tree, vegetation, grass, water
Infrastructure: road, sidewalk, parking, rail track
Objects: car, truck, person, bicycle
Environment: sky, ground, terrain

Images were downscaled to 384×512 for training due to GPU memory constraints (original 4000×6000 px).

Architectures

Model	Params	Accuracy	Inference
UNet	~34.6M	86.4%	Fast
Swin Transformer	—	~82%	Slow
DeepLabV3+ (ResNet-50)	—	~80%	Medium
SegNet (VGG)	—	~75%	Fast

All models trained with Adam (lr=0.001), sparse categorical crossentropy, up to 500 epochs with early stopping; convergence around epoch 300–400.

Key tradeoffs:

UNet’s encoder–decoder with skip connections gave the best overall accuracy with fast inference.
Swin Transformer captured finer details thanks to global attention but at high compute cost.
SegNet was the most memory-efficient via pooling-index upsampling.
DeepLabV3+ with ASPP is best suited for full-resolution inference when memory allows.

Known limitation: all models struggled with small objects (vehicles, pedestrians <60% accuracy) due to downscaling; patch-based inference at full resolution would close this gap.

Technical Stack

TensorFlow 2.x / Keras · NumPy · OpenCV · Google Colab (T4 GPU) · TensorBoard

Project Report

Course: CS5480 — Deep Learning for Vision, IIT Hyderabad (Aug–Nov 2023) · Supervised by Prof. Summohana Channapayya

Transformers | Abhiroop Chintalapudi - AI Researcher

Multimodal Pain Classification (m-PainAttnNet)

Architecture

Multimodal Fusion Strategies

Results

Ablation over architecture components

Modality ablation (full model)

Dataset

Technical Stack

Project Report

Semantic Segmentation of Aerial Drone Images

Dataset

Architectures

Technical Stack

Project Report