Throat photo with exudate regions boxed and labelled with confidence scores. ThroatCare phone app showing a guided throat screening screen.
02 · Computer Vision, 2026

AI-Assisted Pharyngitis Triage

A smartphone-based AI system for remotely screening suspected bacterial pharyngitis (strep throat) from a throat photo. It is designed to help people decide whether a sore throat may require an in-person clinical evaluation.

Problem

Bacterial pharyngitis is a common reason people seek care for a sore throat, and unlike many viral infections, it may require antibiotic treatment. For people in rural or low-resource areas, however, getting evaluated can mean traveling hours to a clinic. I explored whether a smartphone photo could provide useful guidance about when an in-person evaluation may be warranted—without attempting to replace a clinician. The challenge was building around real patient images that are often blurry, poorly lit, or out of frame.

Approach

I collected data from three practicing physicians to understand how they evaluate throat images and which visual signs matter most. That research shifted the system toward the tonsils and visible exudate. I built a multi-stage computer vision pipeline and manually drew 3,778 bounding-box annotations to train a detector that effectively distinguishes exudate from reflections and teeth.

Evaluation

The final system reached 0.80 AUC, 90% sensitivity, and 81% specificity in five-fold cross-validation. Low-confidence cases were intentionally routed for clinical review rather than forcing a prediction. On two separate external throat-image datasets, the model reached 0.87 and 0.89 AUC.

Outcome

I was invited to give a guest lecture at USC on the project as a case study in developing innovative AI products with noisy, real-world data.

Tools

PyTorch · Faster R-CNN · ResNet-18 · Grad-CAM · Weights & Biases · Python · MCP

Next project Where Generative AI Fits at Work