vad-llm

Video Anomaly Detection

We investigate detecting anomalies in video streams (e.g., for surveillance systems, autonomous vehicles, social media content moderation) by designing state-of-the-art deep neural networks and more recently with LLMs, VLMs, and agentic AI systems.

Sponsor: 

NSF

Sample Publications:

  • Karim, H. and Yilmaz, 2026. SemVAD: Fusing Semantic and Vision Features for Weakly Supervised Video Anomaly Detection. Transactions on Machine Learning Research (TMLR) [pdf]
  • Mumcu, F., Jones, M., Yilmaz, Y. and Cherian, A., 2025. ComplexVAD: Detecting Interaction Anomalies in Video. Winter Conference on Applications of Computer Vision Workshops (WACVW).[pdf]
  • Karim, H. and Yilmaz, 2024. Real-Time Weakly Supervised Video Anomaly Detection. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) [pdf]
  • Doshi, K. and Yilmaz, Y., 2023. Towards interpretable video anomaly detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (WACV).[pdf]
  • Doshi, K. and Yilmaz, Y., 2021. Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate. Pattern Recognition[pdf]
  • Doshi, K. and Yilmaz, Y., 2020. Continual learning for anomaly detection in surveillance videos. IEEE/CVF conference on computer vision and pattern recognition workshops.[pdf]

Adversarial Machine Learning

Attackers can deceive AI systems by designing adversarial data samples or cyberattacks. We investigate potential vulnerabilities of various AI systems and propose solutions.

Sponsor: 

NSF

Sample Publications:

  • Mumcu, F. and Yilmaz, Y., 2025. Universal and Efficient Detection of Adversarial Data through Nonuniform Impact on Network Layers. Transactions on Machine Learning Research (TMLR) [pdf]
  • Karim, H. and Yilmaz, Y., 2025. Invisibility Cloak: Hiding Anomalies in Videos via Adversarial Machine Learning Attacks. Winter Conference on Applications of Computer Vision (WACV).[pdf]
  • Mumcu, F. and Yilmaz, Y., 2024. Sequential architecture-agnostic black-box attack design and analysis. Pattern Recognition.[pdf]
  • Mumcu, F. and Yilmaz, Y., 2024. Multimodal attack detection for action recognition models. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).[pdf]
  • Mumcu, F. and Yilmaz, Y., 2024. Fast and lightweight vision-language model for adversarial traffic sign detection. Electronics.[pdf]
  • Mumcu, F., Doshi, K. and Yilmaz, Y., 2022. Adversarial machine learning attacks against video anomaly detection systems. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).[pdf]
noise-amp

Time Series Forecasting

Time series forecasting and anomaly detection problems appear in a wide range of applications, such as environmental monitoring, ship route prediction, energy optimization, etc.

Sponsor: 

NIFA

Sample Publications:

  • Murad, M.M.N. and Yilmaz, Y., 2026. Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series. International Conference on Machine Learning (ICML) [pdf]
  • Murad, M.M.N., Aktukmak, M. and Yilmaz, Y., 2025. Wpmixer: Efficient multi-resolution mixing for long-term time series forecasting. AAAI Conference on Artificial Intelligence [pdf]
  • Murad, M.M.N., Yirenya-Tawiah, D.K., Weller, T. and Yilmaz, Y., 2025. Multi-Resolution Mixer Network for Localization of Multiple Sensors from Cumulative Power Measurements. IEEE Wireless Communications and Networking Conference (WCNC). [pdf]
  • Doshi, K., Abudalou, S. and Yilmaz, Y., 2022. Reward once, penalize once: Rectifying time series anomaly detection. In 2022 International Joint Conference on Neural Networks (IJCNN). [pdf]

Hardware-based Authentication

Hardware signatures can significantly complement software-based authentication. We introduce enhanced RF fingerprints and Physically Unclonable Functions (PUFs) through additively manufactured antenna arrays and digital circuits, respectively.

Sponsor: 

Army

Sample Publications:

  • Ranstrom, T., Jebreil, O., Razak, F.A., Yilmaz, Y. and Mumcu, G., 2026. Analytical and Experimental Validation of Wireless Authentication through Enhanced RF Fingerprints of Chaotic Antenna Arrays. IEEE Transactions on Information Forensics and Security. [pdf]
  • Pendino, A., Nguyen, N., Nouma, S., Wang, J., Yavuz, A., Yilmaz, Y. and Mumcu, G., 2025. Additively Manufactured RF Electronics With Structurally Integrated Physically Unclonable Functions for Wireless System Security. IEEE Access. [pdf]
  • McMillen, J., Razak, F.A., Mumcu, G. and Yilmaz, Y., 2025. Hardware and Deep Learning-Based Authentication Through Enhanced RF Fingerprints of 3D-Printed Chaotic Antenna Arrays. IEEE Access. [pdf]
  • McMillen, J., Mumcu, G. and Yilmaz, Y., 2023. Deep learning-based rf fingerprint authentication with chaotic antenna arrays. IEEE Wireless and Microwave Technology Conference (WAMICON). [pdf]

 

puf
lidar

Remote Sensing

We use drone-based lidar and SAR data, as well as satellite-based data, for landscape characterization and environmental monitoring.

Sponsor: 

Army Corps of Engineers

Sample Publications:

  • McMillen, J. and Yilmaz, Y., 2025. FuseForm: Multimodal Transformer for Semantic Segmentation. Winter Conference on Applications of Computer Vision Workshops (WACVW). [pdf]
  • McMillen, J. and Yilmaz, Y., 2025. SegGen: An Unreal Engine 5 Pipeline for Generating Multimodal Semantic Segmentation Datasets. Sensors. [pdf]

Medical AI

We collaborate with medical doctors and physical theraphists on medical image segmentation, multimodal data fusion, and federated learning.

Sponsor: 

Moffitt Cancer Center

Sample Publications:

  • Ahmed, S., Parker, N., Park, M., Jeong, D., Peres, L.C., Davis, E.W., Permuth, J.B., Siegel, E.M., Schabath, M.B., Yilmaz, Y. and Rasool, G., 2026. Reliable radiologic skeletal muscle area assessment—a biomarker for cancer cachexia diagnosis. Cells. [pdf]
  • Tripathi, A., Waqas, A., Schabath, M.B., Yilmaz, Y. and Rasool, G., 2025. HONeYBEE: enabling scalable multimodal AI in oncology through foundation model-driven embeddings. npj Digital Medicine. [pdf]
  • Abudalou, S., Choi, J., Gage, K., Pow-Sang, J., Yilmaz, Y. and Balagurunathan, Y., 2025. Challenges in Using Deep Neural Networks Across Multiple Readers in Delineating Prostate Gland Anatomy. Journal of Imaging Informatics in Medicine. [pdf]
  • Koutsoubis, N., Waqas, A., Yilmaz, Y., Ramachandran, R.P., Schabath, M.B. and Rasool, G., 2025. Privacy-preserving Federated Learning and Uncertainty Quantification in Medical Imaging. Radiology: Artificial Intelligence. [pdf]
  • Tripathi, A., Waqas, A., Venkatesan, K., Yilmaz, Y. and Rasool, G., 2024. Building flexible, scalable, and machine learning-ready multimodal oncology datasets. Sensors. [pdf]
  • Baldeon-Calisto, M., Wei, Z., Abudalou, S., Yilmaz, Y., Gage, K., Pow-Sang, J. and Balagurunathan, Y., 2023. A multi-object deep neural network architecture to detect prostate anatomy in T2-weighted MRI: Performance evaluation. Frontiers in Nuclear Medicine. [pdf]
database