Paria Mehrbod

Master's student in Computer Science at MILA/Concordia University, specializing in machine learning and data analysis. Proficient in Python, PyTorch, and AWS cloud services. Experienced in language model development and ML research, with multiple publications.

• "Test Time Adaptation through Adaptive Quantile Recalibration," P. Mehrbod, P. Vianna, G. Nanfack, E. Belilovsky, G. Wolf, Submitted
• "Circuit Discovery Helps To Detect LLM Jailbreaking," P. Mehrbod, G. Nanfack, B. Knyazev, E. Belilovsky, Submitted
• "Beyond Cosine Decay: On the effectiveness of Infinite Learning Rate Schedule for Continual Pre-training," V. Singh, P. Janson, P. Mehrbod, A. Ibrahim, I. Rish, E. Belilovsky, B. Thérien, 02/01/25, arXiv preprint arXiv:2503.02844 (2024)
• "Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation," P. Vianna, M. S. Chaudhary, P. Mehrbod, A. Tang, G. Cloutier, G. Wolf, M. Eickenberg, E. Belilovsky, 02/01/24, Conference on Lifelong Learning Agents (CoLLAs), Pisa, Italy

Large-Scale Mechanistic Analysis of LLM Jailbreak Circuits - 07/01/24 - Present

• Developed novel methodologies for identifying neural circuits linked to model jailbreaking in production-scale LLMs (LLaMA-2/3), advancing mechanistic interpretability beyond small transformers.
• Implemented comprehensive jailbreak datasets and circuit discovery algorithms.
• Tech stack: PyTorch, transformer-lens, HuggingFace Transformers

Test-Time Adaptation via Percentile-Guided Neuron Alignment - 05/01/24 - Present

• Designed a novel test-time adaptation method that dynamically adjusts neural network activations using piecewise linear transformations.
• Demonstrated state-of-the-art results across CIFAR-10/100 and ImageNet benchmarks, achieving superior performance without additional training.
• Validated theoretical framework through rigorous experiments across Vision Transformers and ResNets.

Infinite Learning Rate Schedulers for Training Foundation Models - 01/01/24 - 03/31/24

• Investigated novel learning rate scheduling approaches for continual pre-training of large language models.
• Conducted experiments using GPT-NeoX (49M parameters) to analyze model generalization across distribution shifts from English to German domains: Pile (300B tokens), German Common Crawl (∼ 200B tokens).
• Evaluated infinite learning rate schedulers with and without experience replay.
• Tech stack: PyTorch, GPT-NeoX, HuggingFace Transformers

• Concordia Merit Scholarship, 04/01/23, Awarded based on academic excellence and research potential
• IAESTE Exchange Fellow, 08/01/21 - 12/31/21, Secured 1st place in Iran's IAESTE selection exam, awarded fellowship grant for international exchange internship in Austria
• Exchange Fellow Extended, 07/01/22 - 02/28/23, Received exclusive follow-up fellowship grant from employer for continued research collaboration based on previous internship performance