The complete implementation of a decoder-only transformer language model from scratch, trained on the TinyStories dataset. The implementation includes all core components of the transformer architecture, training with teacher forcing, and various inference enhancements including beam search, KV caching, gradient accumulation, and gradient checkpointing. All components are implemented using only basic PyTorch operations as required by the assignment constraints.

This research project systematically investigates the mechanisms of in-context learning (ICL) in transformer-based language models through controlled counterfactual experiments. By testing how models resolve conflicts between pre-trained knowledge and contextual information, we uncover fundamental insights about reasoning flexibility, attention mechanisms, and instructional control in modern AI systems.

Worked on India’s first govt-developed MRI (AIIMS Delhi & SAMEER); Built pipeline from raw K-space -> IFFT -> MRI; Applied Fourier Neural Operator to predict missing datapoints and enhance image quality.

Implemented robust aggregation against Byzantine clients; used Weiszfeld algorithm for geometric median; Ensured secure aggregation & communication efficiency; evaluated on non-IID datasets.

Built end-to-end counter-speech system; implemented n-gram LM, CRF, and Transformer-based GEC from scratch; Trained seq2seq with attention visualizations; Factual Analysis in LLM-generated counterspeech and applied Inference-Time Intervention to improve reliability; Evaluated via BLEU/ROUGE/BERT Score & toxicity reduction.

Worked on India’s first govt-developed MRI (AIIMS Delhi & SAMEER); built pipeline from raw K-space -> IFFT -> MRI; applied Fourier Neural Operator to predict missing datapoints and enhance image quality.

Implemented Transformer from scratch (patch embedding, multi-head attention, positional encodings); Reduced attention complexity via Linformer; Introduced multi-scale patch embeddings.

Applied DP-based RL in Football Simulation, Online Knapsack, Portfolio Optimization environments.

Built abstractive text summarization with Transformers (Hugging Face). Developed full MLOps pipeline with DVC,MLflow, FastAPI, Docker, CI/CD for reproducible deployment.