Learning Track
LLM Research and NLP
From the transformer math to a paper at the ACL Student Research Workshop.
A research-oriented track with four sub-tracks: Transformers and NLP Foundations, Fine-Tuning and Adaptation, Evaluation and Experimental Rigor, and Research Practice and Publishing at ACL SRW. Takes you from the architecture (built from the math up) to a reproducible, SRW-ready short paper. Distinct from the practitioner Artificial Intelligence track.
Sub-Tracks
Tokenization, embeddings, attention, and the transformer from the math up.
Build the transformer from first principles: the tokenization problem, training a BPE/WordPiece/SentencePiece tokenizer, embeddings and positional encoding, scaled dot-product and multi-head attention, the full block, and pretraining objectives. Ends with a from-scratch tiny transformer and a research-readiness lesson.
First lesson: What LLM and NLP Research Is
Adapt real models with LoRA/QLoRA on a student GPU budget.
The adaptation lifecycle, full vs parameter-efficient fine-tuning and the memory math, LoRA and QLoRA hands-on, instruction tuning and preference optimization (DPO/RLHF), data curation and decontamination, and the compute realities of free/cheap GPUs. Ends with a measured fine-tuning experiment and a job-readiness lesson.
First lesson: The Adaptation Lifecycle
The rigor reviewers demand: metrics, contamination, significance, reproducibility.
Why evaluation gates publication: automatic metrics (perplexity, BLEU, ROUGE, BERTScore) and their failure modes, benchmarks and data contamination, human evaluation and LLM-as-judge bias, statistical significance and ablations, and reproducibility. Ends with a reusable evaluation harness and a job-readiness lesson.
First lesson: Why Evaluation Is the Hard Part
Read, reproduce, write, review, and present a paper at the Student Research Workshop.
The research craft: the ACL ecosystem and ACL Rolling Review, reading papers and literature review, scoping a question and finding a mentor, reproducing a paper, writing in the ACL template (with the mandatory Limitations + Ethics sections), surviving peer review and the rebuttal, and presenting. Ends with an SRW-ready short paper and a research-career lesson.
First lesson: The ACL Ecosystem and SRW