Build your (not so) Large Language Model

LLMs are the product of Industry 4.0 that has most influenced the current advancement of GenAI and ignited our imagination with endless application possibilities. In this talk, we will take a step back in time and look at their evolution, from pre-deep learning models to the current ones based on Transformer architecture. Finally, we will try to build a Large Language Model from (almost) scratch and discuss the difference between pre-training and fine-tuning, as well as their advantages and disadvantages.