The Technology behind ChatGPT

In this talk, I will cover the technology behind ChatGPT. I will discuss the core architecture of chatGPT, the training methodology, and the training data. I will cover the basics of language representation, transformers and large-scale pretraining of language models. I will trace the evolution of GPT from its first version to GPT-3, and then GPT 3.5, their similarity and difference. Finally, I will discuss instructGPT and how it utilizes reinforcement learning to deliver the impressive performance that ChatGPT has today.

 

Junling Hu is the founder and CEO of Coach.ai, which delivers AI life coaches. Prior to founding the company, Junling was the Director of Applied AI at Live Person, where she led a team building conversational AI solutions. Prior to that, she had been an AI leader at Samsung and PayPal. Junling is the author of the book The Evolution of Artificial Intelligence. Junling received her Ph.D. in Computer Science from University of Michigan at Ann Arbor.