LLM economics : The cost of leveraging Large Language Models

Most of us are using LLMs and some of us are getting to the point where LLMs are going to production. Honeymoon phase is going to get over soon and practical realities like cost & maintainability are going to become mainstream. However, the cost of running LLMs is not well understood or often not put in perspective. In this talk we will dive deep into what type of costs are involved in building LLM based apps. How do these compare when you run RAG vs Fine tuning, what happens when you use Open Source vs Commercial LLMs? Spoiler- If you wanted to summarize the entire Wikipedia to half its size using GPT-4 8k context window, it would cost a whopping $360K! While there is ample information available online about LLMs and their performances, our session focuses solely on the math-intensive aspect of understanding LLM pricing. We delve into the cost analysis of running popular LLMs, comparing their pricing for a specific task of summarizing Wikipedia. Moreover, we offer valuable knowledge on the levers of pricing in OpenAI and 3rd-party APIs, as well as the costs associated with self-hosted models and fine-tuning. Additionally, we introduce TrueFoundry's innovative solutions, such as a compression API for reducing OpenAI costs and simplified deployment of open-source LLMs through our Model Catalogue and Drop-in APIs. By attending our session, participants gain unique and actionable insights that cannot be easily found online.
Who is this presentation for?
This presentation is for anyone and everyone who is looking to use LLMs either on the enterprise level or for personal use.
Prerequisite knowledge:
None
What you'll learn?
Interact with brilliant minds out there and foster new connections