
Speaker "Joshua Reini" Details Back


-
Name
Joshua Reini
-
Company
Truera
-
Designation
Developer Evangelist
Topic
Evaluating and Tracking LLM Experiments: Building Better LLM Apps with TruLens
Abstract
Building LLM apps that combine powerful LLMs with vector databases, agents and more? If you’re developing with a framework like LlamaIndex or LangChain; an LLM like one from OpenAI or Hugging Face; or using a vector database from Pinecone or Chroma, you will want to learn how to measure the performance and quality of your LLM-based applications using feedback functions. This session explores what feedback functions are, how to use them, and why they can make all the difference.
Who is this presentation for?
LLM application developers.
Prerequisite knowledge:
LLM app development basics.
What you'll learn?
Building LLM apps that combine powerful LLMs with vector databases, agents and more? If you’re developing with a framework like LlamaIndex or LangChain; an LLM like one from OpenAI or Hugging Face; or using a vector database from Pinecone or Chroma, you will want to learn how to measure the performance and quality of your LLM-based applications using feedback functions. This session explores what feedback functions are, how to use them, and why they can make all the difference. This session covers: The challenges with LLM app development today What’s a feedback function and how does it work? How to put feedback functions to good use as you are developing LLM apps Tracking performance, quality, and cost across LLM app versions Demo of TruLens for LLM Apps, an open source software toolkit that uses feedback functions for evaluating LLM apps