New Relic brings instant observability to OpenAI’s GPT-3 Posted on : Mar 15 - 2023

Observability tooling company New Relic Inc. said today it’s launching a first-of-its-kind machine learning operations capability that makes it easy for engineering teams to monitor the performance of OpenAI LLC’s GPT-3 effectively.

The company has announced the availability of a new OpenAI quickstart in its Instant Observability catalog of plugins. With just a couple of lines of code, teams can monitor OpenAI completion queries while tracking both performance cost metrics in real time within New Relic. According to the company, it’s all about helping companies to tap the benefits of new artificial intelligence technologies such as GPT-3 while balancing business needs versus costs.

New Relic sells software that companies can use to monitor their applications, DevOps environments and the infrastructure that powers them. Its Instant Observability offering is a catalog that contains more than 400 quickstarts, or prebuilt bundles of observability tools developed by experts and reviewed by New Relic. They can be installed in just a couple of clicks, enabling almost instant observability for any support service or product.

The need for GPT-3 observability is becoming more pressing due to the buzz around the ChatGPT chatbot that has drawn attention to the incredible capabilities of generative AI. OpenAI’s ChatGPT is able to respond to almost any question in a humanlike manner, and it has proven to be so realistic and useful — if sometimes wrong — that Google LLC is seriously worried it might threaten its dominance of online search.

Microsoft Corp. recently invested $10 billion in OpenAI and announced plans to integrate ChatGPT with its Bing search engine. Meanwhile, dozens of other companies have begun using OpenAI’s GPT-3 model to power applications such as help desk tickets and customer service chatbots, and for content and image generation.

With more companies using GPT-3, there’s a growing need for observability tools that can help them optimize usage and reduce costs, and that’s what New Relic is hoping to tap into with today’s release. With just two lines of code, it says, users can automatically import a monitor module from the nr_openai_monitor library, generating a simple dashboard that shows various GPT-3 performance metrics. Its capabilities include real-time cost tracking, plus metrics such as average response times and other performance metrics around GPT-3 requests that can help teams to use the service more efficiently. View more