LLM insights (beta)

Last updated:

|Edit this page

We've teamed up with various LLM platforms to track metrics for LLM apps. This makes it easy to answer questions like:

  • What are my LLM costs by customer, model, and in total?
  • How many of my users are interacting with my LLM features?
  • Are there generation latency spikes?
  • Does interacting with LLM features correlate with other metrics (retention, usage, revenue, etc.)?

Currently, we support integrations for the following platforms:

How to install

Langfuse

  1. First add Langfuse Tracing to your LLM app (Quickstart).
  2. In your Langfuse dashboard, click on Settings and scroll down to the Integrations section to find the PostHog integration.
  3. Click Configure and paste in your PostHog host and project API key (you can find these in your PostHog project settings).
  4. Click Enabled and then Save.

Langfuse events will now be exported into PostHog once per day.

Helicone

  1. Sign up for Helicone and add it to your app.
  2. Similar to how you set Helicone-Auth header when configuring your LLM client, add two new headers Helicone-Posthog-Key and Helicone-Posthog-Host with your PostHog host and API key (you can find these in your PostHog project settings):
client = OpenAI(
api_key="your-api-key-here", # Replace with your OpenAI API key
base_url="https://oai.hconeai.com/v1", # Set the API endpoint
default_headers= {
"Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
"Helicone-Posthog-Key": "<ph_project_api_key>",
"Helicone-Posthog-Host": "<ph_client_api_host>",
}
)

Helicone events will now be exported into PostHog as soon as they're available.

Dashboard templates

Once you've installed an integration, dashboard templates help you quickly set up relevant insights. Here examples for Langfuse and Helicone.

To create your own dashboard from a template:

  1. Go the dashboard tab in PostHog.
  2. Click the New dashboard button in the top right.
  3. Select LLM metrics – [name of the integration you installed] from the list of templates.

Questions?

Was this page useful?

Next article

Autocapture

Note: Autocapture is available for our JavaScript Web , React , and React Native SDKs. PostHog can capture frontend events automatically using autocapture, such as any click , change of input , or submission associated with a button , form , input , select , or textarea . This means you don't need to manually instrument tracking for individual components, links, buttons, or other parts of your product. Autocapture is enabled by default . Disabling autocapture You can also disable…

Read next article