TableTalk

Helicone AI — The Open-source LLM Observability for Developers | Product Hunt

Hello world! 👋👋 Justin here, CEO of Helicone with my co-founder Cole. We are extremely excited to launch Helicone on Product Hunt! 🧊 Background story Before launching Helicone, we developed several projects with GPT-3/GPT-J, including airapbattle.com, tabletalk.ai, dreamsubmarine.com, and debateai.org. For each project, we used a beta version of Helicone which gave us instant visibility into user engagement and result quality issues. As we talked to more builders and companies, we realized they were spending too much time building in-house solutions like this and that existing analytics products were not tailored for LLM workloads. 🧊 Helicone’s differentiators Other startups (LangChain) are working on similar products like LangSmith with high learning curves, rigid pricing structure and closed source, or are slow-moving giants like Datadog which isn’t exactly purpose-built for Large Language Models. 🧊 Our goals Our goal is simple. As a team of developers, we realized that we want a tool that “just works”, and dead easy to start. We focus a ton on the developer experience, by allowing developers to access all observability and monitoring features with just a header. Our goal is to help you surface insights or errors, so you can focus on problem-solving faster. 🧊 What’s new with this launch ⭐ Improved dashboard & request table: we make it faster to segment your data on the dashboard, view responses and relevant monitoring metrics such as cost, latency, and time to first token (TTFT). ⭐ Introducing Sessions: trace your AI agent and RAG chatbot, debug them or refine your responses in different context in our Playground. ⭐ Introducing Prompts & Experiments: iterate on your prompts, experiment with a different dataset or model and evaluate your responses with Scores. ⭐ Scalability & Reliability: since the beginning, we’ve made significant progress towards 100% log coverage, so that all of your logs will be captured. Thanks for supporting us on the launch! We are always excited to hear your feedback. Feel free to message us or “submit an issue” on GitHub, and we will respond / build it. Github: github.com/Helicone/helicone Docs: docs.helicone.ai Open Stats: us.helicone.ai/open-stats Happy developing! ✌️ And thank you [@mwseibel](https://www.producthunt.com/@mwseibel) for hunting us! 00:00 Intro 00:38 One-line Integration 00:51 Flexing 01:07 Dashboard 02:02 Requests 02:36 Sessions for Agents 03:05 Prompt A/B Testing