Understanding local government dynamics is messy and complicated. When Sunil Rajaraman ran for city council in Northern California, he realized the challenges that citizens faced in finding and understanding information about local issues. He started leveraging AI to generate summaries of city meeting notes, to enhance his own understanding of city dynamics. What started as a personal efficiency exercise quickly turned into service offerings for city governments and real estate developers. Hamlet pairs AI with human oversight to make public information easier to understand.
Hamlet partnered with Focused because of their LangChain & LangGraph expertise. While Hamlet had a talented engineering team, they were looking to build AI expertise in-house. Focused assigned full-time AI developers to not only build products for Hamlet, but also to enable their existing engineering team.
Speaker Identification and AI Accuracy Powered by LangChain and LangSmith
Speaker Search is a product that pinpoints quotes from local officials related to specific issues from public meeting transcripts. The search feature helps teams in commercial real estate, renewable energy, retail, and even political advocacy quickly find relevant discussions in public meetings, making it easier to navigate long transcripts and video recordings and surface the information that matters.
Imagine that you are a real estate developer looking to purchase a parcel of land. Your team spends hours sitting in long local city council meetings in order to gather key information that is critical for your development plan to get approved. That’s where Speaker Search can help – it saves you time. Instead of spending hours in local meetings, or pouring over transcripts, you can easily find the topics you care about using the search interface.
While building Speaker Search, Hamlet's engineering team was struggling to evaluate the accuracy of output; they had no insight into whether or not the application was performing. Focused recommended the LangChain ecosystem. The team used LangSmith to create custom evaluator functions to measure the accuracy of assigning the right speakers to the right quotes. By implementing these functions with LangSmith, the team gained essential visibility into RAG pipeline performance. Using custom evaluators, the team achieved over 90% accuracy.
If we didn’t use LangSmith, it would have been impossible to know if we were improving accuracy. Without LangSmith, you are just kind of guessing.
- Sarah Kainec, Lead Engineer at Focused
LangChain and LangSmith reduced validation cycles from months to days, providing traceability at each processing step of the data pipelines. LangSmith's evaluation capabilities allowed the team to establish baseline metrics, track incremental LLM improvements, and optimize prompt engineering strategies
Hamlet partnered with Focused to productionize their product offerings and build scalable AI solutions that could grow with the company. Within 3 months of working with Focused, Hamlet was able to take product offerings like Speaker Search to market. Hamlet’s unique civic-minded AI solutions secured them funding to fuel their next phase of expansion.
When I look at where our product and engineering effort stands today compared to pre-Focused, the impact is readily apparent. They’ve been the extra rocket booster we needed to achieve our dream of building magical, AI-driven products.
— Mike Grafton, Head of Engineering
