Introducing: Findsight

I just launched Findsight AI, available for free on findsight.ai. Findsight helps you discover non-fiction sources like books, articles, papers, and more by comparing their core claims and ideas.

Join more than 10,000 users, and try it right now on findsight.ai, no sign-up required!

1. What is Findsight?
2. What’s unique about Findsight?
3. How does it work?
4. Notes

What is Findsight?

Findsight is a search engine that lets you explore non-fiction content based on claims — the key points made by sources. You can discover and compare claims from various sources, see how authors tackle issues differently, and navigate to related claims to create your own learning journey. You can also find links to the original book or article. This approach is called syntopical reading.

Findsight combines traditional text and tag search with AI-powered filters that let you find related claims, and even claims that answer questions.

Here’s an example of a result:

Clicking on the title of a work expands this context view. In addition to the highlighted claims (the ones matching the search), you’ll get to see a few more claims surrounding it to get a sense of the narrative of the source. The real fun starts with the explore feature: click the arrow to jump from a claim to related claims. There’s always something new to learn by starting from Random and following a new path.

Findsight’s AI-powered filters enable you to find claims by meaning. Here’s an example of the “answer” filter identifying data that supports the user’s question:

You can find a detailed explanation of Findsight’s search capabilities in the Search Guide on findsight.ai.

What’s unique about Findsight?

Generative AI has inspired exceptional knowledge retrieval tools, but Findsight’s core priorities make it unique:

Top-notch summarization: Findsight employs cutting-edge large language models for high-quality summarization, extracting key claims while minimizing AI-generated inaccuracies — faithfully reflecting the source’s intent.

Robust hybrid search: With a custom search engine tailored for non-fiction, Findsight melds traditional text search and AI-powered retrieval to deliver highly relevant results in most cases.

Discovery, not consumption: Findsight isn’t about consumption; it’s a launchpad to find insights and collect books or articles to explore further. Since a collection of claims can’t capture a source’s entire content, we limit the context returned in searches.

That said, here’s a quick list of related services¹ that you can combine with Findsight to level up your learning:

• Elicit: AI research assistant with a rich search interface for common attributes of scientific papers
• Shepherd: High-quality human-curated book suggestions with a sprinkle of AI-powered topic exploration
• Basmo: Polished app experience for interacting with books using AI chatbots, and share knowledge with other users
• Librarian AI: Discover new books based on ones you already have
• ChatGPT Retrieval: Early-stage project to enable knowledge retrieval with OpenAI’s ChatGPT

Disclaimer: Findsight/Summarity has no affiliation with any of the projects mentioned above. Follow links at your own risk. Not an endorsement.

How does it work?

This is a highly condensed overview of the tech stack behind Findsight - you don’t need to know this to use it. If you’re interested in the implementation details on any of the parts below, drop me an email and I’ll consider writing a separate post.

Findsight’s AI features are powered by OpenAI’s GPT3, GPT3.5 completion models and Ada embeddings, combined with fine-tuned Azure LER/NER models.

Beyond those services, Findsight is a depencency-free application written in Nim. The larger components are:

• a custom OpenAI client with semantic caching and automatic HyDE prompting
• an extraction module that automatically cleans and chunks input text, and adapts prompts and chunk size based on self-supervision
• a custom KNN / vector-similarity index based on SuperBit encoding and using Weave for data-parallel execution
• a hybrid search engine that uses three stages with content-dependent boosting:
  • Alpha: binary attribute filtering (e.g. tags)
  • Beta: BM25-ranked full-text search using a custom search syntax (SafeTS5) that compiles to provably safe (limited complexity) FTS5 queries (uses NPEG)
  • Gamma: a ranked similarity search based on the KNN index, using potentially present Alpha and Beta stages as a pushdown filter
• a book metadata engine that identifies and normalizes ISBN catalog data based on book content
• a mummy-based server using SQLite for persistent storage

Notes