Community contributions
💡
Dspy now works with LanceDB. LanceDB helps DSPy by quickly storing and retrieving information. This lets DSPy find relevant details faster, making prompts more accurate. This integration ensures that prompts meet user needs, leading to a better writing experience.
💡
Prefect released ControlFlow 0.10, a Python framework for building agentic AI workflows. This release introduces a major new feature: persistent memory for AI agents, and ControlFlow memory now integrates with LanceDB.
💡
A heartfelt thank you to our community contributors of lance and lancedb this month: @ankitvij-db @jiachengdb @dentiny @tonyf @dsgibbons @rithikJha @sayandipdutta @gagan-bhullar-tech @PrashantDixit0 @mattbasta @jameswu1991 @bllchmbrs @antoniomdk @ousiax @rahuljo @philz
💡
Check out VARAG, Vision first RAG Engine project powered by LanceDB
Good reads
- Late interaction & efficient Multi-modal retrievers need more than a vector index. In late interaction rankers, the query embedding interacts with all document embeddings using a MaxSim operator that calculates cosine similarity. The results from these calculations are then summed across the query terms.
ColPali is a visual retriever model that combines:
- PaliGemma: This model integrates the vision encoder with the Gemma-2B language model. It also includes projection layers that convert language model inputs into 128-dimensional vectors.
- A late interaction mechanism inspired by ColBert.
- Multi document agentic RAG: A walkthrough. Agentic RAG is a smarter version of regular RAG systems. Instead of just pulling up information, it carefully analyzes the question and uses smart strategies to give a better, more thoughtful answer.
Event recap
- The inaugural Foundation of Compound AI Meetup, hosted by LanceDB and Databricks on September 24th, was a great success! A special thanks to our speakers, Chang, Erik, and Kobie. Stay tuned for our next event in Oct 22nd!
- In Octber, Compound AI Meetup series will take place in Mountain View on Oct 22nd. You will hear from the Cofounder/CEO of Continue, Cofounder/CTO of LanceDB, and Engineering manager from Databricks.
Latest releases
- Starting with Lance v0.18.0, new datasets will now use Lance file format v2.0 by default. This brings much faster scans as well as null support to all data types. This change will propagate to the next LanceDB SDK release in October.
