Developers, Ditch the Black Box: Welcome to Continue Remember flipping through coding manuals? Those quickly became relics with the rise of Google and Stack Overflow, a one-stop shop for developer queries. Now, the
Benchmarking GPT4-o Last week, OpenAI announced its latest LLM, GPT-4o, which follows GPT-4 Turbo. In this blog, we benchmark GPT4-o using a new benchmarking toolkit called "
Columnar File Readers in Depth: Scheduling vs Decoding We've been working on readers / writers for our recently announced Lance v2 file format and are posting in-depth articles about writing a high
New Funding and A New Foundation for Multimodal AI Data I’ve been building data tooling for ML/AI for almost two decades, beginning with being one of the original contributors to pandas. My cofounder
Columnar File Readers in Depth: Parallelism without Row Groups Recently, I shared our plans for a new file format, Lance v2. As I'm creating a file reader for this new format I
Benchmarking Cohere Rerankers with LanceDB Reranking is a process of re-arranging the results of a retriever based on some metric that can be independent of the retrieval scores. In this
Tokens Per Second is NOT All You Need Goodhart’s Law: When a measure becomes a target, it ceases to be a good measure We're excited to have a guest post
Accelerating deep learning workflows with Lance Lance is a columnar data format that is easy and fast to version, query and train on. It’s designed to be used with images,
Practical introduction to Adaptive-RAG Traditional LLMs provide answers based on a fixed knowledge database on which they are trained. This limits their ability to respond with current or specific
Chunking techniques with Langchain and LlamaIndex In our last blog, we talked about chunking and why it is necessary for processing data through LLMs. We covered some simple techniques to perform
Lance v2: A columnar container format for modern data Why a new format? Lance was invented because readers and writers for existing column formats did not handle AI/ML workloads efficiently. Lance v1 solved
Convert any Image dataset to Lance In our previous article, we explored the remarkable capabilities of the Lance format, a modern, columnar data storage solution designed to revolutionize the way we
