The Future of Open Source Table Formats: Apache Iceberg and Lance As the scale of data continues to grow, open-source table formats have become essential for efficient data lake management. Apache Iceberg has emerged as a
Lance File 2.1: Smaller and Simpler Almost a year ago I announced we were going to be embarking on a journey to build a new 2.0 version of our file
Designing a Table Format for ML Workloads In recent years the concept of a table format has really taken off, with explosive growth in technologies like Iceberg, Delta, and Hudi. With so
November Feature Roundup Lance On November 13th, we released Lance 0.19.2. This release includes several new features and improvements, including: * Flexible handling of data for inserts
Columnar File Readers in Depth: Backpressure Streaming data applications can be tricky. When you can read data faster than you can process the data then bad things tend to happen. The
Lance v0.16.1 Feature Roundup In Lance v0.16.1 we introduced several new features, implemented by a combination of LanceDB engineers and community contributors. Lance is an OSS project
The case for random access I/O One of the reasons we started the Lance file format and have been investigating new encodings is because we wanted a format with better support
Lance v0.15.0 Lance 0.15.0 introduces several experimental features, marking major milestones on important projects in our library. It exposes the first public APIs for full-text
Streamlining Our SDKs In the fast-evolving landscape of software development, maintaining multiple SDKs in various languages can be a challenging endeavor, especially for a small startup! Each bespoke
Columnar File Readers in Depth: APIs and Fusion The API used to read files has evolved over time, from simple "full table" reads to batch reads and eventually to iterative "
Lance v2 is now in Beta We've been talking for a while about a new iteration of our file format. We're pleased to announce that the new
Columnar File Readers in Depth: Scheduling vs Decoding We've been working on readers / writers for our recently announced Lance v2 file format and are posting in-depth articles about writing a high
