[MayCXC]

A vec0 full scan reads each row's vector through vec0_get_vector_data(), which looks up the row's chunk, opens the chunk's vectors blob, reads one vector and closes the blob again. A chunk's blob holds chunk_size vectors (3 MiB for 1024 float[768]), and each sqlite3_blob_open() seeks through the blob's overflow pages to reach the row's offset, so one read costs more the larger the chunk, and the opens dominate the scan.

On a 10,000-row float[768] table in a database file:

select sum(length(embedding)) from v;                  -- 844 ms
select sum(vec_distance_cosine(embedding, ?)) from v;  -- 807 ms

The scan already visits rows in (chunk_id, chunk_offset) order. Its rowid statement now also returns each row's chunk_id and chunk_offset, and the cursor keeps one blob handle per vector column, moved to the next chunk with sqlite3_blob_reopen(). A write that replaces the chunk's row expires the handle (sqlite3_blob_read() answers SQLITE_ABORT); the read is then retried once on a newly opened handle, so the scan reads the chunk as it is now. With this change the two queries take 11 ms and 30 ms. Measured again on an idle machine (medians of 3 interleaved runs of 7 queries), they take 10.2 ms instead of 778.9 ms and 28.4 ms instead of 772.0 ms; in memory, 3.5 ms instead of 95.8 ms and 23.7 ms instead of 111.6 ms.

tests/test-fullscan.py checks every row's vectors against point lookups across chunks with deleted rows and two vector columns, and reads a chunk whose shadow row is rewritten in the middle of a scan. Both pass on main too, since the results are unchanged; with the retry removed, the second fails on the first row read after the rewrite. The full suite and make test-asan pass. The commit adds a ### Changed entry under [Unreleased] in CHANGELOG.md.

The same change is open on vlasky/sqlite-vec as vlasky#17 and upstream as asg017#336, where it reads flat columns by chunk and leaves rescore, IVF and DiskANN columns, which keep their vectors in tables of their own, to be read row by row.

[mceachen]

Merged, thanks. Review turned up three problems, fixed in f793711:

• The scan's chunk_id and chunk_offset come from the sort SQLite does when the scan starts: v_rowids has no index on them, so EXPLAIN QUERY PLAN shows USE TEMP B-TREE FOR ORDER BY. A delete, insert or optimize on the same connection during the scan then read stale positions. A deleted row came back with a zeroed vector, a row whose slot was reused came back with the new row's vector, and optimize failed with opening blob failed. The fix looks up each row's position with vec0_get_chunk_position(), which cost about 4 ms on the 10,000-row float[768] scan.
• A failed sqlite3_blob_reopen() was retried on a new handle, which dropped an interrupt raised by a progress handler. Only SQLITE_ABORT is retried now.
• With extended result codes on, a rollback that changed the schema makes sqlite3_blob_read() and sqlite3_blob_reopen() return SQLITE_ABORT_ROLLBACK, which rc == SQLITE_ABORT misses. The checks now compare rc & 0xff.

vlasky#17 and asg017#336 have the same code.