AddressSanitizer SIGABRT in dbdataNext() Due to Negative-Size-Param During .recover

Database Recovery Process Triggering Invalid Memory Copy Operation

The core issue manifests as an AddressSanitizer-detected SIGABRT during execution of SQLite’s .recover command, specifically when processing a malformed database file. The failure occurs in the dbdataNext() function at shell.c line 11991 during a memcpy operation attempting to copy -1 bytes. This indicates fundamental corruption in either the database structure or the recovery logic’s handling of damaged files. The error chain involves multiple SQLite internals: the VDBE execution engine calls into the recovery module’s iterator, which ultimately passes invalid parameters to memory manipulation routines. Key characteristics include allocation patterns (4200-byte region allocated via SQLite’s memory subsystem) and stack trace evidence showing the recovery process transitioning through sqlite3_exec, shellExec, and recoverDatabaseCmd.

Database file analysis reveals a minimal valid SQLite 3 header followed by extensive zero-byte padding. The 4200-byte allocation corresponds to SQLite’s default page size configuration (4096 bytes) with additional overhead. The negative size parameter suggests either underflow in size calculation or corruption of critical header fields that define page payload sizes. Of particular significance is the interaction between the recovery engine’s page parsing logic and the database’s physical storage layout – when processing damaged pages, missing validation checks may allow invalid size computations to propagate through the system.

Root Causes in Page Structure Parsing and Recovery Logic

Three primary factors contribute to this failure mode:

1. Corrupted Page Header Metadata
   Database pages contain header information specifying payload sizes and cell pointers. If the recovery process encounters pages where the "number of bytes of payload" field (bytes 5-7 of page header) contains invalid values due to file corruption, subsequent size calculations may underflow. The dbdataNext() function’s logic for extracting cell data relies on these header values without sufficient sanity checking when operating in recovery mode.

2. Incomplete Error Handling in Recover Virtual Table
   The dbdata virtual table implementation used by .recover prioritizes data extraction over stability. When iterating through pages via dbdataNext(), the code path assumes pages conform to basic structure expectations. Malformed pages containing cell pointers that reference offsets beyond physical page boundaries (as seen in the test0.db file) cause invalid memory access patterns that only manifest during actual byte copying operations.

3. AddressSanitizer-Exposed Undefined Behavior
   The specific crash triggers depend on Clang’s AddressSanitizer detecting the invalid memcpy parameter. In non-instrumented builds, this might silently corrupt memory or return garbage data rather than crashing immediately. The instrumentation converts what would normally be undefined behavior into a fatal error, revealing weaknesses in the recovery engine’s input validation layer.

Resolution Strategy for Damaged Database Recovery

Step 1: Validate Database Structural Integrity
Before attempting recovery, verify basic database consistency:

sqlite3 test0.db "PRAGMA integrity_check"

For severely damaged files, this may fail outright. If the file returns "okay" but crashes on recovery, proceed to deeper structural analysis.

Step 2: Analyze Database Header and Page Layout
Use hexadecimal inspection to examine the first 100 bytes of the database:

hexdump -C -n 100 test0.db

Validate the SQLite header magic string "SQLite format 3\000" at offset 0. For the provided test0.db, the Base64 decoding shows a valid header followed by all-zero content, indicating a database file that was never properly initialized or suffered catastrophic storage failure.

Step 3: Manual Page Inspection
Extract individual database pages for analysis using SQLite’s internal page numbering. For page 1 (the first page after the header):

dd if=test0.db bs=4096 skip=1 count=1 2>/dev/null | hexdump -C

In normal databases, this would reveal the schema root page. All-zero content suggests complete lack of database initialization.

Step 4: Safe Recovery Execution Parameters
When dealing with zero-filled or corrupted files, limit the recovery scope:

sqlite3 -cmd ".parameter init" \
        -cmd ".parameter set @filename test0.db" \
        -cmd ".recover --ignore-freelist --ignore-errors" \
        recovered.db

The --ignore-freelist prevents processing of potentially corrupted page reuse information, while --ignore-errors skips pages with validation errors rather than aborting.

Step 5: Custom Build with Debugging Instrumentation
Recompile SQLite with recovery-specific debugging:

CFLAGS="-DSQLITE_DEBUG -DSQLITE_ENABLE_DBDATA -DSQLITE_DIRECT_OVERFLOW_READ" \
LDFLAGS="-lasan" \
./configure --enable-debug
make sqlite3

Run recovery with tracing enabled:

ASAN_OPTIONS=detect_leaks=0 ./sqlite3 -cmd ".trace stdout" -cmd ".recover" test0.db

This outputs low-level page processing details preceding the crash.

Step 6: Patching dbdataNext() Validation
Modify the SQLite shell.c source to add boundary checks in dbdataNext():

/* Original code causing crash */
memcpy(p->buf, &p->pData[p->iOff], nByte);

/* Patched version with validation */
if( nByte < 0 || p->iOff + nByte > p->nData ){
  sqlite3_log(SQLITE_ERROR, "Invalid nByte %d at offset %d", nByte, p->iOff);
  return SQLITE_CORRUPT;
}
if( nByte > 0 ){
  memcpy(p->buf, &p->pData[p->iOff], nByte);
}

This prevents negative size copies and logs diagnostic information about corrupt regions.

Step 7: Alternative Recovery Methods
When standard recovery fails, use low-level data extraction:

echo "SELECT writefile('table_data', data) FROM sqlite_dbpage WHERE pgno=1;" | \
sqlite3 -cmd ".open test0.db" -cmd ".output /dev/null"

This uses the sqlite_dbpage virtual table to directly access raw page content, bypassing the recovery engine’s parsing logic.

Step 8: Database Header Reconstruction
For files with valid headers but corrupted content, create a new database with matching page size and encoding:

sqlite3 new.db "PRAGMA page_size=4096; VACUUM;"

Then attempt to transplant individual pages from the corrupted file using the sqlite3_dbdata extension’s low-level interface.

Step 9: Leveraging WAL-based Recovery
If the database used Write-Ahead Logging (WAL), recover data from the WAL file:

strings test0.db-wal | grep -a -E '^[a-zA-Z0-9\/+]{20,}=' | base64 -d > wal_data

This extracts base64-encoded blobs that may contain recent database changes not present in the main file.

Step 10: Post-Crash Memory Analysis
When AddressSanitizer reports allocation details (0x6210000524c4 in 4200-byte region), correlate this with SQLite’s memory subsystems:

// Match allocation size to SQLite components
#define DBDATA_PAGE_BUFFER_SZ 4200  // Common in page recovery buffers

Use debug symbols to identify which subsystem allocated the corrupted buffer (sqlite3MemMalloc in the stack trace indicates general-purpose allocation).

Final Mitigation Approach
For production environments dealing with unreliable storage:

1. Implement periodic PRAGMA incremental_vacuum to maintain contiguous page allocation
2. Use PRAGMA page_size_checks=ON to enable runtime page validation
3. Combine .recover with custom SQL functions to filter invalid rows during extraction
4. Maintain WAL files with checksum validation using PRAGMA wal_checkpoint(TRUNCATE)

The fundamental fix requires enhancing SQLite’s recovery module to validate page header fields before memory operations. Users should upgrade to SQLite versions containing commit 39a30c20 (2023-01-03) which added additional bounds checking in the dbdata module. For legacy versions, manual backporting of these validation checks becomes essential when processing untrusted or damaged database files.