Segmentation Fault in sqlite3OsFileSize During Database Backup Initialization

---

Understanding the Segmentation Fault in sqlite3OsFileSize During Backup Operations

Root Cause: Invalid File Handle in Backup File Truncation Routine

The segmentation fault occurs when the SQLite library attempts to truncate a database file during a backup operation. The sqlite3OsFileSize function, which retrieves the size of a database file via the Virtual File System (VFS) layer, dereferences a null or invalid file handle pointer. This happens because the backup subsystem invokes sqlite3OsFileSize on a file object that has not been properly initialized or has already been closed. The invalid file handle is often a result of incomplete error handling in scenarios where the backup target database is not fully opened or configured, particularly when using non-default page sizes (e.g., 65,536 bytes) or executing shell-specific commands like .res.

The crash manifests in the backupTruncateFile function, which is part of the internal backup logic. This function attempts to truncate the target database file to match the size of the source database. If the target file handle is invalid, the VFS layer’s xFileSize method (accessed via sqlite3OsFileSize) reads from a null pointer, triggering a segmentation fault. The AddressSanitizer (ASAN) report highlights a read operation at address 0x30, a classic symptom of dereferencing a NULL pointer offset by 48 bytes (e.g., accessing a struct member at offsetof(struct SomeStruct, member) == 48).

Critical Failure Modes: Null File Handles and VFS Layer Assumptions

1. Null File Handle in Backup Context:
   The backup process assumes that the target database file is fully initialized and associated with a valid VFS file handle. If the target database is not opened correctly (e.g., due to a missing filename, insufficient permissions, or a failed sqlite3_open call), the file handle remains NULL. Subsequent operations on this handle, such as sqlite3OsFileSize, crash the process.

2. Race Conditions During Shell Command Execution:
   The SQLite shell (CLI) command .res 0, which switches the output mode to "list" format, can inadvertently interact with internal shell state. In rare cases, this command may trigger an implicit database operation (e.g., auto-opening a temporary database) that conflicts with ongoing backup processes, especially if the backup was initiated without proper synchronization.

3. Page Size Configuration Conflicts:
   Setting a non-default page size (e.g., PRAGMA page_size=65536) alters the database file’s geometry. If the backup process starts before the target database’s page size is validated or synchronized with the source, the VFS layer may reference an incompletely initialized file structure, leading to invalid pointer offsets.

4. Legacy Code Paths in Backup Logic:
   Older SQLite versions (pre-2010) contained logic in backupTruncateFile that did not robustly validate file handles. The bisect results in the forum thread point to a 2010 commit that introduced regression in error handling, which resurfaced in newer versions due to code refactoring or incomplete fixes.

Resolution Strategy: Validating File Handles and Isolating Backup Initialization

Step 1: Patch the Backup Truncation Logic
Apply the fix from SQLite’s official repository (commit 020968f8), which ensures that sqlite3OsFileSize is only called on valid file handles. This involves adding a null-check before invoking the VFS method:

if( pFile ){ 
  rc = sqlite3OsFileSize(pFile, pSize); 
} else { 
  rc = SQLITE_ERROR; 
}

Step 2: Audit Shell Commands for Implicit Database Operations
Review the SQLite shell’s .res command implementation to ensure it does not inadvertently open or close database connections. Modify the shell to use explicit transactions or deferred operations when changing output modes, preventing conflicts with active backups.

Step 3: Enforce Page Size Compatibility Checks
Before initiating a backup, verify that the source and target databases have compatible page sizes. If the target database is new or empty, explicitly set its page size to match the source using PRAGMA page_size before attaching it to the backup process.

Step 4: Instrument the VFS Layer for Debugging
Compile SQLite with debugging flags (-DSQLITE_DEBUG) and enable logging in the VFS layer to trace file handle states. This helps identify uninitialized file objects:

sqlite3_vfs *vfs = sqlite3_vfs_find(NULL);
vfs->xLog = myCustomLogger; // Custom logging function

Step 5: Stress-Test Backup Initialization Under Edge Cases
Simulate the crash scenario using a test harness that:

• Opens a source database with page_size=65536.
• Creates a minimal table (CREATE TABLE t(E)).
• Repeatedly executes .res 0 while initiating backups in parallel threads.
• Monitors for null file handles in sqlite3_backup_step.

Step 6: Update Error Handling in Backup APIs
Modify sqlite3_backup_step to return SQLITE_ERROR immediately if the target database’s file handle is invalid, preventing further operations on a corrupted backup context.

Step 7: Backport Fixes to Legacy SQLite Versions
If using an older SQLite version, manually apply the null-check patch to backupTruncateFile and recompile. Validate with the provided test case to confirm resolution.

---

Final Code Fix and Validation Workflow

The core fix involves adding a null-check in backupTruncateFile (located in sqlite3.c):

// Before
rc = sqlite3OsFileSize(p->pDestDb->pVfs, pDest, &nDestPage);

// After
if( pDest ){
  rc = sqlite3OsFileSize(p->pDestDb->pVfs, pDest, &nDestPage);
} else {
  rc = SQLITE_ERROR;
}

Validation Steps:

1. Recompile SQLite with ASAN and debug symbols:

   export CFLAGS="-fsanitize=address -g -DSQLITE_DEBUG"
   ./configure && make
   

2. Execute the reproducer script:

   PRAGMA page_size=65536;
   CREATE TABLE t(E);
   .res 0;
   

3. Confirm no segmentation faults occur and ASAN reports clean output.

This comprehensive approach addresses the null file handle scenario, fortifies backup initialization, and prevents similar crashes in edge-case configurations.