Database File Descriptor Leak During Backup OOM Testing in SQLite 3.32.2

File Descriptor Retention in Backup Operations Under Out-of-Memory Conditions

The core problem revolves around incomplete cleanup of database file descriptors when the backup_malloc.test encounters simulated out-of-memory (OOM) conditions during SQLite backup operations. This manifests as test failures in dependent test suites such as mallocAll.test due to unclosed database connections persisting across test boundaries. The issue is observed in SQLite version 3.32.2 but reportedly resolved in version 3.37.0. The original backup_malloc.test script attempts to validate SQLite’s behavior under memory allocation failures during backup processes but fails to explicitly close database handles (db and db2) after finalizing the backup object. Subsequent test cases relying on fresh database environments detect residual file descriptors, leading to false positives in resource leak detection mechanisms.

Database backups in SQLite utilize the sqlite3_backup API to copy data between source and destination databases. The test framework employs fault injection via do_faultsim_test to simulate OOM scenarios, validating error handling paths. When memory allocation failures occur during backup steps (B step 50), the backup process may terminate abruptly without executing standard cleanup routines. While the test explicitly finalizes the backup object with B finish, it does not account for scenarios where database connections remain open due to interrupted execution flows. This creates dangling file handles that persist beyond the test’s execution scope.

The coupling between test cases arises from SQLite’s file descriptor management strategy. Most test suites assume exclusive access to database files and rely on deterministic resource cleanup between tests. When a prior test leaves open connections, subsequent operations on the same database files (e.g., malloc-1.X subtests) encounter locked files or unexpected states, triggering integrity check failures. The problem is exacerbated in OOM simulation environments where exception handling paths may bypass conventional cleanup sequences.

Incomplete Resource Cleanup During Backup Process After Simulated Memory Allocation Failures

The primary technical root cause lies in the interaction between SQLite’s backup API and the Tcl test harness’s error propagation logic. When B step 50 returns SQLITE_NOMEM or SQLITE_IOERR_NOMEM, the test script throws an "out of memory" error using Tcl’s error command. This bypasses the standard control flow where database closure would typically occur after backup finalization. SQLite’s internal cleanup mechanisms for database connections are not automatically triggered when exceptions escape the test harness’s execution context, leading to unclosed file descriptors.

SQLite manages database connections as independent objects with explicit lifecycle controls. The sqlite3_backup object (B) is a transient structure that facilitates incremental data transfer between connections but does not own the underlying database handles. Finalizing the backup with B finish releases resources associated with the backup operation itself but leaves source and destination databases (db and db2) unaffected. This design requires callers to explicitly close database connections after backup completion—a step omitted in the original test script when OOM errors occur.

The fault simulation framework (do_faultsim_test) complicates resource management by injecting failures at strategic points. In the -body phase, OOM errors thrown during B step 50 disrupt the expected sequence of operations, preventing execution from reaching the -test block’s database closure calls. Although faultsim_test_result and faultsim_integrity_check attempt to validate post-error states, they do not enforce connection cleanup when exceptions propagate outside the test’s error-catching mechanisms.

Another contributing factor is the test environment’s handling of database file locks. SQLite employs file locking primitives (e.g., flock, dot-file locks) to enforce concurrent access restrictions. Open database connections maintain active locks on underlying files, preventing other processes or test cases from obtaining exclusive access. When backup_malloc.test fails to close connections, subsequent tests attempting to delete or overwrite test.db/test2.db encounter "file in use" errors, violating test isolation principles.

Ensuring Proper Database Closure and Decoupling Interdependent Test Cases

Step 1: Validate File Descriptor Status After Test Execution
Use operating system utilities (e.g., lsof on Unix-like systems, Process Explorer on Windows) to monitor open file handles during test execution. Insert diagnostic output in the test script to log active database connections before and after each test phase. For SQLite 3.32.2, modify the -test block to include:

puts "Open DB handles: [info commands db*]"

This reveals whether db and db2 persist beyond the test’s execution. Compare results between normal runs and OOM-simulated scenarios to identify orphaned connections.

Step 2: Enforce Database Closure in Exception Handlers
Wrap the backup step and subsequent operations in a try/finally construct to guarantee database closure regardless of error conditions. Modify the -body and -test blocks as follows:

-body {
  try {
    set rc [B step 50]
    if {$rc == "SQLITE_NOMEM" || $rc == "SQLITE_IOERR_NOMEM"} {
      error "out of memory"
    }
  } finally {
    # Ensure intermediate cleanup
    catch {B finish}
    catch {db close}
    catch {db2 close}
  }
}

This ensures that even if an OOM error is thrown, the finally block executes to close connections. However, this approach may interfere with the test’s original purpose of validating memory leak detection, as premature closure could mask resource retention issues.

Step 3: Isolate Test Cases via Forced Connection Closure
Modify the test harness’s setup and teardown procedures to forcibly close all database connections before and after each test. Adjust the -prep block to include aggressive cleanup:

-prep {
  catch {db close}
  catch {db2 close}
  sqlite3 db :memory:
  sqlite3 db2 :memory:
  forcedelete test2.db
  # ... rest of preparation code
}

Using in-memory databases during setup minimizes residual file artifacts. For persistent database tests, implement a global registry of open connections and iterate through them during teardown.

Step 4: Upgrade to SQLite 3.37.0+ for Internal Resource Management Fixes
The SQLite development team indicates that version 3.37.0 resolves this issue without test script modifications. Analyze the commit history between 3.32.2 and 3.37.0 to identify relevant fixes. Key areas include:

• Backup API error recovery: Enhancements to sqlite3_backup_finish() ensuring proper resource release even when memory allocation fails mid-operation.
• Connection lifecycle management: Internal checks that automatically close dormant database handles when their parent objects are destroyed.
• Test framework improvements: Stricter enforcement of test isolation in the faultsim harness, including automatic connection cleanup after simulated failures.

Step 5: Decouple Test Dependencies via Sandboxed Environments
Restructure test suites to operate within isolated namespaces or temporary directories. For each test case:

1. Generate unique database filenames using process IDs or timestamps.
2. Redirect database connections to temporary filesystems.
3. Implement custom error handlers that capture and suppress exceptions without affecting global state.

Example modification for backup_malloc.test:

forcedelete test_$pid.db test2_$pid.db
sqlite3 db test_$pid.db
sqlite3 db2 test2_$pid.db

This prevents cross-test contamination by ensuring each run operates on distinct files.

Step 6: Audit SQLite Configuration for Lazy File Closing
Review SQLite’s compile-time options and runtime settings influencing file descriptor management. Key configurations:

• SQLITE_FCNTL_CLOSEPOLL: Adjusts how aggressively SQLite closes underlying file handles.
• SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE: Disables checkpoint-on-close behavior that might retain locks.
• SQLITE_OPEN_URI: Enables URI filenames with parameters like mode=ro to prevent exclusive locking.

Recompile SQLite with -DSQLITE_DEBUG_FILE_CLOSE to enable diagnostic logging of file closure events, aiding in pinpointing leak sources.

Step 7: Implement Connection Pooling with Mandatory Teardown
For test suites requiring persistent connections, introduce a connection pool with strict teardown policies. Wrap database handles in managed objects that automatically invoke sqlite3_close_v2() when references are dropped. In Tcl, leverage object-oriented extensions or custom reference-counted wrappers:

proc managed_sqlite3 {name file} {
  upvar $name db
  sqlite3 db $file
  trace add command db delete "apply {args} { catch { db close }}"
}

This ensures that when the db command is deleted (e.g., during namespace teardown), the underlying connection is closed.

Step 8: Enhance Fault Injection Harness to Track Resources
Modify the do_faultsim_test infrastructure to log resource acquisition and release. Instrument the Tcl interpreter to intercept sqlite3 and sqlite3_close commands, maintaining a real-time list of open connections. Augment fault simulation reports with resource leakage statistics, failing tests that leave dangling handles.

Step 9: Validate Cross-Version Behavior via Differential Testing
Execute the original and modified test scripts against multiple SQLite versions (3.32.2, 3.36.0, 3.37.0) to identify behavioral changes. Use differential analysis to correlate specific code changes with resolved issues. Focus on commits related to:

• sqlite3_backup_remaining() error handling
• Memory context management during backup interruptions
• VFS layer modifications for file locking/unlocking

Step 10: Establish Test-Specific Resource Quotas
Configure the test environment to enforce strict limits on file descriptors and memory usage. Use OS-level controls (e.g., ulimit -n on Unix) to cap the number of open files. Tests exceeding these quotas will fail immediately, highlighting resource leakage without relying on secondary effects in subsequent test cases.

By systematically addressing the lifecycle management of database connections under error conditions and decoupling test case dependencies, developers can eliminate file descriptor leaks during OOM testing. These solutions balance the need for rigorous fault simulation with the imperative of maintaining test isolation, ensuring accurate detection of memory management issues without false positives from residual state.