SQLite3 CLI Crash on PRAGMA hard_heap_limit=1 with Null Pointer Dereference

Crash Triggered by PRAGMA hard_heap_limit=1 Followed by Invalid Statement

Issue Overview: CLI Crash Due to Null Pointer Dereference During Error Context Handling

The SQLite3 command-line interface (CLI) crashes when executing a specific sequence of statements:

1. PRAGMA hard_heap_limit=1 (sets the maximum heap memory limit to 1 byte).
2. A subsequent line containing 0 (interpreted as an invalid SQL statement).

This crash manifests as a segmentation fault (SEGV) due to a null pointer dereference in the shell_error_context() function. The stack trace reveals that the CLI attempts to read a null pointer (zCode = 0x0) while generating an error message for the failed PRAGMA command. The undefined behavior sanitizer (UBSAN) output explicitly flags this as a "load of null pointer of type ‘char’".

Key Observations from the Discussion:

• The crash occurs in SQLite3 versions containing commit 416602a851 (introduced in late 2021) but not in earlier versions.
• The bug is reproducible on Linux (x86-64 Ubuntu 20.04) using GCC 9.4.0 with debug flags enabled.
• Other platforms (Windows, macOS, WASM) report Error: out of memory followed by graceful termination instead of a crash.
• The crash requires executing the PRAGMA statement followed by an invalid statement (0) in the same input batch (e.g., via a script file).

Technical Context:

• The PRAGMA hard_heap_limit=1 forces an immediate out-of-memory (OOM) error by restricting the heap to 1 byte.
• The CLI’s error-handling logic (shell_error_context()) attempts to highlight the problematic SQL code snippet but fails when zCode (a pointer to the SQL text) is null.
• The commit 416602a851 modified error reporting in the CLI, inadvertently introducing a scenario where zCode is dereferenced without validation.

Root Causes: Invalid Assumptions in Error Reporting Logic

1. Null Pointer Dereference in shell_error_context()
The function shell_error_context() assumes that zSql (the input SQL string) is always valid and non-null. However, after an OOM error, memory allocation failures may corrupt or invalidate pointers, causing zCode (derived from zSql) to become null. The code attempts to access zCode[iOffset] without checking for nullity, leading to a crash.

2. Interaction Between OOM and Subsequent Statements
The CLI processes input line-by-line. When the PRAGMA command triggers an OOM, the CLI’s internal state (including the database connection and error buffers) becomes inconsistent. The subsequent 0 (invalid SQL) forces the CLI to generate an error message, but the OOM leaves insufficient memory to format the error context correctly.

3. Platform-Specific Behavior Differences

• Linux (Crash): The GNU C library (glibc) and GCC’s memory management may not recover gracefully from OOM conditions, leading to null pointers.
• Windows/macOS (Graceful Exit): These platforms terminate the process immediately after detecting OOM, avoiding the null pointer dereference.
• WASM (Fiddle): JavaScript’s memory constraints and exception handling cause abrupt termination, masking the null pointer issue.

4. Bisect-Identified Regression
The commit 416602a851 ("Improve error messages in the CLI") altered how SQL text is stored and referenced during error reporting. A missing null-check in the revised logic allowed zCode to be dereferenced even when invalid.

Resolution: Code Fixes, Workarounds, and Best Practices

1. Apply the Official Patch
The SQLite team resolved this issue in commit d6893183. The fix involves adding a null-check before dereferencing zCode in shell_error_context():

if( zCode != NULL && iOffset>=0 && iOffset<len ){
  // Existing logic to highlight SQL error...
}

Steps to Apply:

1. Update to SQLite version 3.40.0 or later.
2. If using a custom build, cherry-pick commit d6893183 into your codebase.

2. Workarounds for Unpatched Versions

• Avoid Combining hard_heap_limit=1 with Subsequent Statements:
  Execute the PRAGMA command in isolation:

  echo "PRAGMA hard_heap_limit=1;" | sqlite3
  

• Increase Heap Limit to Prevent OOM:
  Use a realistic value (e.g., PRAGMA hard_heap_limit=1048576) to avoid triggering OOM.
• Use .exit or .quit After OOM:
  Force the CLI to terminate immediately after the error:

  PRAGMA hard_heap_limit=1;
  .exit
  

3. Debugging and Analysis Techniques

• Reproduce with UBSAN/ASAN:
  Compile SQLite with sanitizers to detect undefined behavior:

  CFLAGS="-fsanitize=undefined,address" ./configure --enable-debug
  make
  

• Inspect Error Context Generation:
  Add logging to shell_error_context() to trace zSql and zCode values:

  fprintf(stderr, "zSql=%p, zCode=%p\n", zSql, zCode);
  

4. Best Practices for Handling OOM in SQLite

• Avoid Extreme Heap Limits: hard_heap_limit=1 is a stress-test scenario; use it cautiously.
• Validate Input Scripts: Ensure no invalid SQL follows memory-critical commands.
• Monitor CLI State After Errors: Use .trace or .log to capture internal state changes.

5. Platform-Specific Mitigations

• Linux: Use ulimit -v to constrain memory at the OS level instead of hard_heap_limit.
• Windows/macOS: Leverage task managers or activity monitors to detect memory exhaustion.
• WASM: Implement custom OOM handlers in JavaScript to reload the SQLite session gracefully.

6. Understanding SQLite’s Error Handling Flow
After an OOM:

1. The CLI calls sqlite3_exec() or sqlite3_prepare_v2(), which returns SQLITE_NOMEM.
2. shell_exec() invokes save_err_msg() to format the error message.
3. save_err_msg() calls shell_error_context() to generate a contextual snippet of the SQL.
4. With zCode null, the unpatched code crashes at shell.c:17008.

7. Long-Term Code Hygiene

• Add Null Checks in Error-Prone Functions: Audit all CLI functions that dereference user-provided strings.
• Unit Tests for OOM Scenarios: Integrate OOM injection tests into SQLite’s regression suite.
• Document Edge Cases: Explicitly warn users about combining hard_heap_limit with multi-statement scripts.

This guide provides a comprehensive roadmap for diagnosing, resolving, and preventing crashes related to PRAGMA hard_heap_limit=1 and null pointer dereferences in SQLite3. By addressing both immediate fixes and long-term code quality measures, developers can ensure robust error handling in memory-constrained environments.