FORTRAN-SQLite Interface Memory Corruption: String Handling and Pointer Mismatches

Issue Overview: Memory Corruption During Database Handle Acquisition in FORTRAN-C-SQLite Interface

The core problem involves memory corruption manifested as an array bounds write error when a legacy FORTRAN codebase interfaces with SQLite via a C wrapper function. The corruption occurs during database handle acquisition through the open_database_ C function called from FORTRAN. Key technical elements include:

1. Memory Corruption Signature
   The debug tool reports writing 8 bytes to heap address 0x3ccaf68a, exceeding the allocated 128-byte block at 0x3ccaf610 by 122 bytes. This indicates a buffer overflow where 8 bytes are written 2 bytes beyond legal allocation boundaries. The stack trace implicates SQLite’s memory allocation chain (sqlite3Malloc → sqlite3DbMallocZero) during schema initialization (sqlite3SchemaGet), triggered by sqlite3_open.

2. FORTRAN-C Interop Mechanics

   • FORTRAN passes fixed-length CHARACTER*512 strings without null termination
   • Implicit string length argument (f_len) appended to C function calls
   • Use of integer*8 (64-bit integer) to represent C pointers (sqlite3**)
   • Pointer assignment via *fortran_handle = db in C code

3. Critical Code Patterns

   • String Conversion: The fsalloc function converts FORTRAN strings to C strings
   • Pointer Lifetime: Database handle (sqlite3*) stored in FORTRAN-managed memory
   • Memory Management: Absence of free() for temporary filename buffer

The corruption arises from two primary failure domains: improper string handling between FORTRAN and C causing buffer overflows, and pointer management errors in cross-language type representation. These interact with SQLite’s internal memory allocation patterns to produce the observed heap violation.

Possible Causes: String Termination Failures and Pointer Representation Mismatches

1. FORTRAN-to-C String Conversion Without Proper Null Termination

FORTRAN CHARACTER arrays lack implicit null terminators. When passing dbName (512-byte buffer) to C, the wrapper must:

• Allocate f_len + 1 bytes (where f_len is actual string length ≤512)
• Copy f_len bytes from FORTRAN buffer
• Explicitly add \0 terminator at index f_len

Failure Modes:

• Buffer Under-Allocation: If fsalloc allocates only f_len bytes instead of f_len+1, writing the terminator overflows the buffer
• Terminator Omission: If fsalloc copies all 512 bytes (including padding) without trimming to actual length, SQLite interprets garbage bytes after FORTRAN string content as part of the filename

Example Corruption Pathway:

1. FORTRAN passes 122-character filename (no null terminator)
2. fsalloc allocates 122 bytes, copies 122 chars
3. Missing terminator causes sqlite3_open to read past buffer end
4. Heap metadata corruption occurs during subsequent SQLite operations

2. Pointer Type Mismatch in FORTRAN/C Handle Representation

FORTRAN’s integer*8 dbHandle stores the sqlite3* pointer value as a 64-bit integer. The C wrapper interprets this as sqlite3** fortran_handle (pointer-to-pointer). The assignment *fortran_handle = db writes the 8-byte sqlite3* value to memory pointed by fortran_handle.

Failure Modes:

• Invalid Pointer Casting: If FORTRAN’s dbHandle isn’t properly aligned or initialized as pointer storage
• Pointer Size Mismatch: On 32-bit systems where integer*8 exceeds native pointer size (not applicable here given error context)
• Dereference Violation: If fortran_handle points to memory not allocated for pointer storage

Example Corruption Pathway:

1. FORTRAN declares dbHandle as uninitialized integer*8
2. C code treats fortran_handle as valid sqlite3** pointer
3. *fortran_handle = db writes 8 bytes to arbitrary memory location
4. Heap corruption occurs when SQLite later allocates near this address

3. Memory Leak in Temporary Filename Buffer

The t_filename buffer allocated by fsalloc isn’t freed after sqlite3_open, causing memory leaks. While not directly causing the array bounds write, this indicates broader resource management issues that may compound memory corruption under heavy usage.

Troubleshooting Steps, Solutions & Fixes

1. Diagnose and Fix FORTRAN-to-C String Conversion

Step 1: Audit fsalloc Implementation
Obtain the implementation of fsalloc to verify:

• Allocation size is f_len + 1
• Explicit null termination at t_filename[f_len]
• Handling of FORTRAN’s blank-padded fixed-length strings

Common Bug Patterns:

// WRONG: Copies entire FORTRAN buffer including padding
char* fsalloc(char* f_str, int f_len) {
    char* c_str = malloc(f_len);
    memcpy(c_str, f_str, f_len);
    return c_str; // No null terminator
}

// CORRECT: Trim trailing spaces and null-terminate
char* fsalloc(char* f_str, int f_len) {
    // Trim trailing spaces
    while (f_len > 0 && f_str[f_len-1] == ' ') f_len--;
    // Allocate with terminator
    char* c_str = malloc(f_len + 1);
    memcpy(c_str, f_str, f_len);
    c_str[f_len] = '\0';
    return c_str;
}

Step 2: Add Boundary Checks
Insert debug prints or assertions in open_database_:

t_filename = fsalloc(filename, f_len);
assert(t_filename[strnlen(t_filename, f_len+1)] == '\0'); // Validate termination

Step 3: Use SQLite Tracing
Enable SQLite’s file open logging to verify filename integrity:

sqlite3_open_v2(t_filename, &db, SQLITE_OPEN_READWRITE, NULL);
printf("Attempting to open: [%s]\n", t_filename); // Inspect actual filename

2. Validate FORTRAN/C Pointer Handling

Step 1: Ensure Proper Pointer Storage
FORTRAN must reserve memory for pointer storage before C writes to it:

FORTRAN:

integer*8 dbHandle
! Explicitly initialize as null pointer
dbHandle = 0
character*512 dbName
return_value = open_database(dbName, dbHandle)

C:

int open_database_(char* filename, sqlite3** fortran_handle, FsLen f_len) {
    // ... 
    // Ensure fortran_handle points to writable memory
    if (fortran_handle == NULL) return SQLITE_MISUSE;
    *fortran_handle = db;
    return rc;
}

Step 2: Use Interoperability Types
Modern FORTRAN supports C interoperability:

use, intrinsic :: iso_c_binding
type(c_ptr) :: dbHandle
character(512) :: dbName
interface
    function open_database(filename, db) bind(c, name="open_database")
        import :: c_ptr, c_char, c_int
        character(kind=c_char) :: filename(*)
        type(c_ptr) :: db
        integer(c_int) :: open_database
    end function
end interface

Step 3: Pointer Marshaling Verification
Inspect pointer values across language boundary:
C:

printf("C: db handle = %p\n", db);
printf("C: fortran_handle storage at %p\n", fortran_handle);

FORTRAN:

! After call
print *, 'FORTRAN: dbHandle = ', dbHandle

Compare addresses to ensure proper pointer storage.

3. Comprehensive Memory Management

Solution 1: Fix fsalloc Memory Leak
Free temporary filename buffer after sqlite3_open:

t_filename = fsalloc(filename, f_len);
rc = sqlite3_open(t_filename, &db);
free(t_filename); // Critical cleanup

Solution 2: Use SQLite Memory Management
Replace fsalloc with SQLite’s allocator for better diagnostics:

t_filename = sqlite3_malloc(f_len + 1);
// ... copy and terminate ...
rc = sqlite3_open(t_filename, &db);
sqlite3_free(t_filename);

Solution 3: Add Error Handling
Properly propagate error codes and validate handles:

int open_database_(...) {
    if (t_filename == NULL) return SQLITE_NOMEM;
    rc = sqlite3_open(t_filename, &db);
    if (rc != SQLITE_OK) {
        sqlite3_close(db);
        db = NULL;
    }
    *fortran_handle = db;
    free(t_filename);
    return rc;
}

4. Advanced Debugging Techniques

Technique 1: Memory Sanitizers
Compile SQLite and C wrapper with address sanitizer:

CFLAGS="-fsanitize=address" LDFLAGS="-fsanitize=address" ./configure

Technique 2: SQLite Debug Logs
Enable internal logging before sqlite3_open:

sqlite3_config(SQLITE_CONFIG_LOG, debug_log_callback, NULL);

Technique 3: FORTRAN Array Bounds Checking
Compile FORTRAN with array bounds checking:

gfortran -fcheck=all -o app *.f90

Final Corrected Implementation

C Wrapper:

#include <sqlite3.h>
#include <string.h>

int open_database_(char* filename, sqlite3** fortran_handle, int f_len) {
    sqlite3* db = NULL;
    char* t_filename = NULL;
    int rc = SQLITE_ERROR;

    // Validate input pointers
    if (!fortran_handle || !filename) return SQLITE_MISUSE;

    // Convert FORTRAN string to C string
    t_filename = sqlite3_malloc(f_len + 1);
    if (!t_filename) return SQLITE_NOMEM;
    
    memcpy(t_filename, filename, f_len);
    t_filename[f_len] = '\0'; // Explicit termination

    // Trim trailing spaces
    while (f_len > 0 && t_filename[f_len-1] == ' ') {
        t_filename[--f_len] = '\0';
    }

    rc = sqlite3_open(t_filename, &db);
    sqlite3_free(t_filename);

    if (rc != SQLITE_OK) {
        sqlite3_close(db);
        *fortran_handle = NULL;
    } else {
        *fortran_handle = db;
    }

    return rc;
}

FORTRAN Caller:

program main
    use, intrinsic :: iso_c_binding
    implicit none
    type(c_ptr) :: dbHandle
    character(512) :: dbName
    integer(c_int) :: rc

    dbName = 'test.db' // C_NULL_CHAR
    rc = open_database(dbName, dbHandle)
    
    if (rc == 0) then
        print *, 'Database opened successfully'
    else
        print *, 'Error opening database: ', rc
    end if
end program

This implementation addresses all identified failure modes: proper string handling with null termination, correct pointer management, and rigorous memory cleanup. The use of SQLite’s allocator ensures compatibility with its internal memory tracking systems, while explicit error checking prevents invalid handle propagation.