Building SQLite for WASM Fails with “must use ‘struct’ tag to refer to type ‘Wal'” in Non-Amalgamation Configurations

Compilation Error Due to Missing Type Declaration in WAL Module

Issue Overview

A compilation error occurs when building SQLite for WebAssembly (WASM) with the --enable-amalgamation=no flag, specifically in the Write-Ahead Logging (WAL) module. The error manifests as:

/projects/sqlite/repo/bld/../src/wal.c:747:3: error: must use 'struct' tag to refer to type 'Wal'
 747 |  Wal *pWal,        /* The WAL context */

This error arises because the compiler encounters an incomplete or unrecognized declaration of the Wal struct type. The root cause involves mismatched handling of the SQLITE_OMIT_WAL preprocessor directive across header and source files, particularly in non-amalgamation builds.

In standard amalgamation builds (--enable-amalgamation=yes, the default), all SQLite code is combined into a single sqlite3.c file, ensuring headers are included in a specific order. Non-amalgamation builds compile individual source files separately, exposing dependencies on header inclusion order and preprocessor flag propagation. The WASM build environment typically requires SQLITE_OMIT_WAL to be defined due to platform constraints (e.g., lack of shared memory support), but this definition is not consistently applied early enough in wal.c when using split source files.

The error occurs at the point where the Wal struct is referenced before its full declaration is visible. In SQLite’s source organization, wal.h declares Wal conditionally based on SQLITE_OMIT_WAL, but wal.c does not include sqlite3.h (where SQLITE_OMIT_WAL is often defined) until later in the file. This creates a scenario where the compiler parses references to Wal before knowing whether the struct is declared or omitted.

Mismatched Preprocessor Directives and Header Inclusion Order

Inconsistent Definition of SQLITE_OMIT_WAL

The SQLITE_OMIT_WAL macro controls whether WAL-related code is included in the build. When building for WASM, this flag must be defined to disable unsupported WAL features. However, in non-amalgamation builds:

1. Header Inclusion Order: The wal.c file includes sqlite3.h after local headers like wal.h, delaying visibility of SQLITE_OMIT_WAL. If wal.h checks for SQLITE_OMIT_WAL before it is defined, the Wal struct declaration is omitted, causing later references to Wal in wal.c to fail.
2. Build System Configuration: The WASM build environment may define SQLITE_OMIT_WAL in platform-specific configuration files (e.g., sqlite3.h overrides) that are not processed early enough in non-amalgamation builds.

Amalgamation vs. Non-Amalgamation Build Differences

In amalgamation builds:

• All headers are included in a controlled order within sqlite3.c, ensuring SQLITE_OMIT_WAL is defined before wal.h is processed.
• The Wal struct declaration is either included or omitted consistently.

In non-amalgamation builds:

• Each source file (e.g., wal.c) includes headers independently.
• If wal.c does not include sqlite3.h before referencing Wal, the compiler encounters an undeclared type.

Platform-Specific Configuration Challenges

The SQLite build system does not inherently detect WASM/WASI (WebAssembly System Interface) environments. Developers must manually configure flags like SQLITE_OMIT_WAL, but these flags may not propagate correctly to all source files in split builds. For example:

• Autotools-based builds (e.g., configure --with-wasi-sdk=X) may not inject SQLITE_OMIT_WAL into compiler flags.
• Platform-specific sqlite3.h modifications (e.g., unconditional #define SQLITE_OMIT_WAL for WASM) may be overridden by later includes or build steps.

Resolving Header Dependencies and Build Configuration

Step 1: Validate SQLITE_OMIT_WAL Propagation

Confirm that SQLITE_OMIT_WAL is defined throughout the build process:

1. Inspect Compiler Command Lines:
   Ensure -DSQLITE_OMIT_WAL is present when compiling wal.c. For example:

   gcc -c -DSQLITE_OMIT_WAL -I/path/to/sqlite/includes wal.c -o wal.o
   

2. Check sqlite3.h Configuration:
   Verify that sqlite3.h unconditionally defines SQLITE_OMIT_WAL for WASM builds. For example:

   #if defined(__EMSCRIPTEN__) || defined(__wasi__)
   #define SQLITE_OMIT_WAL 1
   #endif
   

Step 2: Adjust Header Inclusion Order in wal.c

Modify wal.c to include sqlite3.h before any local headers:

// Add at the very top of wal.c
#include "sqlite3.h"
// Existing includes
#include "wal.h"

This ensures SQLITE_OMIT_WAL is defined before wal.h is processed, allowing the latter to conditionally omit Wal declarations.

Step 3: Use Split Amalgamation for Large File Handling

If non-amalgamation builds are required to avoid "too many lines" errors in tooling:

1. Generate Split Amalgamation:
   Use SQLite’s split-sqlite3c.tcl script (or equivalent) to divide sqlite3.c into smaller files:

   tclsh split-sqlite3c.tcl
   

   This creates sqlite3-1.c, sqlite3-2.c, etc., which can be compiled individually without losing amalgamation benefits.

2. Update Build System:
   Replace references to individual source files with the split amalgamation outputs.

Step 4: Patch Build Configuration for WASI

For configure-based builds targeting WASI:

1. Modify configure.ac:
   Add logic to detect WASI and automatically define SQLITE_OMIT_WAL:

   AC_TRY_COMPILE([], [
   #if !defined(__wasi__)
   #error "Not WASI"
   #endif
   ], [AC_DEFINE([SQLITE_OMIT_WAL], [1], [Disable WAL for WASI])])
   

2. Rebuild Autotools Configuration:
   Run autoreconf -fi to regenerate configure with the new logic.

Step 5: Workaround via Explicit Compiler Flags

If modifying source files is undesirable, pass SQLITE_OMIT_WAL explicitly:

./configure --enable-amalgamation=no CFLAGS="-DSQLITE_OMIT_WAL"

This ensures the flag is defined globally, regardless of header inclusion order.

Step 6: Revert to Amalgamation Builds

If non-amalgamation builds are not strictly necessary:

./configure --enable-amalgamation=yes

Amalgamation builds avoid header order issues by design.

Step 7: Monitor Upstream SQLite Changes

The SQLite team has experimented with removing hard-coded SQLITE_OMIT_WAL definitions for WASM. Track these changes:

1. Update to Latest Trunk:

   fossil update trunk
   

2. Test for Regressions:
   Ensure WASM builds still function correctly after updates. If SQLITE_OMIT_WAL is no longer defined, re-add it via compiler flags.

Step 8: Custom Source Patching

For environments requiring long-term stability:

1. Apply a Local Patch to wal.c:

   --- a/src/wal.c
   +++ b/src/wal.c
   @@ -16,6 +16,7 @@
    **
    ** This file contains the implementation of a write-ahead log (WAL) used in 
    ** "journal_mode=WAL" mode.
    */
   +#include "sqlite3.h"
    #include "sqliteInt.h"
    #include "wal.h"
   

2. Maintain the Patch:
   Use version control or a patch management system to reapply this change during updates.

Final Recommendations

1. Prefer Amalgamation Builds: They minimize header-related issues and are SQLite’s primary supported configuration.
2. Explicitly Define SQLITE_OMIT_WAL: Use compiler flags to ensure consistent behavior across build types.
3. Contribute Upstream: If split amalgamation or WASM configuration improvements are critical for your project, collaborate with the SQLite team to integrate robust solutions.

By addressing header inclusion order, build flag consistency, and platform detection, developers can resolve the "struct Wal" compilation error while maintaining compatibility with WASM/WASI constraints.