AIX xlc Build Failure Due to va_arg Dereferencing in SQLITE_TESTCTRL_LOGEST

Issue Overview: Syntax Errors During SQLite 3.38.0 Build with xlc on AIX

A compilation failure occurs when attempting to build SQLite 3.38.0 on AIX using IBM’s XL C/C++ compiler (xlc) with the -q64 flag for 64-bit support. The error manifests as syntax warnings at three consecutive lines in sqlite3.c, specifically within the SQLITE_TESTCTRL_LOGEST case block of the sqlite3_test_control function. The compiler flags the following code as invalid:

va_arg(ap, int*)[0] = rLogEst;             
va_arg(ap, u64*)[0] = iInt;               
va_arg(ap, int*)[0] = sqlite3LogEst(iInt); 

The error messages indicate a potential missing semicolon or comma, but the root cause is the compiler’s inability to parse the direct dereferencing of va_arg macros. The user confirmed the issue arises from an older xlc version (V11.1) and provided a local workaround using temporary variables instead of inline dereferencing.

The code in question is part of SQLite’s test control logic, which manipulates variable arguments via va_list. The SQLITE_TESTCTRL_LOGEST case retrieves and modifies LogEst values (logarithmic estimates used in query planning) through a series of va_arg calls. The problem arises because the xlc compiler interprets the va_arg(ap, TYPE*)[0] syntax as a malformed expression, likely due to non-compliance with C99 standards or implementation-specific quirks in how va_arg macros expand.

Possible Causes: xlc Compiler Limitations and Non-Portable va_arg Usage

1. xlc Compiler Non-Compliance with C99 va_arg Semantics

The IBM XL C/C++ compiler for AIX (V11.1) exhibits strict parsing behavior when handling va_arg macros. The C standard specifies that va_arg expands to an expression that has the type specified in its second argument. However, directly subscripting the result of va_arg (e.g., va_arg(ap, int*)[0]) is not guaranteed to be portable. The xlc compiler may interpret va_arg(ap, int*) as an rvalue (non-modifiable temporary), making the [0] subscript operation syntactically invalid. This violates the compiler’s expectations for lvalues in array subscript contexts.

2. Legacy Compiler Version with Macro Expansion Quirks

The xlc V11.1 compiler, released in 2010, predates widespread adoption of C99 features and may handle macro expansions differently than modern compilers. SQLite’s code assumes that va_arg can be treated as an lvalue pointer, which works on compilers like GCC or Clang but fails on xlc due to internal differences in how va_arg is implemented. The compiler’s macro expansion logic might generate intermediate tokens that disrupt the syntactic validity of the expression.

3. Type Mismatch or Ambiguity in va_arg Dereferencing

The va_arg calls in the problematic code retrieve pointers (int*, u64*), which are then immediately dereferenced via [0] subscripting. While this is functionally equivalent to *va_arg(ap, int*), the subscript notation may confuse the xlc compiler’s parser, especially if the macro expansion introduces unexpected tokens. Additionally, the compiler’s type-checking logic might flag the subscript operation as ambiguous if it cannot resolve the type hierarchy correctly.

Troubleshooting Steps, Solutions & Fixes: Code Modifications and Compiler Workarounds

1. Apply the Official SQLite Patch for xlc Compatibility

The SQLite development team addressed this issue in check-in 46d1a6de620f26fe. The patch modifies the SQLITE_TESTCTRL_LOGEST case to use temporary variables instead of inline dereferencing:

case SQLITE_TESTCTRL_LOGEST: {              
  double rIn = va_arg(ap, double);            
  LogEst rLogEst = sqlite3LogEstFromDouble(rIn);     
  u64 iInt = sqlite3LogEstToInt(rLogEst);         
  int *p1 = va_arg(ap, int*);
  u64 *p2 = va_arg(ap, u64*);
  int *p3 = va_arg(ap, int*);
  p1[0] = rLogEst;             
  p2[0] = iInt;               
  p3[0] = sqlite3LogEst(iInt);       
  break;                         
}  

This refactoring eliminates direct dereferencing of va_arg results, resolving the syntax error. To apply this fix:

1. Download the patched sqlite3.c source from the check-in.
2. Replace the existing SQLITE_TESTCTRL_LOGEST case with the modified code.
3. Rebuild the library using the same compiler flags.

2. Manual Code Modification for Legacy Build Environments

If updating to the patched SQLite version is impractical, manually edit the sqlite3.c file as follows:

• Locate the SQLITE_TESTCTRL_LOGEST case (lines 172757–172759 in the original report).
• Replace the three problematic lines with:

int *p1 = va_arg(ap, int*);
u64 *p2 = va_arg(ap, u64*);
int *p3 = va_arg(ap, int*);
p1[0] = rLogEst;             
p2[0] = iInt;               
p3[0] = sqlite3LogEst(iInt);  

This change aligns with the SQLite team’s fix and ensures compatibility with xlc’s parsing rules.

3. Compiler Flag Adjustments and Version Upgrades

While the code modification is the definitive solution, additional steps can mitigate similar issues:

• Test with xlc’s C99 Mode: Ensure the compiler is invoked with -qlanglvl=extc99 to enable C99-compliant parsing.
• Upgrade xlc: IBM XL C/C++ V16.1 or later includes improved C11/C99 compliance and might handle the original code without errors.
• Preprocessor Workarounds: Use conditional compilation to isolate non-portable va_arg usage:

#if defined(__IBMC__) || defined(__IBMCPP__)
  /* Use temporary variables for xlc */
#else
  /* Original inline dereferencing */
#endif  

4. Validation and Regression Testing

After applying the fix:

1. Rebuild the library and verify that the syntax errors are resolved.
2. Execute SQLite’s test suite (e.g., make test or make smoketest) to ensure the SQLITE_TESTCTRL_LOGEST functionality behaves as expected.
3. Monitor runtime behavior in applications using the modified library, particularly any logic relying on sqlite3_test_control with SQLITE_TESTCTRL_LOGEST.

This comprehensive approach addresses both the immediate build failure and underlying portability issues, ensuring robust SQLite operation on AIX with xlc.