LSM1 Read-Only Mode Crashes Due to Custom Page/Block Size Mismatch

Issue Overview: Read-Only Mode Fails to Honor Custom Page and Block Sizes, Leading to Cursor Crashes

When working with the lsm1 database engine in stand-alone mode, a critical issue arises when the database is opened in read-only mode. Specifically, the database fails to honor custom page and block sizes configured during its creation. This mismatch occurs because the read-only mode does not properly initialize the page and block sizes from the database’s metadata, unlike the regular read-write mode. As a result, cursors operating on the database in read-only mode may crash due to incorrect page number computations, which rely on the default block size instead of the custom values stored in the database.

The root of the problem lies in the lsmBeginRoTrans function, which is responsible for initializing a read-only transaction. In this function, the page and block sizes are not set to the values stored in the database’s shared memory or checkpoint metadata. This omission causes the file system layer to operate with default sizes, leading to inconsistencies when the database uses non-default configurations. The issue becomes particularly apparent when the database file is zero bytes in size on disk, as the shared memory values are not propagated to the file system connection.

The proposed fix involves modifying lsmBeginRoTrans to explicitly set the page and block sizes from the database’s checkpoint metadata. This ensures that the file system layer operates with the correct sizes, even in read-only mode. The fix has been tested and validated by multiple contributors, confirming that it resolves the cursor crashes and ensures consistent behavior across read-only and read-write modes.

Possible Causes: Why Custom Page and Block Sizes Are Ignored in Read-Only Mode

The issue stems from the way the lsm1 engine handles database initialization in read-only mode. When a database is opened in read-write mode, the lsm_open function correctly initializes the page and block sizes by reading the values from the database’s shared memory or checkpoint metadata. However, this initialization step is skipped in read-only mode, leading to the following specific causes:

1. Missing Initialization in lsmBeginRoTrans: The lsmBeginRoTrans function, which sets up a read-only transaction, does not include logic to configure the file system connection with the database’s custom page and block sizes. This omission means that the file system layer defaults to predefined sizes, which may not match the database’s actual configuration.

2. Shared Memory Pointer (pShmhdr) Nullification: In read-only mode, the pShmhdr pointer, which points to the shared memory header containing database metadata, is initially null. This prevents the system from loading the checkpoint and retrieving the correct page and block sizes. Although the shared memory values are eventually set, they are not propagated to the file system connection in time for read-only transactions.

3. Checkpoint Loading Overhead: The lsmCheckpointLoad function, which loads the database’s checkpoint metadata, is not called during read-only transaction initialization. This function is essential for retrieving the correct page and block sizes, but its absence in lsmBeginRoTrans means that the file system layer remains unconfigured.

4. Default Sizes in File System Layer: The file system layer (lsmFs) defaults to predefined page and block sizes if no explicit configuration is provided. When the database uses custom sizes, this mismatch leads to incorrect page number computations and cursor crashes.

5. Zero-Sized Database Files: In cases where the database file is zero bytes in size on disk, the shared memory values are not immediately available. The system relies on the checkpoint metadata to determine the correct sizes, but this metadata is not loaded in read-only mode, exacerbating the issue.

Troubleshooting Steps, Solutions & Fixes: Resolving Page and Block Size Mismatches in Read-Only Mode

To address the issue of custom page and block sizes being ignored in read-only mode, the following steps and solutions can be implemented:

1. Modify lsmBeginRoTrans to Load Checkpoint Metadata

The primary fix involves updating the lsmBeginRoTrans function to load the database’s checkpoint metadata and configure the file system connection with the correct page and block sizes. This ensures that the file system layer operates with the same sizes as the database, even in read-only mode. The updated code should include the following logic:

if (LSM_OK == rc && 0 == lsmCheckpointClientCacheOk(pDb)) {
    rc = lsmCheckpointLoad(pDb, 0);
    if (LSM_OK == rc) {
        lsmFsSetPageSize(pDb->pFS, lsmCheckpointPgsz(pDb->aSnapshot));
        lsmFsSetBlockSize(pDb->pFS, lsmCheckpointBlksz(pDb->aSnapshot));
    }
}

This code block checks if the checkpoint client cache is valid and, if not, loads the checkpoint metadata. It then sets the page and block sizes in the file system layer using the values retrieved from the checkpoint.

2. Add Assertions for Validation

To ensure that the page and block sizes are correctly set, add assertions after configuring the file system connection. These assertions verify that the sizes in the file system layer match those in the checkpoint metadata:

assert(LSM_OK != rc || lsmFsPageSize(pDb->pFS) == lsmCheckpointPgsz(pDb->aSnapshot));
assert(LSM_OK != rc || lsmFsBlockSize(pDb->pFS) == lsmCheckpointBlksz(pDb->aSnapshot));

These assertions provide a safeguard against incorrect configurations and help identify issues during development and testing.

3. Handle Zero-Sized Database Files

For databases that are zero bytes in size on disk, ensure that the shared memory values are propagated to the file system connection. This can be achieved by explicitly setting the page and block sizes after loading the checkpoint metadata, as shown in the updated lsmBeginRoTrans function.

4. Optimize Checkpoint Loading

To minimize overhead, use the lsmCheckpointClientCacheOk function to determine whether the checkpoint needs to be loaded. This function checks if the client cache is valid, preventing unnecessary checkpoint loading in every transaction. The optimized code ensures that the checkpoint is loaded only when required:

if (LSM_OK == rc && 0 == lsmCheckpointClientCacheOk(pDb)) {
    rc = lsmCheckpointLoad(pDb, 0);
    if (LSM_OK == rc) {
        lsmFsSetPageSize(pDb->pFS, lsmCheckpointPgsz(pDb->aSnapshot));
        lsmFsSetBlockSize(pDb->pFS, lsmCheckpointBlksz(pDb->aSnapshot));
    }
}

5. Test with Custom Page and Block Sizes

After implementing the fix, thoroughly test the database with custom page and block sizes to ensure that the issue is resolved. Verify that cursors no longer crash in read-only mode and that the file system layer operates with the correct sizes. Testing should include scenarios with zero-sized database files and various combinations of page and block sizes.

6. Validate the Fix in Different Environments

To ensure the robustness of the fix, validate it in different environments and configurations. Test the database with varying workloads, including read-heavy and write-heavy operations, to confirm that the fix does not introduce new issues or regressions.

7. Document the Changes

Finally, document the changes made to lsmBeginRoTrans and the rationale behind them. This documentation should include details about the issue, the proposed fix, and the testing performed to validate the solution. Clear documentation helps other developers understand the changes and ensures that the fix is maintained in future updates.

By following these steps, the issue of custom page and block sizes being ignored in read-only mode can be effectively resolved, ensuring consistent and reliable behavior across all modes of operation.