Resolving SQLite WAL/SHM File Permission Conflicts in Docker Shared Volumes

Understanding SQLite WAL/SHM File Permission Behavior in Multi-Container Environments

The core challenge arises when SQLite databases operating in Write-Ahead Logging (WAL) mode generate auxiliary files (.wal and .shm) with restrictive permissions in Docker environments. These files are critical for concurrent database access across multiple containers. In a typical scenario, PHP and Node.js containers share a Docker volume containing the SQLite database file. When the first container writes to the database, SQLite creates .wal and .shm files with permissions derived from the base database file’s mode, adjusted by the active umask of the process. If these files are created with insufficient permissions (e.g., 0600), subsequent containers running under different user contexts cannot access or modify them, leading to "permission denied" errors or database lock contention.

This behavior is rooted in SQLite’s file creation logic and Unix-like permission inheritance rules. The database engine does not explicitly set permissions for .wal and .shm files but instead relies on the operating system’s file creation defaults. When SQLite opens a database in WAL mode, it uses the robust_open() function (as seen in SQLite’s os_unix.c source) to create these files. The permissions for new files are calculated as (SQLITE_DEFAULT_FILE_PERMISSIONS & ~umask), where SQLITE_DEFAULT_FILE_PERMISSIONS is typically 0644. However, if the base database file already exists, SQLite mirrors its permissions for new WAL/SHM files, minus the umask. This interaction between existing file modes, umask, and container user contexts creates permission mismatches in multi-container Docker setups.

The problem is exacerbated when containers operate under distinct user IDs or group memberships. Docker’s default behavior maps container users to host users via the user namespace, but shared volumes often retain host directory ownership unless explicitly configured. For example, if the PHP container runs as user www-data (UID 33) and the Node.js container runs as user node (UID 1000), SQLite’s WAL/SHM files created by one user will not grant write access to the other. Even with liberal file permissions like 0666, Unix systems enforce user/group ownership checks before honoring permission bits. Thus, the fundamental issue is not merely the file mode but the ownership mismatch across containers.

Diagnosing Permission Mismatches and Ownership Conflicts

Permission conflicts in SQLite WAL/SHM files stem from four interrelated factors:

1. Incorrect umask Settings in Containers
The umask value active during SQLite operations determines how permissions are stripped from newly created files. A umask of 022 (common in many Linux distributions) reduces a default 0644 file mode to 0644 & ~022 = 0644 - 0022 = 0622, but since SQLite uses robust_open(), the calculation is (database_file_mode & ~umask). If the base database file has 0666 permissions and the umask is 0000, the WAL/SHM files inherit 0666. However, if the umask is 0022, the result is 0666 & ~0022 = 0644, which denies write access to non-owner users unless group/other permissions are explicitly granted. Containers often inherit the host’s umask unless overridden, leading to inconsistent file modes.

2. User/Group ID Mismatches Across Containers
Docker containers running PHP and Node.js applications often use different base images with preconfigured user accounts. For instance, the php:apache image uses www-data (UID 33), while node:alpine uses node (UID 1000). When these containers write to a shared volume, files created by one user are inaccessible to the other unless both users share a common group or the files have world-writable permissions. Even with 0777 permissions, security policies like SELinux or AppArmor might block cross-user access.

3. SQLite’s Group Ownership Inheritance Gap
As noted in SQLite’s os_unix.c source code, the robust_open() function does not explicitly set the group ownership of WAL/SHM files to match the base database file. If the database file belongs to group appusers, newly created WAL/SHM files inherit the effective group ID of the creating process, which may not match the database file’s group. This breaks scenarios where containers share a common group but run under different users.

4. Volume Mount Configuration in Docker
Docker volumes mounted from the host inherit the host directory’s ownership and permissions. If the host directory is owned by root:root with 0755 permissions, containers without root privileges cannot write to it. Named volumes managed by Docker may assign arbitrary ownership unless configured with docker volume create --opt o=uid=1000,gid=1000. Misconfigured volume mounts force containers to operate with mismatched user/group contexts.

Configuring Docker and SQLite for Cross-Container WAL/SHM File Access

To resolve permission conflicts, implement the following strategies:

1. Synchronize User/Group IDs Across Containers
Force all containers to use the same user and group IDs when accessing the shared volume. For example, in your docker-compose.yml:

services:
  php-app:
    user: "1000:1000"
    volumes:
      - "app_data:/var/www/data"
  node-app:
    user: "1000:1000"
    volumes:
      - "app_data:/app/data"
volumes:
  app_data:

This ensures both containers operate as UID 1000 and GID 1000. If using custom images, modify Dockerfiles to create a common user:

FROM php:apache
RUN groupadd -g 1000 appgroup && \
    useradd -u 1000 -g appgroup appuser && \
    chown -R appuser:appgroup /var/www
USER appuser

2. Set the umask in Container Entrypoints
Override the default umask in containers to 0002 (allowing group write access) or 0000 (no restrictions). In a custom entrypoint script:

#!/bin/sh
umask 0002
exec "$@"

Update Dockerfiles to copy the entrypoint and set execution permissions:

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

3. Configure Directory Setgid and Inheritable Permissions
On the host or volume directory, enable the setgid bit to ensure new files inherit the parent directory’s group:

chmod g+s /path/to/volume
chown :appgroup /path/to/volume
chmod 2775 /path/to/volume

This forces all new files (including WAL/SHM) to inherit the directory’s group (appgroup). Combined with umask 0002, files are created with 0664 permissions, granting group write access.

4. Pre-Create WAL/SHM Files with Correct Permissions
If SQLite has not yet created the WAL/SHM files, pre-create them with desired permissions:

touch /path/to/database.wal /path/to/database.shm
chmod 0666 /path/to/database.*

SQLite will reuse these files if they exist. However, this is a temporary fix and may not persist across database restarts.

5. Use Docker Named Volumes with Explicit Ownership
Create a named volume with predefined ownership:

docker volume create --driver local \
  --opt type=tmpfs \
  --opt o=uid=1000,gid=1000,mode=1770 \
  app_data

The mode=1770 sets the sticky bit for the directory, ensuring only the owner and group can modify files.

6. Leverage Access Control Lists (ACLs) for Fine-Grained Permissions
On the host filesystem, apply ACLs to grant write access to specific users/groups:

setfacl -R -m u:php-user:rwx,g:node-group:rwx /path/to/volume
setfacl -d -R -m u:php-user:rwx,g:node-group:rwx /path/to/volume

The -d flag ensures default ACLs are applied to new files.

7. Validate SQLite Configuration and File Handles
Ensure SQLite is configured to use WAL mode and that all processes close database connections properly. Lingering file handles may retain outdated permissions. Use lsof to inspect open files:

lsof /path/to/database

By combining user/group synchronization, umask adjustments, and filesystem ACLs, you can enforce consistent permissions for SQLite’s WAL/SHM files across Docker containers. Avoid relying on 0777 permissions, as they expose security risks; instead, use group-based access controls and Docker’s user namespace features to maintain least-privilege principles.