Intermittent OPFS Data Loss in Edge with SQLite-WASM: Causes and Fixes

Understanding OPFS Data Loss in Browser-Based SQLite Applications

The core issue revolves around unexpected data loss in web applications using SQLite-WASM with the Origin Private File System (OPFS) API, particularly in Microsoft Edge. Developers report that OPFS-stored data disappears under unclear conditions, while data in LocalStorage remains intact. This behavior undermines the reliability of applications requiring persistent client-side storage, such as note-taking apps, collaborative tools, or offline-first solutions. The problem is exacerbated by its intermittent nature (occurring roughly monthly) and the lack of clear reproduction steps, making root cause analysis challenging.

Technical Context: OPFS and Browser Storage Tiers

OPFS is part of the File System Access API, designed to provide web apps with a private, sandboxed file system tied to the origin (domain). Unlike LocalStorage (limited to 5–10 MB) or IndexedDB (asynchronous key-value storage), OPFS allows synchronous file operations critical for porting low-level libraries like SQLite to WebAssembly. However, browser vendors implement OPFS with varying storage eviction policies. By default, OPFS data is considered "best-effort" persistent and may be purged under storage pressure unless the app explicitly requests persistent storage permissions. This distinction is often overlooked, as LocalStorage and IndexedDB are subject to different eviction rules.

Root Causes of OPFS Data Loss in Edge

1. Missing or Denied Persistent Storage Permissions

Browsers classify storage into temporary (default) and persistent tiers. Temporary storage can be cleared without user interaction when the browser detects low disk space, prolonged inactivity, or during routine cleanup. Persistent storage requires explicit user consent via navigator.storage.persist(). If an app fails to request this permission or the user denies it, OPFS data remains vulnerable to automated eviction. Edge further complicates this by tying persistent storage grants to other permissions (e.g., notifications), creating unexpected dependencies.

2. Third-Party System Cleanup Utilities

Utilities like CCleaner, AVG PC TuneUp, and BleachBit aggressively delete browser cache and temporary files. These tools often misclassify OPFS data as disposable "Internet Cache" or temporary files, especially when they lack awareness of modern web standards like OPFS. Windows 11’s built-in Storage Sense feature similarly targets temporary files, and its heuristics may erroneously include OPFS directories. Unlike cookies or LocalStorage, OPFS directories are stored as physical files on disk (e.g., in Edge’s WebKit/OriginPrivateFileSystem directory), making them susceptible to file-level cleaners.

3. Browser-Specific Bugs and Storage Quirks

Edge’s implementation of OPFS and its integration with Windows storage management subsystems may have undiscovered bugs. For instance:

• Inconsistent Permission Persistence: Edge might revoke persistent storage grants after browser updates, OS reboots, or profile resets.
• OPFS Directory Corruption: Silent failures during write operations (e.g., due to abrupt browser shutdowns) could leave databases in a corrupted state. However, SQLite-WASM typically retains corrupted files unless explicitly deleted, making this less likely.
• Storage Pressure Algorithms: Edge’s storage eviction logic might prioritize OPFS data over other storage mechanisms under memory constraints, even when persistent permissions are granted.

4. Application Architecture Flaws

While not directly causing data loss, certain design choices amplify the impact:

• Lack of Redundant Backups: Relying solely on OPFS without periodic exports to IndexedDB or cloud sync increases data vulnerability.
• Inadequate Error Handling: Failing to detect NotFoundError exceptions when accessing OPFS files leaves users unaware of data loss until they attempt to reload the app.

Comprehensive Solutions for Preventing OPFS Data Loss

Step 1: Enforce Persistent Storage Permissions

Requesting Permissions:

• Explicit User Gesture: Trigger navigator.storage.persist() after a user interaction (e.g., clicking "Save Data" button) to comply with browser security policies. Example:

  document.getElementById('saveButton').addEventListener('click', async () => {
    const isPersistent = await navigator.storage.persist();
    alert(isPersistent ? 'Storage is persistent!' : 'Storage may be cleared.');
  });
  

• Fallback Strategies: If persistent storage is denied, warn users about data volatility and offer alternatives like exporting to downloadable files or cloud storage.

Monitoring Permission States:

• Use navigator.storage.persisted() to check current status during app initialization:

  async function checkStoragePersistence() {
    const isPersistent = await navigator.storage.persisted();
    if (!isPersistent) {
      showWarningBanner('Enable persistent storage to prevent data loss.');
    }
  }
  

Step 2: Mitigate Third-Party Cleanup Risks

User Education:

• Provide documentation instructing users to:
  • Exclude Edge’s application data directory from cleanup tools.
  • Disable Windows Storage Sense (Settings > System > Storage > Storage Sense).
  • Avoid utilities like CCleaner that lack OPFS awareness.

Developer Workarounds:

• Data Redundancy: Regularly mirror OPFS data to IndexedDB, which is less likely to be purged by file cleaners. Example sync logic:

  async function backupToIndexedDB() {
    const opfsFile = await getOPFSFileHandle();
    const content = await opfsFile.text();
    const db = await openIDB('backups');
    await db.put('backups', content, 'latest');
  }
  

• Checksum Validation: Store SHA-256 hashes of critical databases in LocalStorage and validate them on app startup to detect tampering or deletion.

Step 3: Address Edge-Specific Quirks

Testing Across Browsers:

• Replicate the app in Chrome, Firefox, and Safari to isolate Edge-specific issues. Use feature detection for OPFS:

  if ('storage' in navigator && 'getDirectory' in window.showDirectoryPicker) {
    // OPFS is supported
  }
  

Edge Bug Mitigations:

• File Locking: Use FileSystemSyncAccessHandle for atomic writes, reducing corruption risks during abrupt closures.
• Storage Pressure Monitoring: Listen to quotachange events to warn users before data eviction:

  navigator.storage.addEventListener('quotachange', (event) => {
    console.log('Storage quota changed:', event);
    alert('Free space is low—export data to prevent loss.');
  });
  

Step 4: Implement Robust Recovery Mechanisms

Automated Backups:

• Schedule incremental backups using the Background Sync API or Service Workers:

  // In service worker:
  self.addEventListener('periodicsync', (event) => {
    if (event.tag === 'opfs-backup') {
      event.waitUntil(backupToIndexedDB());
    }
  });
  

Corruption Recovery:

• Embed SQLite’s built-in integrity checks:

  PRAGMA integrity_check;
  

  If corruption is detected, restore from the last known good backup.

Step 5: Collaborate with Browser Vendors

• File Edge Bug Reports: Detail the issue with steps to reproduce, including Edge version, OS build, and installed cleanup tools.
• Engage with Standards Bodies: Advocate for clearer OPFS eviction policies in the W3C Storage Working Group to prevent divergent browser implementations.

By systematically addressing permissions, third-party risks, browser quirks, and recovery workflows, developers can significantly reduce OPFS data loss incidents. Persistent storage permissions remain the first line of defense, but a layered approach combining user education, redundancy, and vendor collaboration is essential for mission-critical applications.