persisting large wasm-sqlite3 datasets in browser (beyond kv-storage)

# kai zhu (3 months ago)

at work, we have browser-app that load-and-persist ~100MB (500k rows) csv-files into wasm-sqlite3 [1], (ingestion-time is ~15s for 100MB csv). we wish to go bigger, but chrome's indexeddb has a hard-limit of 125MB per key-value object (on windows).

i don't have anything actionable. just want people aware of datapoint, and think about javascript-language-design to improve UX-handling with sqlite3-persistence.

fyi, ignoring persistence, chrome can handle wasm-sqlite3 datasets as large as 300MB (1.5 million rows) in memory, before crashing (on 8gb windows10 machine).

[1] sql.js wasm-sqlite3 kripken/sql.js

# Isiah Meadows (3 months ago)

Not sure this pertains to the ECMAScript spec in any way. You may have better luck with WICG, because they are who deal with those specs - those are specific to the web, while JS is used in places where those might not even make sense (like IoT sensors).


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com