Thread by @dvassallo

A to-do web app with a twist: All user data is end-to-end encrypted! 🔐

demo.encrypted.dev

The point of this demo is that it should feel indistinguishable from a regular web app, despite all database queries are running over encrypted data, and inside the browser.

Every to-do item gets encrypted by the browser with an AES-256 secret key that never goes to the server. The user gets data privacy, while the app developer gets spared the liability of handling unencrypted user data. A win-win.

For now, this is just a proof of concept. It shows that web apps like this can work & perform well without server-side db queries. With e2e encryption, all db queries must happen in the browser. The server is there just to handle access control & store the encrypted data.

So, while the user sees this:

The server only sees this:

Data gets stored as a transaction log with encrypted records.

The “update” at seq #3 was to mark item #2 as complete.

The “update” at seq #8 was an edit to put 🔑 in item #5.

The “delete” in seq #9 was to delete an item added at seq #7 (which doesn't show in the screenshot).

Whenever the browser runs a database query, it asks the server for any new transactions since the last seq # it had seen, and then it applies any new transactions to its store. So queries are not downloading the whole database every time.

The browser will also periodically compact the transaction log and re-upload it to the server — encrypted, obviously. This allows updates and deletes to get squashed whenever the transaction log reaches a certain size.

A major focus of the proof of concept was on performance. The app was load tested with 1K, 10K, & 100K to-do items. On a 2.9 GHz i9 MacBook Pro with 100 Mbps network, the app was able to fetch and decrypt 1K items in 0.3s, 10K in 0.8s, and 100K in 6.4s.

Once the app had loaded the data, querying it was nearly instantaneous. Performance is still an area under development though, and more results will be posted soon. But I think the results already look promising, and there’s still a ton of perf opportunities.

What’s next?

The framework will be extracted from this prototype. Anyone will be able to use it to build end-to-end encrypted web & mobile apps with. The framework will be 100% open source & MIT licensed.

After the OSS framework there will be a PaaS:

https://twitter.com/dvassallo/status/1126925437790081024?s=21

If you want to keep up to date with its progress you can subscribe to the mailing list: updates.encrypted.dev/subscribe. (No more than one or two emails a month.) And I’ll also be tweeting updates here on Twitter.

There are still a few things being explored that haven't been validated by this proof of concept:

- Data sharing across users.
- Key rotation or revocation.
- Real-time live queries & push notifications.
- Other ways of distributing the key across devices (beyond copy/pasting).

That’s it for now. Happy to answer questions. BTW, the demo is MIT open source too: github.com/encrypted-dev/…. You are free to try it out, fork it, modify it, build your own app based on it, compete with me using it, etc. You have my blessing.

Next week will explore the possibility of having an SQL interface for the local db. Today the app simply gets an array of items, so you'd have to query programmatically with JavaScript's filter/map/sort/etc. SQL could allow easier filters, aggregations, sorting, projections, etc.

One option could be to use an sqlite impl for the browser, such as this: github.com/kripken/sql.js/. So, when the server returns new transactions, they would get applied to sqlite (instead of the current JavaScript data structure), and then the app would simply query through sqlite.