Intent to Deprecate and Remove Web SQL

3,815 views
Skip to first unread message

Ayu Ishii

unread,
Apr 28, 2023, 4:26:20 PM4/28/23
to blink-dev, Ben Morss
Design docs
https://developer.chrome.com/blog/deprecating-web-sql

[Google Internal] https://docs.google.com/document/d/1bTj_nDqbdvE102sCm3KuwvN5c_HneLNPl9mmPeUjG4M/edit?usp=sharing

[Google Internal] https://docs.google.com/document/d/1CDdEO65pCIo60NM8CWHNNN7EunJ-wd8v1dGUxTOBJrM/edit?resourcekey=0-R0fxP199QQ-8gnMqzmQyrw

Summary
The Web SQL Database standard was first proposed in April 2009 and abandoned in November 2010. It was implemented in WebKit in 2008 and shipped in Chrome and Safari, on both desktop and mobile. Gecko and EdgeHTML never implemented this feature and WebKit unshipped it in 2019. The W3C encouraged those needing web databases to adopt Indexed Database.

Since its release, it has been incredibly difficult to keep our users secure. SQLite was not designed to run untrusted SQL statements, and yet with Web SQL we have to do exactly this. Keeping up with security and stability fixes dictates updating SQLite in Chromium and impacts the feature’s stability. In 2022 alone, we updated SQLite 11 times. This comes in direct conflict with Web SQL’s requirement of behaving exactly as SQLite 3.6.19, and with the lack of a SQL specification in Web SQL, we cannot make any such compatibility guarantees.
With SQLite WASM as an effective replacement for web developers requiring a relational database, we would like to remove Web SQL entirely.

Target timeline

M101 - 123 - Enterprise Policy

M115 - Add deprecation message

M118-123  - Deprecation trial

M119 - Ship removal


Usage and Risk

Overall usage still shows a high percentage of 0.34% of page loads, however our analysis has concluded that very little usage is for actual storage.


Through analyzing sites from HTTPArchives, we found a majority of its usage is from outdated incognito detection (e.g. Criteo, Reddit),  and fingerprinting (e.g. Fingerprintjs, evercookie).


There are JS storage libraries that became popular around the time that Web SQL was introduced which use the feature. Oftentimes their usage is part of a fallback chain, where on modern browsers other storage technologies like localStorage or IndexedDB would be chosen before Web SQL. Examples of such libraries are localForage, cordova-sqlite-storage, Sencha Touch. Many, like localForage and cordova-sqlite-storage, gate its usage on feature detection due to its availability only on Chromium browsers. However older versions of Sencha Touch look as though they may not have been gated. Sencha Touch has since removed its SQL feature which depends on Web SQL in their version released in 2015.


Our conclusion from our HTTPArchives analysis is that we were only able to identify one site that is not gated by feature detection, and one site with significant breakage. We’ve notified open source libraries of Web SQL deprecation, and plan to reach out to site owners we’ve classified as breakage.


Analyzing extensions usage, we’ve identified 74% of extensions that use Web SQL are from JS storage libraries like localForage and cordova-sqlite-storage. However there were a higher number of usages that rely on Web SQL heavily, and many that are not gated by feature detection as well. We have identified these extensions and plan to contact the developers on this deprecation.


Further analysis for the web platform and extensions can be found in our public facing Web SQL usage analysis doc.


For those that would need to migrate, we expect a significant amount of work will be required. Therefore we would like to show deprecation messages early, and make a long deprecation trial available to allow developers to plan for their migration before full removal. We’ve provided steps for testing Web SQL removal for a website, and a guide to SQLite WASM and for migrating a database for developers to follow to start their migration. 


Communications

What we’ve done so far:

  • Worked with internal partners to move major products off of Web SQL (Completed in 2022)

  • Communicated to edu/enterprise partners of its planned removal (Aug, 2022)

    • No usages found from this process

  • Communicated with known external partners using Web SQL on its planned removal

    • All on board with migrating to WASM + SQLite

  • Published an article on the state of Web SQL and its deprecation (Aug, 2022)

  • Published an article on its recommended replacement, SQLite WASM (Jan, 2023)

  • Removed Web SQL in third party contexts in M97

  • Removed Web SQL in non-secure contexts in M110

  • Published an article for migrating a database from Web SQL to SQLite Wasm (Mar, 2023)

  • [InProgress] Communicate to identified developers in extensions / HTTPArchives usage


Related Intents

Intent to Deprecate and Remove Web SQL in 3rd Party Contexts

Intent to Deprecate and Remove Web SQL in Non-Secure contexts


Blink component Blink>Storage>Web SQL
Search tags Web SQL
Risks
Interoperability and Compatibility Removing Web SQL will align Chromium based browser behavior to all other browser engines. Currently Chromium is the only browser engine that supports Web SQL. Because of this, most websites gate the usage of Web SQL by feature detection.
Gecko: N/A Never implemented
WebKit: Unshipped (https://github.com/WebKit/WebKit/commit/761bce943c0696a6bb93116eb0576ed07dbfdc65) Removed in 2019
Web developers: N/A

Security Currently SQLite in Chromium is updated very frequently, sometimes in multiple consecutive milestones. The frequency is defined by stability or security issues found in the SQLite library. Bad security issues have historically surfaced such as Magellan 2.0 that had been publicized in tech news in 2019, among others.The storage team needs to respond quickly to these issues, and update the library when issues are found with help from the SQLite team and Release and Security TPMs. 
Removing Web SQL will permanently remove the attack vector of malicious SQL statements.  
WebView application risks
While we see a 0.02% usage on WebView, we are unable to verify the nature of this usage. However now that Deprecation Trials are supported for WebView, we think the risk of removal is significantly reduced.

Goals for Deprecation Trial

The goal for the deprecation trial is to allow for a 6 month window after removal to let developers remove their usage of Web SQL. We may extend this window depending on feedback from participating developers. Our recommendation is for developers to switch to SQLite compiled to WebAssembly backed by the Origin Private File System. We’ve published guidance for this migration in our developer blog.


Debuggability Planning to add a deprecation message in the console.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes, removal in all
Is this feature fully tested by web-platform-tests? No (Web SQL tested in web_tests)
DevTrial instructions Steps on how to test your site with Web SQL removed.  https://docs.google.com/document/d/1EMJSmKDVGVv0sbsRDz1b8-tTkzv9yi4S30-rzEiK9AQ/edit?usp=sharing

Flag name web-sql-access
Requires code in //chrome? False
Tracking bug https://crbug.com/695592

Link to entry on the Chrome Platform Status https://chromestatus.com/feature/5134293578285056

Yoav Weiss

unread,
May 3, 2023, 6:59:50 AM5/3/23
to blink-dev, Ayu Ishii, Ben Morss
LGTM1

Thanks for the details analysis. The rollout plan seems reasonable, and the low effective breakage gives me hope that this would stick.

Mike Taylor

unread,
May 3, 2023, 9:20:59 AM5/3/23
to Yoav Weiss, blink-dev, Ayu Ishii, Ben Morss

LGTM2 - kudos to the team for the very detailed compat and risk analysis, as well as proactively engaging in outreach. Good luck. :)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0b0f9e3a-c44a-4029-968b-5c3f2d77622fn%40chromium.org.

Mike West

unread,
May 3, 2023, 11:40:35 AM5/3/23
to Mike Taylor, Yoav Weiss, blink-dev, Ayu Ishii, Ben Morss
LGTM3. Good luck with the launch, I've been excited about this for a while and I'm looking forward to reducing our attack surface.

-mike


Ayu Ishii

unread,
May 3, 2023, 12:46:13 PM5/3/23
to blink-dev, Mike West, Yoav Weiss, blink-dev, Ayu Ishii, Ben Morss, Mike Taylor
Thank you for the review & approvals!
We will update the thread if there are changes to the timeline.

Best,
Ayu

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Thomas Steiner

unread,
May 3, 2023, 1:08:22 PM5/3/23
to Ayu Ishii, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor
Here's the PR that updates the removal timeline for the developer-facing article: https://github.com/GoogleChrome/developer.chrome.com/pull/6187

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f1b57ff4-7b66-4587-a57a-fda643c8073fn%40chromium.org.


--
Thomas Steiner, PhD—Developer Relations Engineer (https://blog.tomayac.comhttps://twitter.com/tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----
Version: GnuPG v2.3.4 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck0fjumBl3DCharaCTersAttH3b0ttom.hTtPs://xKcd.cOm/1181/
----- END PGP SIGNATURE -----

Thomas Steiner

unread,
May 12, 2023, 4:10:59 AM5/12/23
to Asier Lostalé, blink-dev, Thomas Steiner, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Ayu Ishii
Hi Asier,

- Does it mean that in M119 WebSQL will not be working by default, but it will still be possible to enable it back with this configuration setting? Would this be the case until M123?

As a site owner, you can take part in the deprecation trial, which will be available until Chrome 123 inclusive. It's supposed to be released on March 4, 2024. The PR is still pending, but here's a link to the relevant section in the staging preview.
 
- Would it be possible to be more precise in terms of dates? Or is still too early for this?

The current date when Chrome 119 is going to be released is October 30, 2023. The PR is still pending, but here's a link to the relevant section in the staging preview.
 
Cheers,
Tom

Asier Lostalé

unread,
May 12, 2023, 9:40:17 AM5/12/23
to blink-dev, Thomas Steiner, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Ayu Ishii
Hi all,

Thanks for communicating the plans for this removal with this additional information.

However, we still have two questions about this:
- Does it mean that in M119 WebSQL will not be working by default, but it will still be possible to enable it back with this configuration setting? Would this be the case until M123?
- Would it be possible to be more precise in terms of dates? Or is still too early for this?

Thanks a lot!

Asier Lostalé

unread,
May 16, 2023, 4:29:32 AM5/16/23
to blink-dev, Thomas Steiner, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Ayu Ishii, Asier Lostalé
Hi Thomas,

Thanks for your reply.

If possible, I'd like to clarify a couple of topics:

- I see there is already an "Allows access to WebSQL APIs" flag that can be used to force access to WebSQL. For how long is this flag planned to be kept? Will it be available from M119 to M123? What about after M123?
- As a site owner, how can I take part of the deprecation trial?

Regards,

Asier

Thomas Steiner

unread,
May 16, 2023, 4:36:24 AM5/16/23
to Asier Lostalé, blink-dev, Thomas Steiner, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Ayu Ishii
On Tue, May 16, 2023 at 10:29 AM Asier Lostalé <asier....@openbravo.com> wrote:
Hi Thomas,

Thanks for your reply.

If possible, I'd like to clarify a couple of topics:

- I see there is already an "Allows access to WebSQL APIs" flag that can be used to force access to WebSQL. For how long is this flag planned to be kept? Will it be available from M119 to M123? What about after M123?

Since the code is going to be removed, the flag will be removed as a consequence as well. Given the current timeline, I would not count for the code to exist after 123. 
 
- As a site owner, how can I take part of the deprecation trial?

Please see this article on origin trials. A deprecation trial works just the other way round: rather than granting your site early access to a future feature, it grants you late access to a past feature.

Cheers,
Tom

Ayu Ishii

unread,
Jun 26, 2023, 4:57:27 PM6/26/23
to blink-dev, tste...@google.com, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Ayu Ishii, Asier Lostalé
Hi blink owners,

With request from partners, we are planning to update the timeline to enable deprecation trial from M117 (previously M118) to provide a larger window to integrate with the trial before full removal in M119.
The rest of the timeline will stay the same.

NEW - Target timeline:

M101 - 123 - Enterprise Policy

M115 - Add deprecation message

M117-123  - Deprecation trial

M119 - Ship removal OLD - Target timeline:

M101 - 123 - Enterprise Policy

M115 - Add deprecation message

M118-123  - Deprecation trial

M119 - Ship removal

Thanks, Ayu

Rick Byers

unread,
Jun 26, 2023, 5:31:14 PM6/26/23
to Ayu Ishii, blink-dev, tste...@google.com, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé
Thanks for the update, no concerns with the change!

BTW, I've added WebSQL as the #1 example case around "ease of adaptation" in our blink compat principles. I tried to briefly summarize the story of how I (and others) were previously opposed to removal on the grounds that it took important functionality away from developers (eg. superhuman showing they couldn't match performance any other way), but are now supportive given the demonstrated viability of SQLLite in WAsm. Feel free to suggest edits and/or additional links.

Rick

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Hans Meyer

unread,
Jun 29, 2023, 10:57:12 AM6/29/23
to blink-dev, Ayu Ishii, tste...@google.com, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé

M117-123  - Deprecation trial


What does this mean in terms of when trial registration will be available at https://developer.chrome.com/origintrials/#/trials/active ?

I'm the author of an extension that's currently using Web SQL Database. I'd like to take advantage of the deprecation trial while I'm completing the IndexedDB implementation but I'm concerned there will not be enough of a window for 1mil+ users to receive the update to a version of the extension that contains the trial token before Sept 14 (119 date for Canary).

Hans

Thomas Steiner

unread,
Jun 30, 2023, 9:48:09 AM6/30/23
to Ayu Ishii, blink-dev, tste...@google.com, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé
Timeline updated in developer-facing comms: https://github.com/GoogleChrome/developer.chrome.com/pull/6725

Ayu Ishii

unread,
Jul 12, 2023, 7:59:41 PM7/12/23
to blink-dev, tste...@google.com, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé, Ayu Ishii
WebSQL Deprecation Trial registration is now available at https://developer.chrome.com/origintrials/#/view_trial/494270059103911937

Guillem Olivé

unread,
Sep 7, 2023, 1:29:36 PM9/7/23
to blink-dev, Ayu Ishii, tste...@google.com, blink-dev, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé
Hi,

I am working by a third-party enterprise and we are still using webSQL functionality. Our application will be refactored to remove it, but it will take some time to make it possible.
We are apprehensive about the disclaimer in the WebSQL Deprecation Trial registration: "I understand that I may need to limit use of this experimental feature to some subset of my users because of the limit whereby an experimental feature will be disabled if it is used on more than 0.5% of Chrome page loads (as a median over a 14 day period).". Will our production application not be able to work if the overall usage increases from 0.34% to 0.51% of page loads?

Thanks!

El dia dijous, 13 de juliol de 2023 a les 1:59:41 UTC+2, Ayu Ishii va escriure:

Panos Astithas

unread,
Sep 8, 2023, 4:24:25 PM9/8/23
to Guillem Olivé, blink-dev, Ayu Ishii, tste...@google.com, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé
Hi Guillem,

The page load limit only applies to trials that introduce new features, not deprecation trials like this one that remove a feature. Apologies for the confusion. 

Thanks,
Panos


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Bobby Langley

unread,
Feb 12, 2024, 7:56:48 PMFeb 12
to blink-dev, Panos Astithas, blink-dev, Ayu Ishii, tste...@google.com, Mike West, Yoav Weiss, Ben Morss, Mike Taylor, Asier Lostalé, Guillem Olivé
Just started using the origin trial in one of our apps that requires a slightly overly extertive lift to change from webSQL to IndexedDB. The expiry says May 28, 2024 -- what is the potential that date could be extended while we work to release our new version?

Etienne Noël

unread,
Feb 13, 2024, 5:24:34 AMFeb 13
to Bobby Langley, Asier Lostalé, Ayu Ishii, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Panos Astithas, Yoav Weiss, blink-dev, tste...@google.com
Hi,

We have been communicating this deprecation for a while now so unfortunately, we will not extend it further.

It’s a significant source of security issues so we will keep the original deprecation timeline.

If that can be useful to your situation, SQLite is available in the browser: 

It might make the migration easier for you if that’s a suitable solution.

Thank you,

Etienne


Etienne Noël

Engineering Manager

Chrome Storage



Antonio MORENO

unread,
Feb 23, 2024, 11:02:48 AMFeb 23
to blink-dev, Etienne Noël, Asier Lostalé, Ayu Ishii, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Panos Astithas, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi all,

Related to this topic, a bit of a simple question: In the initial announcement, it seemed to indicate that Chrome 123 would be the first version without any kind of support for WebSQL. However, at least in the current version of the beta (123.0.6312.4) support is still there, and can be enabled with the corresponding flag. Can you confirm if this will be the case with the stable version of 123, and the feature will be removed only in version 124?

Thanks, and regards,

Antonio.

Ayu Ishii

unread,
Feb 23, 2024, 1:15:46 PMFeb 23
to blink-dev, Antonio MORENO, etien...@google.com, Asier Lostalé, Ayu Ishii, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi Antonio,

That is correct, 123 will be the last supported version via flag or policy.
124 will be the first version WebSQL will no longer be available even with a flag or policy.

Best,
Ayu

Clémence Vaz

unread,
Feb 28, 2024, 10:29:39 AMFeb 28
to blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi all :)

Have you decided a timeline for WebSQL support removal on Android WebViews ?
We've already developed a SQLite alternative but there will be performance impacts on our mobile app as soon as WebSQL is disabled and we need to anticipate and communicate with our clients about the "switch" date.

Thank you very much for your help !

Clémence.

Ayu Ishii

unread,
Feb 28, 2024, 1:00:35 PMFeb 28
to blink-dev, Clémence Vaz, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi Clémence,

We're still working with the WebView team to nail down the exact timeline, but will update this thread once it is decided.

For context for others, there is a bug in WebView removal where the API is still accessible when WebsiteSettings.SetDatabaseEnabled is set to true. While the fix is not hard, we've learned since the initial Intent that the Deprecation Trial, while supported on WebView, cannot be used for all cases of WebView. So we are working with the WebView team to further assess the usage and decide on the timeline of making this change. We will keep this thread updated once this is decided.

Best,
Ayu

Hans Meyer

unread,
Mar 3, 2024, 10:21:18 AMMar 3
to blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi Ayu,

Can you clarify... are you saying the deprecation trial does NOT extend to Chrome 124? I would not have expected that given that the trial's expiration date has been communicated consistently as May 28, 2024.

Thanks,
Hans

Anton Bershanskyi

unread,
Mar 3, 2024, 12:50:43 PMMar 3
to blink-dev, Hans Meyer, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Hi Hans,

Could you please clarify which web page suggests that the WebSQL availability would extend to Chrome 124 inclusive? All news/articles I could find online suggest that the last version supporting WebSQL would be 123:
Also, the DevTools UI already removed WebSQL viewer from "Application" tab. The change will arrive in Chrome Canary next week. WebSQL flag is also considered to be expired (no longer available and will be removed from source code soon).

Sincerely,
Anton.

Hans Meyer

unread,
Mar 4, 2024, 9:24:42 AMMar 4
to blink-dev, Anton Bershanskyi, Hans Meyer, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, past...@google.com, Yoav Weiss, blink-dev, tste...@google.com, Bobby Langley
Fair enough Anton. Admittedly a bit of wishful thinking on my part.

Although FWIW, the origin trial status in dev tools only mentions the expiration date, not version. By no means an excuse, but I'm curious, why express the trial expiration in terms of a date at all, much less a date that is 1.5 months after from when it effectively ends for most users?

Again, no excuses. I agree the versioning was clearly communicated. Just wanted to give some feedback to consider to possibly minimize confusion in the future.

Thanks for your response,
Hans

Panos Astithas

unread,
Mar 5, 2024, 1:30:11 PMMar 5
to Hans Meyer, blink-dev, Anton Bershanskyi, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
The difference between the last trial milestone and the end date is due to the fact that origin trial tokens contain the expiration date by design. Given that the token needs to expire after the ending milestone has passed and given the often lengthy browser update cycle for some users, we ended up with a compromise that expects most clients will have been updated when the second to next milestone after the ending one has been released (4+4 weeks later). It is really aiming at avoiding the origin trial feature being prematurely unavailable to users.

Granted this is not very obvious, but it has worked reasonably well in practice.

Thanks,
Panos

Hans Meyer

unread,
Mar 7, 2024, 3:43:39 PMMar 7
to blink-dev, Panos Astithas, blink-dev, Anton Bershanskyi, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley, Hans Meyer
Thanks for the explanation Panos. That makes sense.

OK so... please forgive me folks. I know this is a crazy thing to ask, I really do, but I need to take the shot.

Is there ANY possibility whatsoever that access to the API can be preserved through 124?

I've been grinding away for a year on a major extension rewrite that I'll be ready to submit to the Web Store early next week. The upgrade path includes auto migration from Web SQL to IndexedDB. This extension has 1 million users and I'm kind of freaking out about the number of them who will miss the upgrade window and lose their data.

I know it's a long shot and this is an 11th hour ask and I'm not special by any means and I could've been more diligent about heeding the deadline... all true... but, well... as embarrassing as it might be, I have to ask.

Respectfully,
Hans

Anton Bershanskyi

unread,
Mar 7, 2024, 4:54:53 PMMar 7
to blink-dev, Hans Meyer, Panos Astithas, blink-dev, Anton Bershanskyi, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Hi Hans,

It's unfortunate that you find yourself in this situation. I'm not a Google or Chromium representative, but official representatives answered this exact question before and consistently rejected any further deadline changes. However, you can use SQL in the browser via a polyfil library (WASM-based SQLite port). One can relatively trivially export data from WebSQL into a different browser store (e.g., OPFS) and then continue using existing WebSQL code via the polyfil. If you need help, I can show you how to prepare two incremental extension releases which do this: (1) small revision (with short review time) which will back up existing data from WebSQL into other datastore while still using WebSQL for all work and (2) larger revision which will use the new datastore and SQLite polyfil with existing WebSQL call sites. After this, you won't have any time pressure and will be able to work on refactoring code like converting WebSQL schema to a web-native storage of your choice.

Sincerely,
Anton.

Thomas Steiner

unread,
Mar 8, 2024, 5:51:36 AMMar 8
to Anton Bershanskyi, blink-dev, Hans Meyer, Panos Astithas, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
In case it's any helpful, I have written a migration guide on how to go from Web SQL to SQLite. It's admittedly only tested on toy data, so your actual mileage may vary, but at least the overall mechanics should be somewhat useful at least.
Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.comtoot.cafe/@tomayac)

Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891

----- BEGIN PGP SIGNATURE -----
Version: GnuPG v2.4.3 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck
0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

Hans Meyer

unread,
Mar 9, 2024, 12:23:09 AMMar 9
to blink-dev, Anton Bershanskyi, Hans Meyer, Panos Astithas, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Thank you Anton and Thomas for your responses. I really appreciate it.

Anton, I did some experimentation with the WASM-based SQLite port today and I'm curious to understand more about the approach you mentioned. In particular:

1) Am I correct in understanding that there's a polyfil that allows the use of the WebSQL API surface with the WASM-based SQLite port? That would be awesome. Do you have a link for this?
2) I'm a little confused about the necessity for and distinction between the two separate releases. Can you clarify?
3) If I will have the new release ready to submit to CWS by this Tuesday anyway, I'm wondering if it's worth the lift/risk of pursuing these interim releases. It seems there is only something gained if the interim releases are ultimately approved sooner than the new release... unless I'm missing something?

Thanks again Anton. I really appreciate your help and I'm happy to continue the convo outside this thread if that's best.

Sincerely,
Hans

Hans Meyer

unread,
Mar 9, 2024, 12:40:02 AMMar 9
to blink-dev, Anton Bershanskyi, Hans Meyer, Panos Astithas, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Does anyone know if the WebSQL db file itself will remain intact?

I'm hoping yes, as that would preserve some manual options for recovery.

Hans



Anton Bershanskyi

unread,
Mar 9, 2024, 6:51:41 AMMar 9
to blink-dev, Hans Meyer, Anton Bershanskyi, Panos Astithas, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Hi Hans,

> Am I correct in understanding that there's a polyfil that allows the use of the WebSQL API surface with the WASM-based SQLite port?

I'm not aware of such polyfil, however I'm not sure why you would need one, considering the similarity of the WebSQL API and WASM port: both are merely wrappers for SQLite 3, so you would just need to change the initialization code to talk to WASM wrapper instead of window.openDatabase().

> I'm a little confused about the necessity for and distinction between the two separate releases. ... I will have the new release ready to submit to CWS by this Tuesday

I just proposed this as a way to speed up migration, but since you already have a WebSQL-free version ready already, you probably can just release that.

> Does anyone know if the WebSQL db file itself will remain intact?

There are no guarantees about on-disk files since they are not meant to be read or edited manually... But based on past experience, there is usually no rush to clean up old code, so the files might stay there for some time. Again, no guarantees. You can recover this files like so:
  1. Go to chrome://profile-internals , find the relevant profile and copy its "Profile Path"
  2. Go to that folder and navigate to ./databases/chrome-extension_<your_extension_id>_<number>
  3. There will be a file which you can open in any SQLite viewer
Also, note the following: in order for user to migrate data, the user will need to start up browser and load every profile with your extension after you publish the extension update but before Chrome 124 is installed. In fact, it is a known issue that when Chrome changes profile data format and deprecates migration code, some profiles do not get migrated because they are used so rarely.

Sincerely,
Anton.

Hans Meyer

unread,
Mar 9, 2024, 9:29:36 AMMar 9
to blink-dev, Anton Bershanskyi, Hans Meyer, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
>  I'm not aware of such polyfil, however I'm not sure why you would need one

Ah ok. I misunderstood. This is actually the biggest part of the lift and risk in my particular case. There's a ton of nested db.transaction()/tx.executeSql() calls that would need to be rewritten and the app in some cases relies directly on SQLResultSet. It's messy.

> the user will need to start up browser and load every profile with your extension after you publish the extension update but before Chrome 124

This is my main concern. I understand some of this is inevitable. Just trying to think of ways to mitigate data loss given current circumstances.

> There are no guarantees about on-disk files

Understood. Just wondering if there were any engineers or engineer-adjacent folks in the thread who were privy to that bit of detail about the work currently being done to extract WebSQL.

For now I'm gonna focus on wrapping up the release and hope for a speedy CWS approval.

Thanks again Anton.

Cheers,
Hans

Anton Bershanskyi

unread,
Mar 9, 2024, 7:12:34 PMMar 9
to blink-dev, Hans Meyer, Anton Bershanskyi, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Hi Hans,

> bit of detail about the work currently being done to extract WebSQL.

You might be interested in this bug titled "Delete WebSQL data after API removal". Also, data deletion will occur only after the actual API code is deleted, which will take some time. For reference, I looked into what it would take and came up with this ~18K LOC patch (and I'm not done writing tests for it yet!).

Sincerely,
Anton.

Hans Meyer

unread,
Mar 11, 2024, 6:13:45 PMMar 11
to blink-dev, Anton Bershanskyi, Hans Meyer, blink-dev, Ayu Ishii, Antonio MORENO, etien...@google.com, Asier Lostalé, Ben Morss, Guillem Olivé, Mike Taylor, Mike West, Yoav Weiss, tste...@google.com, Bobby Langley
Apparently my earlier response got stuck in moderation but for the record this was exactly what I was looking for Anton. Thank you!

Hans
Reply all
Reply to author
Forward
0 new messages