LocalStorage or SQLite data will no longer be backup and can be deleted

16,120 views
Skip to first unread message

Samuel M

unread,
Jan 7, 2012, 10:12:44 AM1/7/12
to phonegap
Hi all,

I hope I am wrong, but after installing iOS5.1 beta2, I see that the
app data of my phonegap app are now stored in the Library/Cache
folder :
See this screenshot my app data in iExplorer :
http://cl.ly/1p3L1U2W1J3y3I1T1r3y
(before it was in Library/WebKit folder).

So now, it means that all WebKit Data (sqlite db, localstorage,
indexedDb) will not be backup in iCloud, and worse, they can be
deleted if the phone has a low memory problem.
Some link :
https://developer.apple.com/icloud/documentation/data-storage/
and the story of InstaPaper (for deleted data stored in Cache) :
http://www.marco.org/2011/10/13/ios5-caches-cleaning

I will work right now on a backup of my users data on my own servers.
I have users using my apps since 2 years, and they have generated a
lot of data. It would be catastrophic if they loose everything. Hope
Apple will fix that. Maybe we should fill a bug report?

Samuel

Phillip Gooch

unread,
Jan 7, 2012, 1:00:21 PM1/7/12
to phonegap
Yes, at least one method should be a safe one to store info.

Samuel M

unread,
Jan 7, 2012, 5:05:32 PM1/7/12
to phonegap
It's very problematic for all phonegap apps using localstorage or
SQLite. What can we do in order to fix that?

russgum

unread,
Jan 7, 2012, 7:48:31 PM1/7/12
to phonegap
Is there a way toe save an SQlite file in the application area instead
or in addition to where it is currently stored? I need to be able to
sync a db file to the cloud so I can share it with other users.

Russ

Ken OKABE

unread,
Jan 8, 2012, 2:10:02 AM1/8/12
to phon...@googlegroups.com
Apple focuses on iCloud especially from iOS5.01
http://developer.apple.com/icloud/documentation/data-storage/

I personally have used HTML5 LocalStorage technique and PhoneGap File
API together, but when I submited to AppleStore, it was rejected due
to dataStorage guideline.

Application/Documents folder looks automatically backed up by iCloud,
and HTML5 localstorage is out of the world of iCloud, so using both
technique may occur inconsistency of the application behavior and
dangerous.

If you use HTML5 localStorage only for cash stuff, it would be no
problem, but I think now it's rather easier to design or safe to use
native storage via fileAPI for whole data which needs to be backup.

> --
> You received this message because you are subscribed to the Google
> Groups "phonegap" group.
> To post to this group, send email to phon...@googlegroups.com
> To unsubscribe from this group, send email to
> phonegap+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/phonegap?hl=en?hl=en
>
> For more info on PhoneGap or to download the code go to www.phonegap.com

Samuel M

unread,
Jan 8, 2012, 9:38:29 AM1/8/12
to phonegap
We can make a phonegap plugin to regularly copy the storage files in
the Documents directory.
And if the DB has been wiped out, copy the backup in the Caches.
But I don't really like the idea...

Samuel M

unread,
Jan 8, 2012, 9:45:03 AM1/8/12
to phonegap
I don't understand why they are doing that to save some Kbytes in
iCloud. In my apps, the SQLite file more the localStorage file is
about 1,6Mo (not too much regarding the free 2Go of iCloud). We should
at least have the choice where to store this files (in Documents or in
Caches)
Do you think we need to fill a bug?

On Jan 8, 2:10 am, Ken OKABE <kenok...@gmail.com> wrote:
> Apple focuses on iCloud especially from iOS5.01http://developer.apple.com/icloud/documentation/data-storage/

russgum

unread,
Jan 8, 2012, 1:28:51 PM1/8/12
to phonegap
Is there any way to just copy the db with javascript without a plugin?
If I could copy it to a dropbox folder and later reload it from a
dropbox folder my problem would be solved. When I looked at the
documentation for native apps it seemed that there were ways to store
the db in the apps folder. I am wondering if it would be better to
just bite the bullet and write a native app in objective C?

Russ

Simon MacDonald

unread,
Jan 9, 2012, 1:30:36 PM1/9/12
to phon...@googlegroups.com
No, you need to write some native code into order to copy from your
asset folder to the file system. I keep hinting for someone on the
list to create a plugin but no one has picked up the torch yet and I'm
off doing other things at the moment.

Simon Mac Donald
http://hi.im/simonmacdonald

Samuel M

unread,
Jan 9, 2012, 3:46:26 PM1/9/12
to phonegap
@Simon: a plugin to do what? To copy the DB file in the Document
folder?

I am looking for a solution to automatically synch a WebSQL DB to a
server. There is this thread on StackOverflow:
http://stackoverflow.com/questions/1744522/best-way-to-synchronize-local-html5-db-websql-storage-sqlite-with-a-server
But every lib is dependant of another framework...
I am ready to pay for a simple solution (I don't have the time to code
it). Could be a great idea of open source project.


On Jan 9, 1:30 pm, Simon MacDonald <simon.macdon...@gmail.com> wrote:
> No, you need to write some native code into order to copy from your
> asset folder to the file system. I keep hinting for someone on the
> list to create a plugin but no one has picked up the torch yet and I'm
> off doing other things at the moment.
>
> Simon Mac Donaldhttp://hi.im/simonmacdonald

Simon MacDonald

unread,
Jan 9, 2012, 3:56:57 PM1/9/12
to phon...@googlegroups.com
Exactly, and Gaurav already wrote the code for Android and iOS.
Someone just needs to wrap this in a plugin for both platforms and
come up with a common JS interface that folks can follow on other
platforms.

http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phonegap.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+GauravSTomarBootstrappingIntelligence+%28Gaurav+S+Tomar+%3A+Bootstrapping+Intelligence%29

Samuel M

unread,
Jan 9, 2012, 4:38:31 PM1/9/12
to phonegap
Ok, but it doesn't solve the problem that the database is no longer
saved by iCloud.
Now the DB file of WebKit is located in the /Caches folder...

On Jan 9, 3:56 pm, Simon MacDonald <simon.macdon...@gmail.com> wrote:
> Exactly, and Gaurav already wrote the code for Android and iOS.
> Someone just needs to wrap this in a plugin for both platforms and
> come up with a common JS interface that folks can follow on other
> platforms.
>
> http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phoneg...
>
> Simon Mac Donaldhttp://hi.im/simonmacdonald
>
>
>
>
>
>
>
> On Mon, Jan 9, 2012 at 3:46 PM, Samuel M <samuel.miche...@gmail.com> wrote:
> > @Simon: a plugin to do what? To copy the DB file in the Document
> > folder?
>
> > I am looking for a solution to automatically synch a WebSQL DB to a
> > server. There is this thread on StackOverflow:
> >http://stackoverflow.com/questions/1744522/best-way-to-synchronize-lo...

KyrePh

unread,
Jan 10, 2012, 1:23:52 AM1/10/12
to phonegap
@Ken OKABE
Interesting... when you say you used HTML5 LocalStorage and the File
API together, how did you specifically use them together that caused
Apple to reject your app?
I'm now assuming that as long as you use the File API only for
persistent storage, and LocalStorage only for temporary stuff/caching,
and don't mix them up, both techniques would be okay to use together
in an app.... right?

On Jan 8, 3:10 pm, Ken OKABE <kenok...@gmail.com> wrote:
> Apple focuses on iCloud especially from iOS5.01http://developer.apple.com/icloud/documentation/data-storage/

Samuel M

unread,
Jan 10, 2012, 1:38:33 PM1/10/12
to phonegap
I think will implement "my own iCloud" to backup the WebSQL data of my
apps before iOS5.1 is released. I can't find another solution..
> > >> >> And if the DB has been wiped out, copy thebackupin the Caches.
> > >> >> But I don't really like the idea...
>
> > >> >> On Jan 7, 7:48 pm, russgum <russ...@gmail.com> wrote:
>
> > >> >> > Is there a way toe save an SQlite file in the application area instead
> > >> >> > or in addition to where it is currently stored? I need to be able to
> > >> >> > sync a db file to the cloud so I can share it with other users.
>
> > >> >> > Russ
>
> > >> >> > On Jan 7, 2:05 pm, Samuel M <samuel.miche...@gmail.com> wrote:
>
> > >> >> > > It's very problematic for all phonegap apps using localstorage or
> > >> >> > > SQLite. What can we do in order to fix that?
>
> > >> >> > > On Jan 7, 1:00 pm, Phillip Gooch <phillip.go...@gmail.com> wrote:
>
> > >> >> > > > Yes, at least one method should be a safe one to store info.
>
> > >> >> > > > On Jan 7, 7:12 am, Samuel M <samuel.miche...@gmail.com> wrote:
>
> > >> >> > > > > Hi all,
>
> > >> >> > > > > I hope I am wrong, but after installing iOS5.1 beta2, I see that the
> > >> >> > > > > app data of my phonegap app are now stored in the Library/Cache
> > >> >> > > > > folder :
> > >> >> > > > > See this screenshot my app data in iExplorer :http://cl.ly/1p3L1U2W1J3y3I1T1r3y
> > >> >> > > > > (before it was in Library/WebKit folder).
>
> > >> >> > > > > So now, it means that all WebKit Data (sqlite db, localstorage,
> > >> >> > > > > indexedDb) will not bebackupin iCloud, and worse, they can be
> > >> >> > > > > deleted if the phone has a low memory problem.
> > >> >> > > > > Some link :https://developer.apple.com/icloud/documentation/data-storage/
> > >> >> > > > > and the story of InstaPaper (for deleted data stored in Cache) :http://www.marco.org/2011/10/13/ios5-caches-cleaning
>
> > >> >> > > > > I will work right now on abackupof my users data on my own servers.

Hugh Johnson

unread,
Jan 10, 2012, 1:44:47 PM1/10/12
to phon...@googlegroups.com
Are both LocalStorage and WebSQL data deleted at the same time? I use both and if only WebSQL is deleted but not LocalStorage my app will have issues.

Ken OKABE

unread,
Jan 10, 2012, 3:10:15 PM1/10/12
to phon...@googlegroups.com
In fact, currently I am in a process to re-submit the rejected app,
and they just gave me the info it's due to iOS Data Storage
Guidelines.

https://developer.apple.com/icloud/documentation/data-storage/

I have used the localStorage technique for some 'index' file(small
data, frequently re-written) which have pointers to big data saved
with phoneGap API(this is stored to ApplicationDocument folder).

So, here is my guess:
They have 2 iOS devices with iCloud feature tuned on, then have my app
sync on the both devices via iCloud.
Obviously, there will be some inconsistency. I don't think my app
crashes or the inconsistency appears obvious, but I assume they
noticed somewhere.

This is the only possibility I can guess in terms of the storage
issue, so I re-designed to use phoneGap file API instead of
localStorage, that change is not trivial to me, because I needed to
manage async call backs for fileAPI.

Having said that


>I'm now assuming that as long as you use the File API only for
persistent storage, and LocalStorage only for temporary stuff/caching,
and don't mix them up, both techniques would be okay to use together
in an app.... right?

Yes, true.
I prefer to use localStorage especially in the middle of logic chain,
since it's sync and easier to handle,
but it turned out that mix usages logically like mine has problem for
iCloud backup. You should be careful.

Ken

Samuel M

unread,
Jan 10, 2012, 3:52:52 PM1/10/12
to phonegap
Your case is another problem.
The fact is, that we can no longer rely on WebSQL to store user data,
because they will be deleted (or at least not saved in iCloud).
It would be great if phonegap add a native bridge to a native SQLite
database (stored in Document folder). I think it has been done in
Android for version < 2.2.
> iCloudbackup. You should be careful.
> >> >> > > indexedDb) will not bebackupin iCloud, and worse, they can be
> >> >> > > deleted if the phone has a low memory problem.
> >> >> > > Some link :https://developer.apple.com/icloud/documentation/data-storage/
> >> >> > > and the story of InstaPaper (for deleted data stored in Cache) :http://www.marco.org/2011/10/13/ios5-caches-cleaning
>
> >> >> > > I will work right now on abackupof my users data on my own servers.

Ken OKABE

unread,
Jan 10, 2012, 4:23:48 PM1/10/12
to phon...@googlegroups.com
My response above is to @KyrePh question to me.

My response to you would be as follows:
'a native bridge' can be done by someone who can write a Phonegap Plugin.
How big is the WebSQL data?
If it's small enough to handle, if I were in your case, I write some
js code to back up to the document folder via fileAPI, and reconstruct
the WebSQL database periodically.

Samuel M

unread,
Jan 11, 2012, 8:05:47 PM1/11/12
to phonegap
Thanks.

I have created a bug report :
https://bugreport.apple.com/cgi-bin/WebObjects/RadarWeb.woa/6/wo/6Lqn29SZtTD1dEAqAUY0ow/15.83.28.0.9

Please, vote for this bug. It's crucial for every apps using
localstorage or WebSQL in phonegap.

ziobudda@gmail

unread,
Jan 11, 2012, 9:58:41 PM1/11/12
to phon...@googlegroups.com
Il giorno 12/gen/2012, alle ore 02:05, Samuel M ha scritto:


Url not works. 

Any "patch" or workaround while Apple fix it ? 

M.

--
Michel 'ZioBudda' Morelli                       mic...@ziobuddalabs.it
Sviluppo applicazioni CMS DRUPAL e web dinamiche (LAMP+Ajax)
0200619074 - 3939890025 (mobile)--  Fax: +39-0291390660
http://www.ziobuddalabs.it                      Skype: zio_budda

ScottP

unread,
Jan 11, 2012, 10:18:45 PM1/11/12
to phonegap
I couldn't access the URL above either.

On Jan 11, 8:05 pm, Samuel M <samuel.miche...@gmail.com> wrote:
> Thanks.
>
> I have created a bug report :https://bugreport.apple.com/cgi-bin/WebObjects/RadarWeb.woa/6/wo/6Lqn...

Samuel M

unread,
Jan 12, 2012, 8:31:48 AM1/12/12
to phonegap
@ziobudda : there is no workaround, Apple really need to fix that
before public release (and we are close, because it's the beta 3), or
it will be catastrophic for all phonegap app using localstorage or
WebSQL.

I got an answer from an Apple guy, and they explain they did that to
save space, because with apps loading a lot of content into a
UIWebView (like twitter), it takes a lot of space :
https://devforums.apple.com/thread/137882?tstart=0
But they completely forgot us, poor phonegap dev, relying into
LocalStorage or WebSQL to store users data.

Sorry for the bug URL. I just learn that the bug report are private,
and there is no way to vote for a bug (http://stackoverflow.com/
questions/144873/can-i-browse-other-peoples-apple-bug-reports).
The only way to make a bug report important and visible, is to create
duplicate bug report for the same bug. I invite you to create a bug
report here :https://bugreport.apple.com

Here is what I wrote :

Summary:
WebKit data (localstorage or local SQLite) are now stored in Library/
Caches folder (instead of Library/WebKit folder). This is a big
problem for all apps using UIWebView and storing user data, because
they will no longer be backup and may be deleted. There are a lot
HTML5 apps (cf. phonegap) using localstorage or SQLite as a critical
feature.

Steps to Reproduce:

In a UIWebView, create a new DB or use localStorage with JavaScript.
You can see that the WebKit data is now stored in Library/Caches :
http://cl.ly/1p3L1U2W1J3y3I1T1r3y

Expected Results:

We should at least specify the directory of WebKit data (and to set it
to Documents/ for critical data)

Actual Results:

The WebKit data are stored in Library/Caches folder, and can be
deleted

Regression:

The WebKit data was stored in Library/WebKit folder, and was correctly
backup by iCloud

Samuel M

unread,
Jan 12, 2012, 8:48:26 AM1/12/12
to phonegap
We really need to make some noise in order to be listened by Apple.
Phonegap apps will not have a good possibility to store data (except
file API). And existing apps will lose user data. We are already on
beta 3, so they may ship the public release soon.

I also forgot to say that the database is deleted when users update
from 5.01 to 5.1 :
https://devforums.apple.com/thread/137882?tstart=0
and
https://devforums.apple.com/thread/134820?tstart=0
> > > > > For more options, visit this group at...
>
> read more »

WebSteve

unread,
Jan 12, 2012, 10:59:38 AM1/12/12
to phonegap
Samuel,
When I click on the link, I get a bunch of drop-down menus. I can't
find the bug report in order to vote or rank it. Please give us
directions so we can participate!

Regards,
Steve

On Jan 11, 5:05 pm, Samuel M <samuel.miche...@gmail.com> wrote:
> Thanks.
>
> I have created a bug report :https://bugreport.apple.com/cgi-bin/WebObjects/RadarWeb.woa/6/wo/6Lqn...

jpburns

unread,
Jan 12, 2012, 11:32:05 AM1/12/12
to phonegap
I'm using the plugIn ApplicationPreferences to save my user settings
(a very small bit of data). I assume this method isn't affected by
Apple's decision, since it uses the native method of saving this info.

WebSteve

unread,
Jan 12, 2012, 11:33:38 AM1/12/12
to phonegap
Samuel M,
Your steps don't follow "the description format required by Apple
Engineering." Not sure how high it will be ranked among the engineers.
If we are going to "make noise," let's make it harmonious.

Regards,
Steve

Samuel M

unread,
Jan 12, 2012, 12:09:17 PM1/12/12
to phonegap
WebSteve,
It's not possible to vote for the bug. Because the Apple bugs report
are private.
The only way to make a bug report important and visible, is to create
duplicate bug report for the same bug. I invite you to create a bug
report here : https://bugreport.apple.com

Samuel M

unread,
Jan 12, 2012, 12:45:58 PM1/12/12
to phonegap
WebSteve,
As I said, it's not possible to vote for the bug, because Apple bugs
are private.
The only way to make a bug report important and visible for Apple, is
to create
duplicate bug report for the same bug. If they see a lot of bug report
for the same bug, they will look at it.
I invite you to create a new bug report here : https://bugreport.apple.com

On Jan 12, 10:59 am, WebSteve <shust...@aeteam.net> wrote:

WebSteve

unread,
Jan 12, 2012, 2:01:14 PM1/12/12
to phonegap
Samuel M, it looks like our messages crossed.

Due to the user confidentiality forms we signed, we are not really
supposed to talk about unreleased aspects of iOS, which includes this
issue. These issues are supposed to be confined to the Apple
Developers Forum. We could get in trouble if Apple decides to take
notice of these posts.

Regards,
Steve

Samuel M

unread,
Jan 12, 2012, 2:27:07 PM1/12/12
to phonegap
Yes, you are right. I wanted to warn phonegap users, because the
problem is critical IMHO..
Let's continue the discussion on thoses threads :

https://devforums.apple.com/message/602112#602112

https://devforums.apple.com/thread/137882?tstart=0

https://devforums.apple.com/message/602863#602863

ziobudda@gmail

unread,
Jan 12, 2012, 5:46:53 PM1/12/12
to phon...@googlegroups.com
Il giorno 12/gen/2012, alle ore 20:27, Samuel M ha scritto:

Yes, you are right. I wanted to warn phonegap users, because the
problem is critical IMHO..

Any possible plugin that resolv it (like a workaround) ? 

Samuel M

unread,
Jan 14, 2012, 2:50:57 PM1/14/12
to phonegap
No workaround I know.
If you want that apple change this behaviour, you should report a bug
at https://bugreport.apple.com
There is no way to vote for an existing bug, the only way is to submit
several reports.

By the way, I am developing a JS lib to synchronize a local SQLite DB
to a server. I will release it in github once finished and tested
(will work all the week end on it).

Sam

ScottP

unread,
Jan 14, 2012, 10:28:03 PM1/14/12
to phonegap
Here's my workaround so far... Using the info in this link:
http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phonegap.html
I modified my AppDelegate.m
Probably could do this in a plugin, but it was quicker for me to just
mod the AppDelegate. Maybe you could use fileWriter that is built in
Phonegap but I'm not sure.
Here's my mod - it looks for a database called 0000000000000001.db in
the Documents folder.
If it is not there then it will look in either Library/Webkit/
databases or Library/Caches and copy the webkit database over to
Documents folder.
That takes care of the DB issue with regards to backup and "cleaning/
deleting". Now just have to worry about accessing the copied DB in
the Documents folder.
I'm pretty new to Obj-C so maybe could use some cleaning up.

NSString *databaseName = @"0000000000000001.db";

//Get Library path
NSArray *libraryPaths =
NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,
NSUserDomainMask, YES);
NSString *libraryDir = [libraryPaths objectAtIndex:0];

//Get Documents path
NSArray *paths =
NSSearchPathForDirectoriesInDomains(NSDocumentDirectory,
NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *databaseFile = [documentsDirectory
stringByAppendingPathComponent:databaseName];

//Get <iOS5.1 webkit db path
NSString *olddatabasePath = [libraryDir
stringByAppendingPathComponent:@"WebKit/Databases/file__0/"];
NSString *olddatabaseFile = [olddatabasePath
stringByAppendingPathComponent:databaseName];

//Get iOS5.1 webkit db path
NSString *newdatabasePath = [libraryDir
stringByAppendingPathComponent:@"Caches/file__0/"];
NSString *newdatabaseFile = [newdatabasePath
stringByAppendingPathComponent:databaseName];

BOOL success;
BOOL oldWebkitDb;
BOOL newWebkitDb;

NSFileManager *fileManager = [NSFileManager defaultManager];

success = [fileManager fileExistsAtPath:databaseFile];
oldWebkitDb = [fileManager fileExistsAtPath:olddatabaseFile];
newWebkitDb = [fileManager fileExistsAtPath:newdatabaseFile];

// Need to copy from <App home>Library/Cache to <app home>/
Documents
// Need to copy from <App home>Library/Webkit to <app home>/
Documents

// If the database in Documents folder already exists then return
without doing anything
if(success)
{
return [ super webViewDidStartLoad:theWebView ];
}
else if(oldWebkitDb)
{
// If the old webkit db is there then proceed to copy the
database to the users Documents

[fileManager createDirectoryAtPath:databasePath
withIntermediateDirectories:YES attributes:nil error:NULL];

// Copy the database from the package to the users filesystem
[fileManager copyItemAtPath:olddatabaseFile
toPath:databaseFile error:nil];

// depending on your ARC setting you could uncomment the release
//[fileManager release];
}
else if(newWebkitDb)
{
// If the new webkit db is there then proceed to copy the
database to the users Documents

// Create the database folder structure
[fileManager createDirectoryAtPath:databasePath
withIntermediateDirectories:YES attributes:nil error:NULL];

// Copy the database from the package to the users filesystem
[fileManager copyItemAtPath:newdatabaseFile
toPath:databaseFile error:nil];

//[fileManager release];
}
else
{
return [ super webViewDidStartLoad:theWebView ];
}

My plan for accessing the copied DB is using the sqlite DB plugin
found here:
https://github.com/davibe/Phonegap-SQLitePlugin
I've installed it and added to my app, and did a quick test to see if
it could access the copied DB and it did. This is the test I used:

function testDB()
{
var db;
db = new PGSQLitePlugin("0000000000000001.db");
db.executeSql('CREATE TABLE IF NOT EXISTS test_table (id integer
primary key, data text, data_num integer)');
db.transaction(function(tx) {
return tx.executeSql(["INSERT INTO test_table (data, data_num) VALUES
(?,?)", "test", 100], function(res) {
console.log("insertId: " + res.insertId + " -- probably 1");
console.log("rowsAffected: " + res.rowsAffected + " -- should be 1");
return db.executeSql("select count(id) as cnt from test_table;",
function(res) {
console.log("rows.length: " + res.rows.length + " -- should be 1");
return console.log("rows[0].cnt: " + res.rows[0].cnt + " -- should be
1");
});
}, function(e) {
return console.log("ERROR: " + e.message);
});
});
}

Then in your device ready function call it.
function onDeviceReady()
{
testDB();
}

It did indeed add the test table and and the test data.
This is what I'm gonna roll with unless Apple responds to my bug
report. But i"m guessing it's on us to "fix" our apps.

Scott

On Jan 14, 2:50 pm, Samuel M <samuel.miche...@gmail.com> wrote:
> No workaround I know.
> If you want that apple change this behaviour, you should report a bug
> athttps://bugreport.apple.com

ScottP

unread,
Jan 14, 2012, 10:32:26 PM1/14/12
to phonegap
I should mention that I only tested my workaround on iOS5.0.
Will try 5.1 when I get a chance.

On Jan 14, 10:28 pm, ScottP <scott.a.padi...@gmail.com> wrote:
> Here's my workaround so far... Using the info in this link:http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phoneg...

Samuel M

unread,
Jan 15, 2012, 11:48:44 AM1/15/12
to phonegap
Great work!
I wasn't aware of the SQLite phonegap plugin. This is great. Is there
any performances issues? The API is the same as WebSQL?

I am finishing the WebSQL backup in a MySQL DB in our server.

The client JS code is working well and is simple to use. Just need to
add 2 lines in the code:

var tableToSync = [{myTable1},{myTable2}];
DBSYNC.initSync(tableToSync, db, {UUID}, 'http://wwww.myserver.com/
backup');

and later in the code (after some modifications:
DBSYNC.syncNow(_syncProgress);

Samuel M

unread,
Jan 16, 2012, 7:45:33 PM1/16/12
to phonegap
@ScottP : I am still coding the server part of the WebSqlSync (it was
more complicated than I thought), but I will share my code if it can
help other people.


I tried to use the SQlite phonegap plugin just by calling new
PGSQLitePlugin("0000000000000001.db"); instead of the standard DB
creation. But unfortunately, it didn't work well. I need to
investigate, but it looks like some callback doesn't work. (of course,
I installed correctly the PG plugin). Did you have more success than
me with your apps (except the small test case)

ScottP

unread,
Jan 16, 2012, 10:11:36 PM1/16/12
to phonegap
Samuel -
var db;
db = new PGSQLitePlugin("0000000000000001.db");
db.executeSql('CREATE TABLE IF NOT EXISTS test_table (id integer
primary key, data text, data_num integer)');

This was basically taken right from the readme except the name (I
replaced it with the default webkit db name, 0000000000000001.db).
Seemed to work to create a DB in the documents folder. I believe that
is the default location for db's created with the plugin - Documents
folder.
Are you getting errors? I had to add the libsqlite3.dylib library to
get the plugin to work.

What didn't work for you?

Scott

ScottP

unread,
Jan 16, 2012, 10:53:51 PM1/16/12
to phonegap
Been playing with the plugin a little more. It's run my queries with
out problems.
Let me know what is not working, I'll try to help.

Sam

unread,
Jan 17, 2012, 2:09:30 PM1/17/12
to phonegap
Hi Scott,

Thanks a lot for your help and for sharing your code.
It's working now. It looks like a little bit slower, and I have some
accents problems (I have apps with DB content in different languages).
Need to investigate that...

FYI, it wasn't working, because the native DB doesn't have the same
API than the WebSQL db.
I made an adapter, so my code works both in safari and in iOS :

_executeSql : function(tx, sql, params, dataHandler, errorHandler) {

if (typeof this.getDB().dbPath !== 'undefined') {
//this is a native DB, the method signature is different:

var sqlAndParams = [sql].concat(params);

var cb = function(res) {
//in WebSQL : result.rows.item(0);
//in the plugin : res.rows[0].cnt
res.rows.item = function(i) {
return this[i];
};

//the result callback hasn't the tx param
dataHandler(tx, res);
};

tx.executeSql(sqlAndParams, cb, errorHandler);
} else {
//Standard WebSQL
tx.executeSql(sql, params, dataHandler, errorHandler);
> > > > > > > > 0200619074 - 3939890025 (mobile)--  Fax:...
>
> read more »

Sam

unread,
Jan 17, 2012, 5:59:08 PM1/17/12
to phonegap
OK, so I have fixed the encoding bug in PGSQLitePlugin (I did a pull
request in github).

Now I am trying to understand your code in AppDelegate. I put your
code in the function didFinishLaunchingWithOptions
But I don't understand why you return SUCCESS if the db file exists in
WebKit/Databases/file__0/
Shouldn't we migrate this previous db file (with user data) in the
Documents directory (where PGSQLitePlugin works)?
Maybe I am just tired and I need a rest...
Thanks in advance for your help Scott,

Sam
> > > > > > > It did indeed...
>
> read more »

ScottP

unread,
Jan 17, 2012, 7:38:17 PM1/17/12
to phonegap
I actually put my code in webViewDidStartLoad. I just followed what
Guarav Tomar had in there from where I cribbed a bunch of the code.
So based on what he had for copying a prepopulated db he returns
without doing any copying if the db is already there.

So in my code I have success as a BOOL checking if the db is there.
If it's already there (success = true) then go ahead and return
without copying.

If success = false then check if the db file exists in WebKit/
Databases/file__0 with a BOOL olddatabaseFile.
If olddatabaseFile = true then copy
else if olddatabaseFile = false continue to check the new webkit
location.

Does that make sense? I think I'm starting to think in circles...
endless loops in my head!
Maybe I have it wrong?
> ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -

Sam

unread,
Jan 18, 2012, 11:24:52 AM1/18/12
to phonegap
Hi Scott! Sorry about the confusion. Your code works perfectly. (I was
a bit tired yesterday ;-)
OK, I think I will send my update this weekend (I will only do 3 days
of beta testing because 5.1 can come soon...)
Thanks to you ScottP and the author of PGSQLitePlugin, we have a
robust solution.

I'll post my JS DB sync code in Github ASAP if any of you are
interested.
> > > > > > > > > toPath:databaseFile...
>
> read more »

ziobudda@gmail

unread,
Jan 18, 2012, 11:32:43 AM1/18/12
to phon...@googlegroups.com
Il giorno 18/gen/2012, alle ore 17:24, Sam ha scritto:

I'll post my JS DB sync code in Github ASAP if any of you are
interested.

YES YES YES

M.
--
Michel 'ZioBudda' Morelli                       mic...@ziobuddalabs.it
Sviluppo applicazioni CMS DRUPAL e web dinamiche (LAMP+Ajax)

ScottP

unread,
Jan 18, 2012, 4:01:51 PM1/18/12
to phonegap
Any suggestions on a localstorage work around. I mostly use it for a
couple of in-app settings, so I may just go with the
applicationPreferences plugin (I believe those get backed up). But I
thought I'd ask anyways.
> > > > > > > > > >     // If the database in Documents folder already exists then return- Hide quoted text -
>
> - Show quoted text -...
>
> read more »

Sam

unread,
Jan 19, 2012, 9:50:17 AM1/19/12
to phonegap
@ziobudda : I will post the github link of the DB Sync, it needs a
little more testing and code cleaning (hope I'll have the time soon)
@ScottP : for localStorage, I think I will use WebSQL DB, with a load
at startup and a cache system to make it synchronous. Like that, it
will be also synchronized with my WebSqlSync code. But
ApplicationPreferences plugin will be fine too.

And for people reading silently this thread with apps using
localStorage or WebSQL, you should read this thread too :
https://devforums.apple.com/message/606269#606269
And report a bug. As an Apple engineer says, "a bug with mutiple dups
generally makes a stronger case than a bug with none."
-> https://bugreport.apple.com/
> > > > > > > > > > >     NSFileManager *fileManager = [NSFileManager defaultManager];...
>
> read more »

IAmNino

unread,
Jan 20, 2012, 2:27:01 AM1/20/12
to phon...@googlegroups.com
I've tried getting this answered in my own thread but maybe I can get my answers here... 

Can someone please clarify this for me? 

As I understand it, the Storage API's database is for some reason (thanks to apple) being stored in a volatile area that does not get synced with the iCloud, and can be deleted as other applications need that cached space. Furthermore, if I want to have data from the user stored for future use and to sync with the iCloud, I need to use the File API. 

Is my understanding of this correct? 

If so, would it be feasible to store small amounts of data as a JSON Object in a txt file for a quick & dirty solution? 

Please advise as this is new territory for me! 

ScottP

unread,
Jan 20, 2012, 8:31:52 AM1/20/12
to phonegap
If by "Storage API database" you mean the WebSQL & localStorage that
is built into webkit then I think yes your are correct, but it may be
better to continue the discussion in the Apple Dev Forums.
See the link in Sam's post above to the Apple dev forums.
Submit a bug to report to apple if this is a problem for you.

Sam

unread,
Jan 20, 2012, 8:58:54 AM1/20/12
to phonegap
We were talking about a beta. You should read this thread :
https://devforums.apple.com/message/606269#606269
We just need that everyone concerned create a bug report. Hopefully
Apple will fix that before public release.

On Jan 20, 2:27 am, IAmNino <vermice...@gmail.com> wrote:

ziobudda@gmail

unread,
Jan 20, 2012, 9:01:57 AM1/20/12
to phon...@googlegroups.com
Il giorno 20/gen/2012, alle ore 14:58, Sam ha scritto:

We were talking about a beta. You should read this thread :
https://devforums.apple.com/message/606269#606269
We just need that everyone concerned create a bug report. Hopefully
Apple will fix that before public release.

If someone say me what I need to write I write it. 

Sam

unread,
Jan 20, 2012, 12:33:30 PM1/20/12
to phonegap
My bug report wasn't well written, but here it is:

Summary:
WebKit data (localstorage or local SQLite) are now stored in Library/
Caches folder (instead of Library/WebKit folder). This is a big
problem for all apps using UIWebView and storing user data, because
they will no longer be backup and may be deleted. There are a lot
HTML5 apps (cf. phonegap) using localstorage or SQLite as a critical
feature.
Steps to Reproduce:
In a UIWebView, create a new DB or use localStorage with JavaScript.
You can see that the WebKit data is now stored in Library/Caches :
http://cl.ly/1p3L1U2W1J3y3I1T1r3y
Expected Results:
We should at least specify the directory of WebKit data (and to set
it
to Documents/ for critical data)
Actual Results:
The WebKit data are stored in Library/Caches folder, and can be
deleted
Regression:
The WebKit data was stored in Library/WebKit folder, and was
correctly
backup by iCloud

Sam

unread,
Feb 2, 2012, 11:04:10 AM2/2/12
to phonegap
Hi,
My Apple bug report is still open. No news from Apple. Our phonegap
apps data are in danger, because even with the PhonegapSQLPlugin, user
that don't upgrade the app will loose their data... If you are
concerned, you should fill a bug report (the more we are, the more
Apple will dare to look at this problem)

For the good news, I just shared my WebSQL syncing lib in github :
https://github.com/orbitaloop/WebSqlSync/
(it's very easy to integrate in your app - just 2 function to call,
but you need to code the server side)
I code it because there was no generic solution in the market (http://
stackoverflow.com/questions/1744522/best-way-to-synchronize-local-
html5-db-websql-storage-sqlite-with-a-server/2313945#2313945)

It's still in beta, but it works fairly well, I use it in my apps (and
i will release it in prod very soon). User data are now in sync with
my server (it's like a backup). And user can have several devices
synchronized (ex. between their iPhone and their iPad).
Hope this help,

Sam

Kerri Shotts

unread,
Feb 3, 2012, 1:11:57 PM2/3/12
to phon...@googlegroups.com
Just FYI, from my own experience:

As of iOS 5.0.1 (my devices are all on this, so can't verify on 5.0), WebKit's localStorage is stored under Library/WebKit which /is/ backed up and sync'd. See: http://stackoverflow.com/questions/7653820/is-ios-5-icloud-backup-save-webkit-data-for-app-using-the-storage-of-a-uiwebview

That doesn't mean that one should rely only on localStorage, so having a sync or export framework isn't a bad idea, but still, it is nice to know that it shouldn't get wiped out when iOS goes on a "cleaning" spree, and that it will be backed up.

Sam

unread,
Feb 3, 2012, 3:44:13 PM2/3/12
to phonegap
The problem is not with 5.0.1 but the next version:
https://devforums.apple.com/thread/137882?tstart=0

(I am the one that ask this stackoverflow question)

On Feb 3, 1:11 pm, Kerri Shotts <kerrisho...@gmail.com> wrote:
> Just FYI, from my own experience:
>
> As of iOS 5.0.1 (my devices are all on this, so can't verify on 5.0),
> WebKit's localStorage is stored under Library/WebKit which /is/ backed up
> and sync'd.
> See:http://stackoverflow.com/questions/7653820/is-ios-5-icloud-backup-sav...

Kerri Shotts

unread,
Feb 3, 2012, 5:50:42 PM2/3/12
to phon...@googlegroups.com
Woops, my bad. Totally misread the version #. D'oh.

Brain isn't on today, I guess.

Kerri Shotts

unread,
Feb 3, 2012, 5:59:08 PM2/3/12
to phon...@googlegroups.com
Thinking aloud now; I wonder what the performance hit would be of creating a localStorage-like native plugin for phoneGap that stored the data in a place that we /know/ will get backed up / sync'd, etc. My app is pretty much using localStorage to store everything, including treating it like a mini-database (without the need for relational data and questions-- so more like a big keyed array). Since it's a fast to access, I don't always offload the contents into another array, so I wonder what the hit would be in going from js to native and back again inside a loop.

Guess there's only one way to find out: build a plugin and see how it goes... ;-) Wish me luck.

Luke Melia

unread,
Feb 3, 2012, 6:08:39 PM2/3/12
to phon...@googlegroups.com
Kerri,

I recently tried something similar with a websql plugin on iOS for the same reason. Performance was unacceptable for us and we rolled it back. Curious to hear what you find out.

Cheers,
Luke

On Friday, February 3, 2012 at 5:59 PM, Kerri Shotts wrote:

Thinking aloud now; I wonder what the performance hit would be of creating a localStorage-like native plugin for phoneGap that stored the data in a place that we /know/ will get backed up / sync'd, etc. My app is pretty much using localStorage to store everything, including treating it like a mini-database (without the need for relational data and questions-- so more like a big keyed array). Since it's a fast to access, I don't always offload the contents into another array, so I wonder what the hit would be in going from js to native and back again inside a loop.

Guess there's only one way to find out: build a plugin and see how it goes... ;-) Wish me luck.

--
You received this message because you are subscribed to the Google
Groups "phonegap" group.
To post to this group, send email to phon...@googlegroups.com
To unsubscribe from this group, send email to
phonegap+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/phonegap?hl=en?hl=en
 
For more info on PhoneGap or to download the code go to www.phonegap.com

Kerri Shotts

unread,
Feb 3, 2012, 10:37:51 PM2/3/12
to phon...@googlegroups.com
After thinking some more, I did something totally different. This is 0.1 stuff, hacked together in a few minutes, and is likely to break in the most unexpected ways. It's also horrible code. BUT, it works for me as a proof of concept, and I will be making it a lot better down the road. Just wanted to see if it would work, and it seems to (in the simulator). So take it with a really, really, really, really, really, really big grain of salt and testing.


So what does that do? Within 500ms of the start of the app, the code loads all values from "localStorage.dat" into localStorage. (Essentially we're overwriting what's already there. But if the values are the same, no foul. If localStorage is empty, then we're just creating what was lost.) Then it sets up a 30s timeout to call a routine that saves all localStorage values back into "localStorage.dat". This same routine sets up another timeout to call itself in 30s (after the save, so if things take awhile there's no risk of overwriting one's self). This should mean that all data in localStorage older than 30s is safe and secure in a location that is backed up. (What about data within the last 30s? Need to add figure that one out. Currently it is vulnerable.)

Right now the code is hacky and essentially writing key,value pairs like this: key[EQ]value[EOK]. There's a million better ways, but it was quick & easy to split, so there. It works as a proof of concept. Large datasets, of course, would take a considerable amount of time to load, /but/ I have another javascript loading nearly 8mb of data in less than 10s, so perhaps it would be within acceptable limits even if localStorage was huge.

All that said, there's got to be a way to this natively without a horrible performance impact. But perhaps this is a bandaid in the interim?

Kerri Shotts

unread,
Feb 3, 2012, 10:39:56 PM2/3/12
to phon...@googlegroups.com
Hi. See my other post a few secs ago. I wondered about the perf-hit and it sounds like it might be too great. I've got a temp bandaid solution that has potential (not even /alpha/ yet) that just works with JS and some phonegap File apis. Still want to work something out with a native persistent storage, though... will let you know if I come up with anything that will work.

ScottP

unread,
Feb 4, 2012, 9:51:07 AM2/4/12
to phonegap
Just to reiterate Sam's point the more bug reports Apple has, the more
likely they are to respond.

Sam

unread,
Feb 27, 2012, 5:24:47 PM2/27/12
to phonegap
The bug report is still open, and the release date of iOS5.1 is soon.
All phonegap users are concerned, LocalStorage or WebSQL are essential
for our apps. But it looks like Apple doesn't see the importance of
the phonegap community.

All bug reports are private, so it's not possible to vote for a bug.
You need to create a duplicate in order to draw the Apple intention on
that problem. They prioritize bugs fixing regarding the number of
duplicates. So please, create your bug report, it's easy :

http://bugreporter.apple.com/


(for more info, you can read this thread: https://devforums.apple.com/thread/137882?tstart=0
)


On Jan 20, 12:33 pm, Sam <samuel.miche...@gmail.com> wrote:
> My bug report wasn't well written, but here it is:
>
> Summary:
> WebKit data (localstorageor local SQLite) are now stored in Library/
> Caches folder (instead of Library/WebKit folder). This is a big
> problem for all apps using UIWebView and storing user data, because
> they will no longer be backup and may be deleted. There are a lot
> HTML5 apps (cf. phonegap) usinglocalstorageor SQLite as a critical
> feature.
> Steps to Reproduce:
> In a UIWebView, create a new DB or uselocalStoragewith JavaScript.

WebSteve

unread,
Feb 28, 2012, 11:01:15 AM2/28/12
to phonegap
Actually this issue has already been addressed. Please see my post:
http://iphonedevlog.wordpress.com/2012/02/28/saving-localstorage-to-a-more-secure-folder-location/

Regards,
Steve Husting

ScottP

unread,
Feb 28, 2012, 11:52:24 AM2/28/12
to phonegap
The solution is only valid if ALL your current users are diligent and
update their apps before the 5.1 release.
If they don't then they would face potential data loss when they
upgrade to iOS5.1.
Read the very first post of the thread Sam posted:
https://devforums.apple.com/thread/137882?tstart=0

So I would not go as far as to say it has been addressed. yes there
are workarounds available for both the webkit sqlite db and
localstorage, but they are limited in their scope if users don't
update apps before upgrading to iOS5.1

On Feb 28, 11:01 am, WebSteve <shust...@aeteam.net> wrote:
> Actually this issue has already been addressed. Please see my post:http://iphonedevlog.wordpress.com/2012/02/28/saving-localstorage-to-a...
>
> Regards,
> Steve Husting

Sam

unread,
Feb 28, 2012, 6:00:08 PM2/28/12
to phonegap
I agree with you ScottP. By experience, I have a lot of users that
don't know how to upgrade their app (but they do know how to rate an
app ;-)
And sorry, but the solution of WebSteve looks like an ugly hack for
me. It can raise a lot of potential problems.

Apple have to take phonegap developers seriously by allowing them to
use a method to store data securely.
So the call to create bug reports is still valid :
http://bugreporter.apple.com/

Kerri Shotts

unread,
Feb 28, 2012, 7:04:19 PM2/28/12
to phon...@googlegroups.com
Speaking as the author of said ugly hack, I agree. It's ugly. There's a lot that can go wrong with it. I shouldn't even have to worry about whether or not it can blow up or not, because I shouldn't have had to come up with it in the first place (that is, the bug shouldn't exist). But it /works/ (for me), and since I can't presume that Apple will fix said bug, it's in my app for the time being. (Sometimes developers don't have the luxury to "hope" that something will get fixed. That's why you end up with ugly hacks.)

[Aside: Speaking as a former test engineer, I can /assure/ you that a lot of programmers only get more stubborn when having to close bug dups. Maybe Apple doesn't work that way, but my previous place of employment did (and it was /big/. Can't name names.). Which meant that dup'd bugs tended /not/ to be fixed (or went unfixed for/ever/) because of stubbornness. Stupid? Yes. But that's the way it was.]

If Apple /does/ fix the bug for 5.1, YAY, and the code goes away. If they don't, (or even worse, they do, but not in 5.1), the code stays. It will be iterated upon and improved, of course, and the latest will always be at http://pastebin.com/u5XXQDub. (Keep in mind: it was developed in less than a couple of hours. If there's a better way under the assumption the bug will remain, have at it.)

As to the hackish-ness, feel free to improve on it & play with it & hack it to pieces. Have at it. If you come up with something better, let me know, and I'll be happy to update the code. :-)
Message has been deleted

Kerri Shotts

unread,
Feb 28, 2012, 7:09:55 PM2/28/12
to phon...@googlegroups.com
Hi Steve,

Not to be pedantic or anything, but this developer's a she. ;-) Thanks for the blog post even so. :-)

Here's hoping Apple fixes it in 5.1 and we won't have to worry about it at all. If not, oh well. I've some ideas for improvement especially with regards to the splitting part (as coded, it can break too easily)... will keep you posted.

Sam

unread,
Mar 1, 2012, 2:01:36 PM3/1/12
to phonegap
Hi,

Sorry, it wasn't my intention to offend anybody (english is not my
native language, so I can be tactless sometime). Every developers make
quick fixes and hacks. It's just that Steve presented this as THE
solution. There is no solution except Apple fix the bug (or it will
break for users with old version)

And speaking about duplicate bug report, it's an Apple engineer who
said that :
"a bug with mutiple dups generally makes a stronger case than a bug
with none."
(in this thread : https://devforums.apple.com/thread/137882?tstart=0 )

Cheers,

Sam
> >http://iphonedevlog.wordpress.com/2012/02/28/saving-localstorage-to-a...
>
> > Regards,
> > Steve Husting

Kerri Shotts

unread,
Mar 1, 2012, 3:20:23 PM3/1/12
to phon...@googlegroups.com
No offense taken. And glad to know that Apple considers dups differently than my previous place of employment. 

Here's hoping Apple takes care of the issue without an intervening public-release version that has the issue. That'd just be a pain to work with.

WebSteve

unread,
Mar 1, 2012, 6:23:01 PM3/1/12
to phonegap
Sam,

Please quote me accurately. I wrote, "Actually this issue has already
been addressed." I never said it was THE solution. Kerri is the only
person I know who has actually tackled the issue and submitted
something that works, even though the "solution" must be narrowly
applied. Good job for her!

Regards,
Steve Husting

Devgeeks

unread,
Mar 1, 2012, 6:33:46 PM3/1/12
to phon...@googlegroups.com
+1

Sam

unread,
Mar 7, 2012, 8:09:37 PM3/7/12
to phon...@googlegroups.com
They did it. Apple has released their app with that bug. 
I already got angry users that loose all their work on my app :-/

ScottP

unread,
Mar 7, 2012, 9:35:33 PM3/7/12
to phonegap

I was really, really hoping they wouldn't. I haven't gotten any bad
feedback from users yet but I'm expecting it.
As of today I know for sure not all of my users have updated their app
with my workarounds.
Not looking forward to having to say sorry to so many users!

Kerri Shotts

unread,
Mar 7, 2012, 10:25:38 PM3/7/12
to phon...@googlegroups.com
Well. That sucks. Not only did they release /with/ the bug, but if they ever fix it, that means there will be at least one revision of the os that has the box. Argh!

Perhaps anyone who loses information could potentially obtain the information from their previous backups? Localstorage would still be there in the backup -- just that you'd have to build an import method to bring it in to make things easier. Just thinking out loud.

seb

unread,
Mar 7, 2012, 11:07:32 PM3/7/12
to phonegap
It's not a bug. Apple did this on purpose. Many Apps are opening
webviews for displaying websites. And all the data (cache, images,
databases) would be backed up with iCloud. Apple wants to prevent that
to save space. It's that simple.
They won't fix it because they don't perceive it as a bug. And since
Phonegap Apps make up only a tiny, tiny fraction of all Apps, Apple
won't care.
So, instead of hoping that Apple will fix it, it would be better to
implement a new solution. I, for myself, will use the file API and
create my on local storage system.

ScottP

unread,
Mar 8, 2012, 8:14:43 AM3/8/12
to phonegap

I understand why they did it and I'm fine with that part. What bothers
me is they don't persist any data that is currently stored across
their update. So when a user updates iOS, any currently stored webkit
data is gone.
I'm looking into if it's possible to get the data from an iTunes
backup as Kerri mentioned, but haven't gotten very far.

marcucio

unread,
Mar 8, 2012, 10:16:17 AM3/8/12
to phon...@googlegroups.com
Join the club... this bug sucks.. no real workaround as I can see to save my users data because a fix in my app will take at least 2 weeks to get approved and in the hands of my users.

I just hope the most of the heat falls on Apple not my apps!

Sam

unread,
Mar 8, 2012, 10:25:57 AM3/8/12
to phon...@googlegroups.com
There is a solution that works most of the time to get back the data. The migration can be done manually:
 - download the software iExplorer (http://www.macroplant.com/iexplorer/, free)
 - connect your device to your computer and run iExplorer
 - in iExplorer, open the Apps directory, then  your App directory, then the Library/WebKit/Databases directory
 - copy the directory Database/file__0 to the Library/Caches directory :
See this screenshot : http://cl.ly/2N3J3R3R0y0k1t3e1C2q

(it's more or less the same for localstorage)

But some users doesn't even see the Library/WebKit/Databases directory. For them, there is an ultimate solution :
- Get an old iTunes backup "/Users/USER/Library/Application Support/MobileSync/Backup".
(eventually use TimeMachine to get an older backup)

- Use "iPhone Backup extractor" (http://supercrazyawesome.com/) and extract the WebKit directory.


Hope this help,

Sam

Sam

unread,
Mar 8, 2012, 10:47:24 AM3/8/12
to phon...@googlegroups.com
Yes, it sucks !

I think we should create a lot of bug report (https://bugreport.apple.com/), because it looks like Apple doesn't know the importance of the phonegap dev community (that's why they didn't fix the bug)

mikem

unread,
Mar 8, 2012, 10:57:54 AM3/8/12
to phon...@googlegroups.com
The big issue is that there really is nothing I can do to fix this, your solution would work but there is no way to tell the people who downloaded my apps to do these steps (and they probably couldn't figure out how to do it either)

Is there a solution yet so this doesn't happen moving forward?

Kerri Shotts

unread,
Mar 8, 2012, 12:56:56 PM3/8/12
to phon...@googlegroups.com
Moving forward, I guess the only option is to assume that localStorage/sqlite is no longer persistent (even though it is supposed to be). Either do away with using it entirely, /OR/ implement some method that persists it to the documents folder. (Which is what I do with localStorage. Sqllite would probably be more painful...)

The absolute best method to do that persistence would be to somehow copy the file in the cache over to the documents folder at periodic intervals (or when it has changed), and read it at startup to populate the cache. Which is sorta what I do, only in Javascript... I may perhaps work on a plugin that does this natively down the road. [I would assume better performance, less risk from that kind of proposition. Plus the native code could be fired on suspend -- rather than with JS, I just write it out every so often.]

But for the current situation... *sigh* Apple was stupid and released with the bug, which means a lot of people are going to have their data wiped. Short of digging back into the filesystem or an old backup, I don't know any way to recover the data. :-( [[short of being jailbroken and using iFile..., but that's not exactly an option we can suggest!]]

ScottP

unread,
Mar 8, 2012, 1:24:52 PM3/8/12
to phonegap
Actually, further back in this thread I adapted the process that
Gaurav Tomar ( http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phonegap.html
) uses to copy a prepopulated database over. My modifications
basically look for the webkit database in both (pre and post iOS5.1)
locations and if it needs then copy it to the documents folder. You
can look at previous posts to see all the code changes I made to
AppDelegate.m.
Then you can use the native sqlite plugin found here:
https://github.com/davibe/Phonegap-SQLitePlugin.

This way you won't need to continually "backup" your sqlite database
since you'll be using the plugin to natively connect to the DB file
you copied to the documents folder already.

So by using your workaround, Kerri for localstorage and this
workaround for sqlite, I think we're ok moving forward...
I'll be honest I haven't looked too closely at your localstorage
workaround, but I'm guessing the code I'm using to copy the webkit DB
over could be modified into a plugin to do just as you propose for the
localstorage.
> >>>> I already got angry users that loose all their work on my app :-/- Hide quoted text -
>
> - Show quoted text -

Kerri Shotts

unread,
Mar 8, 2012, 1:44:04 PM3/8/12
to phon...@googlegroups.com
Scott,

I'm wondering the same thing... I don't see why it couldn't work (logically), but I need to get a better handle on how uiwebview is handling localStorage before I'm secure emotionally (for example: is it only ever in file__0.localstorage? The __0 implies to me that there could be __1, __2...). 

The more I think about it, however, the more I prefer the idea of a plugin/addon that would physically copy the localStorage files rather than do it in Javascript (which is slower, more likely to blow up, not suspend-safe... that is, I can't tell it to write out localStorage on a pause because of the iOS quirk that the pause handler isn't going to handle native code until after a resume...). Plus, given that localstorage is just a sqlite database with one table anyway, I'm thinking I could do my cloud/export work that way too (since I want a method of selectively working with keys)... My coding fingers are tickling now... am definitely going to play around with that and see what comes out.

Kerri Shotts

unread,
Mar 8, 2012, 6:36:16 PM3/8/12
to phon...@googlegroups.com
All,

Tentative code to support NATIVE persistence of localStorage with phoneGap (at least 1.3.0, but I would imagine, it'll be the same on any version, since it doesn't use any PG constructs) is up at http://pastebin.com/5881768B

It differs from my previous solution in that it does NOT use Javascript in any form or fashion to accomplish the data copy, which means it is faster (for large datasets), and less error-prone. (And more elegant. The JS version really felt hackish.)

To use, paste into AppDelegate.m (if you are using MRC, you'll need to make a couple of changes to release the fileManager object; I'm on ARC, and so did not do so). It replaces /part/ of webViewDidStartLoad: -- I've included the remainder of that function in the pastebin, but whatever is in yours is what you should keep. Also note that if you are doing anything special with ResignActive or Terminate to merge the two functions together, as this code uses them in order to save out localStorage back to persistent storage.

It's way fresh, so take it with the grain of salt you need to; I have tested on my pre-5.1 devices, but as I have none at 5.1, I can't promise it works there yet. (Someone who is brave willing to take a go?)

If there are changes/suggestions/etc you have, feel free to make them, and if necessary, I'll adjust the paste as necessary.

ScottP

unread,
Mar 8, 2012, 8:03:53 PM3/8/12
to phonegap

You go girl! :)

On Mar 8, 6:36 pm, Kerri Shotts <kerrisho...@gmail.com> wrote:
> All,
>
> Tentative code to support NATIVE persistence of localStorage with phoneGap
> (at least 1.3.0, but I would imagine, it'll be the same on any version,
> since it doesn't use any PG constructs) is up
> athttp://pastebin.com/5881768B
> >>http://gauravstomar.blogspot.com/2011/08/prepopulate-sqlite-in-phoneg...

mikem

unread,
Mar 8, 2012, 9:23:02 PM3/8/12
to phon...@googlegroups.com
Interesting idea, is this only for localStorage or would this work for webSQL too?
Message has been deleted

Lan

unread,
Mar 8, 2012, 10:39:47 PM3/8/12
to phon...@googlegroups.com
Perhaps, instead of copying it manually on the /Documents folder, we could create a hard link instead (linkItemAtPath:toPath:error:) ? So that the files in /Library/Caches and /Documents point at the same file and changes in the file in /Library/Caches will be visible on the /Documents file? On start of the application and if the file is purged in /Library/Caches, _then_ we can create a hard link _again_, this time, the target dest is in /Library/Caches.

What do you think?

Kerri Shotts

unread,
Mar 8, 2012, 10:40:06 PM3/8/12
to phon...@googlegroups.com
Depends on how you intend on working with it; if you're planning on on just copying the data out (and then using a sqlite plugin), you can use ScottP's process (as noted a few posts above). If you intend on continuing to work with sqlite via the browser, then you'd have to take into account making sure it was copied back and forth appropriately. If you did that, it /should/ work, I suppose, since we're really just talking about moving sqlite databases around. Would be worth a try, anyway, though I don't use sqlite myself for my app, so I'll have to leave that as an exercise for someone else.

mikem

unread,
Mar 9, 2012, 9:25:55 AM3/9/12
to phon...@googlegroups.com
I like that idea but I am not too familiar with the inner workings of it.

One other way would be to write a plugin which overwrites the js sql functions and passes it on to a native implementation of SQL. This seems like a lot of work but I think in the long run this is the best solution. The same goes for localStorage. You can then be assured that the data is written in the correct location and we don't have to worry about backups. 

WebSteve

unread,
Mar 9, 2012, 10:49:38 AM3/9/12
to phonegap
I will integrate this with my 5.0 device and upgrade to 5.1 and see
what happens. I have some other code to debug in the app first. I'll
report the results of the upgrade here.

Thanks for this terrific contribution, Kerri!

Regards,
Steve

WebSteve

unread,
Mar 9, 2012, 2:37:47 PM3/9/12
to phonegap
Kerri,

Your code shows a lot of NSLog output, but I don't see any of them
displayed in Xcode's console section at the bottom of the screen when
I "Run" code on the device. Are they supposed to display
automatically, or do I need to do something to turn that on?

Regards,
Steve

WebSteve

unread,
Mar 9, 2012, 2:52:01 PM3/9/12
to phonegap
Never mind, I see it now. Everything works well on the console.

Regards,
Steve

Kerri Shotts

unread,
Mar 9, 2012, 5:03:20 PM3/9/12
to phon...@googlegroups.com
That's good to hear. :-) To verify everything is working right, you should have an appdata.db file in documents that matches your localstorage file now; and then when you start the app up, the reverse should be true. (To be double sure, I renamed/deleted the existing localstorage file, and it did come over. -- Something, in fact, that has to be done anyway by the code, since the file copy can't apparently overwrite existing files.)

As always, if there's something that could be done better, if anyone finds a bug, I'd be more than happy to hear it. :-)

Sam

unread,
Mar 9, 2012, 8:05:43 PM3/9/12
to phon...@googlegroups.com
It's good to see all this activity and code sharing!
Here is my small contribution:

For WebSQL DB file migration, I have adapted the code of ScottP here :
My addition : If there are 2 DB files (1 in WebKit dir and the other in Caches, it will copy the bigger file into the document folder).

Use this code with the Native SQLite phonegap plugin : https://github.com/davibe/Phonegap-SQLitePlugin (you will be able to work directly in the Document folder).
Regarding this plugin, there are some differences between the WebSQL API, here is an adaptor:

And at least, I wrote a small lib to synchronize the SQlite DB to a server :

Thanks everybody for sharing your solutions

Hope this help,

Sam

Arpit Jain

unread,
Mar 13, 2012, 3:03:38 PM3/13/12
to phon...@googlegroups.com
I might have missed some posts, but instead of going through all this trouble, why can't we just change the location of webkit database folder? 

WebPreferences* prefs = [webView preferences];
[prefs _setLocalStorageDatabasePath:@"~/Library/Application Support/MyApp"];
[prefs setLocalStorageEnabled:YES];

In this way, we can store the database to a folder which is backed up by iCloud and will not loose any data. Any issues in this that I might know of? 

(Other than calling private function, which for Webkit Apple doesn't mind)

Sam

unread,
Mar 13, 2012, 3:10:04 PM3/13/12
to phon...@googlegroups.com
Wow, I didn't know that it was possible. Did you try it? No problem with Apple review regarding this private function?

Arpit Jain

unread,
Mar 13, 2012, 3:43:16 PM3/13/12
to phon...@googlegroups.com
I did not try this function directly. But I opened the .plist file directly for NSUserDefaults and changed the value of Webkit storage key and it worked perfectly. Just submitted an update to Apple couple of days ago. Hopefully, they won't have any issue with this. They generally don't find any issues with using private functions of Webkit variables, or so I have read. 

Arpit Jain

unread,
Mar 13, 2012, 3:44:19 PM3/13/12
to phon...@googlegroups.com

mikem

unread,
Mar 13, 2012, 3:49:30 PM3/13/12
to phon...@googlegroups.com
is this for a Mac app or iOS app?

Chema Roldan

unread,
Mar 13, 2012, 4:52:16 PM3/13/12
to phon...@googlegroups.com
Hi and thank you very much!!! Where I need to put this code?

Regards! 
It is loading more messages.
0 new messages