recommending heavy Gaia users do not upgrade to iOS 5

[Andrew Johnson]

Hey folks,

We learned today that in iOS5, Apple has changed the behavior of the "Cache" folder. Data stored in this folder can be randomly deleted when you start to run low on disk space. This effects Gaia because we store saved maps in this folder. 

We learned about this because the developer of Instapaper ran into the issue and blogged about it: http://www.marco.org/2011/10/13/ios5-caches-cleaning

We are now trying to figure out what to do. There doesn't seem to be an easy answer - on the one hand, we could start saving everything in the Documents folder, but then this data will be backed up to iTunes/iCloud. This becomes problematic because it will make backups take many hours if you have several gigabytes of maps stored. We actually did it this way years ago, but moved to the Cache folder once it became clear that backups were too slow.

We hope that this doesn't become a huge issue, and we will push a release to fix this as soon as we can. 

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[muell...@aol.com]

Hallo Andrew,

thanks for the essential information.
I'm a developer myself (Windows) and i "like" such surprises too.
I think Apple should public such changes BEFORE it offers releases

Regards
Dr. Müller

[SoKal]

Andrew,

Thank you very much for letting us know about this. I'm an engineer, so sort of like Dr. Müller, I feel for the difficult challenge ahead of you to try decide what best to do.

I am in no hurry to upgrade to iOS 5, so no sweat. Maybe not until iOS 5.0.2 or 5.2---not until there's a "2" in there somewhere. You know, like Windows XP SP2. :-) :-) :-)

Again, thank for you taking care of your users.

--Ken

[Morrie]

If those are the only two choices, I vastly prefer the documents
folder, as it would be exceedingly frustrating if a downloaded map got
pushed out of the cache. Cache, by definition, should be deletable if
space is needed, so in some sense it's not the right place anyway for
persistent data. The 'many hours' to back up would only happen the
1st time, right? It's no different from storing many GB of videos or
music, really, so I'd think users should be happy to pay the price of
backing it up, especially if they really need to back up because their
iPad is being repaired or replaced.

[Andrew Johnson]

Hey Morrie,

Yes, we agree. 

If we don't hear from Apple on Monday, we have an update ready to go that moves everything to Documents. We'll ask them to expedite the review, and hopefully the update will be live this week.

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[Andrew Johnson]

Thanks for your note Ken.

I really couldn't do anything else - if someone got lost in the woods because of this bug, and we had not done our best to warn them, I couldn't live with myself. 

I have noticed that many extremely popular apps that are likely affected are making no mention of this on their websites or anywhere else. Hopefully it's either an error of omission or they are somehow using the Documents folder, and it wasn't a deliberate decision not to warn users.

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[satcp]

With many developers wildly enthusiastic about iOS 5 I was one of
those who did the upgrade just after the final release. I checked all
my apps and at first sight everything was okay. It was not until past
weekend when I needed Gaia GPS in offline mode I noticed all
downloaded map data was gone. The downloaded maps are still listed in
Gaia GPS but the actual map data is gone. The downloaded maps are now
just grey areas... I also noticed in iTunes that the available space
on my iPad 2 increased with approximately 15 GB. Since I haven't lost
anything else, those were my maps I'm afraid...

I think I once explained you in a mail how painstakingly I downloaded
all these maps. With the extremely low download speed of some map
sources the total download time of all these maps is probably in the
magnitude of weeks. I don't want you to feel bad about this. Obviously
you can't help it either when Apple decides to change fundamental
behavior. But I would like to take the opportunity to urge you to make
it possible for us to save downloaded maps (and in the same effort
tracks) to a computer. Many apps support file uploads and downloads
through iTunes. I would love to see this feature for Gaia GPS as well,
even if that means the backups will increase in size and time. That
would not only protect us against data loss, but would also save
considerable amounts of space on the iPad/iPhone because we could
store rarely used maps on the computer and only upload them to the
device when needed. If that is what you mean by "saving to the
Documents folder" I would very much welcome this change.

[Andrew Johnson]

Hi there,

We will restore your Saved Maps in the next update as long as you don't delete the broken maps from the Saved Screen.

We are adding a function that will check and see if any of the tiles are missing from a Saved Map and restore them.

In addition, in the next release, all of your maps will be backed up to iCloud or iTunes as you suggest, unless you disable backups via Settings.

We will get the beta for this out to you ASAP, and we'll provide a version that overwrites your Store version as well so you can do the restore right away.

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[satcp]

No need to hurry in my case, Andrew. I already deleted my lost maps...
I will start all over again once the new version that supports backups
is available. This time the downloading of large areas should be less
frustrating thanks to the 100.000 tiles maps, and the "download along
track" should be a great help as well :).

But still thanks for trying to fix this issue for others who might
suffer from the same iOS 5 woes...

[Doug Rosenberg]

Hi Andrew,

 

I don’t know how difficult it is to implement, but it would be great if we could click on a .gpx file in the iPhone Safari browser and have Gaia GPS be included in the “Open in . . ” dialog. Currently several apps appear to support this, including Evernote and Air Sharing. It would be nice not to have to use a computer if I want to download a track when I’m on the road.

 

Doug

[Andrew Johnson]

I don't think it's very hard actually, and that's a good one. It's on my list, but I haven't looked into it yet.

I'll ask this contractor we are working with to check it out for me.

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[Andrew Johnson]

Hey guys,

I wanted to let you know that this feature, along with iTunes drag 'n drop imports, are on the way. They both amount to the same thing, and we should have them in beta next week.

In addition, we'll have the route creator feature for you later this evening. That's locked and loaded, but I have a business meeting I'm off to right now.

On deck is automatically showing tracks on the map, waypoint organization, and cloud sync.

Regards,

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

On Tue, Oct 18, 2011 at 9:21 PM, Doug Rosenberg <do...@rosenbergseattle.com> wrote:

[Doug Rosenberg]

That's what I call customer service!

Sent from my iPhone

[satcp]

I agree with Doug! The level of support is simply amazing!

I just downloaded and installed the new version (5.41) which supports
backups. Thanks for that Anna and Andrew!

I'm a bit confused by the "backup welcome screen", though. It does
seem to suggest that turning of backups in the iCloud settings
disables backups entirely, but as far as I know that only applies to
the online iCloud backups. With the iCloud backup functionality
enabled, iTunes will no longer create backups automatically but you
can still do so manually (right click on the device in the left
column, then select "Back Up"). When doing so, the iTunes backup will
include all data - including Gaia GPS - regardless of the iCloud
setting for Gaia GPS.

When you have the iCloud backup functionality disabled entirely,
iTunes will still create automatic backups as it did before iOS 5 -
thus including all data, including that of Gaia GPS. As far as I know
it's not even possible to exclude individual app "documents" data from
the iTunes backup as is possible with the online iCloud backup.

I just tested the two above situations (restore a manual iTunes backup
with Gaia GPS disabled in iCloud on a completely erased iPad, and
restore an automatically created backup with iCloud entirely disabled,
again on a completely erased iPad) and in both cases all app
"documents" data was restored, including the Gaia GPS map downloads. I
haven't started re-downloading all my lost maps yet, but the small
test maps I included in the backups worked fine in offline mode after
the restores.

Anyway, now I can start re-downloading my maps with confidence.
Thanks!

I can't wait for the upcoming version with iTunes import and export
functionality! :)

[Andrew Johnson]

We're happy that you are happy with the support :)

As for backups, I'm not sure what you are suggesting we do to improve the new setup. I am sort of at a loss for how to deal with iTunes backing up. We're still storing stuff in Caches under iOS4, but in iOS5 all I can really do is move stuff to Documents.

If this backup disabling method doesn't work well, then all I can say is I hope Apple sees the light and starts allowing us to cache documents locally again, without them being backed up or deleted at random.

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[satcp]

On Oct 20, 9:30 pm, Andrew Johnson <and...@gaiagps.com> wrote:

> As for backups, I'm not sure what you are suggesting we do to improve the
> new setup. I am sort of at a loss for how to deal with iTunes backing up.
> We're still storing stuff in Caches under iOS4, but in iOS5 all I can really
> do is move stuff to Documents.

Obviously I can't speak for everyone, but I want the maps to be backed
up! Thus personally I prefer the iOS 5 approach. If it would have been
like that before, I would still have had my downloaded maps (again,
I'm not blaming you!). Naturally, those using iCloud with only 5 GB of
free storage will want to disable the Gaia GPS iCloud backups, but
they can still manually create a full backup through iTunes if wanted.
Those who only use iTunes and have iCloud disabled, might find the
automatic iTunes backups consume more time (and space, but that's not
a concern with modern harddisks), but it's not like Gaia GPS is the
only application that works this way... Besides, storing the data in
the "documents" folder is probably a prerequisite to implement the
upcoming iTunes import/export functionality (I don't know, but I
observed all other apps I have that offer this functionality also get
their data backed up). But once that functionality is implemented in
Gaia GPS, one can store maps safely on his computer and only upload
them to the iOS device when needed. That could save a lot of space on
the iPad or iPhone and thus also in backup time. Just my opinion of
course...

[Andrew Johnson]

I'm not sure what you are suggesting...

We have move all maps to the Documents folder. Do you think we shouldn't be showing the view, advising you to disable the backup?

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447

[satcp]

On Oct 20, 10:16 pm, Andrew Johnson <and...@gaiagps.com> wrote:

> We have move all maps to the Documents folder. Do you think we shouldn't be
> showing the view, advising you to disable the backup?

No, I think the screen is a good idea. Not everyone can figure out how
to disable the iCloud backups for Gaia GPS. What is confusing is that
the screen suggests the actions will disable both iCloud and iTunes
backups, while only the iCloud backups are affected. I would remove
mention of iTunes on the screen.

It would also be nice to disable the screen for those who have the
iCloud backup functionality entirely disabled, but I'm not sure if
it's possible to detect that programmatically.

[Doug Rosenberg]

For my part, I would like to know a little more about how iCloud backup
affects performance, battery life, etc. before deciding whether to turn it
off for Gaia. I was surprised when I looked at that Manage Storage screen
that Gaia was only about fourth in rank among apps using storage,
notwithstanding that I have quite a few maps saved. In that sense, allowing
iCloud backup might not be that bad. On the other hand, I think I've noticed
an occasional performance hit (e.g., apps sometimes taking longer to
respond) as well as somewhat shorter battery life since installing IOS 5 --
probably because the phone is performing iCloud backups, which it didn't do
before. If that is true, I should turn off iCloud backup for most apps.

Does anyone know how iCloud backup works? How often does it backup? Does it
use both WiFi and 3G? Does it transmit new data incrementally, or does it do
a full backup? If I have, say, 1.0GB of storage, and iCloud performs a full
backup daily, that could exceed my 2GB/month data allowance in two days!

I don't connect my iPhone to a computer very often, but most apps don't need
to be backed up that frequently, either. For apps like Gaia GPS, any map
data lost from a crash would be replaceable, although doing so could be time
consuming. It seems to me that iCloud backup makes the most sense for data
that change frequently and are not easily replaceable, e.g., mail, contacts,
photos, etc. Just my opinion.

[satcp]

On Oct 21, 12:47 am, "Doug Rosenberg" <d...@rosenbergseattle.com>
wrote:

> For my part, I would like to know a little more about how iCloud backup
> affects performance, battery life, etc. before deciding whether to turn it
> off for Gaia. I was surprised when I looked at that Manage Storage screen
> that Gaia was only about fourth in rank among apps using storage,
> notwithstanding that I have quite a few maps saved. In that sense, allowing
> iCloud backup might not be that bad.

The way I see iCloud is to synchronize data amongst multiple iOS
devices. I can imagine this makes sense for items like mail, contacts,
and bookmarks, but I'm not that sure about Gaia GPS map data. Do you
often use multiple iOS devices out in the field?

Nevertheless, if you don't have much maps and everything fits well in
the 5 GB of free iCloud storage, it is of course okay to leave the
Gaia GPS iCloud backup enabled. Especially since iTunes will no longer
create automatic backups with iCloud enabled. For those with a large
volume of maps, it's better to disable the Gaia GPS iCloud backup to
save space in the cloud for more useful stuff and perform manual
backups in iTunes every once in while (if you want backups of the
maps).

Anyway, with the upcoming feature in Gaia GPS that will allow us to
import and export tracks and maps, you will have more control over the
backups. You can download all maps to the computer, store them in a
safe location and any maps you don't often use can be removed from the
iOS device to save space on the device, the iCloud backups (if
enabled) and the iTunes backups. Once you need one of these maps,
simply upload them and you are ready to go. A win-win situation :).

> Does anyone know how iCloud backup works? How often does it backup? Does it
> use both WiFi and 3G? Does it transmit new data incrementally, or does it do
> a full backup? If I have, say, 1.0GB of storage, and iCloud performs a full
> backup daily, that could exceed my 2GB/month data allowance in two days!

According to the documentation <http://support.apple.com/kb/HT4859>,
the iCloud backup is performed daily when the device:

- is connected to Internet over Wi-Fi
- is connected to a power source
- is screen locked

That's for the automatic iCloud backups. Note that you can always
perform a manual iCloud backup any time without these restrictions
(thus over 3G and without a power source connected).

The iCloud backups are incremental (the first one is a full backup of
course), although I have no idea how smart the system is in
identifying changes. iTunes is also supposed to make incremental
backups, but sometimes it seems to create a much larger (more time
consuming) backup than the changes I made would imply...

Apart from the backup functionality, iCloud also offers automatic
synchronization. When wifi is not available, iCloud will use the 3G
connection to synchronize contacts, calendar, mail, notes, reminders
and favorites! Those will probably not eat up much of your 3G data
bundle, but much worse is that iCloud will also synchronize purchases
in iTunes (apps, books and music) over 3G when wifi is not available!
The documentation also suggests iCloud can synchronize "documents and
data" over 3G, but that seems to be something different from the
"documents" folder being discussed in this thread for Gaia GPS. As I
understand it, it's new functionality in iOS 5 that allows apps to
store and synchronize certain data in the cloud. Support for this must
be coded in the application.

As you can see, depending on how you use the device(s), iCloud can be
a serious drain on your 3G connection... I think it's best to disable
some functionality to prevent using 3G for all this:

"Settings" --> "iCloud" --> "Documents & Data" --> "Off" or "On" with
"Use Mobile" disabled
"Settings" --> "Store" --> disable "Use Mobile Data"

After disabling those, iCloud will still synchronize over 3G when wifi
is not available, but the potentially "mobile data plan eaters" should
be disabled.

> I don't connect my iPhone to a computer very often, but most apps don't need
> to be backed up that frequently, either.

Since iOS 5 you can sync or backup to iTunes over wifi. No need to
connect the device to the computer although obviously computer must be
on with iTunes running. You can initiate manual sync operations from
the iOS device, but it seems manual backups can only be made through
iTunes. Note that you must enable wifi syncing for each iOS device
individually in iTunes. Anyway, performing a sync or backup over wifi
is much slower than over USB...

[satcp]

Andrew: I noticed some applications like AVPlayerHD (video player)
have a setting to disable iTunes backups. After turning on the option
in the app, uploaded videos no longer show in the app documents in
iTunes. This obviously also excludes them from the iTunes (and iCloud)
backup. I am not an iOS developer, so I do not know how this works.
Perhaps this just moves the videos to the cache or tmp folder, as Gaia
GPS did in the past. The developers may not be aware of the potential
loss of data when the available space decreases, but maybe they found
another way that actually works. This may be worth investigating and
perhaps an upcoming Gaia GPS version can have a "disable backups"
option in the settings as well (personally I would only disable it for
the huge chunks of data like maps, not tracks). That way everyone
would be happy ;).

[Andrew Johnson]

That's interesting, I'll shoot the developer an email and see what he says. I'm guessing that he is moving the data to Documents as you describe, which has the iOS5 issue, but I'll check to make sure.

I eventually got a response back from Apple when I asked them what I should do, and they basically told me I would need to keep all the maps in Caches and let users suffer the consequences. Luckily, they did approve the releases that moved stuff to Documents, despite what Developer Technical Support told me.

Andrew

Andrew Johnson

Co-founder

www.GaiaGPS.com

415 335 7447