Problems getting started with the iOS GrocerySync example

[Chris Dzombak]

I'm having some problems getting started with the iOS GrocerySync
Couchbase Mobile project.

I downloaded the latest project from Github (commit 21cc84e), and I
downloaded the iOS Couchbase Mobile frameworks from
http://www.couchbase.com/wiki/display/couchbase/iOS and put them in
the project's Frameworks folder.

(1) When I run the app in the simulator and point it to a new
IrisCouch database, it seems like the initial setup goes well (a
design document appears on IrisCouch, and usually any items that
already exist in the app appear on IrisCouch), but then any subsequent
changes (like adding new items in the iOS app) fail to replicate with
this error:

[error] [<0.423.0>] Replicator: couldn't write document
`8ebc9c6478a85c1b0b9df95bd7000e01`, revision `1-
b3215128ed18746e1de86d4cfb0cd559`, to target database `http://my-
username.iriscouch.com/grocery-sync/`. Error: `EXIT`, reason:
`{{badmatch,[]},
[{couch_query_servers,new_process,3},
{couch_query_servers,lang_proc,3},
{couch_query_servers,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}`.

I have tried cleaning & rebuilding the project, removing it from the
simulator & reinstalling it, and deleting & recreating the DB on
IrisCouch, and this seems to be a persistent error.

(2) I then tried running the app on the simulator and on my iPhone.
When I point my simulator to my phone, or my phone to the simulator,
for replication (with a URL like http://192.168.1.118:50971/grocery-sync,
with the correct IP and port), I just continuously get these timeout
messages, and no replication occurs:

[error] [<0.108.0>] Error in replication
`098a26f598185375f7c3bad4101fa863+continuous` (triggered by document
`23467daaf284f585b8040eca480010b2`): timeout
Restarting replication in 5 seconds.
1> 2012-02-11 14:32:56.050 GrocerySync[7276:707] SYNC progress: 0 / 0
[error] [<0.108.0>] Error in replication
`af487223a2bd7208e0966f6b139de20b+continuous` (triggered by document
`23467daaf284f585b8040eca48000896`): timeout
Restarting replication in 10 seconds.
1> [error] [<0.108.0>] Error in replication
`098a26f598185375f7c3bad4101fa863+continuous` (triggered by document
`23467daaf284f585b8040eca480010b2`): timeout
Restarting replication in 10 seconds.

Is it possible for two Couchbase Mobile apps to replicate between
themselves?

Thanks for any help,
Chris

[Jens Alfke]

On Feb 11, 2012, at 11:46 AM, Chris Dzombak wrote:

(1) When I run the app in the simulator and point it to a new
IrisCouch database, it seems like the initial setup goes well (a
design document appears on IrisCouch, and usually any items that
already exist in the app appear on IrisCouch), but then any subsequent
changes (like adding new items in the iOS app) fail to replicate

I think the problem is that the design doc that gets replicated to IrisCouch contains an Objective-C validation function. Once the design doc is replicated upstream, IrisCouch’s servers will start using it, which means that when you try to add new documents it’ll run the validation function against them … but IrisCouch has no idea how to run Objective-C functions, so it barfs.

This is probably happening because your IrisCouch database is in “admin party” mode, where even anonymous access gets admin privileges. (Only admins can write design docs to a db.) So the workaround is to delete the database, then set up a real admin account on IrisCouch. You can leave the default settings to allow anonymous write access to databases, so GrocerySync will still work, it just won’t be able to push the design document.

I’m not sure what the proper solution is. IMHO, syncing design documents was a bad idea, but I admit that it’s useful in some circumstances. Probably GrocerySync should add a filter to its replications to weed out design documents.

(2) I then tried running the app on the simulator and on my iPhone.
When I point my simulator to my phone, or my phone to the simulator,
for replication (with a URL like http://192.168.1.118:50971/grocery-sync,
with the correct IP and port), I just continuously get these timeout
messages, and no replication occurs:

Unless you’ve changed the setting in the CouchDB .ini file, Couchbase Mobile binds to interface 127.0.0.1 (loopback) which means it can’t be reached from other hosts. This is for security. If you do want to allow incoming connections, change the binding address to 0.0.0.0 (all interfaces). Just be aware that, unless you set up user accounts, your data is now world-readable and -writeable.

—Jens

[Chris Dzombak]

Thanks for the pointers, Jens.

I will have to do more reading on exactly how CouchDB supports authentication and user management. I do have this replicating with IrisCouch, though, which is great.

I assume this design document comes from "grocery-sync.couch" that's in the example app's Resources folder?

I've also gotten device-to-device replication working by changing the default_ios.ini in Frameworks/Couchbase.framework/CouchbaseResources. (I am aware of the security implications of this change.) However, since this is part of the framework, it won't be tracked by version control. Is there a preferred way to include a custom CouchDB .ini file that doesn't require modifying anything inside the Frameworks directory?

Thanks,
Chris

> Attachments:
> - smime.p7s
>

[J Chris Anderson]

On Feb 13, 2012, at 3:32 PM, Chris Dzombak wrote:

Thanks for the pointers, Jens.  

I will have to do more reading on exactly how CouchDB supports authentication and user management. I do have this replicating with IrisCouch, though, which is great.

I assume this design document comes from "grocery-sync.couch" that's in the example app's Resources folder?

I've also gotten device-to-device replication working by changing the default_ios.ini in Frameworks/Couchbase.framework/CouchbaseResources. (I am aware of the security implications of this change.) However, since this is part of the framework, it won't be tracked by version control. Is there a preferred way to include a custom CouchDB .ini file that doesn't require modifying anything inside the Frameworks directory?

Here are some pointers about custom config files. Let us know how the P2P stuff goes!

Basic instructions on how to register a custom config file: 

https://github.com/couchbase/iOS-Couchbase/blob/master/Framework/Resources/default_ios.ini#L8

An example of an app that has a custom ini file:

https://github.com/couchbase/iOS-Couchbase/blob/master/EmptyApp/Source/EmptyAppDelegate.m#L38

[Cristian]

Chris,

Do you have any pointers on how to set this up the demo to replicate
with IrisCouch? I have the exact same error but I cannot find any
information on how to get it to work besides this thread.

Thanks,

Cris

On Feb 13, 6:32 pm, Chris Dzombak <ch...@chrisdzombak.net> wrote:
> Thanks for the pointers, Jens.
>
> I will have to do more reading on exactly how CouchDB supports authentication and user management. I do have this replicating with IrisCouch, though, which is great.
>
> I assume this design document comes from "grocery-sync.couch" that's in the example app's Resources folder?
>
> I've also gotten device-to-device replication working by changing the default_ios.ini in Frameworks/Couchbase.framework/CouchbaseResources. (I am aware of the security implications of this change.) However, since this is part of the framework, it won't be tracked by version control. Is there a preferred way to include a custom CouchDB .ini file that doesn't require modifying anything inside the Frameworks directory?
>
> Thanks,
> Chris
>
>
>
>
>
>
>
> On Monday, February 13, 2012 at 5:42 PM, Jens Alfke wrote:
>
> > On Feb 11, 2012, at 11:46 AM, Chris Dzombak wrote:
> > > (1) When I run the app in the simulator and point it to a new
> > > IrisCouch database, it seems like the initial setup goes well (a
> > > design document appears on IrisCouch, and usually any items that
> > > already exist in the app appear on IrisCouch), but then any subsequent
> > > changes (like adding new items in the iOS app) fail to replicate
>
> > I think the problem is that the design doc that gets replicated to IrisCouch contains an Objective-C validation function. Once the design doc is replicated upstream, IrisCouch’s servers will start using it, which means that when you try to add new documents it’ll run the validation function against them … but IrisCouch has no idea how to run Objective-C functions, so it barfs.
>
> > This is probably happening because your IrisCouch database is in “admin party” mode, where even anonymous access gets admin privileges. (Only admins can write design docs to a db.) So the workaround is to delete the database, then set up a real admin account on IrisCouch. You can leave the default settings to allow anonymous write access to databases, so GrocerySync will still work, it just won’t be able to push the design document.
>
> > I’m not sure what the proper solution is. IMHO, syncing design documents was a bad idea, but I admit that it’s useful in some circumstances. Probably GrocerySync should add a filter to its replications to weed out design documents.
>
> > > (2) I then tried running the app on the simulator and on my iPhone.
> > > When I point my simulator to my phone, or my phone to the simulator,

> > > for replication (with a URL likehttp://192.168.1.118:50971/grocery-sync,

[Jens Alfke]

On Feb 24, 2012, at 11:39 AM, Cristian wrote:

Do you have any pointers on how to set this up the demo to replicate
with IrisCouch? I have the exact same error but I cannot find any
information on how to get it to work besides this thread.

On IrisCouch, configure your account to have an admin user instead of being in the default “admin-party” mode where anonymous users can do anything.

To do this, I think you need to go to the Futon web interface at /_utils, then look for the “Fix this!” notice in the sidebar and click it. Then set up an admin password.

After you do this, anonymous connections (like the one from your app) will no longer have permission to upload design documents to the server, which is exactly what you want.

Alternatively, you could configure the push replication from the app to use a filter that omits document IDs beginning with “_design/“. If someone has a chance to implement this, I’d love to include the patch :)

—Jens

[Cristian]

Jens,

I actually have the app replicating both ways now. I'll look into
filters and P2P soon. To summarize for someone that needs a quick
adhoc method of enabling replication to IrishCouch:

1. On IrisCouch:
a. Sign up for IrisCouch. You will be given a couch with a URL
like this: "http://yourCouch.iriscouch.com/"
b. Go to http://yourCouch.iriscouch.com/_utils/. Click on "create
database" on the TOP LEFT of the screen to create a database.
Remember, NO CAPITAL LETTERS! You will be able to access this database
through the link: "http://yourCouch.iriscouch.com/yourDatabase"
c. Also, your account to have an admin user instead of being in

the default “admin-party” mode where anonymous users can do anything.

To do this, click on the "fix this" link BOTTOM RIGHT of futon.

2) In xCode:
a) Add the following variables to your header file:

CouchReplication *push, *pull;
CouchDatabase *database;
- (void) startContinuousSyncWith: (NSURL*)otherDbURL;

b) Add this code to your body file and call after your database
has been initialized in your CouchbaseDelegate:

-(void)couchbaseMobile:(CouchbaseMobile*)couchbase didStart:
(NSURL*)serverURL {
// ...Init database...
// Enable continuous sync!
NSString* otherDbURL = @"http://yourCouch.iriscouch.com/
yourDatabase";
if (otherDbURL.length > 0)
[self startContinuousSyncWith: [NSURL URLWithString: otherDbURL]];
}

- (void) startContinuousSyncWith: (NSURL*)otherDbURL {
NSLog(@"-BEGIN- startContinuousSyncWith: otherDbURL: We are about to
setup replication for oldDatabase %@", otherDbURL);
pull = [database pullFromDatabaseAtURL: otherDbURL];
pull.continuous = YES;
push = [database pushToDatabaseAtURL: otherDbURL];
push.continuous = YES;
}

c. Create a custom .ini file which changes the bindings for
127.0.0.1 to 0.0.0.0 per J Chris Andersons post to allow UNIVERSAL
access your embedded server (be careful).

4. Have fun!

Thanks for all the help,

Cristian

>  smime.p7s
> 6KViewDownload

[Jens Alfke]

On Feb 25, 2012, at 12:22 PM, Cristian wrote:

- (void) startContinuousSyncWith: (NSURL*)otherDbURL {

Looks good. There’s equivalent code in the RootViewController already, but it’s driven by the GUI, so if instead you want to hardcode a replication this is a fine way to do it.

    c. Create a custom .ini file which changes the bindings for
127.0.0.1 to 0.0.0.0 per J Chris Andersons post to allow UNIVERSAL
access your embedded server (be careful).

This isn’t necessary for your immediate purposes. The replication you’re doing only involves outgoing connections to iriscouch.com. The only reason to allow incoming connections to your server would be if you want another server to be able to take the active role in replicating with you, i.e. P2P. And for this to work in practice, you’d also need to use Bonjour or something similar to let the other device know your IP address and port number.

—Jens

[Cristian]

Jens,

With the above code I am actually able to replicate both ways with
Couchbase Mobile embedded server.

As for the RootViewController info. I found this code from a variety
of projects. I had no idea about it. : (

Thanks again,

Cristian

On Feb 25, 6:57 pm, Jens Alfke <j...@couchbase.com> wrote:
> On Feb 25, 2012, at 12:22 PM, Cristian wrote:
>
> - (void) startContinuousSyncWith: (NSURL*)otherDbURL {
>
> Looks good. There’s equivalent code in the RootViewController already, but it’s driven by the GUI, so if instead you want to hardcode a replication this is a fine way to do it.
>
>     c. Create a custom .ini file which changes the bindings for
> 127.0.0.1 to 0.0.0.0 per J Chris Andersons post to allow UNIVERSAL
> access your embedded server (be careful).
>

> This isn’t necessary for your immediate purposes. The replication you’re doing only involves outgoing connections to iriscouch.com<http://iriscouch.com>. The only reason to allow incoming connections to your server would be if you want another server to be able to take the active role in replicating with you, i.e. P2P. And for this to work in practice, you’d also need to use Bonjour or something similar to let the other device know your IP address and port number.
>
> —Jens

[Pulkit Singhal]

I get the following error when running the GUI driven replication, can someone help me figure out what these logs really tell me?
I've already setup authN for https://domain.iriscouch.com and am not sure if something there is botched or somewhere else.

2012-03-26 22:32:41.492 TouchDB Demo[69475:f803] CouchLiveQuery: Starting...
2012-03-26 22:32:42.342 TouchDB Demo[69475:f803] CouchConnectionChangeTracker[_replicator]: Started... <touchdb:///_replicator/_changes?feed=longpoll&heartbeat=300000&since=12>
2012-03-26 22:32:42.342 TouchDB Demo[69475:f803] CouchLiveQuery: Starting...
2012-03-26 22:32:42.344 TouchDB Demo[69475:f803] CouchLiveQuery: ...Finished (status=200)
2012-03-26 22:32:42.344 TouchDB Demo[69475:f803] CouchLiveQuery: ...Rows changed! (now 2)
2012-03-26 22:32:42.345 TouchDB Demo[69475:f803] CouchPersistentReplication[1F86..28CA]: state := triggered
2012-03-26 22:32:42.345 TouchDB Demo[69475:f803] CouchPersistentReplication[37E4..436F]: state := error
2012-03-26 22:32:42.976 TouchDB Demo[69475:f803] CouchLiveQuery: ...Finished (status=200)
2012-03-26 22:32:42.976 TouchDB Demo[69475:f803] CouchLiveQuery: ...Rows changed! (now 4)
2012-03-26 22:32:42.989 TouchDB Demo[69475:f803] CouchPersistentReplication[1F86..28CA] = Processed 0 / 0 changes
22:32:43.295| TDRemoteJSONRequest[GET https://domain.iriscouch.com/grocery-sync/_local/818EA23DC64BF02B9BED2DF8574512E37F6E8830]: Got error Error Domain=TDHTTP Code=404 "404 not found" UserInfo=0x6c6ef50 {NSURL=https://fermyon.iriscouch.com/grocery-sync/_local/818EA23DC64BF02B9BED2DF8574512E37F6E8830, NSLocalizedDescription=404 not found, NSLocalizedFailureReason=not found}
2012-03-26 22:32:43.296 TouchDB Demo[69475:13003] CouchTouchDBServer[touchdb:///]: Replication progress changed
2012-03-26 22:32:43.297 TouchDB Demo[69475:f803] CouchPersistentReplication[1F86..28CA] = Idle
22:32:43.615| WARNING*** : TDConnectionChangeTracker[0x6c3f9e0 grocery-sync]: Got status 404
2012-03-26 22:32:43.616 TouchDB Demo[69475:13003] CouchTouchDBServer[touchdb:///]: Replication progress changed
2012-03-26 22:32:43.617 TouchDB Demo[69475:f803] CouchPersistentReplication[1F86..28CA]: error (
    404,
    "404 not found"
)

[Nicolas Lapomarda]

Hey,

Have you tried entering the database URL instead of the root URL of the server? In your case, https://fermyon.iriscouch.com/grocery-sync would be the URL you want to sync to, I think.

Also, I would try creating the emtpy database on the remote server, just in case auto-create isn't enabled for this replication.

Hope this helps,

Nico

On Mar 27, 2012, at 5:17 AM, Pulkit Singhal wrote:

Hi Jens,

I can't get the UI driven replication to take place.

I added:
    NSURLCredential* cred;
    cred = [NSURLCredential credentialWithUser: @"username"
                                      password: @"password"
                                   persistence: NSURLCredentialPersistencePermanent];
    NSURLProtectionSpace* space;
    space = [[NSURLProtectionSpace alloc] initWithHost: @"domain.iriscouch.com"
                                                   port: 443
                                               protocol: @"https"
                                                  realm: @"administrator"
                                   authenticationMethod: NSURLAuthenticationMethodDefault];
    [[NSURLCredentialStorage sharedCredentialStorage] setDefaultCredential: cred
                                                        forProtectionSpace: space];

Before:
CouchTouchDBServer* server = [CouchTouchDBServer sharedInstance];

In the TouchDB-iOS/Demo-iOS/DemoAppDelegate.m file but nothign happens after I run and go to the "Configure Sync" screen and press "Done" after pasting: https://fermyon.iriscouch.com/

Shouldn't it create a default DB named "grocery-sync" and replicate with it?

Please let me know what I may be doing wrong.

Thanks!
- Pulkit

On Saturday, February 25, 2012 5:57:34 PM UTC-6, Jens Alfke wrote:

[Pulkit Singhal]

I looked at the code and it explicitly turns a rootURL into a databaseURL so that's not a problem so far.
I did take you up on your advice to create the database ahead of time but it didn't make a difference.

Looking at the following logs, it seems like after failing to establish the first replication connection, it just goes into an idle state but I don't understand why it failed in the first place, how can I get more info?

CouchPersistentReplication[1F86..28CA] = Processed 0 / 0 changes

TDRemoteJSONRequest [GET https://fermyon.iriscouch.com/grocery-sync/_local/818EA23DC64BF02B9BED2DF8574512E37F6E8830]:

Got error Error Domain=TDHTTP Code=404 "404 not found"

UserInfo=0x6e2f0e0 {NSURL=https://fermyon.iriscouch.com/grocery-sync/_local/818EA23DC64BF02B9BED2DF8574512E37F6E8830,

NSLocalizedDescription=404 not found,
NSLocalizedFailureReason=not found}

CouchTouchDBServer[touchdb:///]: Replication progress changed

CouchPersistentReplication[1F86..28CA] = Idle

The grocery-sync/_local/818EA23DC64BF02B9BED2DF8574512E37F6E8830 does not exist on the iriscouch.com/grocery-sync DB that I created and I don't know why it expects it to? Is it something that has to be manually create in order to start replication?

[Nicolas Lapomarda]

Probably won't help you much, but this is the URL of the 'special' document that keeps track of replication progress, where previous attempts left off, etc.

Nico

[Pulkit Singhal]

Right! That is why its so surprising because I would've expected it to be created if it didn't yet exist, it can't possibly be a manual task.

[Nicolas Lapomarda]

Yup, it is definitely meant to be created automatically. Maybe the experts can chime in here :)

Nico

[Pulkit Singhal]

Thanks Nico :)

Now for the extremely weird stuff. The replication simply wasn't working ... until I enabled LogSync, LogSyncVerbose and LogRemoteRequest. Is it possible that some crucial code is inside an if block which is controlled by these logging boolean flags?

[Pulkit Singhal]

Yup confirmed it by trial and error. Everytime I have enabled LogSync, LogSyncVerbose and LogRemoteRequest and connect to a DB, the replication happens otherwise it fails!

[Nicolas Lapomarda]

If it helps, I remember having issues with the special document, and whenever that happened, the replication stopped for a while, until it was triggered again and then it restarted properly. Since the local document wasn't there or was current, I was seeing documents being pushed that shouldn't have been because they were already in sync, but the replication then went on to complete successfully. 

I wonder if what you're observing is a manifestation of the replication restarting after failure, or if it really is linked to Log code. The experts would have to chime in here, probably Jens.

[Pulkit Singhal]

Ok I can believe that a little more than my own claim about log statements :) But then I think it would be good to know exactly how long the replication stops working for and when to expect it to come back online and not waste time troubleshooting the wrong things.

[Pulkit Singhal]

BTW Nick, I can't find it on the wiki so I must ask, what is the variable and which class has it in order to turn on auto-create of a DB for the replication process?

[Nicolas Lapomarda]

No worries, been there :) It is the create_target instance variable, in CouchPersistentReplication.

Cheers

[Jens Alfke]

On Mar 27, 2012, at 8:55 AM, Pulkit Singhal wrote:

Looking at the following logs, it seems like after failing to establish the first replication connection, it just goes into an idle state but I don't understand why it failed in the first place, how can I get more info?

Turn on logging (-Log -LogSync -LogSyncVerbose). But a later email implies you did that already.

The 404 errors being logged aren’t actually problems. The first time you sync a specific pair of databases, the document on the remote database that stores the last state won’t exist yet. I keep meaning to put in a hack to suppress that specific error message because it shows up a lot but isn’t meaningful.

Yup confirmed it by trial and error. Everytime I have enabled LogSync, LogSyncVerbose and LogRemoteRequest and connect to a DB, the replication happens otherwise it fails!

Yikes. You’re sure you can reproduce this? I’d be surprised if I made a stupid mistake like putting something in a log call that changes state, but I’ve been known to make stupid mistakes sometimes. :( It’d be great if you could narrow it down to which of those three log modes that triggers it.

(Another possibility is that the bug is a race condition, and the logging slows down the code enough to affect whether or not it’s triggered. I hate bugs like that.)

—Jens

[Pulkit Singhal]

Ill try my best to narrow down and reproduce, it seems to have disappeared for now but I'm sure it'll come back to bite me if it exists.

[Pulkit Singhal]

This happened again today and I was able to fix it by deleting the app and reinstalling it. I'm using beta6 now ... I still haven't found a way to reproduce the corner-case, I'll keep trying.