Pull filter / Unknown filter

[Adam Zieliński]

Hi all,

I have a problem with pull filter. I use sync gateway to replicate data and i want to limit download of data to device.

I created _designe/mydoc on server side with filter function:

{

    _id: "_design/mydoc"

    filters: {

        myfilter: "function(doc) {return true;}"

    }

}

when app runs replication (gateway) http://host:4984/appName/_changes?feed=longpoll&limit=50&heartbeat=300000&filter=mydoc/myfilter i receive 

{

    error: "Bad Request"

    reason: "Unknown filter; try sync_gateway/bychannel"

}

When i call [GET] (gateway) http://host:4984/appName/_design/mydoc i receive 

{

    error: "not_found"

    reason: "unknown URL"

}

When i call [GET] (database) http://host:8092/appName/_design/mydoc i receive:

{

    _id: "_design/mydoc"

    filters: {

        myfilter: "function(doc) {return true;}"

    }

}

Is it ok that sync gateway can't see _design doc?

Why i can't use filter?

How should I create _desing doc on server side? buy "create document" on admin panel or maybe in other way?

Regards,

Adam.

[Jens Alfke]

On Jan 8, 2014, at 8:35 AM, Adam Zieliński <azie...@gmail.com> wrote:

Is it ok that sync gateway can't see _design doc?

Why i can't use filter?

The Sync Gateway doesn’t support CouchDB-style filters (or design documents.)

Channels are intended to serve the same purpose as filtered pull replication, although more scalably. If you want a client to sync with only a subset of documents on the server, you’d write a sync function that tags that subset of docs for a particular channel, and give the user access to that channel. (Or you can use multiple channels, for flexibility.) 

The difference is that channels are assigned once when the document is saved, whereas filters have to run every time a client pulls a document, which means filters become too expensive with large numbers of clients.

In general, you shouldn’t expect the gateway to be a CouchDB clone, although it has the same overall design and a similar API. It’s intended to be compatible enough to support the same replication protocol, but the differences in filtering are an incompatibility.

—Jens

[Adam Zieliński]

I have one more question. I found in one of post:

-"So gateway will run sync function again when I update the sync function itself and/or when documents  are  updates which removes/moves channel ?"

-"Yes."

how to implement sync function when we want to chose chanel by date. For example we want to download to device data that was created in last month? Document and sync function will not change, but after some period of time document should be move from current_doc_channel to old_doc_channel? Of course user should only get document from current_doc_channel.

Regards,

Adam.

[Jens Alfke]

On Jan 9, 2014, at 2:21 AM, Adam Zieliński <azie...@gmail.com> wrote:

how to implement sync function when we want to chose chanel by date. For example we want to download to device data that was created in last month? Document and sync function will not change, but after some period of time document should be move from current_doc_channel to old_doc_channel? Of course user should only get document from current_doc_channel.

We don’t have a good solution to that yet … it’s a tricky problem because relative dates don’t naturally fit into our processing model.

Possible workarounds:

(a) Create a channel per month (e.g. “sales_01_2014”) and have the sync function assign a doc to a monthly channel based on its creation date. The client then asks for the most recent monthly channels when it starts a sync.

or,

(b) Give a document an “age_in_months” property that starts at 0. Every month run a housekeeping task that increments this property. The sync function can use age_in_months to determine whether to assign a doc to a channel. (The obvious drawback is that this housekeeping task touches a lot of documents and is likely to get slow as the data set grows.)

I’m inclined to recommend (a). Anyone else know of other possibilities I didn’t think of?

—Jens

[Adam Zieliński]

I can't find how to connect to particular channel. At this moment i connect by:

var pull = {

    target : dbName,

    source : REMOTE_SYNC_URL,

    continuous : true

};

var serverUrl =  localUrl + "_replicate";

$http.post(serverUrl, pull);

How i suppose to change this code to connect to particular channel?

Thanks for your help.

Adam

[Jens Alfke]

On Jan 16, 2014, at 2:06 AM, Adam Zieliński <azie...@gmail.com> wrote:

How i suppose to change this code to connect to particular channel?

http://docs.couchbase.com/sync-gateway/#replicating-channels-to-couchbase-lite :

To replicate channels to Couchbase Lite, you configure the replication to use a filter named sync_gateway/bychannel with a filter parameter named channels. The value of the channels parameter is a comma-separated list of channels to fetch. The replication from Sync Gateway now pulls only documents tagged with those channels.

So you’d change your replication spec to:

var pull = {

    target : dbName,

    source : REMOTE_SYNC_URL,

    continuous : true,

    filter: "sync_gateway/bychannel",

    query_params: {"channels": "CHANNELNAME"}

};

[Adam Zieliński]

I have a problem:

My sync function is:

{ "Sync": "function(doc) {channel(doc.userName);access(doc.userName, doc.userName);}" }

My user: 

{"name":"adam","admin_channels":["adam","public"],"all_channels":["adam","public"]}

Example of doc on server side:

{

  "_sync": {

    "rev": "1-d493679d-5e81-4138-92cb-9aacf01db9b9",

    "sequence": 463,

    "history": {

      "revs": [

        "1-d493679d-5e81-4138-92cb-9aacf01db9b9"

      ],

      "parents": [

        -1

      ],

      "bodies": [

        ""

      ],

      "channels": [

        [

          "adam"

        ]

      ]

    },

    "channels": {

      "adam": null

    },

    "access": {

      "adam": {

        "adam": 463

      }

    }

  },

  "caseType": "real",  

  "type": "casedata",

  "userName": "adam"

}

Logs:

V/CBLDatabase(26228): Making request to

http://adam:---@hostName:4984/bucketName/_changes?feed=longpoll&limit=50&heartbeat=300000&filter=sync_gateway%2Fbychannel&channels=adam

When I call pull replication it doesnt work. Push replication works without any problems. Any idea?

Regards,

Adam

[Jens Alfke]

On Jan 28, 2014, at 10:26 AM, Adam Zieliński <azie...@gmail.com> wrote:

When I call pull replication it doesnt work. Push replication works without any problems. Any idea?

What does "it doesn't work" mean, exactly? Have you tried turning on any client-side sync logging?

What do you get if you use curl to GET that _changes feed URL that you posted?

—Jens

[J. Chris Anderson]

It can make debugging easier to work against the localhost admin port at 4985, since that will give you access to all the documents.

What do you see at 

curl http://localhost:4985/dbName/_changes

(By default the admin port is only available on localhost)

[Adam Zieliński]

Sorry. I have incorrect configuration on gateway. All the doc was created by GUEST because GUEST user was enable. Sync function puts doc in userName channels. When i wanted to pull doc it was nothing to download for GUEST user. That's why push works and pull wasn't.

Adam.

[Jens Alfke]

On Jan 29, 2014, at 7:21 AM, Adam Zieliński <azie...@gmail.com> wrote:

Sorry. I have incorrect configuration on gateway. All the doc was created by GUEST because GUEST user was enable. Sync function puts doc in userName channels. When i wanted to pull doc it was nothing to download for GUEST user. That's why push works and pull wasn't.

Glad you figured it out! This sounds like a problem that the upcoming gateway admin UI could have helped troubleshoot. It should be available pretty soon for experimental use, and it'll be built into the GA release.

—Jens