J. Chris Anderson can you help? Re: filtering changes feed

[Seung Chan Lim]

This is something that is baffling me.

I'm trying to filter my changes feed from the local couch db on phone gap.

I've created a design doc ( _design/utils) with the filters attribute set to a string representation of the following function

function(doc, req) { 

  if (req.query.doc_ids.indexOf(doc._id) != -1) { 

      return true;

  } else {

      return false;

  } 

      }

then called from within phonegap

http://lite.couchbase./status32/_changes?feed=continuous&include_docs=true&filter=utils/by_id&doc_ids=%4B%22id%22%5D

I'm getting a 404

Am I doing something obviously wrong?

slim

[Seung Chan Lim]

By local couch db I mean couch base lite.

[Jens Alfke]

What OS?
And what does the entire design document look like?

—Jens

[Seung Chan Lim]

iOS

<<snip>> indicates removed content

{_id: "_design/utils"

  views: {

      personal_history: {

  map: "<< snip >>"

      },

      interpersonal_history: {

  map: "<< snip >>"

      }

  },

  filters: {

      by_id: "function(doc, req) { 

  if (req.query.doc_ids.indexOf(doc._id) != -1) { 

      return true;

  } else {

      return false;

  } 

      }"

  },

  "roles": ["design document"]

      }

slim

[Jens Alfke]

That’s not valid JSON (some of the keys aren’t quoted) but I’m assuming that’s because you typed it in by hand? Other than that it looks OK.

IIRC there have been some other reports of people having trouble setting up JavaScript filters in design docs, but no one’s actually filed an issue (ideally with a test case or detailed steps to reproduce.)

—Jens

[Jens Alfke]

I just took a minute to debug this. The answer is simply that there isn't any support (yet) for JavaScript-based filters. The JavaScriptCore integration only supports views (map and reduce functions) so far.

I don't think it would be hard to implement (based on CBLJSViewFunction) but no one's done it.

--Jens

[Jens Alfke]

On Jul 29, 2014, at 4:14 PM, Jens Alfke <jens....@gmail.com> wrote:

I just took a minute to debug this. The answer is simply that there isn't any support (yet) for JavaScript-based filters. The JavaScriptCore integration only supports views (map and reduce functions) so far.

Oops, not true, Should have looked at the source more closely. JS filters are already implemented, it’s just that the top-level function that registers JS compilers only registers the view compiler, not the filter compiler. This just took a couple of lines to fix, so I’ve checked it in (commit 812cf64.)

However, the PhoneGap container/plugin will have to be rebuilt (with this commit) before PhoneGap apps can use this. I don’t know how to do that, but Chris does…

—Jens

[Seung Chan Lim]

Ah, most excellent!

Yes, the invalid JSON is just my sloppiness.

Eagerly awaiting Chris to rebuild the phone gap plugin. 

How do I know when this gets done?

slim

[J. Chris Anderson]

On Tuesday, July 29, 2014 5:31:03 PM UTC-7, Seung Chan Lim wrote:

Ah, most excellent!

Yes, the invalid JSON is just my sloppiness.

Eagerly awaiting Chris to rebuild the phone gap plugin. 

How do I know when this gets done?

On my plate for this week is to cut a 1.0.1 release of PhoneGap (I don't think this will have your fixes, but as soon as I have 1.0.1 done I'll start work on 1.1) 1.1 will include the filter stuff. It will also drop support for iOS 6. (Although we could maintain a parallel version that supports iOS 6 if there is strong demand.)

So my guess is that we'll have something you can play with early next week. Here's the bug to follow / ping me on if it takes longer than a week.

Chris

Chris

[J. Chris Anderson]

On Wednesday, July 30, 2014 6:40:31 PM UTC-7, J. Chris Anderson wrote:

Eagerly awaiting Chris to rebuild the phone gap plugin. 

How do I know when this gets done?

On my plate for this week is to cut a 1.0.1 release of PhoneGap (I don't think this will have your fixes, but as soon as I have 1.0.1 done I'll start work on 1.1) 1.1 will include the filter stuff. It will also drop support for iOS 6. (Although we could maintain a parallel version that supports iOS 6 if there is strong demand.)

So my guess is that we'll have something you can play with early next week. Here's the bug to follow / ping me on if it takes longer than a week.

Sent too early. Here is the link. You could also try applying this locally. (In every copy of CBLite.m you can find!) 

[J. Chris Anderson]

On Wednesday, July 30, 2014 6:42:53 PM UTC-7, J. Chris Anderson wrote:

On Wednesday, July 30, 2014 6:40:31 PM UTC-7, J. Chris Anderson wrote:

Eagerly awaiting Chris to rebuild the phone gap plugin. 

How do I know when this gets done?

Here is a 1.0.2 RC build for the phonegap plugin that includes Jens patch. You can download and unzip it, and then use the full path to the unzipped directory, instead of a git URL for the PhoneGap plugin installer.

I'll verify it after lunch but I'm 99% sure it is great so give it a try. Once it passes verification here I'll update the git and cordova repos. Let me know how that goes.

Chris

[Seung Chan Lim]

Thanks Chris.

I'm testing it out. I'm using the main GIT repository URL I've been using since it seems to be up-to-date. If that assumption is incorrect, please let me know.

What I'm seeing is that I no longer get a 404. But I'm not getting any change notifications, either.

This is what I have in my design document (id is _design/utils)

filters: {

      by_id: (function(doc, req) { 

  if (req.query.doc_ids.indexOf(doc._id) != -1) { 

      return true;

  } else {

      return false;

  } 

      }).toString()

  }

The request URL looks like this

http://lite.couchbase./status32/_changes?feed=continuous&include_docs=true&filter=utils/by_id&doc_ids=%5B%22MY_ID%22%5D

Can you see if I'm doing something obviously wrong?

slim

[J. Chris Anderson]

On Saturday, August 2, 2014 8:37:58 PM UTC-7, Seung Chan Lim wrote:

Thanks Chris.

I'm testing it out. I'm using the main GIT repository URL I've been using since it seems to be up-to-date. If that assumption is incorrect, please let me know.

What I'm seeing is that I no longer get a 404. But I'm not getting any change notifications, either.

If you write a function that passes everything does it work? I guess my first debug pass would be to ensure that the JS is getting called and runs on an always true filter.

If that works then it's just a matter of debugging your JS until you get the results you need. If even an always true filter doesn't work, then we need to debug the rest of the code path, so we'll probably end up opening a ticket with whatever you find there.

Chris

[Seung Chan Lim]

Yeah, I just tried a filter that returns true no matter what, and still doesn't work.

I get a change notify from the http socket that does not utilizes the filter, but none from the one that does.

slim

[J. Chris Anderson]

On Sunday, August 3, 2014 2:00:04 PM UTC-7, Seung Chan Lim wrote:

Yeah, I just tried a filter that returns true no matter what, and still doesn't work.

I get a change notify from the http socket that does not utilizes the filter, but none from the one that does.

Any chance you can share (privately is ok) your app so I can try to reproduce? Otherwise I'll start an independent repro.

Thanks,

Chris

[Seung Chan Lim]

Sorry, I can't share the app. :( I'll be more than happy to help in others possible ways as needed.

slim

[J. Chris Anderson]

Just to let you know I'm looking into this and the filter compilation seems to be running fine. It *looks* like the problem is in JSON parameter handling. So it will take more investigation but now I know where to look.

Chris

[J. Chris Anderson]

It looks like this is an area with a recent change, making clear that parameters are not to be parsed as JSON:

https://github.com/couchbase/couchbase-lite-ios/commit/8bf60318ec828d8172a6dfc6275663505b57bf60

It may be that the plugin version you've got still tries to parse the parameters as JSON but in the future it won't.

Here's a hand-built PhoneGap plugin for now (with some extra logging if you enable -LogCBL_Router YES) https://dl.dropboxusercontent.com/u/14074521/com.couchbase.lite.phonegap.zip

These fixes will be in the next release.

Chris

[J. Chris Anderson]

FYI I remembered when debugging this that when you want to see what your filter parameters are, a line like this 

function(doc, req) {

  throw JSON.stringify(req);

}

will make them show up in the Xcode logs. In the case of the first code I tested, anything but numbers and true or "false" would fail to parse as a parameter. Now it works great, but you may have to decodeURIComponent and JSON.parse parameters in your filter function (or switch to a comma separated list or something easier to decode.)

Chris

[Seung Chan Lim]

Hi, I'm just getting around this. (went on a vacation)

So I see that now a dummy filter function that returns true works great. 

I'm still having trouble making my filter function work, and I'm guessing it has to do with the remark you made about decodeURIComponent and JSON.parse.

I tried throw JSON.stringify(req) as you suggested, but I'm not seeing it show up anywhere. Where does it show up? I have my safari developer console open which I use to debug my phonegap app.

Since I call it like this

http://lite.couchbase./status32/_changes?feed=continuous&include_docs=true&filter=utils/by_id&doc_ids=%4B%22id%22%5D

And my filter function looks like this

function(doc, req) { 

   if (req.query.doc_ids.indexOf(doc._id) != -1) { 

       return true;

   } else {

       return false;

   } 

       }

Are you saying I should:

var doc_ids = JSON.parse(decodeURIComponent(requ.query.doc_ids));

if (doc_ids.indexOf(doc._id) != -1)  {

return true;
}

Am I understanding you?

slim