DBAccessException on .queryView()

[matt martin]

I am attempting to query an Ektorp view using the instructions on the wiki but have been getting a DBAccessException. I've been scratching my head on this one for about a week. My code and the stacktrace are below.

TDDatabase db = server.getDatabaseNamed(Global.YOURTEAM_DB_NAME);

boolean status = db.open();

// Map<String, Object> objs = db.getAllDocs(null);

String dDocName = "ddoc";

String viewName = "allgames";

TDView allView = db.getViewNamed(String.format("%s/%s", dDocName, viewName));

allView.setMapReduceBlocks(new TDViewMapBlock()

{

public void map(Map<String, Object> document, TDViewMapEmitBlock emitter)

{

String type = (String) document.get("type");

if (type.equals("game") == true)

{

emitter.emit(null, document.get("_id"));

}

}

}, new TDViewReduceBlock()

{

public Object reduce(List<Object> keys, List<Object> values, boolean rereduce)

{

return null;

}

}, "1.0");

ViewQuery viewQuery = new ViewQuery().designDocId("_design/" + dDocName).viewName(viewName);

CouchDbInstance dbInstance = Global.getLocalCouchInstance(MainMenu.this);

CouchDbConnector connect = dbInstance.createConnector("yourteam", false);

List<String> ids = connect.getAllDocIds();

ViewResult viewResult = connect.queryView(viewQuery);

07-25 21:41:41.876: E/TDDatabase(8841): Exception in TDRouter

07-25 21:41:41.876: E/TDDatabase(8841): java.lang.reflect.InvocationTargetException

07-25 21:41:41.876: E/TDDatabase(8841): at java.lang.reflect.Method.invokeNative(Native Method)

07-25 21:41:41.876: E/TDDatabase(8841): at java.lang.reflect.Method.invoke(Method.java:507)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.router.TDRouter.start(TDRouter.java:367)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.ektorp.TouchDBHttpClient.executeRequest(TouchDBHttpClient.java:207)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.ektorp.TouchDBHttpClient.get(TouchDBHttpClient.java:70)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.ektorp.TouchDBHttpClient.getUncached(TouchDBHttpClient.java:78)

07-25 21:41:41.876: E/TDDatabase(8841): at org.ektorp.http.RestTemplate.getUncached(RestTemplate.java:26)

07-25 21:41:41.876: E/TDDatabase(8841): at org.ektorp.impl.StdCouchDbConnector.executeQuery(StdCouchDbConnector.java:410)

07-25 21:41:41.876: E/TDDatabase(8841): at org.ektorp.impl.StdCouchDbConnector.queryView(StdCouchDbConnector.java:449)

07-25 21:41:41.876: E/TDDatabase(8841): at com.yourteam.MainMenu$1.handleMessage(MainMenu.java:136)

07-25 21:41:41.876: E/TDDatabase(8841): at android.os.Handler.dispatchMessage(Handler.java:99)

07-25 21:41:41.876: E/TDDatabase(8841): at android.os.Looper.loop(Looper.java:150)

07-25 21:41:41.876: E/TDDatabase(8841): at android.app.ActivityThread.main(ActivityThread.java:4333)

07-25 21:41:41.876: E/TDDatabase(8841): at java.lang.reflect.Method.invokeNative(Native Method)

07-25 21:41:41.876: E/TDDatabase(8841): at java.lang.reflect.Method.invoke(Method.java:507)

07-25 21:41:41.876: E/TDDatabase(8841): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)

07-25 21:41:41.876: E/TDDatabase(8841): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)

07-25 21:41:41.876: E/TDDatabase(8841): at dalvik.system.NativeStart.main(Native Method)

07-25 21:41:41.876: E/TDDatabase(8841): Caused by: java.lang.NullPointerException

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.router.TDRouter.compileView(TDRouter.java:1267)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.router.TDRouter.queryDesignDoc(TDRouter.java:1307)

07-25 21:41:41.876: E/TDDatabase(8841): at com.couchbase.touchdb.router.TDRouter.do_GET_DesignDocument(TDRouter.java:1359)

07-25 21:41:41.876: E/TDDatabase(8841): ... 18 more

07-25 21:41:42.256: D/dalvikvm(8841): GC_CONCURRENT freed 346K, 40% free 4381K/7239K, external 0K/0K, paused 3ms+3ms

07-25 21:41:57.001: I/dalvikvm(8841): threadid=4: reacting to signal 3

07-25 21:41:57.011: D/dalvikvm(8841): threadid=1: still suspended after undo (sc=1 dc=1)

07-25 21:41:57.011: I/dalvikvm(8841): Wrote stack traces to '/data/anr/traces.txt'

07-25 21:41:59.543: D/AndroidRuntime(8841): Shutting down VM

07-25 21:41:59.543: W/dalvikvm(8841): threadid=1: thread exiting with uncaught exception (group=0x4001d5a0)

07-25 21:41:59.603: E/AndroidRuntime(8841): FATAL EXCEPTION: main

07-25 21:41:59.603: E/AndroidRuntime(8841): org.ektorp.DbAccessException: com.couchbase.touchdb.ektorp.TouchDBHttpResponse@405e3988

07-25 21:41:59.603: E/AndroidRuntime(8841): URI: touchdb:/yourteam/_design/ddoc/_view/allgames

07-25 21:41:59.603: E/AndroidRuntime(8841): Response Body: 

07-25 21:41:59.603: E/AndroidRuntime(8841): null

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.http.StdResponseHandler.createDbAccessException(StdResponseHandler.java:44)

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.http.StdResponseHandler.error(StdResponseHandler.java:62)

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.http.RestTemplate.handleResponse(RestTemplate.java:107)

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.http.RestTemplate.getUncached(RestTemplate.java:27)

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.impl.StdCouchDbConnector.executeQuery(StdCouchDbConnector.java:410)

07-25 21:41:59.603: E/AndroidRuntime(8841): at org.ektorp.impl.StdCouchDbConnector.queryView(StdCouchDbConnector.java:449)

07-25 21:41:59.603: E/AndroidRuntime(8841): at com.yourteam.MainMenu$1.handleMessage(MainMenu.java:136)

07-25 21:41:59.603: E/AndroidRuntime(8841): at android.os.Handler.dispatchMessage(Handler.java:99)

07-25 21:41:59.603: E/AndroidRuntime(8841): at android.os.Looper.loop(Looper.java:150)

07-25 21:41:59.603: E/AndroidRuntime(8841): at android.app.ActivityThread.main(ActivityThread.java:4333)

07-25 21:41:59.603: E/AndroidRuntime(8841): at java.lang.reflect.Method.invokeNative(Native Method)

07-25 21:41:59.603: E/AndroidRuntime(8841): at java.lang.reflect.Method.invoke(Method.java:507)

07-25 21:41:59.603: E/AndroidRuntime(8841): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)

07-25 21:41:59.603: E/AndroidRuntime(8841): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)

07-25 21:41:59.603: E/AndroidRuntime(8841): at dalvik.system.NativeStart.main(Native Method)

[Marty Schoch]

After looking at the stack trace I see that it did not find the map
reduce block you registered, instead it found a design document
(actual JSON document) and was trying to compile the javascript. This
ultimately led to an NPE (though my line numbers don't match up, so I
can't tell exactly why).

Obviously you intended to use the Java map/reduce block anyway. So
the real question is why didn't it find it.

You install the view in the database Global.YOURTEAM_DB_NAME, then you
try to access it in "yourteam". Can you double check that the
constant matches this literal value?

marty

[matt martin]

Marty,

Thanks for the response. I verified that the strings match and changed "yourteam" to match that constant.  It sounds like it was finding an old design document I had created (the one you mentioned) so I went ahead and deleted the database so as to start out fresh. My stack trace has now changed to:

07-26 18:16:00.002: E/AndroidRuntime(14151): org.ektorp.DocumentNotFoundException: nothing found on db path: touchdb:/yourteam/_design/ddoc/_view/allgames, Response body: null

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.http.StdResponseHandler.createDbAccessException(StdResponseHandler.java:34)

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.http.StdResponseHandler.error(StdResponseHandler.java:62)

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.http.RestTemplate.handleResponse(RestTemplate.java:107)

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.http.RestTemplate.getUncached(RestTemplate.java:27)

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.impl.StdCouchDbConnector.executeQuery(StdCouchDbConnector.java:410)

07-26 18:16:00.002: E/AndroidRuntime(14151): at org.ektorp.impl.StdCouchDbConnector.queryView(StdCouchDbConnector.java:449)

07-26 18:16:00.002: E/AndroidRuntime(14151): at com.yourteam.MainMenu$1.handleMessage(MainMenu.java:139)

07-26 18:16:00.002: E/AndroidRuntime(14151): at android.os.Handler.dispatchMessage(Handler.java:99)

07-26 18:16:00.002: E/AndroidRuntime(14151): at android.os.Looper.loop(Looper.java:150)

07-26 18:16:00.002: E/AndroidRuntime(14151): at android.app.ActivityThread.main(ActivityThread.java:4333)

07-26 18:16:00.002: E/AndroidRuntime(14151): at java.lang.reflect.Method.invokeNative(Native Method)

07-26 18:16:00.002: E/AndroidRuntime(14151): at java.lang.reflect.Method.invoke(Method.java:507)

07-26 18:16:00.002: E/AndroidRuntime(14151): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)

07-26 18:16:00.002: E/AndroidRuntime(14151): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)

07-26 18:16:00.002: E/AndroidRuntime(14151): at dalvik.system.NativeStart.main(Native Method)

It looks like the design document just isn't saving to the database.

[Marty Schoch]

We don't ever actually create a design document with that view,
instead we match requests for views against the list of registered
views. But, the effect is the same here, its just not finding the
view you registered. It still seems like its looking in the wrong
database.

I took your code, and just replaced references to the Global object
with these hard-coded values and it works fine. Can you try this and
see if it works for you:

TDDatabase db = server.getDatabaseNamed("yourteam");

boolean status = db.open();

String dDocName = "ddoc";
String viewName = "allgames";

TDView allView = db.getViewNamed(String.format("%s/%s",
dDocName, viewName));

allView.setMapReduceBlocks(new TDViewMapBlock()
{
public void map(Map<String, Object> document,
TDViewMapEmitBlock emitter)
{
String type = (String) document.get("type");

if (type.equals("game") == true)
{
emitter.emit(null, document.get("_id"));
}
}
}, new TDViewReduceBlock()
{
public Object reduce(List<Object> keys, List<Object>
values, boolean rereduce)
{
return null;
}
}, "1.0");

ViewQuery viewQuery = new ViewQuery().designDocId("_design/" +
dDocName).viewName(viewName);

HttpClient httpClient = new TouchDBHttpClient(server);
CouchDbInstance dbInstance = new StdCouchDbInstance(httpClient);

CouchDbConnector connect =
dbInstance.createConnector("yourteam", false);
List<String> ids = connect.getAllDocIds();
ViewResult viewResult = connect.queryView(viewQuery);

server.close();

[matt martin]

The error was in my getLocalCouchInstance function. It fails when I get the db instance from that function but works when I hard code like in your above code.

Here is the function. Maybe you can shed some light on what I'm doing wrong here. I appreciate the help. You are great.

CouchDbInstance dbInstance = null;

{

TDURLStreamHandlerFactory.registerSelfIgnoreError();

}

TDServer server = null;

String filesDir = mContext.getFilesDir().getAbsolutePath();

try

{

server = new TDServer(filesDir);

org.ektorp.http.HttpClient httpClient = new TouchDBHttpClient(server);

dbInstance = new StdCouchDbInstance(httpClient);

// Try and create a database. Throws exception if db exists.

try

{

dbInstance.createDatabase(Global.YOURTEAM_DB_NAME);

}

catch (DbAccessException e)

{

e.printStackTrace();

}

}

catch (IOException e)

{

e.printStackTrace();

Log.e("touchdb", "Error starting TDServer", e);

}

return dbInstance;

[Marty Schoch]

On Sat, Jul 28, 2012 at 3:09 AM, matt martin <matt....@gmail.com> wrote:
> The error was in my getLocalCouchInstance function. It fails when I get the
> db instance from that function but works when I hard code like in your above
> code.
>
> Here is the function. Maybe you can shed some light on what I'm doing wrong
> here. I appreciate the help. You are great.

I think the problem is that you are creating new TDServer instances
each time. Behind the scenes they are creating different TDDatabase
objects in memory, that work with the same database on disk. There
are some use cases where that is desirable, but this isn't one of
them. Can you change the code to keep a reference to the TDServer and
reuse it on subsequent calls?

marty

[matt martin]

I did just that and it cleaned up my code nicely, thanks. 

As an aside, does TouchDB support back references when fetching documents yet? My game object has a backreference like so:

@DocumentReferences(backReference = "game", fetch = FetchType.LAZY, descendingSortOrder = true, orderBy = "dateCreated")

private Set<MyObject> objects;

Getting the following exception:

07-29 13:17:11.015: E/AndroidRuntime(1517): Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "team" (Class com.yourteam.Game), not marked as ignorable

07-29 13:17:11.015: E/AndroidRuntime(1517):  at [Source: java.io.ByteArrayInputStream@40763688; line: 1, column: 10] (through reference chain: com.yourteam.Game["team"])

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:267)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.std.StdDeserializer.reportUnknownProperty(StdDeserializer.java:649)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:635)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:1355)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:717)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.ektorp.impl.docref.BackReferencedBeanDeserializer.deserialize(BackReferencedBeanDeserializer.java:43)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2723)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1900)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.ektorp.impl.StdCouchDbConnector$3.success(StdCouchDbConnector.java:241)

07-29 13:17:11.015: E/AndroidRuntime(1517): at org.ektorp.http.RestTemplate.handleResponse(RestTemplate.java:107)

07-29 13:17:11.015: E/AndroidRuntime(1517): ... 15 more

[matt martin]

Looks like this is related to it still not finding / using the correct view. I am doing a replication pull before the query. Could this be causing issues? 

[Marty Schoch]

On Sun, Jul 29, 2012 at 6:18 PM, matt martin <matt....@gmail.com> wrote:
> I did just that and it cleaned up my code nicely, thanks.
>
> As an aside, does TouchDB support back references when fetching documents
> yet?

They are not supported because Ektorp transparently generates views in
the background to support them. We have to teach Ektorp how to build
Java views instead of Javascript views.

> My game object has a backreference like so:
>
> @DocumentReferences(backReference = "game", fetch = FetchType.LAZY,
> descendingSortOrder = true, orderBy = "dateCreated")
> private Set<MyObject> objects;
>
> Getting the following exception:
>
> 07-29 13:17:11.015: E/AndroidRuntime(1517): Caused by:
> org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized
> field "team" (Class com.yourteam.Game), not marked as ignorable

This particular exception has to do with a field in the JSON "team"
not being mapped in the Java class.

See http://stackoverflow.com/questions/4486787/jackson-json-unrecognized-field-not-marked-as-ignorable

marty

[Marty Schoch]

On Sun, Jul 29, 2012 at 8:12 PM, matt martin <matt....@gmail.com> wrote:
> Looks like this is related to it still not finding / using the correct view.
> I am doing a replication pull before the query. Could this be causing
> issues?

I'm not sure how you came to this conclusion. Can you provide more
information about what the view returned and what you expected it to
return?

marty

[matt martin]

'team' is an attribute in another document type that is not present in Game. My map function should be filtering out all documents that aren't Game type.

  String type = (String) document.get("type");

Log.i("mapreduce", document.get("_id").toString());

if (type.equals("game"))

{

emitter.emit(null, document.get("_id"));

}

Additionally, I have a breakpoint set in my map function and it's never hit. This leads me to believe that a different map function is being used.

[matt martin]

In the wiki example, the ObjectRepository looks like so:

public class EmailAccountRepository extends CouchDbRepositorySupport<EmailAccount> {
        public EmailAccountRepository(CouchDbConnector couchDBConnector) {
                super(EmailAccount.class, couchDBConnector);
        }
}

When I use this model the UnrecognizedPropertyException is thrown and my map breakpoint is never hit (even in an unindexed view).

If I add an overridden getAll() method to my RepositorySupport class, my map block executes as expected and no exceptions are thrown.

public class GameRepository extends CouchDbRepositorySupport<Game>
{
	public GameRepository(CouchDbConnector db)
	{
		super(Game.class, db);
	}
	
	@Override @GenerateView
	public List<Game> getAll()
	{
		ViewQuery q = createQuery("all").descending(true);
        return db.queryView(q, Game.class);
	}
}

Is the code on the wiki outdated?

On Sunday, July 29, 2012 6:57:17 PM UTC-7, Marty Schoch wrote:

[Marty Schoch]

On Tue, Jul 31, 2012 at 2:27 AM, matt martin <matt....@gmail.com> wrote:
> In the wiki example, the ObjectRepository looks like so:
>
> public class EmailAccountRepository extends
> CouchDbRepositorySupport<EmailAccount> {
> public EmailAccountRepository(CouchDbConnector couchDBConnector) {
> super(EmailAccount.class, couchDBConnector);
> }
> }
>
> When I use this model the UnrecognizedPropertyException is thrown and my map
> breakpoint is never hit (even in an unindexed view).
>
> If I add an overridden getAll() method to my RepositorySupport class, my map
> block executes as expected and no exceptions are thrown.
>
> public class GameRepository extends CouchDbRepositorySupport<Game>
> {
> public GameRepository(CouchDbConnector db)
> {
> super(Game.class, db);
> }
>
> @Override @GenerateView
> public List<Game> getAll()
> {
> ViewQuery q = createQuery("all").descending(true);
> return db.queryView(q, Game.class);
> }
> }
>
> Is the code on the wiki outdated?

I think it just has to do with the behavior of
CouchDbRepositorySupport.getAll(). It first checks to see if a design
document with a particular name exists (the name uses Ektorp's
conventions which I think would be something like _design/Game in this
case). In your case I'm presuming that will not find a document with
this name. It then proceeds to use _all_docs.

This would seem to explain the behavior you're seeing, your breakpoint
in the view isn't hit, and since _all_docs will return documents not
of type "game" it could then cause the mapping exception as well.

When you override the getAll() method you're skipping the check for
the design doc, and directly pointing it at your view.

I can see how the wiki is confusing here, it may work for the wrong
reasons depending on what your data looks like. It would be great if
you could suggest a better way once you have it working for yourself.
We definitely need more people contributing to the wiki.

There is a larger issue that is starting to surface here. Ideally
CouchDbRepositorySupport would behave a little differently for
TouchDB. For example, instead of looking up a design doc in the
database, it could check the list of views registered in memory. In
my mind the right way for us to tackle this is to have a separate
implementation of the CouchDbInstance and CouchDbConnector classes
that are specific to TouchDB. If we do this right, we could start to
remove some of the TDRouter layer as well (I see this as analogous to
what Jens wants to do with TouchDB-iOS and CouchCocoa in the future).
I'd love to dive into this now because it offers potential performance
improvements too (there are places where TouchDB and Ektorp serialize
and deserialize the same data which could have been returned straight
through with TouchDB), but I really see this as an improvement for
TouchDB-Ektorp 1.1.

marty

[Matt Martin]

>I can see how the wiki is confusing here, it may work for the wrong 
>reasons depending on what your data looks like.  It would be great if 
>you could suggest a better way once you have it working for yourself. 
>We definitely need more people contributing to the wiki.  

I'm not sure I will find a way to get it working. I believe it will require, as you say, some modification to the RepositorySupportClass for TouchDB. I'm currently stepping through some of the MapReduce related methods in TDView but the code is all very new to me. I'll see what I can find out in the coming days and let you know if I find a solution.

[Matt Martin]

Can you verify that mapreduce is functioning in TouchDB at all? I have defaulted back to a standard view query.

ViewQuery viewQuery = new ViewQuery().designDocId("_design/" + "ddoc").viewName("games");
  //viewQuery.descending(true); //use this to reverse the sorting order of the view
ViewResult viewResult = connect.queryView(viewQuery);

for (ViewResult.Row row : viewResult)
{
	String id = row.getValue();
	id = row.getValue();
}

I am not able to do much of anything with row. getDoc(), getKey(), and getValue() all return null. This was the same behavior (empty set returned) I saw when performing a MapReduce through the RepositorySupportClass.

[Marty Schoch]

I think its working. I've tried your code, with the view definitions
I think you're using.

NOTE: I'm assuming you changed the view name to "games" from
"allgames". The code did not work until I made that change.

The code I'm running is here:

http://pastebin.com/cq8egGWr

The output it generates is here:

http://pastebin.com/q62dJuGQ

The reason the value is null is because you have a reduce function
defined. And if you have one defined, querying the view will reduce
by default. You can disable reduce if you want by adding
.reduce(false) to the ViewQuery.

If I add the to the ViewQuery, the results change to:

08-02 16:37:56.132: D/TestAppActivity(668): row value is
825d331c-eea7-4aec-91dc-df8ce9583b9e

marty

[Matt Martin]

Setting reduce to false did it, thanks. I would be happy to contribute to the wiki once I get some more experience with the API. The wiki definitely needs more attention and I can appreciate that you're busy as it is.