Files not being listed

[Mo]

I have successfully hosted the app on my workstation.
Everything works fine, except for the files listing.
I tried listing files using basic code in a java class itself, but
that too gave some wierd DataNucleus errors...
Any pointers here will be appreciated ..
Thanks

[tytung]

Hi, Mo

GoogleFileService is writen for Google App Engine only.
What do you mean about hosting the app on your workstation?

Can you descript how to raise errors and what is the content of
errors?
More detail you provide, more possible others will solve your problem.
Thank you!

[Mo]

Hi,

By workstation, I meant .. my desktop/localhost.
As for errors, here's the err message:
------------------------
Sep 1, 2009 12:47:27 PM org.datanucleus.store.StoreDataManager
registerStoreData
INFO: Managing Persistence of Class :
sinica.googlefileservice.server.datastore.GoogleFile [Table : <class
name="GoogleFile"
identity-type="application"
objectid-class="javax.jdo.identity.StringIdentity"
persistence-modifier="persistence-capable"
>
<inheritance strategy="new-table">
</inheritance>
<field name="contentType"
persistence-modifier="persistent"
null-value="none"
default-fetch-group="true"
embedded="true"
unique="false">
</field>
<field name="date"
persistence-modifier="persistent"
null-value="none"
default-fetch-group="true"
embedded="true"
unique="false">
</field>
<field name="fileId"
persistence-modifier="persistent"
primary-key="true"
null-value="none"
default-fetch-group="true"
embedded="false"
unique="false">
</field>
<field name="fileName"
persistence-modifier="persistent"
null-value="none"
default-fetch-group="true"
embedded="true"
unique="false">
</field>
<field name="fileOwner"
persistence-modifier="persistent"
null-value="none"
default-fetch-group="true"
embedded="true"
unique="false">
</field>
<field name="fileSize"
persistence-modifier="persistent"
null-value="none"
default-fetch-group="true"
embedded="true"
unique="false">
</field>
</class>
, InheritanceStrategy : new-table]
Exception in thread "main" java.lang.NoSuchMethodError:
com.google.appengine.api.datastore.DatastoreService.prepare(Lcom/
google/appengine/api/datastore/Transaction;Lcom/google/appengine/api/
datastore/Query;)Lcom/google/appengine/api/datastore/PreparedQuery;
at org.datanucleus.store.appengine.query.DatastoreQuery.performExecute
(DatastoreQuery.java:224)
at org.datanucleus.store.appengine.query.JDOQLQuery.performExecute
(JDOQLQuery.java:85)
at org.datanucleus.store.appengine.query.JDOQLQuery.performExecute
(JDOQLQuery.java:31)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1489)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:
1371)
at org.datanucleus.store.query.Query.execute(Query.java:1344)
at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:221)
at sinica.googlefileservice.client.examples.FileListing.main
(FileListing.java:22)
-----------------

& the code in FileListing.java is
-----------------
PersistenceManager pm = PMF.get
().getPersistenceManager();
String query = "select from " + GoogleFile.class.getName();

System.out.println(query.toString());
List<GoogleFile> entities = (List<GoogleFile>) pm.newQuery
(query).execute(); //Line 22
if (!entities.isEmpty()) {
System.out.println("found something");
}
-----------------

Thanks
Mo

> > Thanks- Hide quoted text -
>
> - Show quoted text -

[tytung]

Did you change any other files from the GoogleFileService-v0.2.zip
except of adding a new file named FileListing.java ?

I modified "file_list.jsp" to fit your code of "FileListing.java", and
It worked well.

Maybe someone else has good suggestion.

> > - Show quoted text -- Hide quoted text -

[X]

I was getting this problem on the local appengine development server too!  I code in GWT, so I needed a modified version of Mr. Tung's fine creation {attribution in code!}; particularly with the blob class in a seperate package from the file meta-data class, and all jsp files converted to jarrable servlets without being in war {drop-in gwt modules should exist wholly in jar}.  As soon as I changed the List<Blob> to List<Long>, where long = id of Blobs, to completely separate the packages, .getExtent() and SELECT * would fail.

I could get individual files if I knew their id, but I couldn't get a list of the files using queries or extents.

I would upload a couple files, play with the datastore indexes or the class files, check the /ls servlet for "all files", and see "There are no uploaded files".  Checking /dl?id=fileID {download.jsp?}, the files are there and download fine.

Then, after uploading many files, wiping the WEB-INF/appengine-generated/local_db.bin, deleting all datastore indexes and uploading a bunch more files...  It eventually just "started" working.  The very first time it started working, there were many files in the datastore viewer { localhost:xyz/_ah/admin }, all with every column of data not-null, but only the most recent upload was showing.  This was AFTER I had added defaultFetchGroup="true" to all data fields.

The relevant portions of my derived class are as follows, NOTE: This is specialized for GWT.:

/**Derived from GoogleFileService, by Ty Tung, <tyt...@gmail.com>*/
@PersistenceCapable(identityType = IdentityType.APPLICATION,detachable="false")
public class xFile implements IsSerializable,Serializable{

    private static final long serialVersionUID = -1417186764297902980L;
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private String name;
    @Persistent
    private String title;

    @Transient
    @Persistent(defaultFetchGroup="true")
    @Extension(vendorName = "datanucleus", key = "gae.unindexed", value="true")
    private ArrayList<Long> chunks;
    @Persistent(defaultFetchGroup="true")
    private Date date;
   
    @Persistent(defaultFetchGroup="true")
    private String owner;

}

Once I get the code cleaned up a little, I'm going to stick it on code.google.com, possible with a primed local_db.bin so users don't get this, "can't see files" bug.  Basically, if any fields are not properly set and accounted for, queries fail, but getObjectById works.  DELETE your manual datastore-index.xml, unless you really need some specific queries...  It takes time to get all the right queries set up every time you add a field or class, and an incorrect file seems to invalidate existing db entries.


...Even after "fixing" {overcoming} this bug once when switching to servlets and gwt, it happened again when I split the file and blob classes into different packages.  Upload lots of files, add defaultFetchGroup to everything, delete your indexes and appengine-generated, and try accessing all of the fields, whether you need them or not.
xFile x = ...;
x.setFileUnits(x.getFileUnits);//trick
is there for a reason.

I'm a little unsure, but my guess is that detached objects with fields that are never referenced anywhere but the class file describing them are treated like "unused imports"...  They are present in the raw datastore, but without ever being "needed" / accessed, they are accidentally trimmed somewhere along the way.  Ty's example accesses all fields in file_list.jsp, even the internal keys.

Remember, the real Appengine is a cloud of computers,
The dev server is an abstract mockup of the real thing.


If you can't get it, I could send some source with a primed db.  I check x _AT_ aiyx _DOT_ info more often than this email...

[Mo]

Hi,

Thanks for your detailed description.
However, I still can't get it working.
I tried by adding defaultfetchgroup, removing indexes etc.. but in
vain...
Will keep trying... & keep you posted.
If you can share the modified pojos (GoogleFile & GoogleUnit) that wud
be great.

Thanks again
Mo

[X]

I'll put together a package between now an Wednesday and let you know.

I forgot to mention... Deploying to appengine seemed to help. And
referencing the file objects as much as you can in other code seems to
help... My test app is up at http://ignorancebane.appspot.com Check
it out, but don't destroy anything! I'll upload my code there or
code.google.com. It's not ready yet or anything, but you can feel
free to hack on whatever you like... The url will be
ignorancebane.appspot.com/dl?x=xBook But that won't work 'til I put
the file there.

I'm just going to tarball my project and send it to you. Won't work
without a little modification, but I can send you detailed
instructions. I'll mod my ant build so you can just replace the
important directories and run it. You can always import the source
folders, and merge wars. I'll let you know by Wednesday.

--
"He whose desires are drawn toward knowledge in every form will be
absorbed in the pleasures of the soul, and will hardly feel bodily
pleasure --I mean, if he be a true philosopher and not a sham one." -
Plato
"Wise Words Woven With Will Wakes Worlds" - Alyxandor Artistocles