[Dspace-tech] More CC license oddities

[Dorothea Salo]

I'm working on getting 1.4.1 in shape on my test server, and I ran into an odd
problem with the Creative Commons licensing.

Symptom: On the first attempt to attach a CC license, DSpace blows up (ISE
message below). The *second* attempt, however, works flawlessly. Same deal if
you try to attach a CC license to an existing item via the Edit menu: first
attempt blows up, second one works.

The root of the blowup appears to be in CreativeCommons.java:

// set the license text bitstream
setBitstreamFromBytes(item, bundle, BSN_LICENSE_TEXT, bs_format,
license_text.getBytes());

But what's odd is that it seems to be a file I/O error, even though it's already
*done* some file I/O in line 160. I'm stumped. Anybody have any bright ideas?

-- URL Was:
http://phobos.gmu.edu:8080/dspace/tools/edit-item?item_id=1107&cc_license_url=http%3A%2F%2Fcreativecommons.org%2Flicenses%2Fby%2F2.5%2F
-- Method: GET
-- Parameters were:
-- cc_license_url: "http://creativecommons.org/licenses/by/2.5/"
-- item_id: "1107"


Exception:
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:828)
at edu.sdsc.grid.io.local.LocalFile.createNewFile(LocalFile.java:486)
at
org.dspace.storage.bitstore.BitstreamStorageManager.store(BitstreamStorageManager.java:332)
at org.dspace.content.Bitstream.create(Bitstream.java:184)
at org.dspace.content.Bundle.createBitstream(Bundle.java:362)
at
org.dspace.license.CreativeCommons.setBitstreamFromBytes(CreativeCommons.java:303)
at org.dspace.license.CreativeCommons.setLicense(CreativeCommons.java:164)
at
org.dspace.app.webui.servlet.admin.EditItemServlet.showEditForm(EditItemServlet.java:325)
at
org.dspace.app.webui.servlet.admin.EditItemServlet.doDSGet(EditItemServlet.java:158)
at
org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)
at org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99)

(etc. etc.)


The only other thing I can find in the logs that might be related is this:

2007-02-08 15:15:50,628 ERROR org.jboss.web.localhost.Engine @
StandardWrapperValve[edit-item]: Servlet.service() for servlet edit-item threw
exception
java.lang.IllegalStateException: Cannot forward after response has been committed
at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:368)
at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:356)
at org.dspace.app.webui.util.JSPManager.showJSP(JSPManager.java:91)
at
org.dspace.app.webui.servlet.admin.EditItemServlet.showEditForm(EditItemServlet.java:364)
at
org.dspace.app.webui.servlet.admin.EditItemServlet.processUpdateItem(EditItemServlet.java:598)
at
org.dspace.app.webui.servlet.admin.EditItemServlet.doDSPost(EditItemServlet.java:244)
at
org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:147)
at
org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105)


Dorothea

--
Dorothea Salo, Digital Repository Services Librarian
(703)993-3742 ds...@gmu.edu AIM: gmumars
MSN 2FL, Fenwick Library
George Mason University
4400 University Drive, Fairfax VA 22031

[John Preston]

I've got this same error from time to time. I thought that it was my hardware RAID5-1 giving some problems when occasionally the asset store would disappear or something. It appears not only at the CC submission stage for me. Sometimes after I skip CC stage and accept the DSpace distribution license. I get the error. A refresh of the page almost always goes through ok.

Mybe its not my hardware then.

John

org.dspace.license.CreativeCommons.setBitstreamFromBytes (CreativeCommons.java:303)

        at org.dspace.license.CreativeCommons.setLicense(CreativeCommons.java:164)
        at
org.dspace.app.webui.servlet.admin.EditItemServlet.showEditForm(EditItemServlet.java:325)
        at
org.dspace.app.webui.servlet.admin.EditItemServlet.doDSGet(EditItemServlet.java:158)
        at
org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151)

        at org.dspace.app.webui.servlet.DSpaceServlet.doGet (DSpaceServlet.java:99)

        (etc. etc.)


        The only other thing I can find in the logs that might be related is this:

2007-02-08 15:15:50,628 ERROR org.jboss.web.localhost.Engine @
StandardWrapperValve[edit-item]: Servlet.service() for servlet edit-item threw
exception
java.lang.IllegalStateException: Cannot forward after response has been committed
         at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java :368)
         at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:356)
         at org.dspace.app.webui.util.JSPManager.showJSP(JSPManager.java:91)
         at

org.dspace.app.webui.servlet.admin.EditItemServlet.showEditForm (EditItemServlet.java:364)

         at
org.dspace.app.webui.servlet.admin.EditItemServlet.processUpdateItem(EditItemServlet.java:598)
         at
org.dspace.app.webui.servlet.admin.EditItemServlet.doDSPost(EditItemServlet.java :244)
         at
org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:147)
         at
org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:105)


Dorothea

--
Dorothea Salo, Digital Repository Services Librarian
(703)993-3742     ds...@gmu.edu     AIM: gmumars
MSN 2FL, Fenwick Library
George Mason University
4400 University Drive, Fairfax VA 22031

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
DSpace-tech mailing list
DSpac...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech

[Jim Downing]

Hi Dorothea,

Dorothea Salo wrote:
> But what's odd is that it seems to be a file I/O error, even though it's already
> *done* some file I/O in line 160. I'm stumped. Anybody have any bright ideas?
>

> Exception:
> java.io.IOException: No such file or directory
> at java.io.UnixFileSystem.createFileExclusively(Native Method)

> (etc. etc.)

My Pavlovian response is to check the permissions on whatever directory
DSpace was trying to create a file in.

> The only other thing I can find in the logs that might be related is this:
>
> 2007-02-08 15:15:50,628 ERROR org.jboss.web.localhost.Engine @
> StandardWrapperValve[edit-item]: Servlet.service() for servlet edit-item threw
> exception
> java.lang.IllegalStateException: Cannot forward after response has been committed
> at
> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:368)

This looks important but unrelated, I'll take a look.

Best regards,

jim

[Dorothea Salo]

Jim Downing wrote:

> My Pavlovian response is to check the permissions on whatever directory
> DSpace was trying to create a file in.

Seems okay. The error is at best sporadic; I'm having no trouble this morning.
(Load-related, maybe?) I've got the log level stepped up to DEBUG, so if any of
that information would be useful, let me know what.

>> The only other thing I can find in the logs that might be related is this:
>>
>> 2007-02-08 15:15:50,628 ERROR org.jboss.web.localhost.Engine @
>> StandardWrapperValve[edit-item]: Servlet.service() for servlet edit-item threw
>> exception
>> java.lang.IllegalStateException: Cannot forward after response has been committed
>> at
>> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:368)
>
> This looks important but unrelated, I'll take a look.

Digging a little further, I stepped through the process bit by bit, examining
the logs after each step. I learned that the above error happens after clicking
the "Add Creative Commons License" on the Edit Item screen. Doing so also
creates the error following:

2007-02-09 09:51:27,226 ERROR org.jboss.web.localhost.Engine @
ErrorDispatcherValve[localhost]: Exception Processing
ErrorPage[exceptionType=java.lang.Exception, location=/internal-error]
java.lang.IllegalStateException
at org.apache.coyote.Response.reset(Response.java:340)
at org.apache.coyote.tomcat4.CoyoteResponse.reset(CoyoteResponse.java:628)
at org.apache.coyote.tomcat4.CoyoteResponse.reset(CoyoteResponse.java:824)
at
org.apache.catalina.valves.ErrorDispatcherValve.custom(ErrorDispatcherValve.java:413)
at
org.apache.catalina.valves.ErrorDispatcherValve.throwable(ErrorDispatcherValve.java:258)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:179)

(etc.; nothing directly DSpace-related in the output)

Hope this helps; let me know if I've left out something important.

[Jim Downing]

Dorothea Salo wrote:
> Jim Downing wrote:
>

Can't spot anything obvious.

> I've got the log level stepped up to DEBUG, so if any of
> that information would be useful, let me know what.
>

There should be a logging output coming out of BitstreamStorageManager
at line 821:

if (log.isDebugEnabled()) {
log.debug("Local filename for " + sInternalId + " is "
+ bufFilename.toString());
}

Examples of this line before successful and unsuccessful file creation
might get us a bit further, I suppose.

>>> The only other thing I can find in the logs that might be related is this:
>>>
>>> 2007-02-08 15:15:50,628 ERROR org.jboss.web.localhost.Engine @
>>> StandardWrapperValve[edit-item]: Servlet.service() for servlet edit-item threw
>>> exception
>>> java.lang.IllegalStateException: Cannot forward after response has been committed
>>> at

> Digging a little further, I stepped through the process bit by bit, examining
> the logs after each step. I learned that the above error happens after clicking
> the "Add Creative Commons License" on the Edit Item screen. Doing so also
> creates the error following:
>
> 2007-02-09 09:51:27,226 ERROR org.jboss.web.localhost.Engine @
> ErrorDispatcherValve[localhost]: Exception Processing
> ErrorPage[exceptionType=java.lang.Exception, location=/internal-error]
> java.lang.IllegalStateException
> at org.apache.coyote.Response.reset(Response.java:340)
> at org.apache.coyote.tomcat4.CoyoteResponse.reset(CoyoteResponse.java:628)
>

This is a strange one. This exception usually happens if some code
writes to the response and then later forwards to a JSP. I can't see
anywhere this happens in the CC code (and as you point out, the trace
above didn't have any DSpace code in the stack). The only place I can
see that it might occur in DSpace is in the controlled vocabulary tag,
which uses the /internal-error servlet in the errorPage directive, which
I think could cause this problem since InternalErrorServlet also
forwards to a JSP.

So, it's Friday afternoon and my brain is fried - anyone else have any
ideas?

cheers,

jim

[John Preston]

When it happens to me (it hasn't happened recently) I tracked it down to the BitStoreManager.store method at line 332

320   // Where on the file system will this new bitstream go?
321  GeneralFile file = getFile(bitstream);
322
323   // Make the parent dirs if necessary
324  GeneralFile parent = file.getParentFile();
325
326  if (!parent.exists())
327  {
328     parent.mkdirs();
329  }
330
331  //Create the corresponding file and open it
332  file.createNewFile();

I don't have that many files, so I don't think the file system has run out of inodes, but I guess I should really see what the current state is.

John

>          at org.apache.coyote.tomcat4.CoyoteResponse.reset (CoyoteResponse.java:628)

>

This is a strange one. This exception usually happens if some code
writes to the response and then later forwards to a JSP. I can't see
anywhere this happens in the CC code (and as you point out, the trace
above didn't have any DSpace code in the stack). The only place I can
see that it might occur in DSpace is in the controlled vocabulary tag,
which uses the /internal-error servlet in the errorPage directive, which
I think could cause this problem since InternalErrorServlet also
forwards to a JSP.

So, it's Friday afternoon and my brain is fried - anyone else have any
ideas?

cheers,

jim

[Jim Downing]

John Preston wrote:
> When it happens to me (it hasn't happened recently) I tracked it down
> to the
> BitStoreManager.store method at line 332
>
> 320 // Where on the file system will this new bitstream go?
> 321 GeneralFile file = getFile(bitstream);
> 322
> 323 // Make the parent dirs if necessary
> 324 GeneralFile parent = file.getParentFile();
> 325
> 326 if (!parent.exists())
> 327 {
> 328 parent.mkdirs();
> 329 }
> 330
> 331 //Create the corresponding file and open it
> 332 file.createNewFile();
>
> I don't have that many files, so I don't think the file system has run
> out
> of inodes, but I guess I should really see what the current state is.

Could you wind up to debug logging and try to find out which files are
failing and which succeeding?

Best regards,
jim

[John Preston]

I have a slight problem. I can't seem to get the error to occur again. It originally was random but fairly frequently random (if you know what I mean).

However, all is not lost as I think I might have figured out the problem in my case (famous last words). In looking at the assetstore directory to give you the ls -l output, I noticed that some of the directories belonged to the root user and group. I then remembered that I run a cron job every night that runs the filter-media and index-all commands, and it runs as user root. Thus any files that were created in the assetstore belonged to the root user and group.

Now it seems that whenever a new file was created in the assetstore at the root level by this cron job, if the next assetstore file to be created in a normal submission by dspace, which runs as user dpsace, group dspace, then it cannot create the other assetstore subdirectories for this new bitstream because the root belongs to user root. When I was refreshing the page, what it was doing was creating a new bitstream id with a new assetstore path, that would at some point after a number of retries create a new root assetstore path which it would have permission to create, and everything would proceed as normal.

When I reset all the files below the assetstore root path (/dspace/assetstore) to user dspace, group dspace, then I can't get the error to happen. Can you check with the other person who was having the same problem and see if maybe they have a cron job run by user root, which is setting some of the assetstore files to be owned by user root.


John

On 2/13/07, John Preston <byhis...@gmail.com> wrote:

On 2/13/07, Jim Downing <oj...@cam.ac.uk> wrote:

PS Just to rule out one potential source of confusion; are you running
the Sun JDK?

Yes, I using jdk 1.6.

I'm not at that machine right now so I'll send you the other data later.

John

John Preston wrote:
> <snip/>
> 2007-02-12 17:33:19,427 DEBUG
> org.dspace.storage.bitstore.BitstreamStorageManager @ Local filename for
> 46571832589170837501358246915571874212 is
> /data1/dspace/assetstore/46/57/18/46571832589170837501358246915571874212
> 2007-02-12 17:33:19,430 WARN
> org.ejamaica.app.webui.servlet.SubmitServlet @
> jgr...@ejamaica.org:session_id=2FED32ED8F48399F4CE0D094F9FE1FB8:ip_addr=
> 192.168.0.171:upload_error:

> java.io.IOException: No such file or directory
>    at java.io.UnixFileSystem.createFileExclusively(Native Method)

>    at java.io.File.createNewFile(File.java:883)

[Blanco, Jose]

Dorothea:

I'm getting ready to run ItemImport for some items that have creative commons licenses, and so I'm creating the license files, but I'm not sure what to put in the license_rdf file.

I have:

License_url - this gets the url, like: http://creativecommons.org/licenses/by-nc-sa/3.0/

Licnese_text - this gets the pagesource of what's at license_url

But what do I put in

License_rdf ?

Many thanks, Jose

[Blanco, Jose]

Nevermind, I see it in the code, it's what's returned from
License_url + rdf

-Jose

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com