Basic collections issues

[Josh Wilson]

I'm working on a new installation of Islandora 7.x-1.1 and having a few different issues. Perhaps they are all related and will look familiar to someone or point to a more general error. Thanks for any help.

So far all I've installed are:
*core Islandora module
*Islandora Basic Collection
*Islandora Basic Image

and dependencies and tools:
*Islandora XML Form API
*Islandora XML Form Elements
*Islandora Schema API
*Objective Forms
*PHP Lib
*Islandora XML Form Builder
*Islandora XML Forms

I'm unable to ingest anything or create new collections without getting errors.

1) In the solution packs configuration screen (admin/islandora/solution_packs), I get a notice that "Some objects must be reinstalled. See objects list for details." The notice appears under both Islandora and Islandora Basic Image: "Modified Datastream: TN." (See this screenshot for clarification: https://www.dropbox.com/s/2zennk81whcup88/islandora_error.png). If I click "re-install objects" it appears to proceed normally but nothing actually gets fixed.

2) I see this warning repeatedly when browsing around the collection pages: "Warning: curl_setopt(): cannot represent a stream of type Output as a STDIO FILE* in CurlConnection->getRequest() (line 628 of [path omitted]\sites\all\modules\islandora\libraries\tuque\HttpConnection.php)."

3) When I try to create a new collection, I get a 500 Internal Server Error. This appears in the log:

Caused by: org.fcrepo.server.errors.GeneralException: Error getting http://[host URL omitted]/sites/all/modules/islandora_solution_pack_collection/images/folder.png
    at org.fcrepo.server.storage.DefaultExternalContentManager.get(DefaultExternalContentManager.java:183) [fcrepo-server-3.4.2.jar:na]
    at org.fcrepo.server.storage.DefaultExternalContentManager.getFromWeb(DefaultExternalContentManager.java:332) [fcrepo-server-3.4.2.jar:na]
    at org.fcrepo.server.storage.DefaultExternalContentManager.getExternalContent(DefaultExternalContentManager.java:149) [fcrepo-server-3.4.2.jar:na]
    ... 57 common frames omitted
Caused by: java.net.UnknownHostException:[host name omitted]
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:195) [na:1.6.0_43]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) [na:1.6.0_43]
    at java.net.Socket.connect(Socket.java:529) [na:1.6.0_43]
    at java.net.Socket.connect(Socket.java:478) [na:1.6.0_43]
    at java.net.Socket.<init>(Socket.java:375) [na:1.6.0_43]
    at java.net.Socket.<init>(Socket.java:249) [na:1.6.0_43]
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) [commons-httpclient-3.1.jar:na]
    at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$1.doit(ControllerThreadSocketFactory.java:91) [commons-httpclient-3.1.jar:na]
    at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$SocketTask.run(ControllerThreadSocketFactory.java:158) [commons-httpclient-3.1.jar:na]
    ... 1 common frames omitted

And this appears in the Drupal log:

RepositoryException: javax.ws.rs.WebApplicationException: org.fcrepo.server.errors.HttpServiceNotFoundException: [DefaultExternalContentManager] returned an error. The underlying error was a org.fcrepo.server.errors.GeneralException The message was "Error getting http://[host url omitted]/sites/all/modules/islandora_solution_pack_collection/images/folder.png" . in RepositoryConnection->parseFedoraExceptions() (line 227 of [path omitted]\islandora\sites\all\modules\islandora\libraries\tuque\RepositoryConnection.php).

4) When I try to ingest a new image, I get a similar error as in #3.

[Nigel Banks]

I haven't seen this before, but this all seems related, it would appear that Fedora can access the file at the given location:

http://[host URL omitted]/sites/all/modules/islandora_solution_pack_collection/images/folder.png

So my thoughts are:

1. The file is actually missing. To check simply navigate to that location on the file system and look. (Unlikely but worth a check).
2. The file can not be reached at that url. To check simply past the url in your browser and see if it loads. (Unlikely but worth a check).
3. Fedora specifically can't reach the file at that url. (Unlikely unless Fedora is on a separate box, if so try to ping the Drupal site).
4. There is a problem in the configuration of CURL/PHP.

[Peter Murray]

I'm going to zero in on one particular piece in your message...

On Apr 16, 2013, at 11:00 AM, Josh Wilson <joshwi...@gmail.com> wrote:
>
> 1) In the solution packs configuration screen (admin/islandora/solution_packs), I get a notice that "Some objects must be reinstalled. See objects list for details." The notice appears under both Islandora and Islandora Basic Image: "Modified Datastream: TN." (See this screenshot for clarification: https://www.dropbox.com/s/2zennk81whcup88/islandora_error.png). If I click "re-install objects" it appears to proceed normally but nothing actually gets fixed.

Does "proceed normally" mean that the warning triangle is replaced by a green checkmark?

It seems like there is something odd going on with the Fedora Commons side of the equation, so it would be useful to remove Drupal from the picture to see if Fedora is okay. Does the fedora-ingest-demos.[sh|bat] run okay? (It is found in the $FEDORA_HOME/client/bin directory.) Can you use the Fedora Flash admin client (http://[hostname]:8080/fedora/admin) to create an object?


Peter
--
Peter Murray
Assistant Director, Technology Services Development
LYRASIS
Peter....@lyrasis.org
+1 678-235-2955
800.999.8558 x2955

[Adam Vessey]

Due to 2), my initial thought is to the version of PHP installed.
"Output vs File *" seems to speak of some API change which breaks how we
use Curl, as we use a similar method for uploading files. If this is the
case, then it could well cause all the other general ingest issues.

Haven't played around too much with different PHP versions myself, but
what do you happen to be running?

3) makes me think that Fedora is perhaps running on another machine,
which has it's ability to make outgoing connections restricted... When
ingesting some datastreams, we tell Fedora the URL of where to find the
given file on the Drupal machine--if it can't resolve a domain name
(Seems to be implied by the "Caused by: java.net.UnknownHostException"
bit), then it can't contact the Drupal machine to get the asset, and the
ingest of that object (or at least that datastream) will fail...

... Care to share a bit more of the general environment you have setup?
It might help reveal some other possibilities.

- Adam

[Josh Wilson]

Thanks for all of your assistance. Leading suspect at this point is a problem with the TN datastream. Maybe mine got corrupted or somehow installed incorrectly (it might be a holdover from the Islandora 7.x-1.0 install). I'm going to investigate that and my Fedora configuration a bit more, but to answer your questions:

--I'm accessing an IIS development server remotely; Fedora 3.4.2; PHP 5.3.19 (curl enabled); JDK 1.6.0_43. Fedora and Islandora are on the same server. Fedora install properties below.

--I'm able to access and use the Fedora admin client through the browser or through remote desktop onto the server. I'm able to ingest demo objects fine using the batch file. (I'd actually already ingested them, so I deleted a few and tried again, and it seems to work fine.)

--Regarding the solution pack configuration, when I click "re-install objects" I see a progress bar, so something seems to be processing, then it returns me to the config page with a triumphant "Successfully reinstalled" message, only the warning triangle is still there, so it apparently has had no effect.

--the folder icon PNG exists and displays fine if I go directly to the URL. However, when I view the root collection page, the folder icon does not display--I only see text for the basic image collection. What could cause a problem like that? I see that the same thing is going on in the Islandora 7 Sandbox as well!

==Contents of Fedora install.properties==

ri.enabled=false
messaging.enabled=false
apia.auth.required=false
database.jdbcDriverClass=com.mysql.jdbc.Driver
ssl.available=false
database.jdbcURL=jdbc\:mysql\://localhost\:3306/isltestfed?useUnicode\=true&amp;characterEncoding\=UTF-8&amp;autoReconnect\=true
database.password=[redacted]
database.mysql.driver=included
fesl.dbxml.home=D\:\\islandora_supp\\dbxml
database.username=[redacted]
fesl.authz.enabled=true
tomcat.shutdown.port=8005
deploy.local.services=false
xacml.enabled=false
database.mysql.jdbcDriverClass=com.mysql.jdbc.Driver
tomcat.http.port=8080
fedora.serverHost=localhost
database=mysql
database.driver=included
fedora.serverContext=fedora
llstore.type=akubra-fs
tomcat.home=d\:\\fedora\\tomcat
fesl.authn.enabled=true
fedora.home=d\:\\fedora
database.mysql.jdbcURL=jdbc\:mysql\://localhost\:3306/isltestfed?useUnicode\=true&amp;characterEncoding\=UTF-8&amp;autoReconnect\=true
install.type=custom
servlet.engine=included
fedora.admin.pass=[redacted]

--
You received this message because you are subscribed to the Google Groups "islandora" group.
To unsubscribe from this group and stop receiving emails from it, send an email to islandora+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Nigel Banks]

Hmm do you have clean-urls turned on? It will be under the admin section of your Drupal site.

On Tuesday, April 16, 2013 5:00:25 PM UTC+2, Josh Wilson wrote:

[Josh Wilson]

Yep, enabled.

--
You received this message because you are subscribed to the Google Groups "islandora" group.

To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.

[Peter .]

Hi Josh,

Do have the Clientside Validation FAPI module enabled? If so there is a bug with this module that messes up the TN streams. Download the latest dev version of the module and it will fix this issue. I was bit hard by this bug.

Peter

[Josh Wilson]

Thanks for the suggestion, Peter. I don't have the Clientside Validation FAPI module in my install.

I've completely re-installed Fedora, v. 3.5 this time around, yet have gotten back to the same problem, same errors described in original message. I'm thinking this has something to do with the XACML policy enforcement as it relates to my particular development setup. Maybe some more details will help?

--I ingested all the demo objects on the backend without incident. I can access them remotely, i.e., http://[dev server ip redacted]:8080/fedora35/objects. I can browse around and view pictures of smiley buckets and the like just fine.

--However, I run into trouble if I try to administer them, i.e., http://[dev server ip redacted]:8080/fedora35/admin/. I can see the objects and search for them, but if I try to view a datastream, I get a popup in the client "Error (401)" and this spits out in the server log (first few lines):

WARN 2013-04-19 11:17:54.729 [http-8080-1] (DatastreamResource) Authorization failed; unable to fulfill REST API request
org.fcrepo.server.errors.authorization.AuthzDeniedException:
    at org.fcrepo.server.security.PolicyEnforcementPoint.enforce(PolicyEnforcementPoint.java:406) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.security.DefaultAuthorization.enforceGetDatastream(DefaultAuthorization.java:639) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.management.DefaultManagement.getDatastream(DefaultManagement.java:1133) [fcrepo-server-3.5.jar:na]

--In what can't be a coincidence, I also cannot Authenticate with Islandora at this point (the "Unable to authenticate when connecting to to Fedora Server (Version 3.5). Please configure the Drupal Filter." message at admin/islandora/configure).

--If I then remove the deny-apim-if-not-localhost.xml policy (this had been modified per the Drupal filter configuration guidelines) from the policies directory, I can use the Fedora Admin client without any trouble, and can pass the Islandora authentication test. Makes sense, right? I've removed API-M restrictions, I should be able to do whatever I want. BUT, I still get the weird Islandora errors described in the original message. Some more detail on those:

--When I go to http://[dev server]/islandora/object/islandora%3Aroot, the page doesn't completely load, like it's waiting for all the data. The folder icon for the basic collection never shows up. A warning is getting generated in this process:

"Warning: curl_setopt(): cannot represent a stream of type Output as a STDIO FILE* in CurlConnection->getRequest() (line 628 of [redacted-path-to-islandora]\islandora\sites\all\modules\islandora\libraries\tuque\HttpConnection.php).

--In the Fedora server log, this crops up:

ERROR 2013-04-19 11:28:12.864 [http-8080-1] (DefaultManagement) <?xml version="1.0" encoding="UTF-8"?><datastreamHistory  xmlns="http://www.fedora.info/definitions/1/0/management/"  xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.fedora.info/definitions/1/0/management/ http://www.fedora.info/definitions/1/0/datastreamHistory.xsd" pid="islandora:sp_basic_image_collection" dsID="TN"><datastreamProfile pid="islandora:sp_basic_image_collection" dsID="TN"><dsLabel>Thumbnail</dsLabel><dsVersionID>TN.0</dsVersionID><dsCreateDate>2013-04-19T14:29:28.261Z</dsCreateDate><dsState>A</dsState><dsMIME>image/png</dsMIME><dsFormatURI></dsFormatURI><dsControlGroup>M</dsControlGroup><dsSize>48</dsSize><dsVersionable>false</dsVersionable><dsInfoType></dsInfoType><dsLocation>islandora:sp_basic_image_collection+TN+TN.0</dsLocation><dsLocationType>INTERNAL_ID</dsLocationType><dsChecksumType>DISABLED</dsChecksumType><dsChecksum>none</dsChecksum></datastreamProfile></datastreamHistory>

Could the issue be that my dev server is having trouble reaching the schema definitions elsewhere, so it fails? (And for some reason it's especially needy about the TN stream?) Any suggestions on how to fix that, or at least allow for it while I'm working in a dev environment?

[Adam Vessey]

There are a number of policies we include with Islandora: https://github.com/Islandora/islandora/tree/7.x/policies
Have you installed them?

I've convinced myself that the "Output as a STDIO FILE*" bit is due to the related to IIS (despite having never used IIS myself, so take it with a grain of salt?)... Seems like the line numbers don't quite seem to like up, but we may have to change the "modes" with which we open files, when passing them to Curl? Might suffice to change the mode on https://github.com/Islandora/tuque/blob/1.x/HttpConnection.php#L621 and https://github.com/Islandora/tuque/blob/1.x/HttpConnection.php#L634...  Settings them to 'rw+' instead of the just 'w' or 'w+' may let it register as a "normal" file pointer, instead of the different "Output" type which IIS seems to give it.

- Adam

[Nigel Banks]

Hmm I think this may be a problem with the php_curl lib on your system, your running windows, and apparently from what I've gathered online the php_curl is build into the php executable so updating only php_curl is not possible (sorry I'm not super familiar with windows configurations).

I'd try reinstalling a later version of PHP 5.3 (I'm not sure islandora will work with PHP 5.4).

I think we need to revaluate how we are returning data streams:

https://github.com/Islandora/islandora/blob/7.x/includes/datastream.inc#L31-L48

There's gotta be a better way?

[Josh Wilson]

Thanks for the ongoing assistance. I don't have a lot of experience with IIS either, unfortunately, but it's likely what we'll have to work with here if we're going to use Islandora.

I had overlooked the additional Islandora XACML policies, actually. (My fault; it's clear enough in the module's readme. I would suggest including this requirement in the online documentation though...) But, I've added them now, stopped/restarted Tomcat, and no difference in behavior.

I'll update PHP and see if I can figure out if curl works in other applications. This is what phpinfo() tells me about my current curl installation, if it sheds any light on the problem: https://www.dropbox.com/s/h6i2q9zxb7852ck/curl.png

[Melissa Anez]

Josh,

Vincent Nguyen put together an Islandora 6.x installer for windows environments a while back: http://wiki.duraspace.org/display/~vnguyen745/Islandora+Installer

Not sure if this will hit your mark since it's for the older version, but it does definitely work as advertised.

- Melissa

[Josh Wilson]

Melissa, thanks for that link. I think it makes sense not to get committed to an Islandora 6 install, but maybe I'll give that a try just to have a different test to run.

I think I've traced my install problem back to something specific with Fedora, and I've posted the question to the Fedora Users group, so I won't duplicate it here. But if I get it solved, I'll post what I learn, since the symptoms show up in Islandora.

[slanger]

Hi Josh:

Did you ever figure out what was going on with your installation? We're experiencing similar problems:

• Windows installation
• We're seeing the error message "Warning: curl_setopt(): cannot represent a stream of type Output as a STDIO FILE* in CurlConnection->getRequest()..."
• The default folder image used for collections exists, but the collections themselves display a broken image rather than the folder PNG.
• We don't have the Clientside Validation FAPI module installed.

Thanks in advance for any insights you can provide.

[Josh Wilson]

I'm afraid I'm still stuck on that issue. I did come back around to thinking it was an Islandora issue, rather than with Fedora: https://groups.google.com/forum/#!msg/islandora/exC6VEbK6zY/S15ltbYsq3YJ

Curiously, you're the second person who has asked me about it today after months of inactivity. The other person who got in touch said he experienced the problem outside of a Windows environment. Also, other people have said they have gotten it to work in Windows without this problem. That suggests it's not necessarily Windows-specific, though perhaps whatever configuration problem is to blame is more likely to be present on a Windows server?

I haven't ruled out being able to solve it, but I haven't put much time into it the past few months either. In speaking with a few different people off the listserv and in person at Drupalcon, I get the impression that there has never been much testing in Windows environments, plus our organization now has some hope of being able to use a Linux environment instead. So for now I'm putting off worrying about installation issues in favor of familiarizing myself with the front end and hammering out workflow problems. To do that I'm just working with a Linux VM instance.

On Wed, Jul 31, 2013 at 3:51 PM, slanger <slan...@gmail.com> wrote:

Hi Josh:

 

Did you ever figure out what was going on with your installation? We're experiencing similar problems:

• Windows installation
• We're seeing the error message "Warning: curl_setopt(): cannot represent a stream of type Output as a STDIO FILE* in CurlConnection->getRequest()..."
• The default folder image used for collections exists, but the collections themselves display a broken image rather than the folder PNG.
• We don't have the Clientside Validation FAPI module installed.

Thanks in advance for any insights you can provide.

 

On Friday, April 26, 2013 5:20:38 PM UTC-4, Josh Wilson wrote:

[slanger]

Thanks for the quick response, Josh! That's interesting that someone encountered the same problem outside of Windows. It's also heartening to hear that successful Windows installations of Islandora actually exist out there. I was afraid we had signed up for an endless array of trouble, since the developers only test Islandora in Linux environments.

 

If I make any important discoveries about this problem, I'll post them here.

[slanger]

Hi Josh:

I believe I've found a solution to this problem, although it required me to patch the Tuque library.

I've also created a series of patches to make Islandora 7 more Windows-compatible.