Documents not being found in migrated installation

[d...@providence.edu]

Hello,

As part of migrating our content from one server to another, I'm moving our two xtf installations. When I try to view documents, I'm getting a FileNotFound/permission denied for the "lazy" version of each file, but I don't know why; the lazy files are there and have read permissions. (Or rather, they were; I deleted them in case re-indexing the documents after fiddling with some permissions would re-create versions that wouldn't have this problem, but I was wrong!)

Can anyone set me straight? The other installation I migrated a few months ago is doing fine, but I do not remember what I did right! The index and search work just fine on this installation; it's just that I cannot view the documents.

Thanks.

[dan haig]

At a guess, just delete the *whole index and re-index everything fresh. Often seems to do the trick.

.d

--
You received this message because you are subscribed to the Google Groups "XTF Users List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xtf-user+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/1245d48a-9a05-43d4-b479-1d840df1531bn%40googlegroups.com.

[Steven D. Majewski]

I recently ran into a similar problem, and it turned out to be due to SElinux running on the new server. That seems to introduce another whole set of permissions. For now, I’ve just turned it off until I RTFM and learn how to configure and administer It properly. 

  I’ve also, in the past, has issues with symlinks in or to a data directory, when I forgot to copy a tomcat config xml file that enabled symlinks. 

Sent from my iPhone

On Sep 11, 2020, at 2:50 PM, d...@providence.edu <d...@providence.edu> wrote:



--

[d...@providence.edu]

Dan, I deleted the entire index directory this time instead of just the specific batch of lazy files, and then reindexed, but no dice. I'd run the textindexer as root, so they are all owned by root after reindexing, while the files in data/tei are owned by a tomcat user; could this be an issue? In the functioning installation, the lazy files are owned by the tomcat user, but I'm pretty sure I just chowned them over after indexing rather than running the index as tomcat.

sdm7g - can you elaborate on that? I have two XTF installations, one of which is working fine and the other of which is giving me this error, so I'm not sure how a server-wide problem would be affecting them differently.

[RC Martin Haye]

The files need to be writable by Tomcat, because the DynaXML servlet builds additional xsl:key cross-references on the fly.

--Martin

---

From: xtf-...@googlegroups.com <xtf-...@googlegroups.com> on behalf of d...@providence.edu <d...@providence.edu>
Sent: Friday, September 11, 2020 1:23 PM
To: XTF Users List <xtf-...@googlegroups.com>
Subject: Re: [xtf-user] Documents not being found in migrated installation

 

To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/2f58a93b-bcff-400d-ac27-ed7d150e0311n%40googlegroups.com.

[Steven D. Majewski]

I don’t understand it yet myself: need to read the docs to figure out what I need to fix to be able to run it under SELinux. But check and see if it’s running, and if it is, try turning it off and see if it then works (after also restarting tomcat).

In my case, I had a working XTF webapp. I moved and reconfigured some files, and it stopped working.
I put things back where they were initially and it still refused to work, despite same owner, group and permissions ( owner: tomcat, group: tomcat ). I believe SELinux has it’s own permissions.
’Sudo service selinux stop’ made it work again. Apparently, there is also “permissive” and “enforcing” modes. I haven’t had a chance to read the docs yet. Just discovered that was the source of the problem. The clue was periods after the permission codes in ‘ls -l’ listing.

[dan haig]

Sorry for the lag - Martins knows the reasons best, but it is often the case that throwing away your index and running a fresh one will cure a few kinds of perplexing problems. GIve it a whirl.

.d

To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/2f58a93b-bcff-400d-ac27-ed7d150e0311n%40googlegroups.com.

[d...@providence.edu]

Hm, okay, yes, chowning the index folder to the tomcat user (rather than root) and making it writable solved this issue without having to re-index. But I'm guessing that this problem will recur every time I need to reindex the files. What settings/permissions/usergroups/etc. should I tweak in order to avoid this issue? (Or should I not be running textIndexer as root? In the installation on the old server, we run it as root without running into this issue, but I don't know what other settings might be in place.

Thanks!

[Bridger Dyson-Smith]

I just always run `chown -R tomcat:tomcat index/` after running the textIndexer, but there are probably at least 11 other ways to do this.

To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/275840b2-cf8c-496c-951a-a0a9ceaa9ed8n%40googlegroups.com.

[Steven D. Majewski]

I run textIndexer from a cronjob as user tomcat, along with an update from git|svn. — Steve. 

To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/CAD-FYmRpKbvJaZvf1xi2O1vAEmN2kydmytupcbpEQLhmyrZZ5Q%40mail.gmail.com.

[d...@providence.edu]

makes sense - although I'm not actually sure what the credentials for the (in this case) tomcat8 user are or how to find them. (IIRC, in a sandbox installation I had up for testing a few years ago, which has since been destroyed, I remember I had a dedicated "tomcat-runner" user that ran tomcat and that I used to run textIndexer; possibly specifically for this reason?)

[Steven D. Majewski]

Tomcat is usually a non-login account. If you sudo to root, you can then do:

‘Crontab -u tomcat -e’ to edit crontab, or from root, you can then sudo to user tomcat: 

’sudo -u tomcat -s’  ( or, alternate shortcut:  ’sudo sudo -u tomcat ./bin/textIndexer… ‘ )

To view this discussion on the web visit https://groups.google.com/d/msgid/xtf-user/42787a7a-b1bc-4bdd-89c5-f9e4d3eaebacn%40googlegroups.com.

[d...@providence.edu]

Getting back to this, it seems like as long as the index directory has proper permissions, new lazy files added to it will have the proper permissions. (I deleted the entire index directory and reindexed, and found that when it was owned by root, the pages wouldn't load, but after chowning it to the tomcat user, deleting individual lazy files and trying to access those pages just resulted in the files being re-created, owned by tomcat.)

So, no need for me to add "make sure to change the permissions after indexing" to my "if I get hit by a bus" guide to running the site.

Thanks all!