File changes not being reflected when browsing to XTF - Caching Issue?

12 views
Skip to first unread message

Patrick Etienne

unread,
Nov 16, 2010, 2:47:29 PM11/16/10
to xtf-...@googlegroups.com, Christine de Catanzaro, Heather Jeffcoat
xtf devs - 

I'm having a bit of an issue and am hoping for some advice.

The goal of my task is to update existing files (specifically ead type xml files), reindex them, then have the updates be reflected when browsing the xtf servlet. The process we have is automated, but for simplicity, I'll leave out those automated details.

Contained within the root/data/ead directory are folders for each of the ead xml files (and their individual files). What I'm doing is creating a new ead xml file of an already existing name (let's say: MS001-ead as the folder and MS001-ead.xml as the folder's only file). This new MS001-ead.xml (with slightly different content) is copied to the MS001-ead directory replacing the already existing MS001-ead.xml. After this action has been performed, we run the textIndexer "textIndexer -clean -index default". After browsing to the appropriate ead file in the servlet, we can see that the changes are not reflected within the page, however, checking the filesystem shows that the changes are present in both the actual ead file as well as the "*.lazy" version of the ead for the lazy tree.

At first I figured it was just browser caching issues so cleaned browser caches and tried multiple different browsers. The changes still weren't showing up. Next I thought that there must be some caching system that I'm unaware of (something besides building lazy trees), but I've not been able to find any documentation for any such system. I've also thought maybe there was a flag for the textIndexer that would act as some sort of cache refreshing options or in some other xtf associated script, but could not find anything.

My question for the list is,
Is there some sort of caching mechanism within XTF that needs to be refreshed in order to have file updates reflected within the html content?

I have not yet tried deleting xtf related info from the servlet container's "work" directory. This would probably do the trick, but I wanted to see if there was some local xtf control that I was missing.

Any tips, hints, resources, or suggestions would be greatly appreciated!

 - Patrick E.

--
Patrick K. Etienne
Systems Analyst
Georgia Institute of Technology
Library & Information Center
(404) 385-8121


Martin Haye

unread,
Nov 16, 2010, 6:22:48 PM11/16/10
to xtf-...@googlegroups.com, Christine de Catanzaro, Heather Jeffcoat
Hi Patrick,

That's really strange behavior. I'm curious, what happens if you (temporarily) rename the index directory? Does the system still function? In which case, it must be holding onto open files which shouldn't ever happen.

For the record there should be no results caching within XTF.

Does restarting the servlet container (e.g. Tomcat) resolve it? That would be another clue, though clearly it wouldn't be a good long-term solution to be constaintly restarting the container.

--Martin

Patrick Etienne

unread,
Nov 23, 2010, 3:56:31 PM11/23/10
to xtf-...@googlegroups.com
Martin (et al),

I've gotten together with my sysadmin to do a little further testing with this issue. First, I did remove the index directory (I temporarily moved it up a few levels outside of the xtf hierarchy of the filesystem). This did not seem to have any noticeable impact on the xtf application (site still could be browsed without any trouble). I attempted restarting the servlet container, but the old data was still present (even after a reindex). I tried using an incremental rather than clean reindex, this didn't make a difference in the shown-in-webpage (vs. on the filesystem) content (it still had old data). What my sysadmin and I did notice is that it seemed as though the application (tomcat or xtf, I didn't really understand which) was "holding on to open files" (these were my sysadmin's words). Specifically, they were "*.cfs" file paths which no longer existed on the file system but were present in memory. I'm guessing that this has something to do with the issue.

We are running Tomcat 6.0.26 as our servlet container, but could upgrade to 6.0.29 (though it's probably not likely that this will make a difference). I tried to confirm what version of XTF we are running, but alas the CHANGES.txt, INSTALL.txt, and README.txt all link out to the xtf website rather than containing specific release information.

I'm not sure what else to try at this point. Any tips, hints, resources, or recommendations would be appreciated.

 - Patrick E.

PS - Some sample content of the open but non-existent files:
We are running Tomcat 6.0.26 with the latest version being 6.0.29, so an upgrade to Tomcat is available.


patrick  24702   0.0 -0.2    29684   4940  p1- S    16Nov10   0:31.49 /usr/bin/perl -w ./xtf_proc.pl
patrick  26623   0.0 -3.4   276720  71936  p2- S     2Nov10  27:41.02 /System/Library/Frameworks/JavaVM.framew


bash      24036       patrick  cwd     VDIR       14,3        272 38898345 /private/var/tomcat_root/xtf/webapps/xtf/bin
perl      24702       patrick  cwd     VDIR       14,3        136 38895641 /private/var/tomcat_root/xtf/bin
perl      24702       patrick    1w    VREG       14,3   37974937 70202023 /private/var/tomcat_root/xtf/bin/xtf_proc.log
perl      24702       patrick    2w    VREG       14,3   37974937 70202023 /private/var/tomcat_root/xtf/bin/xtf_proc.log
java      26623       patrick    1w    VREG       14,3    1117116 38897336 /private/var/tomcat_root/xtf/logs/catalina.out
java      26623       patrick    2w    VREG       14,3    1117116 38897336 /private/var/tomcat_root/xtf/logs/catalina.out
java      26623       patrick    7w    VREG       14,3       1303 40785302 /private/var/tomcat_root/xtf/logs/catalina.2010-11-02.log
java      26623       patrick    8w    VREG       14,3          0 40785303 /private/var/tomcat_root/xtf/logs/localhost.2010-11-02.log
java      26623       patrick    9w    VREG       14,3          0 40785304 /private/var/tomcat_root/xtf/logs/manager.2010-11-02.log
java      26623       patrick   10w    VREG       14,3          0 40785305 /private/var/tomcat_root/xtf/logs/host-manager.2010-11-02.log
java      26623       patrick   17r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_mm.cfs): No such file or directory
java      26623       patrick   19r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_md.cfs): No such file or directory
java      26623       patrick   20r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_mc.cfs): No such file or directory
java      26623       patrick   21r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_ms.cfs): No such file or directory
java      26623       patrick   22r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_mh.cfs): No such file or directory
java      26623       patrick   24r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_lv.cfs): No such file or directory
java      26623       patrick   25r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_ml.cfs): No such file or directory
java      26623       patrick   27r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_lk.cfs): No such file or directory
java      26623       patrick   29r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_mj.cfs): No such file or directory
java      26623       patrick   30r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_mi.cfs): No such file or directory
java      26623       patrick   31r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_0.cfs): No such file or directory
java      26623       patrick   32r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9d.cfs): No such file or directory
java      26623       patrick   33r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9c.cfs): No such file or directory
java      26623       patrick   36r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_m6.cfs): No such file or directory
java      26623       patrick   37r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_fe.cfs): No such file or directory
java      26623       patrick   38r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_lv.cfs): No such file or directory
java      26623       patrick   39r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_cb.cfs): No such file or directory
java      26623       patrick   41r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_65.cfs): No such file or directory
java      26623       patrick   42r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_32.cfs): No such file or directory
java      26623       patrick   44r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_ih.cfs): No such file or directory
java      26623       patrick   45r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_fe.cfs): No such file or directory
java      26623       patrick   47r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_98.cfs): No such file or directory
java      26623       patrick   50r    VREG       14,3    1405139 86271654 /private/var/tomcat_root/xtf/webapps/xtf/index/spellDict/edmap.dat
java      26623       patrick   57r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_65.cfs): No such file or directory
java      26623       patrick   59r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9b.cfs): No such file or directory
java      26623       patrick   61r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9a.cfs): No such file or directory
java      26623       patrick   62r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_99.cfs): No such file or directory
java      26623       patrick   63r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_98.cfs): No such file or directory
java      26623       patrick   64r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_65.cfs): No such file or directory
java      26623       patrick   65r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_32.cfs): No such file or directory
java      26623       patrick   67r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_0.cfs): No such file or directory
java      26623       patrick   68r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_nb.cfs): No such file or directory
java      26623       patrick   69r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_ma.cfs): No such file or directory
java      26623       patrick   71r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_a6.cfs): No such file or directory
java      26623       patrick   73r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9u.cfs): No such file or directory
java      26623       patrick   74r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_9j.cfs): No such file or directory
java      26623       patrick   75r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_m9.cfs): No such file or directory
java      26623       patrick   77r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_65.cfs): No such file or directory
java      26623       patrick   78r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_32.cfs): No such file or directory
java      26623       patrick   79r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_nb.cfs): No such file or directory
java      26623       patrick   82r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_m7.cfs): No such file or directory
java      26623       patrick   83r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_m6.cfs): No such file or directory
java      26623       patrick   85r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_lk.cfs): No such file or directory
java      26623       patrick   86r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_ih.cfs): No such file or directory
java      26623       patrick   88r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_cb.cfs): No such file or directory
java      26623       patrick   89r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_98.cfs): No such file or directory
java      26623       patrick   91r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_32.cfs): No such file or directory
java      26623       patrick  121r    VREG                                stat(/private/var/tomcat_root/xtf/webapps/xtf/index/_nb.cfs): No such file or directory

On Tue, Nov 16, 2010 at 6:22 PM, Martin Haye <marti...@ucop.edu> wrote:
Hi Patrick,

That's really strange behavior. I'm curious, what happens if you (temporarily) rename the index directory? Does the system still function? In which case, it must be holding onto open files which shouldn't ever happen.

For the record there should be no results caching within XTF.

Does restarting the servlet container (e.g. Tomcat) resolve it? That would be another clue, though clearly it wouldn't be a good long-term solution to be constaintly restarting the container.

--Martin



On 11/16/10 11:47 AM, "Patrick Etienne" <patrick...@library.gatech.edu> wrote:

--
You received this message because you are subscribed to the Google Groups "XTF Developer list" group.
To post to this group, send email to xtf-...@googlegroups.com.
To unsubscribe from this group, send email to xtf-devel+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xtf-devel?hl=en.

Martin Haye

unread,
Nov 23, 2010, 4:08:44 PM11/23/10
to xtf-...@googlegroups.com
Hi Patrick,

It just gets more mysterious, eh? I don't think the Tomcat version will make any difference at all. I'm starting to wonder if there might be two XTF directories involved instead of just one. For instance, if you make a minor (but visible) change to your display stylesheets, does that change show up in the web interface? If not, that's a really sure sign that you're not touching the right directory.

Also, check your textIndexer.conf file (under tomcat/webapps/xtf/conf) and make sure that its <src> path is correct.

I seriously doubt that the Tomcat version has any bearing.

--Martin



On 11/23/10 12:56 PM, "Patrick Etienne" <patrick...@library.gatech.edu> wrote:

Martin (et al),

I've gotten together with my sysadmin to do a little further testing with this issue. First, I did remove the index directory (I temporarily moved it up a few levels outside of the xtf hierarchy of the filesystem). This did not seem to have any noticeable impact on the xtf application (site still could be browsed without any trouble). I attempted restarting the servlet container, but the old data was still present (even after a reindex). I tried using an incremental rather than clean reindex, this didn't make a difference in the shown-in-webpage (vs. on the filesystem) content (it still had old data). What my sysadmin and I did notice is that it seemed as though the application (tomcat or xtf, I didn't really understand which) was "holding on to open files" (these were my sysadmin's words). Specifically, they were "*.cfs" file paths which no longer existed on the file system but were present in memory. I'm guessing that this has something to do with the issue.

We are running Tomcat 6.0.26 as our servlet container, but could upgrade to 6.0.29 (though it's probably not likely that this will make a difference). I tried to confirm what version of XTF we are running, but alas the CHANGES.txt, INSTALL.txt, and README.txt all link out to the xtf website rather than containing specific release information.

I'm not sure what else to try at this point. Any tips, hints, resources, or recommendations would be appreciated.

 - Patrick E.

PS - Some sample content of the open but non-existent files:
We are running Tomcat 6.0.26 with the latest version being 6.0.29, so an upgrade to Tomcat is available.


patrick  24702   0.0 -0.2    29684   4940  p1- S    16Nov10   0:31.49 /usr/bin/perl -w ./xtf_proc.pl <http://xtf_proc.pl/>

Patrick Etienne

unread,
Nov 23, 2010, 4:17:58 PM11/23/10
to xtf-...@googlegroups.com
Thanks for the tips Martin, I'll definitely do a bit more poking around using those suggestions. I'll post the resolution if I'm about to come up with anything significant!

 - Patrick E.

On Tue, Nov 23, 2010 at 4:08 PM, Martin Haye <marti...@ucop.edu> wrote:
Hi Patrick,

It just gets more mysterious, eh? I don't think the Tomcat version will make any difference at all. I'm starting to wonder if there might be two XTF directories involved instead of just one. For instance, if you make a minor (but visible) change to your display stylesheets, does that change show up in the web interface? If not, that's a really sure sign that you're not touching the right directory.

Also, check your textIndexer.conf file (under tomcat/webapps/xtf/conf) and make sure that its <src> path is correct.

I seriously doubt that the Tomcat version has any bearing.

--Martin



On 11/23/10 12:56 PM, "Patrick Etienne" <patrick...@library.gatech.edu> wrote:

Martin (et al),

I've gotten together with my sysadmin to do a little further testing with this issue. First, I did remove the index directory (I temporarily moved it up a few levels outside of the xtf hierarchy of the filesystem). This did not seem to have any noticeable impact on the xtf application (site still could be browsed without any trouble). I attempted restarting the servlet container, but the old data was still present (even after a reindex). I tried using an incremental rather than clean reindex, this didn't make a difference in the shown-in-webpage (vs. on the filesystem) content (it still had old data). What my sysadmin and I did notice is that it seemed as though the application (tomcat or xtf, I didn't really understand which) was "holding on to open files" (these were my sysadmin's words). Specifically, they were "*.cfs" file paths which no longer existed on the file system but were present in memory. I'm guessing that this has something to do with the issue.

We are running Tomcat 6.0.26 as our servlet container, but could upgrade to 6.0.29 (though it's probably not likely that this will make a difference). I tried to confirm what version of XTF we are running, but alas the CHANGES.txt, INSTALL.txt, and README.txt all link out to the xtf website rather than containing specific release information.

I'm not sure what else to try at this point. Any tips, hints, resources, or recommendations would be appreciated.

 - Patrick E.

PS - Some sample content of the open but non-existent files:
We are running Tomcat 6.0.26 with the latest version being 6.0.29, so an upgrade to Tomcat is available.


patrick  24702   0.0 -0.2    29684   4940  p1- S    16Nov10   0:31.49 /usr/bin/perl -w ./xtf_proc.pl <http://xtf_proc.pl/>
Reply all
Reply to author
Forward
0 new messages