Persistence of data
Steve Bennett
How do you get data to remain in Simal in between Jetty restarts?
Err...for that matter, where is data in Jetty stored? :)
Steve
Sander van der Waal
The data is stored in the Jena triple store. Jena supports different
types of data
stores, by default it is TDB which is stored on disk in the folder
determined by
the setting "simal.repository.dir"/"simal.repository.filename".
There's an issue with Elci data storage when you're running from within
Eclipse.
This relates to a 'feature' of Eclipse and is explained in issue 344 [1].
HTH,
Sander
[1] http://code.google.com/p/simal/issues/detail?id=344
> Steve
>
>
Ross Gardler
If your data is not persisting then something strange is happening, so we'll need some more info.
Sent from my mobile device.
> --
> You received this message because you are subscribed to the Google Groups "Simal contributors" group.
> To post to this group, send an email to simal-con...@googlegroups.com.
> To unsubscribe from this group, send email to simal-contribut...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/simal-contributors?hl=en-GB.
>
Steve Bennett
> To clarify what Sander is saying. You shouldn't need to do anything to persist data.
>
> If your data is not persisting then something strange is happening, so we'll need some more info.
Ok, I'll try and narrow it down. I'm running it on a remote linux
machine. Mostly when I shut down jetty manually (ctrl+c) and restart
it (mvn jetty:run), all is well. But last night, I shut down my shell
connection, which apparently left Jetty running but unattended. I
connected today, shut it down by sending SIGINT, then restarted. The
test data that I had imported the previous day (using the handy link
built in) was gone. (I hadn't created any other records).
Incidentally, is there a better way of deploying new CSS files than
copying them into the src/... directory then killing and restarting
Jetty?
Steve
Sander W G van der Waal
> contri...@googlegroups.com] On Behalf Of Steve Bennett
> Sent: 02 February 2011 02:56
> To: simal-con...@googlegroups.com
> Subject: Re: [Simal] Persistence of data
>
> On 2 February 2011 13:17, Ross Gardler <rgar...@apache.org> wrote:
> > To clarify what Sander is saying. You shouldn't need to do anything to
> persist data.
> >
> > If your data is not persisting then something strange is happening, so
> we'll need some more info.
>
> Ok, I'll try and narrow it down. I'm running it on a remote linux
> machine. Mostly when I shut down jetty manually (ctrl+c) and restart
> it (mvn jetty:run), all is well. But last night, I shut down my shell
> connection, which apparently left Jetty running but unattended. I
> connected today, shut it down by sending SIGINT, then restarted. The
> test data that I had imported the previous day (using the handy link
> built in) was gone. (I hadn't created any other records).
That's strange. I have had some issues with persistence before.
Following issue 344 [1] I implemented a shutdown hook to make sure
the model is closed properly and persisted. Sending a SIGINT should (according
to 'Google') let the JVM terminate gracefully so the shutdown hook should have
been executed and the data should have been saved to disk. If you still
have the logs please check for a log message on INFO level similar to :
"Trying to gracefully shut down the repository."
However, I will create an issue for this because persisting the data
should not depend on the shutdown of the application in the first place,
so we need to know in-depth how Jena is handling this.
Sander
Steve Bennett
<sander.v...@oucs.ox.ac.uk> wrote:
> been executed and the data should have been saved to disk. If you still
> have the logs please check for a log message on INFO level similar to :
> "Trying to gracefully shut down the repository."
Thanks, where are the logs stored?
Steve
Ross Gardler
Unless you have customised the log4j.properties they just dump to
stdout. I've often thought we should have them write to file though.
Ross
Sander van der Waal
-Dlog4j.configuration=file:/opt/simal/etc/log4j.xml
Where the log4j.xml is just standard Log4J config, nothing special.
I'm not sure about writing them to file by default. How would we handle
file paths (being platform-specific) ?
Sander
> Ross
>
>
Ross Gardler
log4j handles all that for us, e.g.
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=${user.home}/logs/simal.log
log4j.appender.R.MaxFileSize=100KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
Ross
Steve Bennett
<sander.v...@oucs.ox.ac.uk> wrote:
> the model is closed properly and persisted. Sending a SIGINT should (according
> to 'Google') let the JVM terminate gracefully so the shutdown hook should have
> been executed and the data should have been saved to disk. If you still
> have the logs please check for a log message on INFO level similar to :
> "Trying to gracefully shut down the repository."
I'm not sure I've seen that message. When I press Ctrl+C (SIGINT,
no?), I just get two lines:
2011-02-03 12:06:14.826::INFO: Shutdown hook executing
2011-02-03 12:06:15.484::INFO: Shutdown hook complete
Although, I have previously seen a message about "destorying[sic] the
repository". I'm not sure what causes the different shutdowns.
Steve
Ross Gardler
> On 3 February 2011 01:56, Sander W G van der Waal
> <sander.v...@oucs.ox.ac.uk> wrote:
>> the model is closed properly and persisted. Sending a SIGINT should (according
>> to 'Google') let the JVM terminate gracefully so the shutdown hook should have
>> been executed and the data should have been saved to disk. If you still
>> have the logs please check for a log message on INFO level similar to :
>> "Trying to gracefully shut down the repository."
>
> I'm not sure I've seen that message. When I press Ctrl+C (SIGINT,
> no?), I just get two lines:
>
>
> 2011-02-03 12:06:14.826::INFO: Shutdown hook executing
> 2011-02-03 12:06:15.484::INFO: Shutdown hook complete
The line Sander is referring to is logged by line 188 of UserApplication
(in the web module) [3] in the onDestroy() method.
This looks like a bug to me, see below.
> Although, I have previously seen a message about "destorying[sic] the
> repository". I'm not sure what causes the different shutdowns.
Naturally I made that typo so we could easily search [1] the code for it
at a time like this ;-)
This shows the message is written in the JenaSimalRepository.destroy()
method. This is called in a number of places [2]. Most of which are in
tests, which don't apply here.
One of the locations though is called from the above mentioned method.
Sander says this is intended to save the data to disk, but actually it
does the reverse. It deletes all the data. It's entirely possible that
I'm reading this wrong. I should be in bed now. If not, what gives?
Ross
Sander W G van der Waal
> contri...@googlegroups.com] On Behalf Of Ross Gardler
> Sent: 03 February 2011 02:02
> > On 3 February 2011 01:56, Sander W G van der Waal
> > <sander.v...@oucs.ox.ac.uk> wrote:
> >> the model is closed properly and persisted. Sending a SIGINT should
> (according
> >> to 'Google') let the JVM terminate gracefully so the shutdown hook
> should have
> >> been executed and the data should have been saved to disk. If you still
> >> have the logs please check for a log message on INFO level similar to :
> >> "Trying to gracefully shut down the repository."
> >
> > I'm not sure I've seen that message. When I press Ctrl+C (SIGINT,
> > no?), I just get two lines:
> >
> >
> > 2011-02-03 12:06:14.826::INFO: Shutdown hook executing
> > 2011-02-03 12:06:15.484::INFO: Shutdown hook complete
>
> The line Sander is referring to is logged by line 188 of UserApplication
> (in the web module) [3] in the onDestroy() method.
I think the 'Shutdown hook' message are from Jetty. When I just tested
locally I got:
2011-02-03 20:15:17.032::INFO: Shutdown hook executing
INFO uk.ac.osswatch.simal.wicket.UserApplication - Trying to gracefully shut down the repository.
INFO uk.ac.osswatch.simal.model.jena.simal.JenaSimalRepository - Destorying the SimalRepository
2011-02-03 20:15:17.782::INFO: Shutdown hook complete
> This looks like a bug to me, see below.
>
> > Although, I have previously seen a message about "destorying[sic] the
> > repository". I'm not sure what causes the different shutdowns.
>
> Naturally I made that typo so we could easily search [1] the code for it
> at a time like this ;-)
>
> This shows the message is written in the JenaSimalRepository.destroy()
> method. This is called in a number of places [2]. Most of which are in
> tests, which don't apply here.
>
> One of the locations though is called from the above mentioned method.
>
> Sander says this is intended to save the data to disk, but actually it
> does the reverse. It deletes all the data. It's entirely possible that
> I'm reading this wrong. I should be in bed now. If not, what gives?
Why do you think it deletes all the data? According to the Jena docs:
"It is important to flush all caches to make the file state consistent
with the cached states because some caches are write-behind so unwritten
chnages may be held in-memory. Caches are flushes when a model or dataset
is closed." [1]
Which is why I (try to) make sure that model.close() is called when the
application shuts down. Dereferencing the model and subsequently the
repository may cause problems however and surely there is something wrong
somewhere. I'll need to dig further. Any ideas are welcome of course.
Sander
[1] http://openjena.org/wiki/TDB/JavaAPI#Caching_and_synchronization
> Ross
>
> [1]
> http://www.google.com/codesearch?q=destorying+package%3Ahttp%3A%2F%2Fsimal\
> .googlecode\.com&origq=destorying&btnG=Search+Trunk
>
> [2]
> http://www.google.com/codesearch?q=destroy%28%29+package%3Ahttp%3A%2F%2Fsim
> al\.googlecode\.com&origq=destroy%28%29&btnG=Search+Trunk
>
> [3]
> http://www.google.com/codesearch?hl=en&lr=&q=gracefully+package%3Ahttp%3A%2
> F%2Fsimal\.googlecode\.com&sbtn=Search
>
> --
> You received this message because you are subscribed to the Google Groups
> "Simal contributors" group.
> To post to this group, send an email to simal-
> To unsubscribe from this group, send email to simal-
> contributors...@googlegroups.com.
Ross Gardler
My bad. I just read the method name and assumed destroy means destroy data.
Ross