Pisces, CSS3, QuakeML, SeisComP

[Claudio Satriano]

Hi, looks like I'm the first here :)

As you might have guessed from the title, I have quite a few questions...

I'm starting a research project, where I'm going to create a seismic catalog with more than 200 events per day.

These events comprise phase picks, focal mechanism or moment tensor (for the biggest ones) and I may decide to mesure and store other parameters, like corner frequency, stress-drop, etc...

I definitively need a database for that, and I'm considering Pisces.

Looking at your SRL paper, it seems that it should have no problem in handling such a large dataset (you present a test case with ~55,000 events).

I was wondering why you choose to use the CSS3 schema, instead of one based on the QuakeML representation.

Also, do you have any plan to import/export QuakeML?

Finally, what are the advantages of Pisces over a solution based on the SeisComP database engine?

Cheers,

Claudio

[Jonathan MacCarthy]

Hi Claudio!

Yes, it's rather quiet in here:-)  I sometimes get some emails about Pisces directly that I should probably post here, but haven't yet.

Looking at your SRL paper, it seems that it should have no problem in handling such a large dataset (you present a test case with ~55,000 events).

For the SRL paper, I used SQLite and only a few tables and joins.  For a larger, long-term database, something like PostgreSQL may be a better database backend.  Pisces uses SQLAlchemy, which supports a number of very capable database backends.

 

I was wondering why you choose to use the CSS3 schema, instead of one based on the QuakeML representation.

I chose CSS3 for a few reasons:
1. I'm familiar with CSS3 from using BRTT's Antelope Seismic Information System for several PASSCAL (https://www.passcal.nmt.edu/) temporary seismic deployments, and a similar database schema is used where I work.  This is probably the main reason.

2. CSS3 seems more simple than QuakeML and StationXML.  My goal with Pisces was to make it easier for researchers to use SQL databases in their research, and I didn't think people would do that if they had to read XML or learn about object databases.  I don't fully understand them myself.

Also, do you have any plan to import/export QuakeML?

I don't have immediate plans to support QuakeML, but that's only because I don't use it or know much about it.  I'm open to learning more about it and what QuakeML/StationXML integration might look like.  My only experiences with either specification involve some very hideous-looking text files I get back from various data servers, and some endlessly-nested ObsPy objects.

 

Finally, what are the advantages of Pisces over a solution based on the SeisComP database engine?

I'm not familiar with SeisComP, but maybe it would be helpful if I tell you why I chose to write Pisces instead of using another system:

1. Several seismic data management systems are written in C++ or C.  I think a lot of seismologists (including me) don't really know C or C++.  I always had trouble using these other systems because it was hard for me to read the code, and it would take a huge amount of effort to make or contribute any fixes or improvements to those codes.  Python is great for this because it is so readable.  I wanted anyone to easily make and contribute back useful changes.  Also, the user experience in Python is great, with tools like iPython and Matplotlib to work with.

2. Pisces uses packages like SQLAlchemy and NumPy, which have huge non-seismologist communities around them.  They are well-maintained, have wonderful documentation, and are always improving.  Most of the problems I've ever had in Pisces can be diagnosed and fixed using stackoverflow, google, and the project documentation for NumPy and SQLAlchemy.  With other data management systems, my only help came from home-grown documentation by a small group of developers and nearly-empty mailing list archives.  I realize that exact same thing can be said of Pisces, but I think of Pisces more like an _example_ of how to use databases to manage geophysical data, instead of a finished product.  At this point, Pisces is basically some SQLAlchemy models for CSS3 tables and few convenience functions.  I would like this to grow, but right now it is (hopefully) simple enough that users can understand what it does, and they don't have to wait for me or anyone else if they want to fix something or add something.

If you are considering Pisces for your project, I might suggest trying it out with a very small data set and an SQLite database.  Pisces is still Beta-type software, and I'm ironing out kinks all the time.  I know that some of the query functions need updating, for example:-)  If you find that it is useful and not too frustrating, then maybe it would be worthwhile to figure out the QuakeML import/export issue sooner rather than later.

Thanks again for your interest, and I hope this was helpful.

Best,

Jon

PS.  A new release is in the works, which will include table-building scripts and other improvements.

Cheers,

Claudio

--
You received this message because you are subscribed to the Google Groups "pisces-db" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pisces-db+...@googlegroups.com.
To post to this group, send email to pisc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pisces-db/81c34fbb-d56f-45c3-ac2a-6a302b9805bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonathan MacCarthy]

Claudio,

I also just saw these CSS-QuakeML converters:
https://github.com/NVSeismoLab/commons/tree/master/nsl/converters

I thought someone must have written converters before...

Best,

Jon

[Claudio Satriano]

Hi Jon,

thanks for all the information.

I'm curious to see whether Pisces fits my needs, so I will definitively give it a try.

The nice thing I see in perspective is that it looks like a project I can contribute to, which is only marginally the case with SeisComP.

Moreover, it is Python ;)

I'll let you know as soon as I start fiddling with it (probably in a couple of weeks).

Cheers,

Claudio

[Stephen Bannister]

hi jon,

 I've dived in a bit using pisces - and I found it easy to use thanks very much - especially with your SRL article
and examples to follow.  Following on from Claudio's query I agree totally with your own thoughts : QuakeML is fine
for formal data-structure ;  but gets quite impractical to use/read quite quickly ;  the CSS flat table approach is
quite straight-forward ; and as you point out - you can create your own special-purpose extensions quite quickly.

PS.  A new release is in the works, which will include table-building scripts and other improvements.

I really look forward to any new release you may put out !    I  am now using pisces-db in my practical
day-to-day projects;  combined closely with obspy.  Thanks very much for your development work !!


cheers
stephen

[jkmacc]

Thank you for the kind words, Stephen.  I'm glad that Pisces is working for you so far.  Regarding QuakeML/StationXML, I've recently considered trying to write XML importers/exporters, as these formats seem to be the way many data centers are providing data.  I wish they would also reconsider SEED/miniSEED, and instead use HDF5 files, but that's another discussion:-)  Either way, I think it would only help SQL database users if they could easily download and build databases from these XML formats.

Best,
Jon

[Claudio Satriano]

Hi John,

if you’re looking for an HDF5-based file format, you should take a look at ASDF (http://asdf.readthedocs.org/en/latest/).

Claudio

---------------------------------------------------

Claudio Satriano

satr...@ipgp.fr

http://www.ipgp.fr/~satriano

Equipe de Sismologie

Institut de Physique du Globe de Paris 

1 rue Jussieu, 75238 - Paris Cedex 05

Office : + 33 1 83 95 77 26

Fax : + 33 1 71 93 77 16