How to read/load again an old Rpi shake miniSEED file with SWARM?

553 views
Skip to first unread message

Juan Antonio

unread,
Jan 25, 2017, 6:57:50 PM1/25/17
to RaspberryShake
Buenas noches. 

As Rpi Shake deletes old data every 7 days, i stored old files in a hard disk in order to don´t lost.
I´m trying to open/read an outdated stored file in SWARM again (open file option, SEED/miniSEED file), but I can´t. 

What could be the problem? Any advice to read it?. 

Thanks

Juan Antonio

   

Branden Christensen

unread,
Jan 25, 2017, 7:03:27 PM1/25/17
to RaspberryShake
Juan:


I can confirm that feature in Swarm never worked very well. Maybe some one else on the list can recommend another program for viewing miniSEED files? I think seisgram2k would work for this too.


Branden

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.
To view this discussion on the web visit https://groups.google.com/d/msgid/raspberryshake/0c52873d-1412-4653-9a65-2c6865f5b734%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Tom Parker

unread,
Jan 25, 2017, 7:19:23 PM1/25/17
to raspber...@googlegroups.com
I haven't received other reports of Swarm having trouble reading miniSEED files. Can you describe the problems you've had? I'd like to get them fixed.

On Wed, Jan 25, 2017 at 3:03 PM, Branden Christensen <branden.c...@osop.com.pa> wrote:
Juan:


I can confirm that feature in Swarm never worked very well. Maybe some one else on the list can recommend another program for viewing miniSEED files? I think seisgram2k would work for this too.


Branden
On Jan 25, 2017 18:57, "Juan Antonio" <jaalam...@gmail.com> wrote:
Buenas noches. 

As Rpi Shake deletes old data every 7 days, i stored old files in a hard disk in order to don´t lost.
I´m trying to open/read an outdated stored file in SWARM again (open file option, SEED/miniSEED file), but I can´t. 

What could be the problem? Any advice to read it?. 

Thanks

Juan Antonio

   

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspber...@googlegroups.com.

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Juan Antonio

unread,
Jan 25, 2017, 7:22:46 PM1/25/17
to RaspberryShake, branden.c...@osop.com.pa
Gracias, Branden. 

With seisgram2k i tested several days but neither was able. 

I think can read easily old files generated by Rpi Shake it is a main function to take advance of collected data.     

Juan Antonio  


El jueves, 26 de enero de 2017, 1:03:27 (UTC+1), Branden Christensen escribió:
Juan:


I can confirm that feature in Swarm never worked very well. Maybe some one else on the list can recommend another program for viewing miniSEED files? I think seisgram2k would work for this too.


Branden
On Jan 25, 2017 18:57, "Juan Antonio" <jaalam...@gmail.com> wrote:
Buenas noches. 

As Rpi Shake deletes old data every 7 days, i stored old files in a hard disk in order to don´t lost.
I´m trying to open/read an outdated stored file in SWARM again (open file option, SEED/miniSEED file), but I can´t. 

What could be the problem? Any advice to read it?. 

Thanks

Juan Antonio

   

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.
To post to this group, send email to raspber...@googlegroups.com.

Branden Christensen

unread,
Jan 25, 2017, 7:23:46 PM1/25/17
to RaspberryShake
Hi Tom:


Buenas noches.

Sorry, I never filed this bug report. After a certain size the miniSEED load feature just hangs. Daily miniSEED files are generally too big.

If you have trouble reproducing this, I will help. Let me know.


Yours, Branden

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Angel Rodriguez

unread,
Jan 25, 2017, 7:35:57 PM1/25/17
to raspber...@googlegroups.com
Hi Juan Antonio,

In the next few days there will be an offering that will read and allow you analyze day long miniseed files.

Angel


Shake forum on google.groupsSome useful links:
--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Angel Rodriguez

unread,
Jan 25, 2017, 7:37:58 PM1/25/17
to raspber...@googlegroups.com
Hi Tom,

The problem might be that the files are a day long.  

Angel


Shake forum on google.groupsSome useful links:

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Juan Antonio

unread,
Jan 25, 2017, 7:57:08 PM1/25/17
to RaspberryShake, branden.c...@osop.com.pa
Hello Tom 

The file I was trying to load has only 1.356 Kb. because I started to record the Rpi very late this day. I attach  4 screen and the file (sequenced) to try read this file. As you can see on top file it not recognized and when i click on it, nothing happens

I tried with otehr days, but are bigger files.

This only happens me with files not stored on Rpi Shake. If i try load an external copy of an existent day the  open file option works and open another Heliocorder screen. But logically my interest is on external files 

Gracias 
Juan Antonio 

To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.

To post to this group, send email to raspber...@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.

To post to this group, send email to raspber...@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.
Trying read MiniSEED from 2017.016 file 0.png
Trying read MiniSEED from 2017.016 file 1.png
Trying read MiniSEED from 2017.016 file 2.png
Trying read MiniSEED from 2017.016 file 3.png
AM.R5C96.00.SHZ.D.2017.016

Angel Rodriguez

unread,
Jan 25, 2017, 8:27:42 PM1/25/17
to raspber...@googlegroups.com
HI All,

I have lost track of the thread.  If the files are moving from a linux box to a windows box, could this be a file termination issue?

Angel


Shake forum on google.groupsSome useful links:
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.

Branden Christensen

unread,
Jan 25, 2017, 10:07:13 PM1/25/17
to RaspberryShake
Hi guys:


The problem is that when you move files from Linux to Windows the file permissions change. You need to change the file permissions so that they can be read. Swarm is not able to open the files because the files are not enabled on windows for reading. This is a tonteria de windows. 


Branden

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Juan Antonio

unread,
Jan 26, 2017, 1:20:48 PM1/26/17
to RaspberryShake, branden.c...@osop.com.pa
Thanks for your comments. 

1. Permissións changed: Unfortunately I tested with several files to give full permissions on windows and doesn´t work for me. 

2. I´m thinking that other possible cause could be that configuration of  WINCSP is changing some header or format when export the file to windows. I run SWARM whithout Rpi Shake connection, only to open old files in windows, and gave me a continuous header error (see attached file)       

  
Saludos 

Juan Antonio

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.
To post to this group, send email to raspber...@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.
Error Console when I try open an externally stored MiniSEED file in SWARM.png

Richard

unread,
Jan 26, 2017, 2:07:13 PM1/26/17
to RaspberryShake, branden.c...@osop.com.pa
hi juan,

not sure if this is the problem, but try this:

in WINSCP, explicitly specify that the copy should be done as a binary transfer.  if it is being done as an ASCII transfer, you are losing the first bit of every byte resulting in a corrupted file that nothing could successfully read.

richard

Juan Antonio

unread,
Jan 26, 2017, 2:20:55 PM1/26/17
to RaspberryShake, branden.c...@osop.com.pa
Thanks Richard. 

The default option is binary, but I tested too with the other options (text and automatic) without success. 

Testing the same file on Seisgram2k the error is: 
 
 ERROR: reading: D:\F. Conocimiento\Electrónica y Python\5. Raspberry Shake\1. My RpiShake Data saved from\1. Datos 20170116 al 20170122\Datos\AM.R5C96.00.SHZ.D.2017.019: net.alomax.seis.SeisException: ERROR: reading SEED volume identifier block: control header type code not "D", "R", "Q", "M": not SEED Data Record:: bytes read=  "          "


Juan Antonio    

Branden Christensen

unread,
Jan 26, 2017, 2:23:43 PM1/26/17
to RaspberryShake
Juan Antonio:


Let's try and rule out a problem with the miniSEED files. Please go ahead and email in an example file that you cannot read and I will try to open it with another program here on my Linux box. 


Best, 


Branden Christensen
Director, OSOP


Get yours now! Visit shop.raspberryshake.org


To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.

Branden Christensen

unread,
Jan 26, 2017, 7:04:25 PM1/26/17
to RaspberryShake
Juan:


Thank you for sending me the miniSEED file off the list (Tom, please let me know if you would like the mseed file off the list)


Some observations:


1. The permissions are good:

branden@branden-AO722:~/Desktop$ ls -lh AM.R5C96.00.SHZ.D.2017.020 
-rw-rw-r-- 1 branden branden 6.4M Jan 26 14:55 AM.R5C96.00.SHZ.D.2017.020

2. I can use other miniSEED-compatible programs to view the trace file. 

3. swarm will not open the file. (Tom I can send you the terminal output too. This problem is different from the one I mentioned before about the large miniSEED files. Please let me know how I can help.)


Yours, Branden

Juan Antonio

unread,
Jan 26, 2017, 7:17:03 PM1/26/17
to RaspberryShake, branden.c...@osop.com.pa
Thanks All. 

It is very important for me open these files in SWARM in order to use the less programs as possible. 

 Saludos 
Juan Antonio 

Tom Parker

unread,
Jan 27, 2017, 2:40:55 PM1/27/17
to raspber...@googlegroups.com
I dug through the miniSEED file and it looks corrupted to me. I'm not a miniSEED expert though, perhaps it's a valid style that I haven't encountered before? What I see are two NULL sections, each followed by a reset of the sequence numbers. The NULL sections seem to represent gaps in the data.

Swarm uses Philip Crotwell's seisFile library for reading miniSEED files. I'll check with Philip to see if he knows a better way of handling this. Some apps do seem able to pass over these bad sections.

--Tom

To post to this group, send email to raspberryshake@googlegroups.com.

Tycho Aussie

unread,
Jan 27, 2017, 3:50:38 PM1/27/17
to RaspberryShake, branden.c...@osop.com.pa
Please send me the Miniseed and I will try to open it with Obspy. 
In the mean time, I have a windows compatible program that will open the miniseed files from my Raspberry Shake. 
It's called DIMAS and if you email me I will send you the link to the download. All you need to do is install it's folder in C:\ and you'll be good to go.

-Daniel Burk

Juan Antonio

unread,
Jan 27, 2017, 4:37:32 PM1/27/17
to RaspberryShake
Thanks Tom.

Could be the issue was provoked by the way I extract the file from Rpi Shake? It happens to all my old files extracted. 

I use WINSCP with default options (binary. I´m trying to extract it whith a scp command via puTTY terminal, but i´m noob on Debian and i was not able still to extract at least one file.   

I used several SCP ways i saw, but nothings work to me. 

scp opt/archive/data/2017/AM/R5C96/SHZ.D/AM.R5C96.00.SHZ.D.2017.020  mys...@192.168.1.39:/c/DataShake

The error reported was

opt/archive/data/2017/AM/R5C96/SHZ.D/AM.R5C96.00.SHZ.D.2017.020: No such file or directory

And the files is there inside Shake  

Regards


Juan Antonio

--
Some useful links:
 
Manual: http://manual.raspberryshake.org/
Shop: https://shop.raspberryshake.org/
Website: http://raspberryshake.org/
Do It YourSelf Page: http://raspberryshake.org/do-it-yourself
---
You received this message because you are subscribed to the Google Groups "RaspberryShake" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshak...@googlegroups.com.
To post to this group, send email to raspber...@googlegroups.com.
Visit this group at https://groups.google.com/group/raspberryshake.

Richard

unread,
Jan 27, 2017, 5:21:37 PM1/27/17
to RaspberryShake
your scp is failing due to the missing '/' at the beginning of the /opt/archive path.

try again, but put '/' before opt and you may see different results.

richard

Juan Antonio

unread,
Jan 27, 2017, 7:50:45 PM1/27/17
to RaspberryShake

Angel Rodriguez

unread,
Jan 27, 2017, 8:39:29 PM1/27/17
to raspber...@googlegroups.com
By gosh and by golly, that's what I said two days ago!  I'm going to celebrate with a beer!

Angel
To unsubscribe from this group and stop receiving emails from it, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.

Mercalli

unread,
Jan 28, 2017, 3:51:22 PM1/28/17
to RaspberryShake
Reverse the order, /opt/data/archive/ ...

Juan Antonio

unread,
Jan 28, 2017, 7:48:38 PM1/28/17
to RaspberryShake
Thanks Mercalli. A basic mistake of mine. 

I taked advance of your comments and I made some tests: 
 

1. I Run next pscp command (puTTY) from windows cmd to extract all files of Rpi Shake in one execution

pscp mys...@192.168.1.39:/opt/data/archive/2017/AM/R5C96/SHZ.D/*.* C:/DatosShake

2. As result 8 (all 21st to 28th days) files were stored in my windows computer
3. I tested all files in SWARM (I unconnect Rpi Shake because if SWARM detect file into Shake open it): Results: 21 to 23 failed, 24 to 28 success

4. Next I tested again files extracted other days. They all failed

5. Then i made the same extraction (made with pscp) with WINSCP and tested too. Results: 21st to 23rd failed, 24th to 28ht success    

I question me some things
  • Why results change for some days with the same (WINSCP) method days ago? (here i has doubts because i was playing hanging some parameters 
  • Why differents results  (3 failures vs 5 success) for files extracted form the same place, in the same way and at the same time?
  • It is curious that failed files were first 3 days. ¿Could be older days were trated by SWARM in f¡differents way and have some kind of mark that affect the extracuin proccess?     
I´ll do more test, but if you have more ideas, welcome

Regards 
Juan Antonio 

Mercalli

unread,
Jan 28, 2017, 7:52:47 PM1/28/17
to RaspberryShake
I transferred the files from my RaspberryShake to my Linux box (distro: Open Suse Leap 42.1) using the utility "sshfs." (I can list the command lines if anyone is interested.) The files open directly in SWARM. Both the SEED/miniSEED file records, and SWARM seem to be OK in the Linux world.

My previous post may not be clear. I was just try to tell Juan he got the error "No such file or directory" because he did not enter the path in the correct order.


On Wednesday, January 25, 2017 at 4:57:50 PM UTC-7, Juan Antonio wrote:

Mercalli

unread,
Jan 28, 2017, 8:00:10 PM1/28/17
to RaspberryShake
We both posted at the same time. I am sorry, I have no Windows OS computers, so I am not much help. I believe you are making very good progress.


On Wednesday, January 25, 2017 at 4:57:50 PM UTC-7, Juan Antonio wrote:

Juan Antonio

unread,
Jan 28, 2017, 8:12:50 PM1/28/17
to RaspberryShake
Thanks a lot Mercalli.

Only your path advice has been very useful for me to unblock me. I though i copied the path and i didn´t occurs to me review it. 
I suppose the problem is in extraction proccess from Linux to windows world.

Regards 

Mercalli

unread,
Jan 29, 2017, 11:46:45 AM1/29/17
to RaspberryShake
I am attaching one of my SEED/miniSEED files I transferred from my RaspberryShake. (It is 4.0mb.) If you want to, you can download it and try to open it with SWARM in the Windows OS. I am not sure what that will tell us, but it may tell us something.
AM.R24B9.00.SHZ.D.2017.024

Richard

unread,
Jan 29, 2017, 12:09:55 PM1/29/17
to RaspberryShake
Hi Juan,

I'm not entirely sure what types of data analysis functionality you are looking for, but I can also provide another alternative, my program PQL.  For trace viewing, it provides lots of options you may find interesting.

You can find this here:
File Name: PQL.v2010.307.setup.exe

This is a Windows machine installer program that does what you'd expect.  I have tested opening your file (the one with gaps) you are having trouble with and it opens fine.

cheers, hope this might be of help to you,

richard

Juan Antonio

unread,
Jan 29, 2017, 1:00:41 PM1/29/17
to RaspberryShake
Thanks Mercally

Your files load rigthly. I attach a screenshot.  
I repeated extraction today of 22 to 29 (21 was erased by RS) with WINSCP, and 22 and 23 continues failing, 24 t029th (new one) extract and load ok.

Finally I have files from 16th to 29th day. 16 to 23 fail vs 24 to 29 success.
I can accept that I did some configuration change in WINSCP and because before some files were bad and after ok, but after that all files extracted from RS should be ok,    
   
I attach an image of WINSCP current configuration.(transfer mode = Binary, End of line = LF  (the other option is CR/LF)    


Saludos 

Juan Antonio 
SWARM 1 24 2017 Read Rightly.png
WINSCP configuration.png

Juan Antonio

unread,
Jan 29, 2017, 1:07:26 PM1/29/17
to RaspberryShake
Thanks a los Richard

By now I´m exploring options and learning. I´ll use with big interest and i´m sure will be useful for me. I hope give some feedabck soon 


Best Regards  

Juan Antonio

unread,
Jan 29, 2017, 1:16:19 PM1/29/17
to RaspberryShake
Hello Richard. 

This Url ask me for authentication.  How could do it in other way? 

Regards. 
Juan Antonio 

Richard

unread,
Jan 29, 2017, 3:58:13 PM1/29/17
to RaspberryShake
Hi Juan Antonia,

Sorry, I forgot it was password protected.  You can use the following credentials to download this file:

User Name: PQLXtars
Password:  getPQLX

cheers,

richard

Juan Antonio

unread,
Jan 29, 2017, 4:28:14 PM1/29/17
to RaspberryShake
Thanks Richard. I´ll test next days 

Regards
Juan Antonio

richard boaz

unread,
Jan 31, 2017, 7:53:59 AM1/31/17
to raspber...@googlegroups.com
your scp is failing due to the missing '/' at the beginning of the /opt/data path.

try again, but put '/' before opt and you may see different results.

richard
You received this message because you are subscribed to a topic in the Google Groups "RaspberryShake" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/raspberryshake/NgwHjipcEII/unsubscribe.
To unsubscribe from this group and all its topics, send an email to raspberryshake+unsubscribe@googlegroups.com.
To post to this group, send email to raspberryshake@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages