Data gaps on WWS

27 views
Skip to first unread message

giacomo.u...@gmail.com

unread,
Jul 14, 2021, 9:10:11 AMJul 14
to Earthworm Community Forum
Dear All

I installed EarthWorm 7.10 and WWS 1.3.13 on Ubuntu 20.

Few months ago i configured Earthworm for the connection to 10 SCNL (100 Hz) and 
i) the data gaps on the WWS mysql database are few
ii) the average latency on WWS is good.

Few days ago i added 60 more SCNL (100-200 Hz) and 
i)  intermittent and fixed length data gaps appeared on the WWS mysql database (see picture) and on all the SNCLs 

Please, are 70 SNCL above the maximum limit of channels for the WWS?

Thanks in advance
Kind regards
Giacomo

datagapsexample.png

Baker, Lawrence M

unread,
Jul 14, 2021, 1:43:52 PMJul 14
to giacomo.u...@gmail.com, Baker, Lawrence M, 'Meremonte, Mark E' via Earthworm Community Forum
Giacomo,

Please describe more about your system.  What hardware are you running, and what kind of storage device?

There should be no trouble running Winston for 70 channels of data on a suitable system.  I have been running an Earthworm 7.7 system with 72 channels of 200 Hz, 32-bit data for years.  That is on a rack mount server with four cores and about 8 GB RAM and spinning hard disks.  It does not have gaps.  The OS is CentOS, maybe 6.x?

I have also configured an ARM SBC with a Winston WaveServer for 36 channels of 200 Hz, 32-bit data.  That was a BeagleBone Black with a MicroSD card for storage.  I had to be very careful when building that system to optimize the setup.  In particular, the Linux open source Java was too slow to keep up.  I switched to the Oracle Java for ARM embedded systems.  I have not used that system in years.  Linux Java may have improved, and Oracle Java is definitely a problem these days with their restrictive license terms.

Below are the portions of the ImportEW.config and WWS.config files I use for the 72-channel system, which is still in service.  I don't think the "Fast" option in ImportEW.config has any effect.

Larry Baker
US Geological Survey
650-329-5608
ba...@usgs.gov

===== ImportEW.config =====
import.host=127.0.0.1
import.port=16005
import.receiveID=alive
import.sendID=alive
import.heartbeatInterval=30000
import.expectedHeartbeatInterval=30000
import.timeout=60000
import.dropTableDelay=10
import.log.name=../log/ImportEW.log
import.log.numFiles=10
import.log.maxSize=1000000

# include the database connection parameters
@include Winston.config

# Filters apply to incoming TraceBufs
# rejects packets from more than 10 seconds in the future
# Not used for structural monitoring
#filter=TimeFilter
#TimeFilter.class=gov.usgs.winston.in.ew.TimeFilter
#TimeFilter.order=1
#TimeFilter.past=-100
#TimeFilter.future=10
#TimeFilter.action=reject
#TimeFilter.log=1

# rejecs packets older than the max days allowance
filter=MaxDays
MaxDays.class=gov.usgs.winston.in.ew.MaxDaysFilter
MaxDays.order=0
MaxDays.action=reject
MaxDays.log=1

# Not used for structural monitoring
#filter=McCalPulse
#McCalPulse.class=gov.usgs.winston.in.ew.McCalPulseFilter
#McCalPulse.order=0
#McCalPulse.threshold=500
#McCalPulse.terminal=false

# a final filter that accepts all tracebufs
filter=All
All.order=3
All.class=gov.usgs.winston.in.ew.SCNLFilter
All.scnl=* * * *
All.action=accept

# Default options
options=Default
# Disable RSAM calculations for structural monitoring
#Default.rsam.enable=true
Default.rsam.enable=false
Default.rsam.delta=10
Default.rsam.duration=60
Default.timeThreshold=60
Default.traceBufThreshold=60
Default.maxBacklog=200
# Store 6 months of structural monitoring data
#Default.maxDays=0
Default.maxDays=180

# Fast options (for writing tracebufs immediately to the database)
options=Fast
Fast.timeThreshold=1
Fast.traceBufThreshold=1
Fast.maxBacklog=200
Fast.rsam.delta=10
Fast.rsam.duration=60
Fast.applyTo=AU.* * * *, TAPA * * *

===== WWS.config =====
log.name=../log/WWS.log
# Serve all interfaces for structural monitoring
#wws.addr=127.0.0.1
wws.port=16022
# Disable keepalive for structural monitoring
#wws.keepalive=true
wws.keepalive=false
wws.handlers=4
wws.maxConnections=50
wws.idleTime=7200
wws.allowHttp=true
wws.httpMaxSize=10000000
wws.httpRefreshInterval=300
wws.maxDays=0

@include Winston.config


On Jul 14 2021, at 6:10:11 AM, giacomo.u...@gmail.com <giacomo.u...@gmail.com> wrote:

 

 This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.  



--
--
You received this message because you are subscribed to the Google
Groups "Earthworm Community Forum" group.
 
To post to this group, send an email to earthwo...@googlegroups.com
 
To unsubscribe from this group, send an email to
earthworm_for...@googlegroups.com
 
For more options, visit this group at
http://groups.google.com/group/earthworm_forum?hl=en

---
You received this message because you are subscribed to the Google Groups "Earthworm Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to earthworm_for...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/earthworm_forum/a5dd10c7-d015-491c-a6da-18ba9c18776fn%40googlegroups.com.
<datagapsexample.png>

Victor Preatoni

unread,
Jul 15, 2021, 8:01:24 AMJul 15
to Earthworm Community Forum
Ciao Giacomo.

what acquisition modules are you using? Have you created unique MODULE IDs for each data acquisition modules?.

Also you should check EW waveServerV tanks and see if gaps are just a Winston issue or there are gaps also In EW tanks.

Regards,
Victor

giacomo.u...@gmail.com

unread,
Jul 15, 2021, 10:21:03 AMJul 15
to Earthworm Community Forum
Dear  Larry
Thanks a lot for the infos

The server i'm using has 12x Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz, that seems performant enough.

I tried to use the ImportEW and WWS configurations you sent (see at the bottom), among other things not very different from my configurations,  but the data gaps issue remains. I nalso oted that the higher the sampling rate the larger the data gaps (i have 3 out of 70 channels with 1000 Hz sampling rate).

I think the problem is not related to the WWS configuration rather the mysql database (MyISAM engine) which is not able to manage such data flux. is it reasonable?

Thanks again
Giacomo

ImportEW.config
import.host=127.0.0.1
import.port=16005
import.receiveID=GECO
import.sendID=GECO
import.heartbeatInterval=30000
import.expectedHeartbeatInterval=30000
import.timeout=60000
import.dropTableDelay=10
import.log.name=ImportEW.log
import.log.numFiles=10
import.log.maxSize=1000000
import.createValarmView=false

# include the database connection parameters
@include Winston.config

# Filters apply to incoming TraceBufs
# rejects packets from more than 10 seconds in the future
filter=TimeFilter
TimeFilter.class=gov.usgs.volcanoes.winston.in.ew.TimeFilter
TimeFilter.order=1
#TimeFilter.past=-100
TimeFilter.future=10
TimeFilter.action=reject
TimeFilter.log=1

# rejecs packets older than the max days allowance
filter=MaxDays
MaxDays.class=gov.usgs.volcanoes.winston.in.ew.MaxDaysFilter
MaxDays.order=0
MaxDays.action=reject
MaxDays.log=1

filter=McCalPulse
McCalPulse.class=gov.usgs.volcanoes.winston.in.ew.McCalPulseFilter
McCalPulse.order=0
McCalPulse.threshold=500
McCalPulse.terminal=false

# a final filter that accepts all tracebufs
filter=All
All.order=3
All.class=gov.usgs.volcanoes.winston.in.ew.SCNLFilter
All.scnl=* * * *
All.action=accept

# Default options
options=Default
Default.rsam.enable=true
Default.rsam.delta=10
Default.rsam.duration=60
Default.timeThreshold=60
Default.traceBufThreshold=60
Default.maxBacklog=200
Default.maxDays=0

# Fast options (for writing tracebufs immediately to the database)
options=Fast
Fast.timeThreshold=1
Fast.traceBufThreshold=1
Fast.maxBacklog=200
Fast.rsam.delta=10
Fast.rsam.duration=60
Fast.applyTo=AU.* * * *, TAPA * * *



WWS.config
wws.addr=XXXX
wws.port=16088
wws.keepalive=false
wws.handlers=4
wws.maxConnections=50
wws.idleTime=7200
wws.allowHttp=true
wws.httpMaxSize=10000000
wws.httpRefreshInterval=300
wws.maxDays=0
#wws.slowCommandTime=15000
@include Winston.config

giacomo.u...@gmail.com

unread,
Jul 15, 2021, 10:27:01 AMJul 15
to Earthworm Community Forum
Ciao Victor

I'm using 2 slink2ew modules, one is connected to n. 10 SNCLs and the other to 60 SNCLs.

I also configured the ewmseedarchiver.d module to archives mseed files and the files do not contain data gaps. So the issue affect only WWS.

Regards
Giacomo

Baker, Lawrence M

unread,
Jul 15, 2021, 4:53:41 PMJul 15
to giacomo.u...@gmail.com, Baker, Lawrence M, 'Meremonte, Mark E' via Earthworm Community Forum
Giacomo,

Your computer is certainly more than capable.

Do you see any errors in the log files?  ImportEW is the module that loads data into the MySQL database.  If ImportEW cannot keep up, I would expect to see dropped packets or messages about getting lapped, I assume in the Earthworm export module that feeds ImportEW.  If that is the case, you might try to increase the RingSize in the export module.  My export module setup is:

# ServerIPAdr and ServerPort specify the local IP address and port number
# on which this module will be ready to accept a client connection.

 ServerIPAdr     127.0.0.1     # Export messages via this ethernet card   
                               # 0.0.0.0 is useful because the export listens 
                               # on all available IP addresses, from loopback 
                               # (127.0.0.1) to your actual external IP address.
                               # 0.0.0.0 may be sufficient, or you may want to 
                               # be more specific about what IP address you 
                               # export at.
 ServerPort      16005         # Well-known port number to export msgs on
 MaxMsgSize       4096         # maximum size (bytes) for input/output msgs
                               # 4096 good start for Tracebufs, 128 may be 
                               # sufficient for most other message types
 RingSize         1000         # number of msgs to buffer for export 

 SendAliveText   "alive"       # string sent to client as heartbeat 
 SendAliveInt      30          # seconds between alive msgs sent to client.
                               #  0=> no heartbeat
 RcvAliveText    "alive"       # text of client's heartbeat (we get this)
 RcvAliveInt       150         # seconds between client's heartbeats to us.
                               #  0 => no heartbeat

Larry Baker
US Geological Survey
650-329-5608
ba...@usgs.gov


Baker, Lawrence M

unread,
Jul 15, 2021, 11:54:11 PMJul 15
to Baker, Lawrence M, giacomo.u...@gmail.com, 'Meremonte, Mark E' via Earthworm Community Forum
Giacomo,

Also, here's my notes for editing the MySQL configuration file:

Edit the MySQL Server configuration file, /etc/my.cnf, to use /home/mysql for its data directory and MyISAM as the default storage engine.

# vi /etc/my.cnf

#datadir=/var/lib/mysql
datadir=/home/mysql
# Winston requires the MyISAM engine
default-storage-engine=MYISAM
myisam-recover=BACKUP,FORCE

I cannot remember why I chose MYISAM.  It might have had to do with either the recovery option, or maybe the pattern of many more writes than reads.

Larry Baker
US Geological Survey
650-329-5608
ba...@usgs.gov


Reply all
Reply to author
Forward
0 new messages