Using over of the memory

[許銘凱]

Dear Admin,

  I run the openquake to calculate the hazard curve and hazard map. It shows that using over of the memory.I have no idea why it show the message and how to solve the problem.

And can you tell what is meaning of the MainProcess ?

Thanks

kensheu@kensheu-VirtualBox:~/oq-engine$ ./bin/oq-engine --nd --rh=./CGS141029_m0_5.0/job_500.ini 

[2015-02-17 10:41:47,995 hazard job #71 - PROGRESS MainProcess/2137] **  pre_executing (hazard)

[2015-02-17 10:41:48,124 hazard job #71 - PROGRESS MainProcess/2137] **  initializing sites

[2015-02-17 10:42:10,154 hazard job #71 - PROGRESS MainProcess/2137] **  initializing site collection

[2015-02-17 10:42:10,191 hazard job #71 - PROGRESS MainProcess/2137] **  initializing sources

[2015-02-17 10:42:10,366 hazard job #71 - INFO MainProcess/2137] Submitting task _filter_sources #1

[2015-02-17 10:42:11,086 hazard job #71 - INFO MainProcess/2137] Sent 3M of data

[2015-02-17 10:42:11,087 hazard job #71 - INFO MainProcess/2137] spawned 1 tasks of kind _filter_sources

[2015-02-17 10:42:11,087 hazard job #71 - INFO MainProcess/2137] _filter_sources 100%

[2015-02-17 10:42:11,087 hazard job #71 - INFO MainProcess/2137] Received 0M of data

[2015-02-17 10:42:11,087 hazard job #71 - INFO MainProcess/2137] Considering 4 of 4 sources for model CGStaiwan_source_model-141027.xml('b1',), TRT=Interface subduction zone

[2015-02-17 10:42:11,087 hazard job #71 - INFO MainProcess/2137] Submitting task _filter_sources #1

[2015-02-17 10:42:13,123 hazard job #71 - INFO MainProcess/2137] Sent 3M of data

[2015-02-17 10:42:13,123 hazard job #71 - INFO MainProcess/2137] spawned 1 tasks of kind _filter_sources

[2015-02-17 10:42:13,123 hazard job #71 - INFO MainProcess/2137] _filter_sources 100%

[2015-02-17 10:42:13,123 hazard job #71 - INFO MainProcess/2137] Received 0M of data

[2015-02-17 10:42:13,124 hazard job #71 - INFO MainProcess/2137] Considering 12 of 12 sources for model CGStaiwan_source_model-141027.xml('b1',), TRT=Intraslab subduction zone

[2015-02-17 10:42:13,124 hazard job #71 - INFO MainProcess/2137] Submitting task _filter_sources #1

[2015-02-17 10:42:23,579 hazard job #71 - INFO MainProcess/2137] Sent 3M of data

[2015-02-17 10:42:23,579 hazard job #71 - INFO MainProcess/2137] spawned 1 tasks of kind _filter_sources

[2015-02-17 10:42:23,579 hazard job #71 - INFO MainProcess/2137] _filter_sources 100%

[2015-02-17 10:42:23,580 hazard job #71 - INFO MainProcess/2137] Received 0M of data

[2015-02-17 10:42:23,580 hazard job #71 - INFO MainProcess/2137] Considering 62 of 62 sources for model CGStaiwan_source_model-141027.xml('b1',), TRT=Active Shallow Crust

[2015-02-17 10:42:24,417 hazard job #71 - INFO MainProcess/2137] splitting <TrtModel #0 Interface subduction zone, 45 source(s)>

[2015-02-17 10:42:37,198 hazard job #71 - INFO MainProcess/2137] splitting <TrtModel #1 Intraslab subduction zone, 332 source(s)>

[2015-02-17 10:42:40,462 hazard job #71 - INFO MainProcess/2137] splitting <TrtModel #2 Active Shallow Crust, 2662 source(s)>

[2015-02-17 10:42:40,585 hazard job #71 - INFO MainProcess/2137] Total weight of the sources=1067260.5

[2015-02-17 10:42:40,585 hazard job #71 - INFO MainProcess/2137] Expected output size=8093690.0

[2015-02-17 10:42:41,159 hazard job #71 - PROGRESS MainProcess/2137] **  executing (hazard)

[2015-02-17 10:42:41,166 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #1

[2015-02-17 22:11:22,580 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #2

[2015-02-18 06:54:40,078 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #3

[2015-02-18 10:36:53,755 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #4

[2015-02-18 14:07:27,725 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #5

[2015-02-18 14:41:33,057 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #6

[2015-02-18 16:19:22,623 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #7

[2015-02-18 17:57:19,312 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #8

[2015-02-18 19:57:12,642 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #9

[2015-02-18 22:13:33,824 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #10

[2015-02-19 00:30:31,473 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #11

[2015-02-19 02:53:16,070 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #12

[2015-02-19 05:38:21,497 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #13

[2015-02-19 08:29:14,028 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #14

[2015-02-19 11:45:49,095 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #15

[2015-02-19 15:10:34,963 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #16

[2015-02-19 19:03:29,180 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #17

[2015-02-19 23:35:51,170 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #18

[2015-02-20 01:14:48,453 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #19

[2015-02-20 06:20:54,568 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #20

[2015-02-20 12:07:58,427 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #21

[2015-02-20 18:44:16,763 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #22

[2015-02-21 02:07:19,736 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #23

[2015-02-21 10:12:18,359 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #24

[2015-02-21 19:33:56,657 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #25

[2015-02-22 05:34:02,534 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #26

[2015-02-22 16:30:44,673 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #27

[2015-02-22 18:08:59,770 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #28

[2015-02-22 19:54:26,678 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #29

[2015-02-22 21:35:47,323 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #30

[2015-02-22 23:25:19,311 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #31

[2015-02-23 01:22:25,263 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #32

[2015-02-23 03:17:10,806 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #33

[2015-02-23 05:11:58,713 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #34

[2015-02-23 08:16:54,976 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #35

[2015-02-23 11:02:54,704 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #36

[2015-02-23 13:08:09,253 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #37

[2015-02-23 14:43:00,947 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #38

[2015-02-23 16:22:45,054 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #39

[2015-02-23 16:22:45,056 hazard job #71 - WARNING MainProcess/2137] Using over 80% of the memory!

[2015-02-23 18:12:26,385 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #40

[2015-02-23 18:12:26,386 hazard job #71 - WARNING MainProcess/2137] Using over 82% of the memory!

[2015-02-23 20:52:56,744 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #41

[2015-02-23 20:52:56,746 hazard job #71 - WARNING MainProcess/2137] Using over 83% of the memory!

[2015-02-24 01:25:27,398 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #42

[2015-02-24 01:25:27,443 hazard job #71 - WARNING MainProcess/2137] Using over 84% of the memory!

[2015-02-24 03:26:51,978 hazard job #71 - PROGRESS MainProcess/2137] **  Submitting task compute_hazard_curves #43

[2015-02-24 03:26:51,978 hazard job #71 - WARNING MainProcess/2137] Using over 85% of the memory!

[Daniele Viganò]

Dear User,

On Tuesday, February 24, 2015 at 4:47:22 AM UTC+1, 許銘凱 wrote:

Dear Admin,

  I run the openquake to calculate the hazard curve and hazard map. It shows that using over of the memory.I have no idea why it show the message and how to solve the problem.

The WARNING means that you are using more than the 80% of the memory available in your computer/virtual machine. Usually this means that the computation that you are running is too big for the hardware you have.

If you are running OpenQuake in a virtual machine (i.e. VirtualBox) you can increase the amount of RAM allocated for the virtual machine. On VirtualBox that can be done under "Settings" > "System" > "Base memory".

 

And can you tell what is meaning of the MainProcess ?

MainProcess is the oq-engine main process, sometime called also "master" process. Is the process that manage and supervise the calculation workflow (see https://github.com/gem/oq-engine/wiki/Calculation-Workflow)

Daniele

[許銘凱]

Dear Admin,

When I run the same data and job.ini in different PC, the number of the  MainProcess shows different. For example,one PC show that:

[2015-02-17 10:41:47,995 hazard job #71 - PROGRESS MainProcess/2137] **  pre_executing (hazard)

and another PC show that:

[2015-02-17 10:41:47,995 hazard job #71 - PROGRESS MainProcess/12842] **  pre_executing (hazard)

I have question about  why it show different and what is the meaning of these numbers. Is it means that the first PC will run faster than the another one? 
許銘凱於 2015年2月24日星期二 UTC+8上午11時47分22秒寫道：

[Daniele Viganò]

Dear User,

On Tuesday, February 24, 2015 at 9:09:24 AM UTC+1, 許銘凱 wrote:

Dear Admin,

When I run the same data and job.ini in different PC, the number of the  MainProcess shows different. For example,one PC show that:

[2015-02-17 10:41:47,995 hazard job #71 - PROGRESS MainProcess/2137] **  pre_executing (hazard)

and another PC show that:

[2015-02-17 10:41:47,995 hazard job #71 - PROGRESS MainProcess/12842] **  pre_executing (hazard)

I have question about  why it show different and what is the meaning of these numbers. Is it means that the first PC will run faster than the another one?

No, it's just the Unix Process ID (http://en.wikipedia.org/wiki/Process_identifier) a number that identifies the process in an unique way. The number depends on how many processes are running or did run on the machine.