long FLUDAG runs fail

[dagmcPractice]

Hi everyone ,every time I try to run FLUDAG for big time periods, the run seems to stop without any error.

This problem makes proccess keep running without actually simulating something.

you can see it in the picture attached.

any help will be appriciated.

thank you

[Andrew Davis]

Does it work for shorter time periods ok? Are you running on a shared system that has job time limits? Can you share an input? Can you share any outputs?

--
You received this message because you are subscribed to the Google Groups "DAGMC Users and Collaborators" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dagmc-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dagmc-users/40145301-4cb5-4666-a27b-d5dd7b46bda2n%40googlegroups.com.

[dagmcPractice]

Thanks for the prompt reply.

Yes when it runs with 10k particles it works just fine. Sometimes we can get it work for longer periods of 30 minutes and the run finishes with no issues, but usually the run goes idle earlier, and strangely it always happen at full round minutes, as you can see from the attached picture.

We run on a local machine with 60 or so cores and there is no job time limit as far as I know.

The idle processes don't show any errors, and there is no corresponding error message in the output.

Thanks

ב-יום רביעי, 1 בדצמבר 2021 בשעה 19:38:03 UTC+2, Andrew Davis כתב/ה:

[Andrew Davis]

Could you attach the *.out and *.err files from a run that works ok and a run that fails? There maybe some useful information in there. If you run ulimit -a that will tell you if you do indeed have any time limits currently for your $USER. I agree, Its very odd that its indeed that it seems to be an integer number of minutes.

Is your geometry watertight, i.e. have you run make_watertight and check_watertight to ensure that the geometry is properly sealed. What particles are you running and what energy ranges, just to give me a feel for the likely amount of secondaries.

To view this discussion on the web visit https://groups.google.com/d/msgid/dagmc-users/77e5f64b-8d88-4fab-86ac-0c4a93ec5e96n%40googlegroups.com.

[Andrew Davis]

Another thing that could be done, it to build the DAGMC stack (but DAGMC specifically) in debug mode (-DCMAKE_BUILD_TYPE=DEBUG) and then when you do run (it will run much slower) you will be able to see stopped jobs, and attach gdb to that process and debug from there to see what is going on. However, it would be worth doing what I said above, before this as it could be an oversight in the pre-processing or the input deck.

[dagmcPractice]

Thanks for your help.

I'm attaching two example output files that we are getting, one for a cycle that finished ok (working.out) and one for a cycle that got stuck in interrupted sleep mode (not_working.out).

We also tried your suggestion with the debug flag, and it seems that the process got stuck in a lock state and the debugger prints out this:

Single stepping until exit from function __lll_lock_wait_private,

I'm not sure I know how to get meaningful information using gdb

Thanks

ב-יום רביעי, 1 בדצמבר 2021 בשעה 22:44:43 UTC+2, Andrew Davis כתב/ה:

[Andrew Davis]

Hi

Can you confirm that your geometry is watertight? If you can share the h5m I can do some diagnostics at this end.

Thanks

Andy

To view this discussion on the web visit https://groups.google.com/d/msgid/dagmc-users/0fe1be9f-059c-4427-bd26-e62cfca2d45an%40googlegroups.com.

[Andrew Davis]

The *.log files might have something useful too?

[dagmcPractice]

:attaching the relevant information from the .log file 

and the h5m file

sh: -c: line 0: syntax error near unexpected token `&'
sh: -c: line 0: `flair --help | grep -i web >>& .flair.check'
sh: -c: line 0: syntax error near unexpected token `&'
sh: -c: line 0: `flair --help | grep -i version >>& .flair.check'
Building acceleration data structures...
Time to initialise the geometry0  

ב-יום שני, 6 בדצמבר 2021 בשעה 16:31:23 UTC+2, Andrew Davis כתב/ה:

[Andrew Davis]

Hi

I don't think that geometry corresponds to the geometry that you attached previously? The geometry you sent here has two volumes in it, but your previous FLUKA outputs indicate 28 ASSIGNMA cards, which means that there should be 28 volumes. It also looks like you don't have a blackhole assigned to any volume, which will definitely cause problems. Quite possibly leading to the bevhaviour you're seeing, where low numbers of particles leads to no leakage, longer runs will lead to particles trying to go on forever.

Thanks

Andy

To view this discussion on the web visit https://groups.google.com/d/msgid/dagmc-users/251cdcb5-de96-4403-ba41-d7f7385b0ab0n%40googlegroups.com.