Exodus File Output Incorrect

[Shane Keniley]

Hey all,

I'm getting some strange output file behavior that I don't really understand. Sometime yesterday I was adding an ADInterfaceKernel to handle surface charge and everything was working fine, but after rerunning a test I noticed that the exodus file was suddenly different. Auxiliary variables don't seem to be output to the exodus file correctly. For example, in my DBD test case I have three regions: block 0 (left dielectric), block 1 (gas region), and block 2 (right dielectric), and I have an auxiliary variable "x" and "Efield" which each exist in all three regions. "x" just stores the spatial coordinate for plotting cell data, and Efield is self explanatory. Obviously things like electron temperature and current density only exist in the gas region, but when I try to look at the file in Paraview, I do not see "x" as a variable that can be plotted or used in the X Array Name field, and I see variables like e_temp and tot_gas_current in the dielectric regions even though they are explicitly defined in block = 1 in the input file.

The values of these misplaced variables are always zero, as far as I can tell. Depending on the order of the AuxVariables in the input file I sometimes miss some variables in any of the three blocks. In fact, the way I noticed this issue was that I wrote a python script to extract Exodus variables and plot them, and the aux variable Efield was not found. I mention this because I think it means this isn't a paraview issue. I also tried entering print statements in the relevant AuxKernels and they all seem to be accessed normally. The simulation runs normally

I'm getting this issue on both Ubuntu 18.04 and macOS Catalina (10.15.3). Since seeing this issue I've updated conda (conda update --all), pulled the most recent master branch for Moose, ran "make clobberall" to make sure I wasn't using deprecated links, and recompiled everything on both operating systems. It does not seem to have an effect. I honestly do not remember what I did when this change came about; I was playing around with ADInterfaceKernels and ADMaterials and never touched the Moose source code. This behavior can be seen on my dielectric_charge branch in the /tests/surface_charge/argon_dbd.i file and its output.

Was there some recent update to moose that affected how variables were plotted? I've never seen an issue like this before. It doesn't seem to affect nonlinear variables at all so I can extract the data and reconstruct the number densities, efield, etc. if necessary, but this will be an issue for an upcoming PR because the gold exodus files will be unpredictable if this continues (unless I exclude all AuxVariables I suppose). I'd like to fix this but I'm not sure where to even begin looking.

-Shane Keniley

[Alexander Lindsay]

Have you tried running valgrind on any of the input files that demonstrate this odd behavior?

--
You received this message because you are subscribed to the Google Groups "zapdos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/7e47916b-d3e9-4518-b920-37e7adcf64c6o%40googlegroups.com.

[Shane Keniley]

Running with: 

mpiexec -n 2 valgrind --leak-check=full --log-file=dbd_test_log ../zapdos-opt -i argon_dbd_testcase_ad.i

I get the attached log file. I'm not well versed in using valgrind but it says 0 bytes were lost, so I'm guessing it did not detect a leak anywhere. This issue also affects other tests, but not all. It looks like the files that include multiple blocks are the issue. I've been adding and subtracting AuxVariables from one of the input files for a while now and I don't really have any consistent behavior here, but the order of the AuxVariables in the input file is affecting which variables get mixed up. Really confusing.

To unsubscribe from this group and stop receiving emails from it, send an email to zapdos...@googlegroups.com.

[Alexander Lindsay]

I'm not worried about leaks. I'm more worried about uninitialized values or invalid read/writes. Did you see any of those errors?

It's not quite clear to me; do  you get random behavior for a given input file *run in serial* from run to run? That kind of thing can only be explained by a valgrind-type error.

If you have an input file that gives bad behavior every time you run in serial, please share or push to a branch.

Alex

To unsubscribe from this group and stop receiving emails from it, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/1d8b649c-86b3-4a5c-afef-42af64d5a02bo%40googlegroups.com.

[Shane Keniley]

Sorry, I know this isn't very clear.  I don't see any uninitialized values or invalid read/writes. This issue occurs in serial and parallel. 

To clarify, the output isn't random. For a given input file the same output file will be generated every time I run it. The issue is that it's not saving the auxiliary variables properly, but it's saving everything consistently. If I try to apply an AuxVariable (let's say 'Test') in block = 2, in paraview looking at Cell data in block 2 I'll see Ar_density (which only exists in block = 1). If I add another AuxVariable ('Test2') to block = 2, in paraview I'll see 'Ar_density' and 'Ar*_density' instead of 'Test1' and 'Test2'. When I said random I meant that I cannot figure out why one variable is replaced with another, but the output is consistent between runs unless you add or subtract AuxVariables.  

I already uploaded one relevant input file argon_dbd.i in the tests/surface_charge/ directory of my dielectric_charge branch. This also affected other input files though. The 1d_dc/1d_dc test fails on this branch, and it looks like I'm seeing the same issue (aux variables that exist in the gold file do not exist in the newly created output file). I think the issue is isolated to simulations that utilize multiple blocks.

To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/1d8b649c-86b3-4a5c-afef-42af64d5a02bo%40googlegroups.com.

[Alexander Lindsay]

So if you threw up a PR we would see failures on CIVET for the 1d_dc case?

I’ll try to take a look sometime this week

On Jun 16, 2020, at 5:35 PM, Shane Keniley <keni...@illinois.edu> wrote:



To unsubscribe from this group and stop receiving emails from it, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/71806da1-ae1a-4436-afc4-dbbb5d6386aco%40googlegroups.com.

[John Haase]

I'm pretty sure that I saw this error as well and it was related to declaration order in the input file

[Shane Keniley]

Yeah, it definitely is affected by input file order. That's interesting. Do you recall what the proper order should be? 

[John Haase]

Sorry I don't, this is dragging up stuff from 2-3 years ago

--
You received this message because you are subscribed to a topic in the Google Groups "zapdos-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zapdos-users/qVFEcsDuDP8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/98eac958-472d-47be-8f6e-5b590ca50eeeo%40googlegroups.com.

[Shane Keniley]

So I've been ignoring it for a while, but I am still having this issue. The only thing I've seemed to figure out is that it really only occurs when I have multiple blocks. 

I've uploaded a small example here in a new "aux_fail" branch: 

https://github.com/keniley1/zapdos/tree/aux_fail/problems/aux_test

On both of my computers (Ubuntu 18.04, Mac OSX Catalina) I'm getting the same result: the simulation runs without error, but when I view the output file in Paraview I see the AuxVariables in the wrong blocks. In this example I see emliq_density and H2O assigned to block 0, even though they are very clearly defined in block = 1 in the input file. Input file order does seem to have some kind of effect, but I haven't found an order that fixes the issue. Only AuxVariables are affected; nodal variables seem to be just fine. Using either print statements or a debugger I can examine the blocks where each kernel is applied and everything seems to be perfectly fine during the simulation, e.g. the kernel for emliq_density is clearly being used in block 1. It's only being output to the wrong block. 

I really have no clue how what to do about this. Any advice would be appreciated. Should I post this on the main Moose user group as well, maybe? 

[Alexander Lindsay]

I suspect that this is a MOOSE issue. Best case scenario would be if you could reproduce this with a MOOSE only input file and then submit a GitHub issue. However, if that’s too difficult to do, then you can just try reposting on the moose mailing list. However, I can’t guarantee that anyone other than me will respond over there haha. 

On Aug 14, 2020, at 4:56 PM, Shane Keniley <smkfi...@gmail.com> wrote:

So I've been ignoring it for a while, but I am still having this issue. The only thing I've seemed to figure out is that it really only occurs when I have multiple blocks. 

--

You received this message because you are subscribed to the Google Groups "zapdos-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/65447220-7817-41fd-90c8-8829ffbfc535n%40googlegroups.com.

[Shane Keniley]

Might be a dumb question, but how can I update libmesh locally? Would I need to go through the manual installation steps for libmesh, or is there an easier way? I tried updating the code in the submodule (/projects/moose/libmesh) and then running the update_and_rebuild_libmesh script, but that didn't seem to work.

I only ask because your PR still hasn't gone through (I guess it's waiting on CIVIT?). If that will still be hanging out in purgatory for a while, I'd like to at least fix this locally so I can have access to AuxVariables again. 

To unsubscribe from this group and stop receiving emails from it, send an email to zapdos...@googlegroups.com.

[Alexander Lindsay]

The update and rebuild libmesh script will run “git submodule update” so it will wipe out any libmesh commits you haven’t checked into moose. Run “update_and_rebuild_libmesh.sh —skip-submodule-update” to avoid this. 

On Sep 2, 2020, at 3:02 PM, Shane Keniley <smkfi...@gmail.com> wrote:



To unsubscribe from this group and stop receiving emails from it, send an email to zapdos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zapdos-users/eeac51a8-0734-4083-bea2-81aaeed57639o%40googlegroups.com.