Example 5a (Beam elements and Solid elements together) not running

[Samyog]

Hello everyone,

I had started a discussion to resolve a similar problem.

https://groups.google.com/d/msg/mastodon-users/R-mhtd_o-UA/Hh9plPvNAAAJ

It is still not resolved. I am assuming that some MOOSE modules are not compiled in my local server.

Link to Example 5a is https://mooseframework.inl.gov/mastodon/examples/example5a.html

A similar problem was encountered by a friend who has access to INL's HPC. He had updated MASTODON on HPC yesterday (May 13). Upon running the tests, 148 out of 150 passed the test but two of them failed. I have attached screenshots of the failed tests and the error while running example 5a (shared by him). Does anybody know about this or has anybody else faced this problem?

Thank you,

Samyog

 

[Chandu Bolisetti]

Hi Samyog, 

This is definitely quite odd. Is this happening both in HPC and on your local machine? 

Chandu

[Mriganabh]

Hi Dr. Chandu,

I updated Mastodon on HPC yesterday and 2 out of 150 tests failed as shown in the image attached in samyog's question. I have attached a screenshot of the git log.

Thanks,

Mriganabh

[Chandu Bolisetti]

How old is your moose? Can you run 

git submodule update moose

And then update libmesh by going to moose/scripts and running 

./update_and_rebuild_libmesh.sh

Then compile mastodon. 

Chandu

[Christopher Wong]

I've seen these time out failures on the isolator objects before too, specifically, when I am testing in HPC. Run the tests in serial, and I believe you won't get those failures anymore. 

I'm not exactly sure what the time out failures indicate, but I know they have nothing to do with the test's specific success criteria. When it comes to HPC clusters, theres no guarantee of full compatibility since you aren't the admin, especially when you request multiple cores.  

On Thursday, May 14, 2020 at 4:30:12 PM UTC-6, Chandu Bolisetti wrote:

How old is your moose? Can you run 

git submodule update moose

And compile mastodon again? 

[Mriganabh]

Hi Chandu and Christopher,

I am doing what you recommended. I will also recompile Mastodon using only 1 core. However, I had done something very similar by following the instructions to update Mastodon. The set the commands were - 

---------

git fetch origin

git rebase origin/master

git submodule update

./moose/scripts/update_and_rebuild_libmesh.sh

--------

Followed by following commands to compile - 

--------

make -j 4

./run_tests -j 4

---------

[Mriganabh]

Hi Chandu and Christopher,

Chandu: I am still facing the same error after doing what you recommended. 

Chris: I tried compiling mastodon in serial with only 1 core. This time 143 of them passed but 27 were skipped (as shown in the attached in the image)

Thanks,

Mriganabh

[Chandu Bolisetti]

Hi Mriganabh, 

Looks like Chris was right. Note that skipped tests are not always an indicator of a problem. 

In any case, it looks like you can proceed with running MASTODON on HPC in general. I am still not sure why the isolator test was failing, but if you have a specific problem while running your own models, we can look into that. 

Best regards,

Chandu

[Mriganabh]

Hi Chandu,

I am still facing the same problems while running the example 5a which has both beam & solid elements. The initial accelerations are computed but at the very first time step, I get the error (as shown in the attached image). Mastodon seems to be up-to-date on my HPC account. 

Do you know why is this happening?

Thanks,

Mriganabh

[Chandu Bolisetti]

Gotcha..is this happening for all examples with beam elements then? What about 5b or example 9? 

Also is this happening only for multiple processors or on serial too? 

Chandu

[Mriganabh]

Hi Chandu,

I ran example 5b just now but the simulation aborts due to non-convergence. However, there is no such error as "out of memory trying to allocate -xx bytes". 

As for example 9, I ran the SSI_Response input file and it works fine.

Thanks,

Mriganabh

On Friday, May 15, 2020 at 11:29:47 AM UTC-4, Chandu Bolisetti wrote:

Gotcha..is this happening for all examples with beam elements then? What about 5b or example 9? 

[Christopher Wong]

Mriganabh, I think you should submit an actual job request when you run these files. HPC clusters will usually allow you to run small executions with no official job ticket, but, again, then there's no guarantee of the resources they'll allocate for you in this case.

Also, running those examples isn't so expensive, I don't see a need for supercomputing them. At the very least, you should run them on your local machine as well so you can identify wether the problem is with your MASTODON clone or if it's with your HPC node. All of those examples have been tested and verified, so I'm going to guess your problem is the latter.

[Mriganabh]

Hi Chris,

We were facing the same error while running the example - 5a on our local server, that's the reason I tried it on HPC. Since the error was identical on my local server and HPC for example - 5a, I presumed there must some module which needs to be installed in order to run problems with both solid & beam elements.

Thanks,

Mriganabh

[Chandu Bolisetti]

Mriganabh, 

There are no special modules to run beam elements. If you have MOOSE, example 5a should run fine. It looks like we're running into issues specific to the OS. Let's try to debug this on your local machine then. What OS (including version) are you on? I suggest you run the MOOSE tests and tensormechanics tests and send us the results. Here's how you do it: 

MOOSE tests:

cd moose/test

make -j 8

./run_tests

Tensormechanics tests: 

cd moose/modules/tensormechanics/

make -j8

./run_tests

Please send the log once you run these tests. 

Chandu

[Som Dhulipala]

Hi Mriganabh,

As Chandu suggested, please run the MOOSE and tensormechanics tests before running MASTODON tests. Before running any tests, please do the following:

First, let's get MOOSE running:

- Goto MOOSE folder

- Save your work on the current branch (git add. then run git commit -m "Message")

- create a new git branch in MOOSE and run git clean -dxf

-  fetch the latest MOOSE version from MOOSE/devel (git fetch remote devel if your remote is set to MOOSE)

- update and rebuild libmesh (JOBS=8 ./scripts/update_and_rebuild_libmesh.sh)

- Then goto MOOSE/test and run make -j8

- JOBS=8 ./run_tests

Now let's get tensormechanics running:

- In MOOSE folder goto modules/tensormechanics

- make -j8

- JOBS=8 ./run_tests

Finally, let's get MASTODON running:

- Goto MASTODON folder

- Save your work on the current branch (git add. then run git commit -m "Message")

- create a new git branch and run git clean -dxf

- fetch the latest MASTODON version from MASTODON/devel (git fetch remote devel if your remote is set to MASTODON)

- run make -j8

- JOBS=8 ./run_tests

Hopefully, this helps.

--
https://mooseframework.org/mastodon
https://github.com/idaholab/mastodon
---
You received this message because you are subscribed to the Google Groups "mastodon-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mastodon-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/f4a4edc7-e6e8-4258-9b11-02f9cb9a271b%40googlegroups.com.

--

Som Dhulipala, Ph.D.

Postdoctoral Research Associate,

Facility Risk Group,

Idaho National Laboratory

[Mriganabh]

Hi Chandu and Som,

Unfortunately, I do not have administrative permissions on the local server where I have Mastodon installed. I have requested the team with the permissions to run the tests. In the meantime, I decided to run these tests on my HPC account and see if we can figure the cause of the issue. 

I have the results of the tests and also the OS on HPC.

Thanks,

Mriganabh

To unsubscribe from this group and stop receiving emails from it, send an email to mastodo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/f4a4edc7-e6e8-4258-9b11-02f9cb9a271b%40googlegroups.com.

[William M. Hoffman]

Mriganabh, it looks like your only failures were timeout failures. So the code is working fine, those tests just ran out of time.  I think mastodon is working as expected on your machine.

 

I am not sure what the problem with example 5a is. I am unable to run ex05a in parallel. I am not sure why. It works fine if I do not use mpi. Is that the case for you?

 

Will Hoffman

 

--
https://mooseframework.org/mastodon
https://github.com/idaholab/mastodon
---
You received this message because you are subscribed to the Google Groups "mastodon-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mastodo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/f4a4edc7-e6e8-4258-9b11-02f9cb9a271b%40googlegroups.com.

 

--

Som Dhulipala, Ph.D.

Postdoctoral Research Associate,

Facility Risk Group,

Idaho National Laboratory

--
https://mooseframework.org/mastodon
https://github.com/idaholab/mastodon
---
You received this message because you are subscribed to the Google Groups "mastodon-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mastodon-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/4514fe6f-27e9-4c5f-9f23-abc33c1f73bf%40googlegroups.com.

[Mriganabh]

Hi Will,

That's right. I tried running the example 5a without mpirun and it works fine. Looks like there is some issue with running ex05a in parallel.

Thanks,

Mriganabh

To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/4514fe6f-27e9-4c5f-9f23-abc33c1f73bf%40googlegroups.com.

[Chandu Bolisetti]

Hi Mriganabh,

I have the same issue with example 5a. But example 5b and example 9 run just fine both on serial and parallel. Example 9 has both solid and beam elements, so I don't think the problem is with having solids and beams together. I am trying to see what's going on with 5a and keep everyone posted. You should proceed with your other simulations regardless. Let us know if you have any other issues. 

Chandu

[Mriganabh]

Hi Chandu,

Thank you for the clarification.

Mriganabh

[Som Dhulipala]

Hi Mriganabh,

As Chandu pointed out, the issue with example 05a is rotational degrees of freedom are being assigned to solid elements. I removed these dofs and created a PR in MASTODON.

If you want to run 05a, remove the block 15 from all rotational dofs in blocks variables and auxvariables.

Som

To unsubscribe from this group and stop receiving emails from it, send an email to mastodon-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/9125c24c-c239-41f4-a194-ab9412fb9c7d%40googlegroups.com.

[Chandu Bolisetti]

Here's the PR. It should be merged today and Ex05 should work fine once you update mastodon tomorrow. 

https://github.com/idaholab/mastodon/pull/301

I am marking this as complete now. Please let us know if you have other issues.

To unsubscribe from this group and stop receiving emails from it, send an email to mastodon-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mastodon-users/9125c24c-c239-41f4-a194-ab9412fb9c7d%40googlegroups.com.