Adding results to OP2 file

[Philip Moseley]

Hello pyNastran users,

I'm interested in writing a script that would perform the following steps:

1. Import an existing OP2 file with results in several subcases
2. Perform some processing on those results across subcases to calculate a new dataset of scalars
3. Perform one of the following 2 options:
   1. Add the new results to the existing OP2 object and rewrite the OP2 file
   2. Write a new OP2 file with just the new result data

Steps 1 and 2 should be pretty straightforward. But looking at the examples and the codebase, I haven't found the right way to perform the 3rd step. I need the results in an OP2 file to take back to Patran to visualize (although an HDF5 would be even better).

Am I missing some user-friendly functions to add new result objects? Thanks in advance,

Philip

[Steven Doyle]

There are a few methods to create additional results, but it’s only for a few result types.  The displacement object has add_static/modal/transient_case (there may be others).  It’s much easier to just copy the structure from a small op2 and edit that.

I would definitely write it as an op2 though. I’m pretty sure Patran can’t read the pyNastran hdf5 format.

--
You received this message because you are subscribed to the Google Groups "pyNastran Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pynastran-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/c78340a8-508e-4aa3-882b-4dbbe22b9252n%40googlegroups.com.

[Philip Moseley]

Thanks Steve, as I've looked more into it this definitely seems to be the easiest approach.

Now, instead of trying to create a new OP2 file from scratch, I'm reading my MSC nastran OP2, processing the data, overwriting some of the existing data with my new calculations, and then writing out to a new OP2 file.

This is working fine with my small test files, and I'm able to open the resulting OP2s in Patran. But when I run tests on a "real" OP2 weighing in around 10gb, I'm having performance issues. Reading the OP2 is very fast (<30 seconds), but re-writing the OP2 never finishes. I killed it after a 1.5 hours, on a machine with 128gb ram and a NVMe hard drive. Obviously the first thing I tried was commenting out my calculations, at this point the script is nothing more than read_op2 and write_op2.

I'm running off of HEAD (I needed the MSC 2019 or 2020 support). I get the following trace whenever I kill during the write (always the same line). Are there known performance issues with write_op2 on larger files? It doesn't appear to be an obvious problem with my original OP2 data, Patran is able to open it without any issue, and when I print out some of the data using pyNastran it looks like it's probably been read in correctly.

 RealSolidStressArray - isubcase=1 - CTETRA

Traceback (most recent call last):

  File ".\calculate_fatigue.py", line 149, in <module>

    main()

  File ".\calculate_fatigue.py", line 97, in main

    model.write_op2(args.out_file)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 84, in write_op2

    nastran_format=nastran_format)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 133, in _write_op2

    case_count = _write_result_tables(obj, fop2, fop2_ascii, struct_3i, endian, skips)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 287, in _write_result_tables

    date, is_mag_phase=False, endian=endian)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\tables\oes_stressStrain\real\oes_solids.py", line 601, in write_op2

    j = where(eids3 == deid)[0]

KeyboardInterrupt

[Steven Doyle]

The op2 reader uses numpy’s frombuffer function to directly load the data into ints and again into floats.  A couple of reshapes later and you have the elements and results.  There’s also a ~100x slower version (for validation) that uses struct.unpack.

The op2 writer uses struct.pack, so I’m not surprised it’s slow.  I’d actually be surprised if it’s fast, though that’s kind of egregious.  I suspect it’s not the pack though and rather that weird where call at the bottom of the stack.  That feels like an N^2 error to me.

If anyone knows the corresponding method for writing as frombuffer and how to pack mixed data types, that’s the right solution...

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/9400fffb-d00a-4207-8bed-7796ddec23e4n%40googlegroups.com.

[Philip Moseley]

Yes, the slowdown appears to largely come from the "j = where(eids3 == deid)[0]" line.

I let read_op2/write_op2 run overnight on my 10gb OP2 file, and after about 13 hours it still hadn't finished.

If I create a lookup table, such as:

        eids3_lookup = dict()

        for idx,val in np.ndenumerate(eids3):

            if(val in eids3_lookup):

                eids3_lookup[val].append(idx[0])

            else:

                eids3_lookup[val] = [idx[0]]

And replace that line with "j = eids3_lookup[deid]", it finishes in 25 minutes. 30 minutes still isn't great, but at least it's starting to approach usability. That being said, I don't actually know what is in these arrays, so I'm not 100% sure this solution is correct.

Another possibility for me is to decrease the number of subcases that I write out. How would you go about deleting subcases from an OP2 object after it's been read in?

Thanks,

Philip

[Steven Doyle]

That might be useful to speed up the F06 writing.  I gotta look through it in more detail.  It's just a mapping of element id to coordinate id to account for element_cid having a different shape than element_node.  Probably the right way is to reshape it to 3d, broadcast the columns, and reshape it back to being 2d, so the shapes are the same.  I used a really dumb way to look up the coordinate id and I also did it 4 extra times (for a CTETRA and more for others).

I took a look at the OP2 though.  There's probably a better way to write the data (e.g., fewer copies), but the new way is actually vectorized for the RealSolidArray (stress/strain) and the RealTableArray (displacement, spc_forces, eigenvectors, etc.).  There's a lot more things that aren't.

I just pushed a commit.  I think I got it all correct, but it's possible I missed something.  Hopefully, it'll be <5 minutes?  Depends on the other results you have.

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/a97f4ef4-3e79-4eb1-8573-1fb8aa08e9e5n%40googlegroups.com.

[Philip Moseley]

Looks promising, but I get some shape errors with my test OP2s:

 RealSolidStressArray - isubcase=1 - CHEXA

Traceback (most recent call last):

  File ".\calculate_fatigue.py", line 145, in <module>

    main()

  File ".\calculate_fatigue.py", line 92, in main

    model.write_op2(args.out_file)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 84, in write_op2

    nastran_format=nastran_format)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 133, in _write_op2

    case_count = _write_result_tables(obj, op2_file, fop2_ascii, struct_3i, endian, skips)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\writer\op2_writer.py", line 287, in _write_result_tables

    date, is_mag_phase=False, endian=endian)

  File "C:\Users\moseley\Desktop\python_v3770\python-3.7.7.amd64\lib\site-packages\pynastran-1.4.0+dev.no.checksum.error-py3.7.egg\pyNastran\op2\tables\oes_stressStrain\real\oes_solids.py", line 616, in write_op2

    data_out[:, :4] = element_wise_data

ValueError: could not broadcast input array from shape (100,4) into shape (200,4)

[Steven Doyle]

Hmmm...that's really strange.  It works on my tests.  The code thinks you have 100 or 200 elements in your model, but it can't make up it's mind :)  Are you using SORT2?

Do you have a small example?  Like a beam with a force on it?

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/c49c997c-7a92-4283-83c2-9e55e1e59f93n%40googlegroups.com.

[Philip Moseley]

Yes, here is the mini model that generated that error. The BDF is from MSC Apex, the OP2 output is from MSC Nastran 2020.

It's 2 simple plates (1 solid, 1 shell), glued together, with a gravity load:

You received this message because you are subscribed to a topic in the Google Groups "pyNastran Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pynastran-discuss/0ePesy-y7YU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pynastran-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/CADJnEGG0C1%2B%2Ba3dGZEs-p4yvVbdy4fETeVWfC25fnhxB-8vYyg%40mail.gmail.com.

[Steven Doyle]

That one works for me.

I just ran:

>>> test_op2 .\simple_plates_nastran2020.op2 -o

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/CAKhG9Z1J-Hc%3DSXk3aa44se_WO20bXXAaV95PpnYn8Nrz-LGeXA%40mail.gmail.com.

[Philip Moseley]

Ok, great, it was an error on my end. Sorry!

So, running just read_OP2/write_OP2 works correctly for me on both that small test model and the large model. As you estimated, the large model writes ~4gb of stress-only results out in about 4 minutes. Down from 13+ hours, good for a >150x speedpu. Bravo!

It looks like my error was related to assigning a new numpy array to a pyNastran stress result, ie something like:

    model.chexa_stress[1].data = new_stress

Previously this seemed to give the correct output, but with the last pull it gave the error above. I've now replaced that line with something like:

   np.copyto(model.chexa_stress[1].data, new_stress)

And now it seems to give once again the correct output, without any errors.

In both cases the array shapes are identical, but it looks like maybe I had a reference issue.

In any case, great work, thank you!

Philip

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/CADJnEGHYk%2BDkBJR1OspznOOVX%3DxXvaX_Yfo54Yuc-0Bm2_yqyQ%40mail.gmail.com.

[Steven Doyle]

There's more to go :)

I did a lot of micro optimizations for the OP2 reading.  I probably won't do that too much, but another 2-10x doesn't seem that hard.  Before that though, there's a lot of other results that should get their 150x speed up (like real stress/strain plates & complex stress/strain solids/plates).

To view this discussion on the web visit https://groups.google.com/d/msgid/pynastran-discuss/CAKhG9Z3efYnvNkfbVNES5Joo3TcxUx1dtL54nkhpzRPdfuX5rA%40mail.gmail.com.