Embed metadata in VSP step file (open thread)

[Trevor Laughlin]

A little background...

I am working on a project utilizing VSP and I'd like to take advantage of the VSP step file capability to retain the true OML shape rather than using the Hermite file (wireframe), which is dependent on the tessellation parameters set by the user. In my application, I need to know the name of the component (as in the VSP window) and the type (wing, fuse, etc). This information is provided in the Hermite file, but not in a STEP file.

The simple idea I had to keep all the info that I require in one file was to put some type of metadata in a block comment in the step file itself. I tested this using an online step checking tool (http://www.steptools.com/translate/ap203check) and it didn't seem to throw it off. See example below.

The geometric sets that are already exported already group the surfaces into VSP components, now I just need to know the component name and type.

Before I charge down this path that (I think) will suit my needs, I wanted to open this up to any other use cases and/or ideas that might be useful to anyone else. My intention would be to implement this in the VSP source itself, and if done well enough to get it into the official release. I'm a few weeks out from starting this task, so feel free to share your thoughts.

Regards,

Trevor

[Rob McDonald]

Trevor,

Thanks for the email and for sending it to the list -- hopefully others who have similar needs will chime in.

I like the idea of using comments to embed arbitrary metadata -- it is potentially a great way to sidestep limitations of the STEP standard itself.

I don't know if you've dug into the VSP source at all yet, but we don't actually write the STEP file ourselves directly -- no fprintf or std::cout << to be found.  Instead, we rely on the STEPCode library (http://stepcode.org/) to do the heavy lifting. 

STEP files as engineers use them are actually part of a much larger set of standards.  Those standards are published as a set of specifications.  STEPCode actually is a tool for reading those specification files and auto-generating data structures and code that adhere to the spec.  So, in the build process for STEPCode, we actually compile some code, then read the spec, generate more code, and then compile that into the library we use.

From VSP's point of view, we start with a STEP class, to which we add various entities.  Most of them are boiler plate simply required to make a valid STEP Part 21 AP103 file.  The part we actually use to represent the geometry can be quite small.

So, at some point, we add a bunch of points and we add them to a surface entity and that surface is added to a geometric set, etc.  Through that process, we don't have any real control on the order that entities are output (as far as I can tell, it is the order added, but no guarantees).

Only after adding lots of entities to the STEP class, we issue the write to file command at the very end.

Enough rambling, whats the point...

I don't know if STEPCode allows you to add in comments at all -- I also don't know if it allows them to be added in a way that will deterministically end up associated with particular entities in the file.  Likewise, if you were reading the files with STEPCode, I don't know if it would allow you to read in comments and associate them with particular entities.

I would ask Mark on the STEPCode Google Group these sorts of questions before I invested too much into this approach.  It may be easy - it may be impossible - it might be something that can be supported with some reasonable modifications to STEPCode.  No matter what, that is where I would start the search for this implementation idea.

I'm not sure how you're reading STEP files -- it isn't something to be taken lightly.  You might check out STEPCode (I think it has a Python wrapper).  Otherwise, most open source projects that support STEP do so through OpenCascade -- no idea if it would support metadata through comments.

Most commercial programs that can read STEP files use one of a handful of commercial libraries provided for this purpose -- I don't know if those would support comments either.

Enough for now,

Rob

--
You received this message because you are subscribed to the Google Groups "OpenVSP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openvsp+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Trevor Laughlin]

Thanks for all the info and suggestions Rob. I appreciate your time.

At this point I'm definitely shooting from the hip with my ideas, just based on my own needs and looking at the raw step file text and a little bit of the VSP source. I'll look further into STEPCode and perhaps post the idea on their Google Group to see what their initial thoughts are.

I noticed in the VSP source method AddSurf, located in STEPutil.cpp, it creates the geometric set, and in that process assigns a name to the set, which appears to be blank. If generic names can be assigned to entities in the step file (geometric sets and Bezier surfaces in this case), perhaps this can contain the metadata? This looks like it has the potential to be done within VSP if the data can be passed to these methods and the names set. The name of the geometric set would be the component type (wing, fuse, pod, etc), and the name of each surface in the geometric set would be the VSP component name? This assumes names to not have to be unique, and this is a proper use of the STEP standard since it not would be in a commented section.

I should add that I'm not using a formal STEP file reader right now, just a from scratch tool to scan the step file for the needed entities (Cartesian points and Bezier surfaces for now), which may skew my perspective. At some I'll need to revisit this, but I didn't want to get hung up on step files yet since, as you mentioned (and I'm quickly understanding why), this isn't something to be taken lightly.

Thanks,

Trevor

PS-A little more context for those who are interested...

The application I am working on is a parametric structural modeling tool. To date I have always used the Hermite file which limits me to 1st degree curves/surfaces, and I always hated throwing out the fidelity of the true OML. Yes, the user could always increase the tessellation if they were concerned about capturing the OML, but that just adds a pre-processing requirement that I would like to do away with. For my specific application, in most cases the Hermite file is fine considering: most of the structural geometry I am defining is planar and insensitive to discontinuities in 1st degree curves/surfaces, and mesh sizes for aircraft loads models are on the order of inches and do not capture local curvature anyway (and they don't need to as long as they get the stiffness and internal load distribution right). But, since my core geometry engine is capable of higher order geometry, well...why not....

[Rob McDonald]

On Wed, Dec 2, 2015 at 6:04 AM, Trevor Laughlin <trevor.w...@gmail.com> wrote:

Thanks for all the info and suggestions Rob. I appreciate your time.

At this point I'm definitely shooting from the hip with my ideas, just based on my own needs and looking at the raw step file text and a little bit of the VSP source. I'll look further into STEPCode and perhaps post the idea on their Google Group to see what their initial thoughts are.

I noticed in the VSP source method AddSurf, located in STEPutil.cpp, it creates the geometric set, and in that process assigns a name to the set, which appears to be blank. If generic names can be assigned to entities in the step file (geometric sets and Bezier surfaces in this case), perhaps this can contain the metadata? This looks like it has the potential to be done within VSP if the data can be passed to these methods and the names set. The name of the geometric set would be the component type (wing, fuse, pod, etc), and the name of each surface in the geometric set would be the VSP component name? This assumes names to not have to be unique, and this is a proper use of the STEP standard since it not would be in a commented section.

You are correct, there are 'name' parameters in the STEP file -- even without metadata, we should modify VSP to at least carry through the component name given by the user.  We can't control if downstream programs use it, but we should provide it.

I do not think STEP will have a problem with duplicate names -- it currently allows duplicate empty strings after all.

The surfaces written to STEP files are a little more involved than you might first expect.

Vehicle (the main VSP data structure) loops over all Geoms, checks if they are in the desired Set, calls Geom::GetSurfVec, and then calls STEPutil::addSurf on each surf.

The user-supplied name is provided at the Geom level.  The Geom type also exists at the Geom level.  No STEP entity exists at that same level.

A Geom has a vector of surfaces for two reasons:

1) Many user-defined Geoms (no built-in ones yet) are constructed of multiple surfaces. PodMan, SeatGroup, and BoxGroup are examples of this.

2) Symmetrical copies, planar or radial symmetry creates potentially many surfaces.

If differentiating names for these surfaces is important, you could append a _i number to the Geom's name.

STEPutil::addSurf calls VspSurf::ToSTEP_BSpline_Quilt on every surface to convert it into a vector of STEP entities.  Those get added to the geometric set for that VspSurf.

You might be wondering how something that came in as a single surface got turned into a vector of STEP surfaces -- this is driven by the option splitsurf.  If set to true, each (presumably watertight) surface is split into (decidely not watertight) surfaces for the STEP file.  The splits are along all the feature lines you see on screen in VSP.  This option is useful to do things like separate the top and bottom wing surfaces for meshing, loads, boundary conditions, etc.

Overall, a single named Geom in the VSP Geom Browser could end up as n*m*p STEP entities.  n - multiple components in a geom, m - symmetrical copies, p - split surfaces.

The p-level all get individual names in VspSurf::ToSTEP_BSpline_Quilt.

The p-surfaces are all grouped into a geometric set -- that gets an individual name in STEPutil::AddSurf.  The geometric set is wrapped in two layers of STEP boiler plate - each one gets a name.

So, we could simply name all n*m*p surfaces by the Geom's name, or we could name_i where i=0:n*m, or we could name_i_j where j=0:p.

I don't know which CAD and analysis programs care about these names, and what they do with them.  For example, would SolidWorks keep the _j names for a bunch of surfaces within a geometric set?  Would SolidWorks use the geometric set's name, or the name from one of the layers of boiler plate that it is wrapped in?

In addition to component type name, OpenVSP also carries two flags at the VspSurf level that describe the component's function.  One flag communicates to CFDMesh - normal, negative, transparent.  Negative surfaces poke holes when CFDMesh runs -- transparent surfaces act like wakes and actuator disks.

The second flag describes another kind of intent -- wing, body, or disk.  This flag can be used by CFDMesh or DegenGeom.  Wings are things that might get a wake attached to a trailing edge.  Disk tells DegenGeom to extract the disk parameters from the shape.

Custom Geom's can set these flags differently for each surface in the Geom.  So, a combined nacelle/disk or body/fins component will be handled intelligently.

You might want to key off of these flags as they are more functionally based vs. the geom names.

Rob

 

I should add that I'm not using a formal STEP file reader right now, just a from scratch tool to scan the step file for the needed entities (Cartesian points and Bezier surfaces for now), which may skew my perspective. At some I'll need to revisit this, but I didn't want to get hung up on step files yet since, as you mentioned (and I'm quickly understanding why), this isn't something to be taken lightly.

Thanks,

Trevor

PS-A little more context for those who are interested...

The application I am working on is a parametric structural modeling tool. To date I have always used the Hermite file which limits me to 1st degree curves/surfaces, and I always hated throwing out the fidelity of the true OML. Yes, the user could always increase the tessellation if they were concerned about capturing the OML, but that just adds a pre-processing requirement that I would like to do away with. For my specific application, in most cases the Hermite file is fine considering: most of the structural geometry I am defining is planar and insensitive to discontinuities in 1st degree curves/surfaces, and mesh sizes for aircraft loads models are on the order of inches and do not capture local curvature anyway (and they don't need to as long as they get the stiffness and internal load distribution right). But, since my core geometry engine is capable of higher order geometry, well...why not....

[Trevor Laughlin]

Thanks Rob, this sounds promising. This might be a good chance for me to actually try building VSP myself and scratching the surface of the code. Hopefully I'll work out a solution and be able to make a contribution to the source.

Trevor

[Trevor Laughlin]

Update in case anyone finds this useful and/or has ideas:

Ralph Latham at ASDL has modified the VSP STEP export methods to include some useful metadata in the STEP entity names. I have tested the Windows executable he compiled with existing VSP3 models. The current format is:

'component id:component type:component name'

Component id is the unique identifier as assigned by VSP, component type is currently an integer describing general type (wing, body, etc), and component name is the name as given by the user in VSP. All metadata is currently at the component level and is colon delimited. For now this same metadata is written to the STEP entity name for each component's geometric set, surface(s), geometrically bounded surface shape representation, and shape representation relationship. An example is below:

Using the component id allows distinction between different components whose name might be the same.

This is a first cut and wanted to post this in case anyone else had a use for this and/or any suggestions. In this future we can probably use the STEP hierarchy more formally to also transfer the hierarchy of the VSP model, but what's shown serves our application for now.

I think the changes Ralph made to the source were pretty straightforward, so maybe this is something that can be incorporated into the official VSP release if there is a need? He should be able to submit a pull request on GitHub.

Thanks,

Trevor

[Rob McDonald]

Nice progress,

I've talked with other users who could use this feature, so I know there will be interest in getting it into mainstream OpenVSP.  There are a few things that I'd like to make sure you are considering.

As you've noted, STEP files have numerous levels of hierarchy.  I'm not sure which one is the best one to use for this purpose -- Ralph used the big hammer and used all of them (that might be the clever & right solution).

I don't know if that approach will be compatible with various CAD programs.  Will they complain about duplicate names at various levels?  Will they complain if different components have duplicate names?  I would suggest you create some test files and post them to ask the user community to test on as many platforms as possible.  One file could have duplicate names like you now use.  Another file could modify each name to indicate which level of STEP hierarchy it came from _A _B _C _D etc.  The latter file could tell us which CAD program pulls its component names from which level of STEP file...

OpenVSP Geom objects can result in significant multiplicity in a STEP file.  Each Geom can have symmetrical copies.  Geom's can have multiple inherent surfaces (no built-in ones do yet, but this is common for custom geoms (BoxGroup).  When Split Surfaces is selected during export, each surface will be split along all feature lines.

You should test this labeling to make sure it does what you want in all of the above (and combined) situations.  When you have duplicate symmetrical copies, do you want to uniquely identify them (_1, _2, _3)?

If you are getting your Geom type information from Geom::GetType().m_Type, I suspect it will not be very useful for your needs.  First, all custom geoms will have the same type number.  Second, a multi-component custom geom won't differentiate between components.

Instead, I suspect that you really want to identify things that are wing-like from things that are body-like.  Someone could even write a custom component that embodied both types of surfaces.  Imagine a drop tank with fins, or a propeller with a spinner...

OpenVSP handles this by another 'type' ID, this one assigned to each VspSurf.  Right now, there are three allowed types (normal, wing, disk).  This type is accessed via VspSurf::GetSurfType() and is intended to give downstream analysis programs a hint of how to treat that particular surface.  More types can be added to meet the needs of other downstream analysis programs.

I do think that this feature (done right) will be useful to most users, but since CAD programs have such different expectations when reading STEP files, I think this feature will always need to be optional like the other STEP file export options.

Rob

[Trevor Laughlin]

Thanks Rob.

We've considered your feedback and tweaked our first cut to leverage more metadata like the surface type.

• VSP Geom metadata will be contained in the geometric set STEP entity in the form of 'geom id:geom type:geom symmetry:geom name'.
• VSP surface metadata will contained in the b spline surface with knots STEP entity in the form of 'surface id:surface type'. As you suggested, this could provide a tip on how the surface should be treated.
• For now we're leaving out metadata from other entities (e.g., shape representation relationship) until we better understand how to use the STEP hierarchy to get more bang for our buck.

It's all pretty straightforward in the case of a wing or fuselage, but like you said it gets a little uncertain if you are using a custom component. For now our application is limited to "wing like" or "fuselage like" components, so if a custom component like a transport fuse is used, the user will just have to tell the program if it's a wing or fuse.

One thing I noticed that I did not expect was that separate geometric sets are created for each surface in a component that is made up of more than one surface (like the PodMan or Seat component). At first  I expected a single geometric set referencing multiple surfaces. We can work with this, but just not sure if that was the intended behavior. This is the reason we are trying to add a 'geom symmetry' flag in the metadata to determine if geometric sets with the same metadata are symmetric copies (True) or another surface in the component (False). I made the assumption that there is a symmetry flag that can be accessed in the VSP source at the Geom level.

Thanks again for all your feedback.

Trevor

[Rob McDonald]

The STEP standard itself doesn't seem to dictate anything about what each level of hierarchy should be used to represent.  My understanding is that various CAD programs use them differently.

The choices I made in OpenVSP were the simple ones to make at the time.  For the most part I was following this AP203 Recommended Practices document:

http://www.steptools.com/support/stdev_docs/express/ap203/recprac203v8.pdf

Changing around some of the layers shouldn't be very difficult in code.  If another scheme makes more sense in most CAD and meshing programs, then I'm certainly happy to change our approach.

Geom has m_SymPlanFlag, m_SymAxFlag, and m_SymRotN.  These are best understood by reading 

Since we allow multiple levels of symmetry, these are bitflags as defined in APIDefines.h.

Geom::m_SurfIndxVec and Geom::m_SurfSymmMap might be useful to you.  They're set up in Geom::UpdateSymmAttach().

Rob

[Charlie Stirk]

FYI, STEP has a recommended practice for User Defined Attributes (UDA), their metadata, and sets of UDA.  

https://www.cax-if.org/documents/rec_prac_user_def_attributes_v14.pdf

UDA can be assigned to parts, or a section of the part shape (only solids and surfaces).    

CAD and translation vendors record the STEP capabilities of their implementations here.

https://www.cax-if.org/vendor_info.php?file_id=1&comp_id=3

OpenVSP could use UDA for the surface metadata, and then it would be available to downstream applications that conform to the CAX-IF recommended practices.   

[Trevor Laughlin]

Thanks Charlie, this is really great info.

As of now we're exporting the metadata to the name of the geometric set entity for each VSP component as a json formatted string. Although, as things mature I see this as the appropriate way forward.

Thanks,

Trevor

[Rob McDonald]

Charlie,

Great information, thanks very much.

Now that I know there is a recommended way to add user data to STEP, anything that gets merged into the main OpenVSP release will be based on that approach.

I had found the cax-if.org website before, but hadn't been able to find anything particularly useful.  Both of your two links are better than anything I found on my own.  I'll have to go back and do some deeper digging.

I wish there was a lower barrier of entry way to utilize the Implementor Forum.  A paid membership is too much for an open source project like OpenVSP that only writes STEP files -- and uses a library for that.  Perhaps we should look into a STEPCode membership that can then be shared somehow between users of STEPCode...

Rob

[Trevor Laughlin]

Hi Rob,

Any chance the version we have implemented (the json string) could be incorporated into main release? Maybe with an option to turn on/off on the VSP export screen? I agree Charlie's suggestion is definitely the way forward, but in the short-term it would serve our purpose. We have tested models in commercial tools (HyperMesh and SolidWorks) and haven't seen any problems (although we had issues when we didn't split the surfaces, but I don't think that is a VSP issue). I think the changes to the code were minimal, but I didn't do the work so I can't provide details.

Regards,

Trevor

[Rob McDonald]

Trevor,

You guys can do whatever you need with the versions you are using for
research and development. If you are developing on VSP, rebasing a
git branch and recompiling when new versions are released should take
very little effort.

If only your team is using this feature, there isn't much reason to
include it in the main OpenVSP release. My reason to include it in
the main release is because we want to make the feature available for
others to use.

On the other hand, if we release it with the name-based metadata and
other people start to count on it, then it becomes something that I
need to support long term. From my point of view, there isn't really
such a thing as a short-term decision.

The modifications to add the metadata to names should have been pretty
minimal. I haven't finished studying the recommended practice
document, but the changes to implement metadata with that approach
should only be slightly more complex.

Rob

[Trevor Laughlin]

Thanks Rob, this sounds like a good plan.

I think the UDA feature could be useful for many users. When I was scanning the document provided by Charlie I was envisioning a "UDA Form" in VSP where the user could select typical metadata per component (id, name, type, symmetry, etc.) or even surface and also define their own custom UDA and maybe link it to a VSP parameter. Not sure what this would take, but it may give me a reason to dive into the code and attempt some development work myself.

Thanks for all your feedback on this topic.

Trevor

[Rob McDonald]

We've thought about adding a general attribute capability to OpenVSP a
few times. Generally speaking, it would be a user-accessible (GUI and
API) mechanism for associating name-value pairs with components and
also subsurfaces.

We also have a name-value subsystem in the Results Manager that helps
unify accessing the results from various analysis runs via the API.

I wouldn't slap this project on as a requirement for adding metadata
to STEP files. I think you guys can work with something more modest.

Using the 'name' fields to embed the component names is obviously a
good thing. Perhaps one of the recommended practices can give
guidance as to which level of the hierarchy we should target when
naming a component.

Using the STEP UDA RP stuff seems like the way to go to identify
whether a given surface is a body- or wing-type surface and other
general information.

It can probably be used to identify the symmetry ancestry of a
component - and that information is generally useful enough that it
makes sense to add it for everyone. However, there may also be a
structural way to imply that information to the STEP file. It seems
like all those levels of hierarchy ought to be useful for something...

If you want to embed metadata that is specific to your application,
you can always make it policy that users embed that metadata in the
component names. For example something like "Wing::metadata". No
code needed, just policy.

Rob

[Charlie Stirk]

Re: symmetry of components, hierarchies and names

The STEP recommended practice for external references and the PDM schema usage guide from the CAX-IF describe how STEP handles assembly bill of materials, and relating part shapes to assembly components.   

https://www.cax-if.org/documents/rec_prac_ext_ref_v21.pdf

https://www.cax-if.org/documents/pdmug_release4_3.pdf

The CAX-IF is working to combine these documents into an updated series of recommended practices, but the drafts are only available to CAX-IF members.   

[Trevor Laughlin]

I opened a pull request (https://github.com/OpenVSP/OpenVSP/pull/99) that adds the component name to the geometric set entity during STEP file export. Like you said, users can add names with data already included in their own format (e.g., Wing::Type::SomethingElse) and parse that string as needed. For me, just the name is all that is needed and I can get some extra "reference geometry" I had also embedded in the STEP file from the new airfoil export file.

I'm open to suggestions to put the name somewhere else (e.g., shape representation?), but it seems like a good place to start and some low-hanging fruit to get some nice functionality.

Regards,

Trevor

[Rob McDonald]

Thanks Trevor,

It looks like the STEP related changes are in two commits (one simply
adds quotes omitted from the first one) -- and the rest are appveyor
commits (with a tiny bit of crossover).

If the history is simple, I can review as-is, but often history that
has been merged multiple times -- in the middle of development -- is
too complex to follow. My preference is for a clean rebased branch --
one per topic. I think we're OK here.

For IGES, I went forward with a build-it-yourself label. It may be
overkill, but it allows enumeration of (split) surfaces -- or just
names -- or ID's. It also allows user-specification of a delimeter
(just in case some tools don't like certain chairs in their names). I
would like to mirror this as closely as possible for STEP -- not that
it can't be improved upon, but that I think they should match
(whatever they are).

The IGES stuff starts here:
https://github.com/OpenVSP/OpenVSP/blob/master/src/geom_core/Vehicle.cpp#L2793

and continues here:
https://github.com/OpenVSP/OpenVSP/blob/master/src/util/VspSurf.cpp#L2254

You would probably want to copy/paste IGESOptionsScreen.* and make
required modifications for STEP.

Rob


On Wed, Apr 11, 2018 at 1:07 PM, Trevor Laughlin

[Trevor Laughlin]

Hi Rob,

Thanks for info. It was on my TODO list to go back and clean-up that PR and remove the appveyor portion. On my fork it was building fine but for an unknown reason it now fails and not stable enough for a PR. Appveyor seems to be quite touchy.

So, I'll rebase and get a clean commit with just adding the component name to the STEP file as a starting point. I'll look at the IGES format and see if I can match it or at least figure out a consistent scheme. In my own fork I piled in a bunch of metadata into a json formatted string and made it optional on the STEP export screen, but I don't think that is the best approach for general use.