Re: Open source manufacturing and design packaging
Bryan Bishop
> On Sat, Dec 13, 2008 at 11:05 PM, Bryan Bishop wrote:
>> I see. What exactly is taking up 100 GB? That's rather extreme. Are
>> these designs? And if so, of what?
>
> This is the commercial product I have been working on for the past three
> years. It's a >$1mil 3000+ piece machine in the biotech space. When I say
> the whole data vault or data set that defines the system I'm including
> solidmodels, drawings, specifications, test plans, test reports, schematics
> and board layouts, software source code, artwork, assembly work
> instructions, purchasing specifications, etc. It's very large project
> compared to say a bicycle, phone, or something on that scale, but small
> compared to a car or an airplane. Just depends.
Importantly, I think that all of those documents should be accounted
for in the type of system-wide repository packaging format that we're
both discussing. All of that should be computationally accounted for
and integrated together. Advantages of this sort of integration
platform (besides being open source and awesome like that) would be
being able to keep exact track of everything going on in the project.
My father used to work for Freescale Semiconductor, and before that,
Motorola. He has told me one too many horror stories about the
manufacturing plants and how changes on the shop floors wouldn't be
told to management and such, so nobody ever really knew the entire
state of the system. I'm sure this is somewhat the same with giant
design projects, such as the scale that NASA has to do; one wrong
measurement on a bolt, and .. it's not pretty.
>> Yes, my hopes are similiar. I'd like to see "open source hardware"
>> projects packaged just like debian packages, so that we could do
>> things like "apt-get install a phone". It's a good metaphor to pass
>> around. :-)
>
> I sort of see the metaphor but I would line the analogy up like this:
>
> Source Code (c, c++, python, etc) ~ CAD models, drawings, analysis,
> simulations, etc.
> Compiler (gcc, gc++, etc) ~ Manufacturing of the product/system.
> Binary (and src for local compile) distribution (.rpm, .deb, etc. in
> repository) ~ Buy or make an open source phone? I see you are thinking STL,
> which is sort of like compiling, but unlike software that would be usuable
> you can't make a phone call on an STL file of an OS cellphone.
> Source repository (cvs, svn, git, etc.) ~ PLM/PDM systems such as PDMWorksE,
> Wildfire, Agile, etc. (no great open source equivalent that I know of, maybe
> what you are planning is way to go?).
Hm. These analogies are difficult. The binary distribution packages
(rpm, deb, pm, etc.) sometimes also contain source code. Since open
source hardware projects are already beginning to store hardware
designs in their svn, git, cvs, mercurial repositories, I was figuring
that these would be what you would want to package for distribution.
But it's important to note that the physical instantiations are also
another layer of abstraction worth thinking about. A few days ago on
the open manufacturing mailing list, there was some discussion going
around about interoperability and international container cargo
standards (as opposed to random/various parcel sizes currently known
to the snail mail system).
Btw, there's a really awesome CAD editing program that is open source
known as BRLCAD ( http://brlcad.org/ ). It originated 20 years ago
from Muuss, the same guy who wrote 'ping'. It's completely featured,
runs on a large number of platforms, and I'm loving it. There's about
400 command line tools that are packaged with it, everything from
translating various CAD formats between each other, to being able to
build models by calling commands through the linux shell, or in the
inline shell while editing objects graphically with Tcl syntax. I'm
considering making a proposal here shortly to the open manufacturing
mailing list to include the dot g file format (what brlcad stores CAD
information in) within the repository standard.
Here's a screenshot of brlcad with the OpenMoko phone:
http://brlcad.org/gallery/s/screenshots/iges-openmoko-conversion.png.html
Re: using STL. Basically the idea is that each packed hardware project
would have metadata, the dot g file, other CAD data, but also some
scripts. For instance, if scripts were packaged to generate the CAD
data per the configuration and parameters that a user wanted (or
defaults), then that would allow STL to be generated, or really
whatever information the user needs to use. Personally I very much
want to include the ability to generate human readable instructions
for making something, instead of just 3D STL/IGES and gcode. Imagine
also being able to print out a standard set of instructions for
building whatever the hardware is: kitchen cooking recipes are a nice
example. Meanwhile there are industrial equivalents where food is
'cooked' with more computationally literate recipes. I want both forms
of recipes being generated from the same information. etc. Some of
these recipes would call for things that an amateur just won't have
laying around the house - how many of us have injection molding
equipment, for instance? I guess the amateur would have to hope that
somebody put an "injection molding machine" into the repository, plus
serializable instructions that he can understand on how to make one
for himself :-). Or from where to 'order' it or who might be able to
make him one (somebody with a fully-fledged fablab?). etc.
Re: the compiler. Ben Lipkowitz and I have put some thought into this,
and we're calling it the "matter compiler". In optimal situations this
would be like flexible manufacturing systems ("factory! reorder and
reorganize to make product #341!"), or taking what you have in terms
of machinery and tools, and then making what you need through lining
it up in the right ways. But, on another layer of abstraction, I think
there's also "software" 'compilers' going on here. If anything,
brlcad's tool suite certainly looks as complex as a compiler, and
other software packages such as an 'apt' clone, and tools to integrate
these packaged hardware projects together, will accumulate to what I
suspect might look like a 'compiler' (but not the 'matter compiler'
idea).
> Mind you I'm not rejecting what you are doing just trying to see how it fits
> in.
I'd be cautious of using conventional programming model analogies with
all of this. It might work out in the end, but it could end up hurting
us significantly by confusing too many individuals. On the other hand,
an overall architecture/roadmap wouldn't hurt either. Earlier this
year Ben and I were working on something like this, but I feel it's
largely out-dated and should be ignored:
http://heybryan.org/mediawiki/index.php/Skdb#todo.3F
(What follows is an excerpt from that page, it's mostly just
ramblings, so it's ignorable I guess.)
=========================================
stuff for userspace
autospec - validates the units for the packages, validates the yaml,
gets additional python plugins if necessary when the yaml specifies
some weird class that is not yet installed.
Are the 'python plugins' (that specify additional classes) to be
provided in package (.skdb) files? tentatively: yes. Then why wouldn't
this be specified in the metadata for the skdb file as a 'dependency'?
agx-get - gets packages from the database, gets all metadata (local
user cache in ~/.skdb/mimedb/ or something), allows the user to search
for packages, uses a sources.list type file to connect to skdb or
other metarepo installations, torrent functionality would be nice, and
must be able to process the metadata files and give the user options
to download particular parts (or everything) related to a .skdb file
-- this will typically involve downloading a new skdb package in the
first place in order to understand the skdb file in its entirety (for
example, different types of packages will most likely have different
types of metadata specified, different types of resources, with data
worth parsing on the user-end for options related to seeking out the
package-contents).
disambiguator - lets user specify what sort of file formats need to be
processed, what the input/outputs have to be, and finds puzzle pieces
that match these specs
agx-sim - the simulator
Packages (.skdb) would describe how they are to be simulated, if at
all. I am sure somebody would like to come up with a standardized
simulation interface. There are a few problems (computationally) that
have to be addressed, like whether or not it should be a binary sort
comparison, n!, or something else. Hard to tell, but the important
aspect is that the architecture allows for this sort of thing.
agx-make - considering fablab facilities (/dev stuff), configuration,
layout, orientations, inventory, number of humans available, etc.,
generate an overall recipe for the entire lab to execute and make the
product ('go command' should be avoidable via passing a parameter)
Generating the programming for the machinery will be somewhat
high-level: this would not be an immediate compiling down to "turn the
servo" sort of code, but rather something that each of the machines
with their pre-installed firmware can understand.
Generating the programming in the first place: run through each of the
(.skdb) files in the overall design (itself a .skdb), load up the
modules geared towards interpreting these variations on (.skdb) (i.e.,
some have particular data formats that can only be parsed in certain
ways), assemble a list of all code (within the packages) that is to be
called/evaluated. The code within each of the packages should 'call a
standard API' to make things happen. This API will abstract the
actual, physical hardware away from the (.skdb) python scripts so that
a variety of configurations can be used without having to go in by
hand and making hundreds of versions of (.skdb) internal python
scripts.
standard-API: all (.skdb) scripts will know how to call specific
functions and actions of the machinery, and then the 'print server'
will figure out the relationship between those functions and the
actual drivers and /dev/ while also providing garbage collecting (so
that wires aren't tripped over and so on).
Alternative to the standard-API idea: what if the python scripts in
(.skdb) files printed out code, and this code could then be compiled
for the specific fablab config that the user has?
Note: whether or not a bug report is from agx-sim, agx-make, or
autospec is irrelevant; all of this data should be fed back into skdb.
Especially user-reports, bug reports, usecases, feedback, etc.
=========================================
>> At the moment the projects that I am hosting are scattered all over
>> the place. There's an old repository at this site:
>> http://repository.designengineeringlab.org/
>
> I was wondering around what is a .cdd file?
"ConceptDraw". Proprietary crap. :-(
>> http://fennetic.net/git/gitweb.cgi (check skdb.git)
>
> I see the web based structure but it is there a way to get/check out the
> repository?
git clone http://fennetic.net/git/skdb.git
> You have certainly put much thought and effort into this I'm most impressed.
> It is exciting to see all those projects started out there, seems they are
> all still stuck that there is no OSS ProEngineer.
Btw, I'm cc'ing this to the openmanufacturing mailing list. It would
be great if you could join and we carry on the discussions there.
Bryan Bishop
> On Thu, Dec 18, 2008 at 3:47 PM, Bryan Bishop wrote:
>> Importantly, I think that all of those documents should be accounted
>> for in the type of system-wide repository packaging format that we're
>> both discussing. All of that should be computationally accounted for
>> and integrated together. Advantages of this sort of integration
>> platform (besides being open source and awesome like that) would be
>> being able to keep exact track of everything going on in the project.
>>
>> My father used to work for Freescale Semiconductor, and before that,
>> Motorola. He has told me one too many horror stories about the
>> manufacturing plants and how changes on the shop floors wouldn't be
>> told to management and such, so nobody ever really knew the entire
>> state of the system. I'm sure this is somewhat the same with giant
>> design projects, such as the scale that NASA has to do; one wrong
>> measurement on a bolt, and .. it's not pretty.
>
> management). Typically encompass or integrate PDM ("product document
> management", these are often multiple datavaults for different packages, for
> example PDMWorks for Solidworks files, SVN for software, Omnify for Orcad,
> etc. It should be noted that the lines between PLM and PDM are very grey),
> change management (ECR, ECO, ECN.), device history record (really a function
> of ERP/MRP but live date is in the PDM/PLM system), etc. MRP and ERP
> functionally strictly speaking should not be big concern but if one was to
> invision a significant project that a real company was to product
> integration woudl be critical. You are correct in that it is not pretty as
> all these software products have different (and sometime divergent) goals so
> integration is quite often manual. A good example is a manual signature and
> scanned PDF for change control on FDA develop product because double
> signature is not often integrated in the system (PDM, PLM, or ERP most of
> which have change control functionality).
Interesting. Most of these terms are familiar to me, but only from a
marketing point of view. I haven't yet dealt with any of the ERP
systems (not even Ned Lilly's (he's on the list somewhere)), even
though I've been meaning to install HERMES2, xTuple, etc. Could you
possibly help me hunt down some links for PLM/PDM, ECR, ECO, ECN,
ERP/MRP, FDA, etc. ? I am aware of only a handful of commercial
systems (SAP stuff?). Although, on the other hand, I suspect that even
if I was to find more information about these systems, it wouldn't
really be helpful since they are closed-source, proprietary and not
something I can play around with. So nevermind. :-)
> But I digress, the question of GIT verses SVN at this point both are a
> compromise in mind. I did play with git a little, the interface is not
> terribly inititive. My concern is that it will be a big barrier for
> non-software people to use. Also one of the biggest advantanges of GIT is
> the natural handling of branching. Branching works well in software because
> it is text and diff is easy. With hardware branching is a nightmare to deal
> with. The only commerical system that I know supporting it is ProPDM (from
> PTC) it was a feature that is off by default. In a past life we turned it on
> because we thought it would be most helpful in dealing with prototype
> cycles. Even with ProE built in diff and merge tools it is very difficult to
> visualize and do the integration. We ended up turning it off and living in
> linear world. I'm not saying I've given up on it just that afaik not ready
> for prime time on the hardware side. Again I digress. Maybe one of these
> backend will work in the longer term with a modified frontend specific to
> multi-disapline projects.
Later in the email you mention how CGS is outdated/old (which I'll get
to in a bit). In BRLCAD specifically, it's my understanding that the
dot g file is really just the accumulation of the commands sent to
mged to alter the object under study. So, this sequence or list of
steps to make the object is much like a program. And if that's the
case, this would then allow branching by keeping track of the source
of the model as if it's source code, thus getting 'hardware
branching'. Granted, this isn't full project branching at all, and
it's not integrated with documentation branching or any of the other
parts of the overall project system that would be important to keep,
but I am wondering what you think about it.
Do you think the packaging format work should be put off until there
is more comprehensive backend hardware branching and project
management tools?
>> Hm. These analogies are difficult. The binary distribution packages
>> (rpm, deb, pm, etc.) sometimes also contain source code. Since open
>> source hardware projects are already beginning to store hardware
>> designs in their svn, git, cvs, mercurial repositories, I was figuring
>> that these would be what you would want to package for distribution.
>> But it's important to note that the physical instantiations are also
>> another layer of abstraction worth thinking about. A few days ago on
>> the open manufacturing mailing list, there was some discussion going
>> around about interoperability and international container cargo
>> standards (as opposed to random/various parcel sizes currently known
>> to the snail mail system).
>>
>> Btw, there's a really awesome CAD editing program that is open source
>> known as BRLCAD ( http://brlcad.org/ ). It originated 20 years ago
>> from Muuss, the same guy who wrote 'ping'. It's completely featured,
>> runs on a large number of platforms, and I'm loving it. There's about
>> 400 command line tools that are packaged with it, everything from
>> translating various CAD formats between each other, to being able to
>> build models by calling commands through the linux shell, or in
>> theCadenace
>> inline shell while editing objects graphically with Tcl syntax. I'm
>> considering making a proposal here shortly to the open manufacturing
>> mailing list to include the dot g file format (what brlcad stores CAD
>> information in) within the repository standard.Cadenace
>>
>> Here's a screenshot of brlcad with the OpenMoko phone:
>> http://brlcad.org/gallery/s/screenshots/iges-openmoko-conversion.png.html
>
> I have been aware of BRLCAD but never tried it, you inspired me to download
> and compile it. Unfortunately I don't share your impression of it, it is
> possible that my short experiment was inadequate to see it's functionality.
> Constructive solids went out about 15 years ago for history based
> parametrics except in some specialize industrial design packages, which I
> would call more just plane history-less modeling. My experience with CS
> modeling is also limited to one project I did on a hydrofoil design about 15
> years ago, it was very painful. I would be very surprised if openmoko was
> modeled in there. Maybe with a really good frontend and some tweaking it
> will evolve into some thing more (there are more than a few commercial
> packages that evolved from constructive solids to parametric, but legacy
> made them awkward compared to ground up products like Solidworks). I'm more
> encouraged of free-cad (see sourceforge) getting there sooner.
Some documents on BRLCAD's priorities etc.:
http://www.gvu.gatech.edu/~jarek/papers/SolidModelingWebster.pdf
http://brlcad.org/BRL-CAD_Priorities.png
BREP and parametrics are in progress.
As for the user interface issues, that's up in the air- I think that
for the true modeling glory details, artists use blender; then, as
long as whatever tools they use can export on over to BRLCAD (if
that's even needed), things are fine. The dot g format does everything
STEP does [and then some] so I still think it's worth considering.
>> Re: using STL. Basically the idea is that each packed hardware project
>> would have metadata, the dot g file, other CAD data, but also someCadenace
>> scripts. For instance, if scripts were packaged to generate the CAD
>> data per the configuration and parameters that a user wanted (or
>> defaults), then that would allow STL to be generated, or really
>> whatever information the user needs to use. Personally I very much
>> want to include the ability to generate human readable instructions
>> for making something, instead of just 3D STL/IGES and gcode. Imagine
>> also being able to print out a standard set of instructions for
>> building whatever the hardware is: kitchen cooking recipes are a nice
>> example. Meanwhile there are industrial equivalents where food is
>> 'cooked' with more computationally literate recipes. I want both forms
>> of recipes being generated from the same information. etc. Some of
>> these recipes would call for things that an amateur just won't have
>> laying around the house - how many of us have injection molding
>> equipment, for instance? I guess the amateur would have to hope that
>> somebody put an "injection molding machine" into the repository, plus
>> serializable instructions that he can understand on how to make one
>> for himself :-). Or from where to 'order' it or who might be able to
>> make him one (somebody with a fully-fledged fablab?). etc.
>
> Hmm, I think I see what you are trying to do here. I have seen this handled
> different ways first hand (and a few via demos). There are commerical PLM
> packages that create web portals for you manufacturers to grab the
> information from and others that directly link your ERP to your suppliers
> ERP for just in time stuff. What works well from a homegrown point is to
Really? Can you show me some of these? I've been trying to find these
for a while now and I just can't seem to find the places where people
let ERP systems wire together. :-( There are some open source B2B apps
out there that do some of this, but I have yet to find something like
"here's a public listing of companies you can automatically query for
RFQ's through some software system".
> create released zip files containing all information required to make
> something. If the PDM package manages file relationships the release package
> building can automated trigger on the change managment process. So for
> example if you have a circuit board that is being release then the zip file
> might contain the bill of materials, gerber files, pdf of the mechanicals,
What's a gerber file? Wikipedia says: "A Gerber File is a file format
used by printed circuit board manufacturing machines to lay out
electrical connections such as traces, vias, and lands. In addition,
the file contains information for drilling, and milling the completed
circuit board."
Huh. Didn't know that.
> maybe the solidmodels and STEP file of the mechanicals, native and pdf of
> the schematic(s), PDF's of all the component datasheets, build instuctions
For electronics, datasheets can be repesented in the ECIX file format.
I've been looking for something like this for other engineering
disciplines, but alas I haven't found anything yet.
> document(s), test plan/proceedure, and the test results form. STL files are
> going away as a standard since they through out so much info most rapid
> houses perfer STEP or even native files. No machine shop, sheetmetal or
> tooling house will take STL files, most work from native or at minimum STEP
Ah, okay. Some of the machines on my university campus take STL files
- like the laser cutter, and then RepRap takes STL too, and some
things that go into the average/typical fablab.
> (except for sheetmetal where you really want to work native). Until/if there
> is an open CAD standard STEP is your best bet.
BRLCAD ;-). I still don't see why the other open source CAD packages
don't just contribute to BRLCAD. Most of the other projects are
one-man teams or something. I mean, if there's some tool that BRLCAD
is missing, submit it - it's a collection of related tools for doing
CAD, in the traditional style of GNU (lots of little programs that do
their job well, chained together to do really big things).
Happy new year,
Samantha Atkins
Not completely. Googling on "open source ERP" (also PLM, PDM) gives
several hits. I worked for an ERP secondary player some years back and
know there is nothing about the software that could not be done in Open
Source. All the middleware (well most of it) to make it work is in
JBOSS and various Apache software projects.
- samantha