Concurrency in CadQuery

[Duncan Smith]

Working on a tool to generate Celtic knot weaves.

It's quite slow, taking about a minute to create all of the component parts for this example, then another minute to union them together. However, this is using only one of the 24  logical processors I have.

The parts are independent until combined, so, naively, replacing:

for c in combined:
     c.makePart(weaveConfig)

with:

with concurrent.futures.ThreadPoolExecutor(max_workers=24) as executor:
    executor.map(lambda x: x.makePart(weaveConfig), combined)

...which creates the seperate part for each element of the knot should be much better, but is actually slower.

Anyone know what's going on here, does CQ or the underlying library have a global mutex? Or is there some trick I can do to improve parallel performance? Or am I hitting python (3.8.5 as it happens) limitations?

Thanks!

  Duncan

[Adam Urbanczyk]

AFAIK you won't be able to do much on the python level. Bool ops are already parallelized in OCCT. Can you profile your code to see where is the bottleneck?

[Duncan Smith]

  

I have no doubt the iterative loft and union I have is less than optimal performance wise, but sweeping along compound curves introduces a twist I don't want, this is the solution I've arrived at.

The concurrent version looks like this. Not sure what it tells me though, other than there's a lot of time in thread.lock, but kinda guess that.

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   1141/1    0.005    0.000   19.385   19.385 {built-in method builtins.exec}
        1    0.000    0.000   19.385   19.385 LoadFreeCadFile.py:1(<module>)
        1    0.022    0.022   18.007   18.007 LoadFreeCadFile.py:87(main)
        1    0.001    0.001   17.985   17.985 LoadFreeCadFile.py:20(loadFile)
       92   11.972    0.130   11.972    0.130 {method 'acquire' of '_thread.lock' objects}
        1    0.000    0.000    9.670    9.670 _base.py:635(__exit__)
        1    0.000    0.000    9.670    9.670 thread.py:230(shutdown)
       10    0.000    0.000    9.670    0.967 threading.py:979(join)
       10    0.000    0.000    9.670    0.967 threading.py:1017(_wait_for_tstate_lock)
       17    3.319    0.195    3.320    0.195 shapes.py:946(_bool_op)
       15    0.038    0.003    2.457    0.164 cq.py:3195(union)
       15    0.000    0.000    2.388    0.159 shapes.py:3100(fuse)
        1    0.000    0.000    2.305    2.305 _base.py:575(map)
        1    0.000    0.000    2.305    2.305 _base.py:600(<listcomp>)
       16    0.000    0.000    2.304    0.144 thread.py:158(submit)
       16    0.000    0.000    2.304    0.144 thread.py:193(_adjust_thread_count)
       10    0.000    0.000    2.303    0.230 threading.py:834(start)
       20    0.000    0.000    2.303    0.115 threading.py:270(wait)
       10    0.000    0.000    2.303    0.230 threading.py:540(wait)
        1    0.009    0.009    1.737    1.737 cq.py:275(split)
       70    0.003    0.000    1.510    0.022 __init__.py:1(<module>)
    896/8    0.004    0.000    1.382    0.173 <frozen importlib._bootstrap>:986(_find_and_load)
    891/8    0.002    0.000    1.382    0.173 <frozen importlib._bootstrap>:956(_find_and_load_unlocked)
    859/8    0.003    0.000    1.380    0.173 <frozen importlib._bootstrap>:650(_load_unlocked)
    741/8    0.002    0.000    1.380    0.173 <frozen importlib._bootstrap_external>:777(exec_module)
   1175/8    0.001    0.000    1.376    0.172 <frozen importlib._bootstrap>:211(_call_with_frames_removed)
        1    0.000    0.000    1.357    1.357 __init__.py:34(export)
        1    1.357    1.357    1.357    1.357 shapes.py:420(exportStl)
   495/74    0.001    0.000    1.164    0.016 {built-in method builtins.__import__}
        4    0.000    0.000    1.133    0.283 __init__.py:2(<module>)
        2    0.000    0.000    0.932    0.466 shapes.py:3091(cut)
        1    0.000    0.000    0.791    0.791 shapes.py:534(BoundingBox)
        1    0.000    0.000    0.791    0.791 geom.py:845(_fromTopoDS)
        1    0.791    0.791    0.791    0.791 {built-in method OCP.BRepBndLib.AddOptimal_s}
  858/855    0.001    0.000    0.642    0.001 <frozen importlib._bootstrap>:549(module_from_spec)
    96/94    0.000    0.000    0.625    0.007 <frozen importlib._bootstrap_external>:1099(create_module)
    96/94    0.623    0.006    0.625    0.007 {built-in method _imp.create_dynamic}
 1094/283    0.001    0.000    0.591    0.002 <frozen importlib._bootstrap>:1017(_handle_fromlist)
        1    0.000    0.000    0.554    0.554 geom.py:1(<module>)
        4    0.000    0.000    0.324    0.081 __init__.py:3(<module>)
        1    0.000    0.000    0.302    0.302 dxf.py:1(<module>)
        1    0.000    0.000    0.237    0.237 knots.py:1(<module>)
        1    0.232    0.232    0.232    0.232 {built-in method nt.startfile}
        3    0.000    0.000    0.214    0.071 assembly.py:1(<module>)
        1    0.000    0.000    0.211    0.211 solver.py:1(<module>)
        1    0.000    0.000    0.193    0.193 pyplot.py:4(<module>)
        1    0.000    0.000    0.167    0.167 options.py:3(<module>)
2074/1915    0.021    0.000    0.166    0.000 {built-in method builtins.__build_class__}
      741    0.005    0.000    0.160    0.000 <frozen importlib._bootstrap_external>:849(get_code)
      880    0.004    0.000    0.145    0.000 <frozen importlib._bootstrap>:890(_find_spec)

[Roger Maitland]

Duncan, a while ago I tried to create a Celtic Knot with a sweep and also found that I couldn't work around the asymmetric twist introduced. So I revised the design and created the following by building up from cq.Edge objects. This version only has one union - for the outer and internal loops - and is very fast.  This version is limited to a polygon in cross section but with some further work in defining the faces, arcs in the cross section could be introduced.  I hope this is helpful.

Cheers,

Roger

"""

Celtic Trinity Knot

name: celtic_knot.py

by: Gumyr

date: December 19th 2021

desc: This python/cadquery code generates a celtic trinity knot.

license:

Copyright 2021 Gumyr

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

"""

from math import sin, cos, pi, atan2, sqrt

import cadquery as cq

MM = 1

class CelticTrinityKnot:

"""

A Celtic Trinity Knot

Parameters:

radius: float = 1.0 - radius of the center of the outside arc

thickness: float = 0.05 - vertical displacement of the arc centers

cross_section_radius: float = 0.2 - size of the cross section

cross_section_side_count: int = 3 - number of sides in the cross section polygon

Properties:

cq_object: cq.Solid - the celtic knot object

"""

center_fraction = 0.5 # Control the shape of the knot - range 0.0-1.0

def arc_path(self, t, radius) -> cq.Vector:

"""Define points in the path of the arcs"""

start_angle = atan2(-self.arc_center[0], radius - self.arc_center[1])

arc_angle = 4 * pi / 3 - 2 * start_angle

arc_radius = sqrt(self.arc_center[0] ** 2 + (radius - self.arc_center[1]) ** 2)

angle = pi / 2 - start_angle - t * arc_angle

return cq.Vector(

self.arc_center[0] + arc_radius * cos(angle),

self.arc_center[1] + arc_radius * sin(angle),

self.thickness * sin(2 * t * 2 * pi),

)

def make_knot_outside(self) -> cq.Solid:

"""Create the outside arcs and fuse them into a single object"""

arc_edges = [

cq.Edge.makeSplineApprox(

[self.arc_path(t / 40, self.radius + r) for t in range(41)]

).translate(cq.Vector(0, 0, h))

for r, h in self.cross_section_pts

]

arc_faces = [

cq.Face.makeRuledSurface(arc_edges[i], arc_edges[(i + 1) % len(arc_edges)])

for i in range(len(arc_edges))

]

knot_outer_faces = [

f.rotate(cq.Vector(0, 0, 0), cq.Vector(0, 0, 1), a)

for f in arc_faces

for a in range(0, 360, 120)

]

return cq.Solid.makeSolid(cq.Shell.makeShell(knot_outer_faces))

def circle_path(self, t, radius) -> cq.Vector:

"""Define points in the path of the central circle"""

angle = t * 2 * pi

return cq.Vector(

radius * cos(angle),

radius * sin(angle),

1.5 * self.thickness * sin(3 * angle + pi / 2),

)

def make_knot_inside(self) -> cq.Solid:

"""Create the inside circle as a single object"""

circle_edges = [

cq.Edge.makeSplineApprox(

[

self.circle_path(t / 40, self.radius * sin(pi / 4) + r)

for t in range(41)

]

).translate(cq.Vector(0, 0, h))

for r, h in self.cross_section_pts

]

circle_faces = [

cq.Face.makeRuledSurface(

circle_edges[i], circle_edges[(i + 1) % len(circle_edges)]

)

for i in range(len(circle_edges))

]

return cq.Solid.makeSolid(cq.Shell.makeShell(circle_faces))

@property

def cq_object(self) -> cq.Solid:

"""The celtic knot object"""

return self.make_knot_outside().fuse(self.make_knot_inside())

def __init__(

self,

radius: float = 1.0,

thickness: float = 0.05,

cross_section_radius: float = 0.2,

cross_section_side_count: int = 3,

):

"""Store inputs and calculate some key values"""

self.radius = radius

self.thickness = thickness

self.cross_section_radius = cross_section_radius

self.cross_section_side_count = cross_section_side_count

center_radius = self.radius * CelticTrinityKnot.center_fraction

self.arc_center = (

center_radius * cos(5 * pi / 6),

center_radius * sin(5 * pi / 6),

0,

)

self.cross_section_pts = [

(v.X, v.Y)

for v in cq.Workplane("XY")

.polygon(self.cross_section_side_count, self.cross_section_radius)

.vertices()

.vals()

]

if __name__ == "main" or "show_object" in locals():

celtic_knot = CelticTrinityKnot(

radius=20 * MM,

thickness=1 * MM,

cross_section_radius=4 * MM,

cross_section_side_count=3,

).cq_object

cq.exporters.export(celtic_knot, "celtic_knot.step")

cq.exporters.export(celtic_knot, "celtic_knot.stl")

show_object(celtic_knot, name="celtic_knot")

--
cadquery home: https://github.com/CadQuery/cadquery
post issues at https://github.com/CadQuery/cadquery/issues
run it at home at : https://github.com/CadQuery/CQ-editor
---
You received this message because you are subscribed to the Google Groups "CadQuery" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cadquery+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cadquery/0afe5d4e-0046-40ab-9e9c-9bc07f0284f2n%40googlegroups.com.

[Duncan Smith]

Thanks, Roger, that is interesting, though it will take me a while to get my head round that approach. Splines I'll need to take another look at. I've developed an aversion to them from trying to use them in freecad.

Th advantage of the iterative loft is being able to arbitrarily set the profile, which I find useful:

[Adam Urbanczyk]

I meant profiling without threads. AFAIR OCP does not release the GIL, so Python threads won't help you.