Fully custom DRC rules using python

[Daniel Treffenstädt]

Hi there,

So I have been thinking a lot about how to properly integrate more complex DRC rules into KiCad, I have some use cases that the current custom rules cannot express properly.

Some examples can be seen here: https://forum.kicad.info/t/custom-drc-rules-more-granular-control-for-ecss-specifications/41686

-- For context --

As an example, these three requirements need more complex code than is possible with the custom DRC rules:

1. Fan out from (PTH/SMD) pad to vi: This is essentially the length of open track between a pad and the nearest via. The required length is dependent on pad type and track width.

2. Maximum difference in connecting track width for two-pad components.

3. Asymmetric track attachment points for two-pad components. Basically this means that tracks connected to two-pad components should point in the same direction, ideally only attach to the head of the component.

Requirements two and three have to do with the reflow process. If components are improperly attached, they can get misaligned during soldering, in the worst case they can tombstone.

Requirement one is mostly important  for classic space hardware which usually comes without solder mask. If a via is too close to a pad, it can act as a sink for solder paste and reduce the amount available for the actual pad.

-- For context --

Recently I have had some time to look at the issue a bit more in depth and decided to write up a plugin that offers an interface to write complex custom DRC rules like the ones stated above. The code can be found here: https://gitlab.com/d.treffenstaedt/extra_drc_plugin

Keep in mind that this is little more than a first draft and the code is mostly exploratory.

Now to the actual point I am trying to make here: I think a system like this could improve the user experience of KiCad and make the DRC system a whole lot more flexible than it already is.

From your perspective - is this something worth putting more time into and work towards making this a part of KiCad itself rather than a plugin? Have similar ideas been proposed before?

Thank you for your time, I hope there is some interest in this.

Best Regards,
Daniel

Some visuals:

[Jon Evans]

Hi Daniel,

> is this something worth putting more time into and work towards making this a part of KiCad itself rather than a plugin?

It would be better in my opinion to use this only as a prototyping tool to identify new DRC checks that should be added to KiCad in C++.   DRC checkers are performance-critical, especially on larger boards, and involving Python in the critical path is not likely to give the kind of performance we expect.

The C++ design rule system is fairly easy to extend and also has a lot of performance optimizations that would be difficult to leverage across the boundaries necessary between the KiCad code and any plugin.

Best,

-Jon

--
You received this message because you are subscribed to the Google Groups "KiCad Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to devlist+u...@kicad.org.
To view this discussion on the web visit https://groups.google.com/a/kicad.org/d/msgid/devlist/0f3e5777-40ce-4681-8227-439e78103b4en%40kicad.org.

[Tomasz Wlostowski]

On 04/11/2023 17:50, Jon Evans wrote:
> Hi Daniel,
>
> > is this something worth putting more time into and work towards
> making this a part of KiCad itself rather than a plugin?
>
> It would be better in my opinion to use this only as a prototyping tool
> to identify new DRC checks that should be added to KiCad in C++.   DRC
> checkers are performance-critical, especially on larger boards, and
> involving Python in the critical path is not likely to give the kind of
> performance we expect.
>
> The C++ design rule system is fairly easy to extend and also has a lot
> of performance optimizations that would be difficult to leverage across
> the boundaries necessary between the KiCad code and any plugin.
>

Hi Daniel,

Thank you for your interest in contributing to KiCad, but here I second
Jon. DRC engine performance is critical in particular for the P&S
routing and component placement. The rule language has been designed
with performance in mind and that's why it has certain limitations.
Trust me, we had considered using Python expressions for DRC in the
past. Running a worst-case O(n^2) clearance queries during shove routing
is already a huge bottleneck.

Also, DRC is a core feature of KiCad. Just like the router or footprint
editor. It must be IMHO an integral part of KiCad, not relying on a
plugin. Imagine the case where someone sends a design where design rules
are provided by plugins to a fab which doesn't have the same set (and
same versions of plugins).

If you'd like to contribute to the DRC engine, we are open for
discussions & patches in C++, as Jon mentioned already it's pretty
straightforward to extend/add new checks, and the examples you provided
don't look particularly difficult to implement.

Regards,
Tom

> Best

> -Jon
>
> On Sat, Nov 4, 2023 at 12:29 PM Daniel Treffenstädt
> <d.treff...@gmail.com <mailto:d.treff...@gmail.com>> wrote:
>
> Hi there,
>
> So I have been thinking a lot about how to properly integrate more
> complex DRC rules into KiCad, I have some use cases that the current
> custom rules cannot express properly.
> Some examples can be seen here:

> https://forum.kicad.info/t/custom-drc-rules-more-granular-control-for-ecss-specifications/41686 <https://forum.kicad.info/t/custom-drc-rules-more-granular-control-for-ecss-specifications/41686>

> *Now to the actual point I am trying to make here:* I think a system

> like this could improve the user experience of KiCad and make the
> DRC system a whole lot more flexible than it already is.
> From your perspective - is this something worth putting more time
> into and work towards making this a part of KiCad itself rather than
> a plugin? Have similar ideas been proposed before?
>
> Thank you for your time, I hope there is some interest in this.
>
> Best Regards,
> Daniel
>
> Some visuals:
>

> no_parameter_rule.png
>
> three_parameter_rule.png

>
> one_parameter_rule.png
>
> --
> You received this message because you are subscribed to the Google
> Groups "KiCad Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to devlist+u...@kicad.org

> <mailto:devlist+u...@kicad.org>.

> To view this discussion on the web visit

> https://groups.google.com/a/kicad.org/d/msgid/devlist/0f3e5777-40ce-4681-8227-439e78103b4en%40kicad.org <https://groups.google.com/a/kicad.org/d/msgid/devlist/0f3e5777-40ce-4681-8227-439e78103b4en%40kicad.org?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "KiCad Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to devlist+u...@kicad.org

> <mailto:devlist+u...@kicad.org>.

> To view this discussion on the web visit

> https://groups.google.com/a/kicad.org/d/msgid/devlist/CA%2BqGbCCnp470E%2Bs%3DGvua4A32gP79p-1cLzHL0uAYSxXAC8XkkA%40mail.gmail.com <https://groups.google.com/a/kicad.org/d/msgid/devlist/CA%2BqGbCCnp470E%2Bs%3DGvua4A32gP79p-1cLzHL0uAYSxXAC8XkkA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Daniel Treffenstädt]

Hi Jon,

Thank you for your feedback.

> It would be better in my opinion to use this only as a prototyping tool to identify new DRC checks that should be added to KiCad in C++.

That sounds like something I could get behind. I have started implementing some of the rules in Python, re-implementing them in C++ would be pretty straightforward once the implementation has matured.

However implementing all (or most) rules from the ECSS-Q-ST-70-12C specification would result in quite a specialised rule set not really that applicable for non-space related projects.

Maybe some of the rules would need to be hidden behind something akin to a checkbox >enable ECSS compliant rules< (other industries probably have similar problems I assume).

Maybe this would be a different discussion altogether, but what is your opinion of the UI I have drafted?

When more rules are added, especially more complex ones with more than one parameter, the current DRC page could get overloaded.

The way I currently do it in this small project is each rule defines its own icon + description + parameter set, the UI is generated from that.

Maybe an approach like this could benefit the current custom DRC rule section, also to integrate it more into the existing rules?

I often find myself writing a custom rule set then reusing it between boards with slight modifications and handing them over to colleagues working with KiCad.

It is relatively easy to accidentally bend the rules so they don't actually do what was intended. A generated UI with parameters could make it easier for them to work with it.

Best Regards,
Daniel

[Jon Evans]

> implementing all (or most) rules from the ECSS-Q-ST-70-12C specification would result in quite a specialised rule set not really that applicable for non-space related projects.

I'm not sure how practical this line of thought would be, but my first thought is: can we implement enough rule primitives in the design rule language to implement all of these space-related rules without implementing the specific checks themselves as individual primitives?  In other words, can the ECSS rules be composed out of smaller or less-specific building-blocks that might be more generally useful?

> Maybe this would be a different discussion altogether, but what is your opinion of the UI I have drafted?

> When more rules are added, especially more complex ones with more than one parameter, the current DRC page could get overloaded.

The "constraints" page is only aimed at the most basic rules.  More complex rules are currently only specified through the custom rules language, and have no GUI.

We could decide to add more GUI for advanced rules, however it would not go on the basic constraints page.

My recommendation would be to focus on implementing the checks you need in the custom rules language first, and make the GUI question a separate question.

> Maybe an approach like this could benefit the current custom DRC rule section, also to integrate it more into the existing rules?

I am not sure what you mean by integrate here; the custom rules and basic constraints are both already integrated together into the DRC system.

We have discussed before the concept of auto-generating UI for editing custom rules, but it hasn't been prioritized at the moment.  Such a system should probably be a layer on top of the custom rules language, essentially parsing it and serving as a "visual editor" for it.

> It is relatively easy to accidentally bend the rules so they don't actually do what was intended. A generated UI with parameters could make it easier for them to work with it.

I'm not sure I think that a GUI like you propose is an answer to rules not doing what was intended.  I think there are other non-GUI features that would help here more, such as an interactive preview of condition predicates.

-Jon

[Daniel Treffenstädt]

> can we implement enough rule primitives in the design rule language to implement all of these space-related rules without implementing the specific checks themselves as individual primitives?

I don't think this is answerable in general, but I assume for most requirements this would be possible. I will have to do some proper analysis of the requirements to answer that fully though.

I will try and deconstruct each requirement into a minimum set of constraints and conditions and try to find commonalities. Unfortunately the ECSS is not really written in a way that makes this easy, so it will take some time.

> My recommendation would be to focus on implementing the checks you need in the custom rules language first, and make the GUI question a separate question.

Noted, however for the sake of not leaving it completely unanswered here:

> I am not sure what you mean by integrate here; the custom rules and basic constraints are both already integrated together into the DRC system.

I simply meant visual integration.

> I'm not sure I think that a GUI like you propose is an answer to rules not doing what was intended.

I was thinking in much simpler terms than that:

I was thinking of exposing parameters like clearances or net names of a custom rule since those might have to be adjusted on a per-project basis, the rule definition itself could remain untouched.

This could make it easier to have one maintainer of custom rules who can become an expert so not every engineer has to directly interact with the s-expressions.

> I think there are other non-GUI features that would help here more, such as an interactive preview of condition predicates.

That would be useful indeed.

[Jon Evans]

> I assume for most requirements this would be possible. I will have to do some proper analysis of the requirements to answer that fully though.

I will try and deconstruct each requirement into a minimum set of constraints and conditions and try to find commonalities. Unfortunately the ECSS is not really written in a way that makes this easy, so it will take some time.

If you find cases where you are not sure if it is possible, please bring them up here.  Ultimately if we can find a way to express all the constraints in a declarative fashion rather than in an imperative fashion (Python program) it will be a good sign that we can fit it all into the DRC language and take advantage of its performance characteristics.

Best,

Jon

[Daniel Treffenstädt]

So this was actually less work than anticipated. I wrote down a quick document describing the additions that would be necessary to support the requirements.

I might just start reading into the KiCad DRC code and see if I can't come up with solutions for the low hanging fruit.

[Clemens Koller]

Hello, Daniel, Jon, Thomasz,

maybe a bit OT, but is that an idea regarding the performance critical DRC:

Would it be possible or an approach to implement a DRC "descriptive language"
to define something like:

(condition “A.Type == ‘pad’ && A.parent.GetField(‘example’) == ‘value’”)

and compile that into optimized machine language to be run by the DRC?

The rules are set only a few times within the design time of a pcb, whereas
the DRC is needed permanently during the layout process for a full online
check.

IMO LTSpice, now QSPICE, is following that approach to get the max
performance out of it's ciruit simulation, too.

Just my five cents.

Regards,

Clemens

On 04/11/2023 17.50, Jon Evans wrote:
> Hi Daniel,
>
> > is this something worth putting more time into and work towards making this a part of KiCad itself rather than a plugin?
>
> It would be better in my opinion to use this only as a prototyping tool to identify new DRC checks that should be added to KiCad in C++.   DRC checkers are performance-critical, especially on larger boards, and involving Python in the critical path is not likely to give the kind of performance we expect.
>
> The C++ design rule system is fairly easy to extend and also has a lot of performance optimizations that would be difficult to leverage across the boundaries necessary between the KiCad code and any plugin.
>
> Best,
> -Jon
>

> On Sat, Nov 4, 2023 at 12:29 PM Daniel Treffenstädt <d.treff...@gmail.com <mailto:d.treff...@gmail.com>> wrote:
>
> Hi there,
>
> So I have been thinking a lot about how to properly integrate more complex DRC rules into KiCad, I have some use cases that the current custom rules cannot express properly.

> Some examples can be seen here: https://forum.kicad.info/t/custom-drc-rules-more-granular-control-for-ecss-specifications/41686 <https://forum.kicad.info/t/custom-drc-rules-more-granular-control-for-ecss-specifications/41686>

>
> -- For context --
> As an example, these three requirements need more complex code than is possible with the custom DRC rules:
> 1. Fan out from (PTH/SMD) pad to vi: This is essentially the length of open track between a pad and the nearest via. The required length is dependent on pad type and track width.
> 2. Maximum difference in connecting track width for two-pad components.
> 3. Asymmetric track attachment points for two-pad components. Basically this means that tracks connected to two-pad components should point in the same direction, ideally only attach to the head of the component.
>
> Requirements two and three have to do with the reflow process. If components are improperly attached, they can get misaligned during soldering, in the worst case they can tombstone.
>
> Requirement one is mostly important  for classic space hardware which usually comes without solder mask. If a via is too close to a pad, it can act as a sink for solder paste and reduce the amount available for the actual pad.
> -- For context --
>
>

> Recently I have had some time to look at the issue a bit more in depth and decided to write up a plugin that offers an interface to write complex custom DRC rules like the ones stated above. The code can be found here: https://gitlab.com/d.treffenstaedt/extra_drc_plugin <https://gitlab.com/d.treffenstaedt/extra_drc_plugin>

>
> Keep in mind that this is little more than a first draft and the code is mostly exploratory.
>

> *Now to the actual point I am trying to make here:* I think a system like this could improve the user experience of KiCad and make the DRC system a whole lot more flexible than it already is.

> From your perspective - is this something worth putting more time into and work towards making this a part of KiCad itself rather than a plugin? Have similar ideas been proposed before?
>
> Thank you for your time, I hope there is some interest in this.
>
> Best Regards,
> Daniel
>
> Some visuals:
>

> no_parameter_rule.png
>
> three_parameter_rule.png

>
> one_parameter_rule.png
>
> --
> You received this message because you are subscribed to the Google Groups "KiCad Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to devlist+u...@kicad.org <mailto:devlist+u...@kicad.org>.
> To view this discussion on the web visit https://groups.google.com/a/kicad.org/d/msgid/devlist/0f3e5777-40ce-4681-8227-439e78103b4en%40kicad.org <https://groups.google.com/a/kicad.org/d/msgid/devlist/0f3e5777-40ce-4681-8227-439e78103b4en%40kicad.org?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups "KiCad Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to devlist+u...@kicad.org <mailto:devlist+u...@kicad.org>.
> To view this discussion on the web visit https://groups.google.com/a/kicad.org/d/msgid/devlist/CA%2BqGbCCnp470E%2Bs%3DGvua4A32gP79p-1cLzHL0uAYSxXAC8XkkA%40mail.gmail.com <https://groups.google.com/a/kicad.org/d/msgid/devlist/CA%2BqGbCCnp470E%2Bs%3DGvua4A32gP79p-1cLzHL0uAYSxXAC8XkkA%40mail.gmail.com?utm_medium=email&utm_source=footer>.