Good Cyclomatic Complexity Number for Go

[Henry Adi Sumarto]

Hi,

I am trying to come up with a good cyclomatic complexity number for typical Go codes. Go is a unique language where its explicit error handling results in a higher cyclomatic number and yet its simple syntax and orthogonal design allows Go to still be readable even at a higher cyclomatic number. I am interested in knowing whether McCabe's number of 10 is still a good general rule for typical Go projects.

In my own codes, I find McCabe's number of 10 is a bit limiting for complex functions. While I can refactor the functions to get to 10 or less, it generally results in a less readable code. There is usually too much fragmentation with important parts of the codes being described elsewhere in various sub-functions. Although Go codes are still readable at 15, I think 12 is a good ideal number for typical Go functions. I am trying to test whether 12 is indeed the ideal cyclomatic number for Go or whether McCabe's 10 is still the ideal.

If you have time, feel free to participate and share your findings. The tool I use to measure cyclomatic complexity is https://github.com/fzipp/gocyclo.

Thanks.

Henry

[Markus Zimmermann]

Maybe there should be an option to not include the error handling in the number crunching?

[Henry Adi Sumarto]

Theoretically speaking, you shouldn't discount the error handling because it represents an alternate execution path. However, the difference between other languages and Go is that in other languages you can typically put one big try/catch over a block of code and it adds only +1 to the number, while in Go each function that returns an error must be checked and that adds many more +1s to the number. You can extract out a block of codes and put it into a separate function to mimic the try/catch block. However, in my own experimentation, it isn't quite an elegant solution and the codes generally become harder to read. I am thinking that the only viable alternative is to determine a new guideline number for Go since Go isn't quite like any other languages.

The question is what is a good number that is applicable to typical Go projects? The whole objective of cyclomatic complexity is to ensure readable and maintainable codes.    

On Monday, September 28, 2015 at 10:44:30 AM UTC+7, Markus Zimmermann wrote:

[Egon]

On Monday, 28 September 2015 05:26:48 UTC+3, Henry Adi Sumarto wrote:

Hi,

I am trying to come up with a good cyclomatic complexity number for typical Go codes. Go is a unique language where its explicit error handling results in a higher cyclomatic number and yet its simple syntax and orthogonal design allows Go to still be readable even at a higher cyclomatic number. I am interested in knowing whether McCabe's number of 10 is still a good general rule for typical Go projects.

In my own codes, I find McCabe's number of 10 is a bit limiting for complex functions. While I can refactor the functions to get to 10 or less, it generally results in a less readable code. There is usually too much fragmentation with important parts of the codes being described elsewhere in various sub-functions. Although Go codes are still readable at 15, I think 12 is a good ideal number for typical Go functions. I am trying to test whether 12 is indeed the ideal cyclomatic number for Go or whether McCabe's 10 is still the ideal.

AFAIK, there is no conclusively proven "software metric" that works. Regarding cyclomatic complexity https://en.wikipedia.org/wiki/Cyclomatic_complexity#Correlation_to_number_of_defects.

Instead I would suggest using peer-review for understandability.

The only way I would use software metric, is to show significance of change... when any metric (e.g. SLOC, Halstead ... etc.) changes significantly, let two people review it.

By picking an arbitrary limit you may be breaking up the code unnecessarily and making it harder to understand. (http://number-none.com/blow/john_carmack_on_inlined_code.html)

+ Egon

[Egon]

Found an additional quote from James Coplien:

Different styles of programming will need to measure different things. Different styles of business have different needs; for example, the encapsulation metrics in a very good high-availability real-time system are likely to be low, while those in a simple calculator program may be higher.

You can't name me a metric that a tool can generate that can't be gamed. I somehow feel that you are not going to be happy whatever tool you get. But I could be wrong, depending on what you want to use the metrics for.

Note: I'm not saying metrics are completely useless; rather that they should be treated as an imprecise information source about a very complex situation. You should be able to show that it really is an effective and useful measurement, and doesn't cause any other problems.

+ Egon

[Benjamin Measures]

> However, the difference between other languages and Go is that in other languages you can typically put one big try/catch over a block of code and it adds only +1 to the number

This is incorrect unless only the last statement in a try block can throw an exception.

The reality is that adding a try-catch block adds an unknown (hidden) cost to the cyclomatic complexity unless all the throw sites can be enumerated to arbitrary depth (due to unchecked exceptions).

Explicit error handling is more upfront (honest) and less combinatorial.

[Markus Zimmermann]

On Monday, September 28, 2015 at 6:18:04 AM UTC+2, Henry Adi Sumarto wrote:

Theoretically speaking, you shouldn't discount the error handling because it represents an alternate execution path. However, the difference between other languages and Go is that in other languages you can typically put one big try/catch over a block of code and it adds only +1 to the number, while in Go each function that returns an error must be checked and that adds many more +1s to the number. You can extract out a block of codes and put it into a separate function to mimic the try/catch block. However, in my own experimentation, it isn't quite an elegant solution and the codes generally become harder to read. I am thinking that the only viable alternative is to determine a new guideline number for Go since Go isn't quite like any other languages.

The question is what is a good number that is applicable to typical Go projects? The whole objective of cyclomatic complexity is to ensure readable and maintainable codes.    

I understand the problems of calculating complexity metrics of source code. I can understand that you want to find a good configuration for the tool you mentioned. However, IMHO this tool lacks some options and features e.g. if you think that the error handling does not add complexity, you should be able to apply that as a custom rule. In general I do not agree to use some magic complexity limit and configuration for every project. As Egon wrote, different styles need to be handled differently. Also, you should use complexity metrics as guidance on what **could** be refactored first but please do not enforce low numbers. It will make people crazy.

Since I am working on some (new) tools around source code quality and testing: I am in general interested in what the Go community wants in these areas. If you have some wishes, please do not hesitate to share them with me.

[Robert Johnstone]

You are partially correct.  Each function that can throw should considered to implicitly have a branch associated with it, so each function that can throw should add +1 to the number.  However, the idea that every possible throw site needs to be enumerated is unduly pessimistic.

@Henry : Benjamin's point is still sound.  The issue appears to be that your tools are failing to capture an important source of complexity in the other languages, not that the number is unusually high for Go.

[Henry Adi Sumarto]

Cyclomatic complexity isn't specific to the tool I mentioned earlier. Cyclomatic complexity was developed by Thomas McCabe in 1976 as a way to measure complexity. You may disagree with what the tool does or doesn't do, but the tool merely implement McCabe's algorithm as is. McCabe developed cyclomatic complexity and came up with the upper limit of 10 after an extensive research. You may read his paper about his various theorems and all if you are into that sort of things. People may agree or disagree with McCabe's approach but cyclomatic complexity has generally been used as an aid to locate where in the code that is likely to need refactoring. It is not the absolute rule, but it is generally a good guide. In addition, when given an upper limit of cyclomatic complexity, people will make a conscious attempt to employ a simpler algorithm to accomplish the same tasks. 

However, my problem is that McCabe's upper limit of 10 doesn't seem to be applicable to Go codes. My own experiment shows me that 12 is probably the ideal, but I am not 100% sure about this. Furthermore, C language must have existed during McCabe's time and -given C's popularity- he was likely to be familiar with it when he developed cyclomatic complexity. If the upper limit of 10 works for C language, I wonder why it doesn't work with Go. 

Anyhow, my intention is to gather feedback from people across different projects, different background, and different coding styles to see what they consider a good upper limit is. Hopefully, whatever the result is, it will be useful to the Go community in general.

[Robert Melton]

On Mon, Sep 28, 2015 at 2:39 PM, Henry Adi Sumarto <henry.ad...@gmail.com> wrote:

However, my problem is that McCabe's upper limit of 10 doesn't seem to be applicable to Go codes.

We haven't had an issue with 10 as our soft upper limit... doing a quick review, the overwhelming majority of our codebase (74k LOC) falls in the 3-8 range (per gocyclo), and we have a handful of functions that break north of 15... and a couple north of 25.  This was never a real heavy focus of our dev, it just happened the be the default listed in the help of gometalinter and when we saw something over 10 -- we did a double-check to make sure it was sane.  

I am curious what experiments you did that made you think 12 is ideal?  I guess for our codebase, we could probably consider 8 our soft cap, cause we have very few between 9 and 15. 

--

Robert Melton | http://robertmeta.com

[Henry Adi Sumarto]

We took a number of complex functions from our codebase, and refactor them until everyone agrees that they are readable and cannot be further reduced without hurting readability. We managed to get some functions to below 10, and some remain in the 10 -12 range. Hence, we established the upper limit of 12. 

On the other hand, considering that we hand-picked these functions for their complexity, it is possible that the upper limit of 12 applies only to corner cases (or at least the corner cases of our codebase), which is why I am curious to know what everyone else's experience regarding this, particularly as applied to Go code.

Btw, have you tried to refactor the ones with 25 and see how far you can go?

[Egon]

I went through few of the code pieces that I had and mostly it remained under 12.

Most common inflating was this:

...

switch r.Method {

case "GET":

     page, err := context.GetPage(id)

     switch err {

     case nil:

          return http.StatusOK, page

     case ErrPageNotExist:

          return http.StatusNotFound, NewMissingPage(id)

     default:

          return http.StatusInternalServerError, NewErrorPage(id)

     }

case "PUT":

...

The CC for this piece is already 6.

I would probably use +1 cyclo for switch, if it's all cases are clearly terminating (ending with return or panic). It's quite common to have a dispatch based on some name and it harms to have those case handling separated.

+ Egon

[Robert Melton]

On Mon, Sep 28, 2015 at 11:06 PM, Henry Adi Sumarto <henry.ad...@gmail.com> wrote:

We took a number of complex functions from our codebase, and refactor them until everyone agrees that they are readable and cannot be further reduced without hurting readability. We managed to get some functions to below 10, and some remain in the 10 -12 range. Hence, we established the upper limit of 12.

I don't know enough about the theory of cyclomatic complexity, is the idea for it to be applied rigidly?  We have treated it like a useful "double check that function" marker and not much more. 

Btw, have you tried to refactor the ones with 25 and see how far you can go?

I was able to get them both under 10 (9 and 5).  One of them will actually get mainlined, it is a fairly straightforward improvement I just hadn't revisited in some time.  The other one is far more debatable, I don't like it -- co-worker thinks it is an improvement.  After passing it around and discussing it -- with our little group, there is a very high correlation between "function size" < "lines visible on screen" and happiness...  Those using setups that allow less lines visible on screen (via font size, editor tooling, screen resolution, whatever) favor the smaller functions (which are a side-effect of lower cyclo count), those of us who can display a lot more lines of code easily tend to be more comfortable / prefer the bigger but singular functions.

So what did we learn... I have no idea... honestly just makes me feel that cyclomatic complexity might be more of a style argument than a science argument, but I will have to read more.  (My lack of coming to any good resolution in this also reminded me of the Burn After Reading ending: https://youtu.be/SlA9hmrC8DU?t=2m33s -- has cursing). 

[Henry Adi Sumarto]

On Tuesday, September 29, 2015 at 10:10:12 PM UTC+7, Robert Melton wrote:

I don't know enough about the theory of cyclomatic complexity, is the idea for it to be applied rigidly?  We have treated it like a useful "double check that function" marker and not much more. 

It depends on what you believe in. Some people swears by it. Some think it is a waste of time. I like to think of it as more of a guide, but it's really nice to have a measurable goal when you are refactoring.

 

Btw, have you tried to refactor the ones with 25 and see how far you can go?

I was able to get them both under 10 (9 and 5).  One of them will actually get mainlined, it is a fairly straightforward improvement I just hadn't revisited in some time.  The other one is far more debatable, I don't like it -- co-worker thinks it is an improvement.  After passing it around and discussing it -- with our little group, there is a very high correlation between "function size" < "lines visible on screen" and happiness...  Those using setups that allow less lines visible on screen (via font size, editor tooling, screen resolution, whatever) favor the smaller functions (which are a side-effect of lower cyclo count), those of us who can display a lot more lines of code easily tend to be more comfortable / prefer the bigger but singular functions.

 

I guess I would fall into the second group ... lol. I prefer squeezing everything on my screen. The screen is never large enough for me. The key argument to cyclomatic complexity is that higher complexity correlates to more bugs. I personally think the key value of CC is when you can get to a lower number with as few extra functions as possible. In my own case, we end up simplifying a lot of our lexer logic.

It looks like McCabe's number works fine in your case. I'll take a look at my code again and see if my functions in the 11-12 range can be further reduced to 10 or less. I still don't like the idea of further splitting them up into more functions.  

[Henry Adi Sumarto]

Thanks for the feedback. Usually switch statement can be substituted with lookup tables. However, it depends on the situation. Sometimes lookup tables leads to more clutters. Earlier I wondered how optimized the switch implementation in Go was, so I wrote a little test earlier comparing a switch statement against lookup table. It appears that using lookup table is slightly faster. (The code is http://play.golang.org/p/In2GVeNsC0 but you cannot run it in Go playground. You may need to copy it and compile it in your machine.)

Btw, do you have problems in getting to 10 or less, or is 12 a good number in your case?

[Konstantin Shaposhnikov]

Running gocyclo on the Go source code gives some interesting statistics (excluding tests):

- 91.5% of functions have CC <= 10

- 93.3% of functions have CC <= 12

- 37% of functions have CC = 1

- the highest CC is 475

[Egon Elbre]

On Tuesday, 29 September 2015, Henry Adi Sumarto <henry.ad...@gmail.com> wrote:

Thanks for the feedback. Usually switch statement can be substituted with lookup tables. However, it depends on the situation. Sometimes lookup tables leads to more clutters. Earlier I wondered how optimized the switch implementation in Go was, so I wrote a little test earlier comparing a switch statement against lookup table. It appears that using lookup table is slightly faster.

I would've used a LUT if I didn't have to dispatch on behavior. I know I could create a map of funcs, but it would be pretty much as the current switch, but with an additional variable and a lot of func declarations.

 

(The code is http://play.golang.org/p/In2GVeNsC0 but you cannot run it in Go playground. You may need to copy it and compile it in your machine.)

Btw, do you have problems in getting to 10 or less, or is 12 a good number in your case?

There's no technical problem, but it would probably reduce code readability rather than help it. 10 or 12 either would be a good guideline, but not a rule.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/HNNUjE5VWos/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[diego....@gmail.com]

Cyclomatic complexity can be helpful in keeping us away from writing complex code that can be more error prone, more complex to be tested and harder to reason about. I would give a special attention for the latter one as it implies in low readability and high maintenance costs. For example, an if/else statement is commonly used as:

if THIS then
 do some logic in order to return THAT


if THAT then
 do some logic in order to return THIS


if SOMETHINGELSE then
 do some logic in order to return THAT and THIS
else
 do some logic in order to return THIS and THAT

Usually each block contains some logic in order to do something. Naturally each piece of logic requires attention from a developer as one must understand what a block is doing, what is its purpose, and finally, if both answers to the previous questions match. If the expected result is what is being produced. Now imagine a single function with 20 conditional statements, where each one produces a single differente result, and you have to explain (and remember) what is everything that function does under which conditions. Probably you already got the point.

However, note that cyclomatic complexity doesn’t try to determine how complex each path contained in a control flow graph is. It doesn’t try to understand each block of your program’s architecture, instead, it focus on how most of the programs are written and assumes that all cases are equal. In order to be helpful it assumes that all execution path contains a specific and unique piece of logic, as most of the time, it is the case. I have been thinking about that point since I started programming in Go. Following the error handling pattern we use in Golang, check this other example:

THIS, ERR := fnGetThis()
if ERR then
 return error


THAT, ERR := fnGetThat()
if ERR then
 return error


SOMETHINGELSE, ERR := fnGetSomethingElse()
if ERR then
 return error


WHATEVER, ERR := fnGetWhatever()
if ERR then
 return error


do some logic in order to produce the expected result using THIS, THAT, SOMETHINGELSE and WHATEVER

It seems to me that none of the if statements that only return an error and has no extra logic, can be error prone, nor is difficult to reason about, nor makes it harder to explain the purpose of that whole block. It does one thing, once everything it needs works. 

I believe Golang could have a better cyclomatic complexity analysis by discarding the cases where the only thing that is done is to return an error and nothing else, as the real complexity to generate those errors is not written there.

[Lee Painton]

From McCabe's paper on the topic: http://www.literateprogramming.com/mccabe.pdf

These results have been used in an operationalenvironrnent by advising project members to limit their software modules by cyclomatic complexity instead of physical size. The particular upper bound that has been used for cyclomatic complexity is 10 which seems like a reasonable, but not magical, upper limit. Programmers have been required to calculate complexity as they create software modules. When the complexity exceeded 10 they had to either recognize and modularize subfunctions or redo the software. The intention was to keep the "size" of the modules manageable and allow for testing all the independent paths (which will be elaborated upon in Section VII.) The only situation in which this limit has seemed unreasonable is when a large number of independent cases followed a selection function (a large case statement), which was allowed. It has been interesting to note how individual programmer's style relates to the complexity measure. The author has been delighted to fmd several programmers who never had formal training in structured programming but consistently write code in the 3 to 7 complexity range which is quite well structured. On the other hand, FLOW has found several programmers who frequently wrote code in the 40 to 50 complexity range (and who claimed there was no other way to do it). On one occasion the author was given a DEC tape of 24 Fortran subroutines that were part of a large real-time graphics system. It was rather disquieting to fmd, in a system where reliability is critical, subroutines of the following complexity: 16, 17, 24, 24, 32, 34, 41, 54, 56, and 64. Mter confronting the project members with these results the author was told that the subroutines on the DEC tape were chosen because they were troublesome and indeed a close correlation was found between the ranking of subroutines by complexity and a ranking by reliability (performed by the project members).

[Alex Efros]

Hi!

On Fri, Apr 13, 2018 at 12:41:35PM -0700, Lee Painton wrote:
> The only situation in which this limit has seemed unreasonable is when a
> large number of independent cases followed a selection function (a large
> case statement), which was allowed.

Another situation is function which does validation of complex data
received from outside - like checking flags in main() or http.Request in
http handler func or similar RPC handler func.

Refactoring these to fit any cyclomatic limit also makes code worse and
not worth it. But increasing global limit because of these exceptional
cases doesn't makes any sense - instead, just add pragma to such func:
func validateSomething() { // nolint:gocyclo

Just try to avoid mixing such a validation logic with another logic in
same function.

--
WBR, Alex.

[diego....@gmail.com]

Refactoring these to fit any cyclomatic limit also makes code worse and 
not worth it. But increasing global limit because of these exceptional 
cases doesn't makes any sense - instead, just add pragma to such func: 
    func validateSomething() { // nolint:gocyclo 

I feel the same as you in these cases you mentioned. The solution you proposed made me think that if we could have a pragma like the one you described, but for specific branches/path, that could be helpful. Example:

if { // gocyclo:skip

Having that, the linter wouldn't need to have a logic to decide whether it should compute a specific branch or not. Would be up to the developer to decide, easy to be configured, and not obscure in any way.