[Investigation] Integration testing framework to be used in Go
40 views
Skip to first unread message
eph...@redhat.com
Jan 28, 2019, 11:40:06 AM1/28/19
to Aerogear
Looking for some tips from the Aerogear community on this. [testify|https://github.com/stretchr/testify] and the standard library are used in other aerogear projects ([aerogear-app-metrics|https://github.com/aerogear/aerogear-app-metrics] and I have investigated pros and cons. Any feedback from those of who have used both/either?
I've come to the conclusion that we do not need a 3rd party library but would be happy to hear from someone who may have used one and can point out the benefits I may have overlooked.
Jira link(s)
What
Investigate possible Golang testing frameworks to use in [aerogear/mobile-security-service-server|https://github.com/aerogear/mobile-security-service-server].
Unit Testing Libraries
testing (standard library)
- Package testing provides support for automated testing of Go packages. It is intended to be used in concert with the “go test” command, which automates execution of any function of the form.
- Documentation: https://golang.org/pkg/testing/
- Pros:
- Easy to use.
- Native Go library so there is no need to install additional dependencies.
- Boiler plate code can be generated in Visual Studio Code
- A lot of examples and documentation available.
- Contains a benchmarking facility.
- Cons:
- More verbose than using a 3rd party library like stretchr/testify
- Doesn't provide mocking functionality out of the box.
- Additional links:
testify
- Documentation: stretchr/testify
- Supports Go 1.11
- Pros:
- Easy to use and implements the familiar Assert style seen in other languages
- Extends the standard Go testing library so both can be used interchangeably
- Provides a mocking mechanism
- Partially used in other Aerogear projects (aerogear-app-metrics
- Cons:
- Seems unnecessary since the standard library can do the same things just in a different way. This just lightly wraps the standard library
- 3rd party dependency not officially supported by Go.
- The standard Go testing library deliberately doesn't use assertions. The authors of The Go Programming Language make some good arguments for Go's style over Assertions. See Page 302.
- Tests can feel like they're written in a different language (RSpec/Mocha for instance)
- Errors can be cryptic "assert: 0 == 1"
- Pages of stack traces can be generated
- Tests stop executing after the first assert fails - masking patterns of failure
Conclusion
My opinion as that we do not need to use an external library for unit testing in Go. The only benefit that I can see is a slight abstraction of the standard library into Assert style functions. The Go team purposely ditched this style (listed in testify > cons).
Here's an interesting talk on the subject: https://youtu.be/yi5A3cK1LNA?t=438
eph...@redhat.com
Jan 28, 2019, 11:41:32 AM1/28/19
to Aerogear
To view the discussion - https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/aerogear/8p5oljDM6gk
David Ffrench
Jan 28, 2019, 11:45:16 AM1/28/19
to Enda Phelan, Aerogear
I agree with your findings Enda, I don't believe there is any need to use an external library for unit testing in Go. Does Go provide integration testing out of the box?
DAVID FFRENCH
senior software engineer, RED HAT MOBILE
Communications House, Cork Road
Waterford, Ireland
 |
--
You received this message because you are subscribed to the Google Groups "Aerogear" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aerogear+u...@googlegroups.com.
To post to this group, send email to aero...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aerogear/7383da73-aee8-4aa6-912b-d527a303c1e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Wei Li
Jan 28, 2019, 11:46:20 AM1/28/19
to Enda Phelan, Aerogear
+1 on sticking with the Go standard testing library. From my personal experience, you don't need another one.
--
You received this message because you are subscribed to the Google Groups "Aerogear" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aerogear+u...@googlegroups.com.
To post to this group, send email to aero...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aerogear/f6d257f7-3668-48ee-8d8e-0243aac29277%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
eph...@redhat.com
Jan 28, 2019, 12:00:45 PM1/28/19
to Aerogear
Yes it seems to support it.
Dara Hayes
Jan 29, 2019, 5:25:03 AM1/29/19
to Enda Phelan, Aerogear
+100 on your findings Enda! I was always a little annoyed that testify was snuck into the tests for aerogear-app-metrics :)
--
You received this message because you are subscribed to the Google Groups "Aerogear" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aerogear+u...@googlegroups.com.
To post to this group, send email to aero...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aerogear/f6d257f7-3668-48ee-8d8e-0243aac29277%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
cma...@redhat.com
Feb 22, 2019, 6:16:08 PM2/22/19
to Aerogear
Hi @folks,
My vote is +1 for use /testify/assert. Sorry, for not agree with and I really hope that this email gives a chance to change your mind or at least encorage others share their opinion as well. The hope never dies. :-)
In order to try to make you folks more receptive and opening mind, I'd like to highlight that Kubernets and OpenShift are using /testify/assert to do their tests as many others OpenSources projects since it is one of the most popular and used pkg in GO. It seems that the CONS shared here, over to this adoption, are NOT totally true. The book The Go Programming Language speaks about about Go's style over Assertions in other languages which means that all argumentations added are NOT necessarily valid for this scenario. Note that the arguments in this thread came from the blog post <https://blog.alexellis.io/golang-writing-unit-tests/>.
Before of anything I'd like to just ask for you compare and check with your own eyes, heart and mind the following practical examples.
Note that both files are doing the same but one is using the pkg and the other not.
Example A:
Example B:
What CONS shared here are not true for the pkg?
- "Tests stop executing after the first assert fails" it is NOT the behaviour faced with
- "Pages of stack traces can be generated" shows NOT be the case here either
- Regards the CONS "3rd party dependency not officially supported by Go", our projects are using many deps and we should not be looking for to re-write the wheel which will be the only way to avoid to use them
- The book text is "with cryptic failure messages like "assert: 0 == 1" and NOT "Errors can be cryptic "assert: 0 == 1"". It means that Assertion approach may not force the dev to add a descriptive message for the errors failures and they tried with the Go style force it, however, it doesn't mean that the same cannot occur in "Go style" :-). So, IMHO It can be considered an orthodoxy CONS but not a big deal for us at all.
- Also, since it is one of the most used and popular GO libs, I personally cannot agree with con argumentation added as a con that the code shows written in a different language and if shows, what is the big deal?
IMHO, what are the CONS and PROS?
CONS:
- In the book, The Go Programming Language, has some argumentations against the Assertions approach in other languages which few may be applied regards the adoption of the testify/assert as for example it will not force the dev to add msg which can describe better the test executions. ( E.g allow failure messages like "assert: 0 == 1" ).
PROS:
- Kubernets and OpenShift use /testify/assert to implement their tests as many other OpenSource projects
- It is one most popular libs in GO. See here https://godoc.org/
- Less code is required == more test in less time
- Make our tests be smaller, cleaner, easier and more understandable.
- Make easier to receive contributions from devs that came from other languages, (highly important to us)
- The tests with asserts are more intuitive for the biggest part of devs
- Less effort to write tests encourage the projects has more tests. Implementations missing tests are a common scenario since many devs don't like to write them.
- Set of comprehensive testing tools for use with the normal Go testing system
- Provides additional features
- All Go testing system and tools will still available and working
- The usage of this lib did not create any impediment or impact on any project.
- The cons are not a really big thing for us and for the biggest part of the community
Cheers,
Craig Brookes
Feb 25, 2019, 5:19:41 AM2/25/19
to Camila Macedo, Aerogear
I don't really want to get into a flame war here, this is a little like tabs v spaces. Regardless of whether a lib is used or not, what actually matters is we write good tests for our code.
However I will address some of your points with my own opinions gathered throughout my time writing go code (quite a while now).
- Kubernets and OpenShift use /testify/assert to implement their tests as many other OpenSource projects
They do but it is quite limited and not surprising that in a huge open source project standards sometimes shift across working groups.
In kubernetes there are 165 files using testify vs 6528 using the testing library so there is a clear bias here.
grep "testify" -r pkg/ | grep "_test" | wc -l
grep "testing" -r pkg/ | grep "_test" | wc -l
- Less code is required == more test in less time
This is subjective it firsts means the developer must understand the DSL provided by the library rather than just understand regular go code.
- Make our tests be smaller, cleaner, easier and more understandable.
It makes them smaller, however more understandable is subjective. Less lines of code does not mean easier to understand what the code is doing.
- Make easier to receive contributions from devs that came from other languages, (highly important to us)
Disagree with this. If they can write go code they can read existing standard go tests written with the std lib. The testify assertion library may well work very differently to other languages' assertion libraries which can lead to subtle bugs. We need to be careful not to carry over biases from other languages / frameworks to new ones. It is important to understand the driving to work with the language rather than trying to force it to be like the language you prefer or came from.
- The tests with asserts are more intuitive for the biggest part of devs
Again this is subjective. If you can read regular go code you can read the tests. Go is optimised to be read not succinct.
- Less effort to write tests encourage the projects has more tests. Implementations missing tests are a common scenario since many devs don't like to write them.
You should optimise not for more tests but instead optimise for important tests that properly exercise methods and interactions.
- Less effort to write tests encourage the projects has more tests. Implementations missing tests are a common scenario since many devs don't like to write them.
This is not a language problem this is fundamental approach problem from these developers.
These are just some of my thoughts. Take from them what you will
Craig
--
You received this message because you are subscribed to the Google Groups "Aerogear" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aerogear+u...@googlegroups.com.
To post to this group, send email to aero...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aerogear/c10eb65e-2363-463d-8ace-1f2394b158f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Craig Brookes
Integreatly
@maleck13 Github
David Ffrench
Feb 25, 2019, 5:35:16 AM2/25/19
to Craig Brookes, Camila Macedo, Aerogear
Hey Camila,
You make very good points. Thank you very much for the code examples between the two testing methods using our own tests. Personally, I find it easier to read the tests wrote with assert, likely because I come from a Node.js assert testing background.
I can guarantee we are not the first group to debate this topic for Go and this debate is always going to be somewhat subjective. For me, consistency is key. I want to be able to switch to another internal Go project, such as the other Mobile Services and Integreatly and be able to write tests in the same way. Even in OpenShift, there are only 61 references[1] to assert while there are 955 references[2] to the standard testing library. Personally, I would prefer to use your proposed method. However, thinking about the larger picture, taking into account the other projects in cloud services and staying consistent, it is my recommendation to use with the Go standard library in the Mobile Security Service.
Best Regards,
DAVID FFRENCH
senior software engineer, RED HAT MOBILE
Communications House, Cork Road
Waterford, Ireland
|
To view this discussion on the web visit https://groups.google.com/d/msgid/aerogear/CA%2B-sgVh-GXu8RdmTqkC5izppzKayBWa%3D8xpZzwxH5oLONXKwvw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages