Migrating from Mockito 2.28 to 3.7 and many tests immediately fail

[John Guin]

Hello all,

I'm trying to get our test suite (java 11, maven, and a weird mishmash of Junit4 and Junit5) migrated from Mockito 2.28.2 to 3.7.7.

I have changed only the references in the POM from 2.28.2 to 3.7.7.  mockito-core also got changed to mockito-inline.

Test behavior is immediately erratic.  The "NotAMock" errors are easy to understand since the tests in question need to be updated.

But other tests intermittently fail with NPEs or ClassNotFound errors.  It's not 100% repro, either - sometimes testFoo passes, other times it throws an NPE.

There is probably something low level I have missed, but I have beat my head against stackoverflow, the mockito site and the search engines to the point that I cannot narrow anything down.

So, does anyone have any advice on how to proceed?

Should Mockito 3.7 be a simple upgrade from 2.28?  Or will every test that used 2.28 need to be manually updated?

And is mockito-inline a superset of mockito-core?  I intend to adopt mockito-inline so that we can use it to mock static methods - using powermockito to do this ties us to junit4 and we want to migrate off that as well.

Any pointers or tips appreciated - thanks!

[Tim van der Lippe]

Could you try using `mockito-core` first and see if that works? If that works, then the culprit is the inline mockmaker. If that doesn't work, could you try out different versions until you bisect to a version that breaks?

In general Mockito should be backwards compatible, but we might have missed something. I would recommend upgrading small parts at a time (e.g. a small version bump, changing the mockmaker) and test all of that in isolation. If you bundle all the changes at once, it becomes harder to figure out what is going wrong.

Op do 11 feb. 2021 om 20:30 schreef 'John Guin' via mockito <moc...@googlegroups.com>:

--
You received this message because you are subscribed to the Google Groups "mockito" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mockito+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mockito/7cc607a5-e93c-40e6-8acc-a7093dbb7969n%40googlegroups.com.

[John Guin]

Changing only mockito 2.28.2 to 3.7.7 (core) results in the same behavior.  I'll keep digging.

[John Buffington]

We have a few projects that have this issue but they are very big (old) classes with > 20k tests. I'm still digging into solutions but for now am using the retry plug in of gradle to work around that.

[John Guin]

I noticed that we were using powermock 2.0.4 (to help with mocking static methods).  I updated that to 2.0.9 based on this issue:

 

https://github.com/powermock/powermock/pull/1049  which supposedly updates powermock to be compatible with Mockito 3.x

 

that update was successful and allows for 100% of my legacy tests to run and pass with powermock 2.0.9 and mockito-core 3.7.7

 

But I still see hundreds of failures when I change only mockito-core to mockito-inline.

 

A key assumption I making, which may be wrong, is that mockito-inline is a superset of mockito-core.  Is this correct?

 

I may also have missed docs on what versions of other components are needed.  Examples would be powermock (now on the latest I could find), junit-platform, and others.  Is this documented anywhere?

 

Thanks for the help gang – I really appreciate it,

John

--
You received this message because you are subscribed to a topic in the Google Groups "mockito" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mockito/BlY1PyabJ94/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mockito+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mockito/218136e0-610f-428b-8753-b823414b6ac2n%40googlegroups.com.

[John Guin]

Some progress:

 

1. I moved the location of the <dependency> node for mockito-inline to come AFTER the dependency node for powermockito.  This, along with the changes to update mockito core to 3.7.7 and powermock to 2.0.9 now allow all existing tests to build and run without any updates needed.  Yay.  But having the order of dependencies in the maven POM be sensitive to changes doesn’t feel right…
2. I’m now trying to black box try the 16 different orders in which the dependencies for mockito-inline, mockito-core, mockito-junit-jupiter and Powermockito need to be added in our pom.  Again, it seems fragile that the order they are listed in the POM matters, but that is the behavior I am seeing.

 

 

 

From: John Guin <john...@smartsheet.com>
Sent: Friday, February 12, 2021 11:13 AM
To: 'moc...@googlegroups.com' <moc...@googlegroups.com>
Subject: RE: [mockito] Migrating from Mockito 2.28 to 3.7 and many tests immediately fail

 

I noticed that we were using powermock 2.0.4 (to help with mocking static methods).  I updated that to 2.0.9 based on this issue:

 

https://github.com/powermock/powermock/pull/1049  which supposedly updates powermock to be compatible with Mockito 3.x

 

that update was successful and allows for 100% of my legacy tests to run and pass with powermock 2.0.9 and mockito-core 3.7.7

 

But I still see hundreds of failures when I change only mockito-core to mockito-inline.

 

A key assumption I making, which may be wrong, is that mockito-inline is a superset of mockito-core.  Is this correct?

 

I may also have missed docs on what versions of other components are needed.  Examples would be powermock (now on the latest I could find), junit-platform, and others.  Is this documented anywhere?

 

Thanks for the help gang – I really appreciate it,

John

 

From: moc...@googlegroups.com <moc...@googlegroups.com> On Behalf Of John Buffington
Sent: Friday, February 12, 2021 8:56 AM
To: mockito <moc...@googlegroups.com>
Subject: Re: [mockito] Migrating from Mockito 2.28 to 3.7 and many tests immediately fail

 

We have a few projects that have this issue but they are very big (old) classes with > 20k tests. I'm still digging into solutions but for now am using the retry plug in of gradle to work around that.

--
You received this message because you are subscribed to a topic in the Google Groups "mockito" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mockito/BlY1PyabJ94/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mockito+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mockito/218136e0-610f-428b-8753-b823414b6ac2n%40googlegroups.com.

[John Guin]

Tvande...   changing back to mockito-core does indeed work, so this points at the inline mock maker.  I tried adding a MockMaker  to my classpath per https://javadoc.io/static/org.mockito/mockito-core/3.8.0/org/mockito/Mockito.html#39 but that did not change behavior.

Any other workarounds you know about?  I'm really trying to replace Powermockito (and its ability to mock static methods) with Mockito and this seems to be just about the last blocking issue.

Thanks for the help!

[Tim van der Lippe]

Sorry I think it mostly depends on your situation. One option you could explore is putting tests in a separate Maven/Gradle project and porting tests over one by one.

Op di 23 feb. 2021 om 16:19 schreef 'John Guin' via mockito <moc...@googlegroups.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/mockito/1be5a91a-b54d-471d-874f-6fc39c4d8136n%40googlegroups.com.

[John Guin]

Thanks for the quick reply – I’ll see if I can narrow this down any further.

To view this discussion on the web visit https://groups.google.com/d/msgid/mockito/CAAYcR_J7JC8yFjADyObdadpf45fHnNanFn_9JEankmMi04%2BwQw%40mail.gmail.com.

[Evgen Kamensky]

I have the same trouble with migration now.
Some failed test solve with https://github.com/powermock/powermock/wiki/mockito#mockito-mock-maker-inline

But have some flaky test when use Robolectric-Mockito-PowerMock in one test. I solve it when stop using org.mockito.Mock with org.powermock.api.mockito.PowerMockito.mock together in one test.

среда, 24 февраля 2021 г. в 00:54:01 UTC+3, john...@smartsheet.com:

[John Guin]

So the upshot of this was there was a conflict with having BOTH Powermockito 2.0.9 AND Mockito 3.7.7 in use.

I found this when I updated our POM to use Mockito 3.7.7 : this was the only change to the POM.

Then if I added this import to ANY test, any test using mocking would fail:

import org.mockito.MockedStatic;

The important clue here was that I was not actually using MockedStatic at this point - only the Mockito version had changed.

I figured the compiler would have thrown out unused import classes, but that was not the case.

long story short:  I bit the bullet and in one MR removed PowerMockito 2.0.9, updated to Mockito 3.7.7 and rewrote all the tests that used Mockito.

I really wish we could have used both during a transition period, but that simply was not possible.

In any case, we are now on Mockito 3.7.7.