Dependency problems during test-execution after upgrade to ReSharper 10

[Sebastian P.]

I'm just in the process of upgrading my ReSharper plugin from ReSharper 9.x to ReSharper 10.

First of all: the upgrade process was *way* easier than the upgrade from R#8 to R#9... I only had to adapt a single breaking change in of of your interfaces. Thanks for that!

Unfortunately, while it was super easy to get the project to a compiling state again, I observer strange dependency errors now when executing one of the test projects [3]. The error message is

Loading an assembly “JetBrains.UsageStatistics, Version=104.0.0.0, Culture=neutral, PublicKeyToken=1010a0d8d6380325” which is not listed in assembly references of assembly “KaVE.RS.Commons, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null” when trying to resolve the type JetBrains.UsageStatistics.FeedbackSettings. A type reference points into an assembly not listed among assembly references.

You will find the complete stack trace at [1]. The "Jetbrains.UsageStatistics" dependency is listed among the references of the mentioned project [2] though.

When I remove the [SetupFixture] in the assembly setup fixture [4], no dependency error is reported anymore, but all test finish with the exception "System.InvalidOperationException : The test host is not running.", which makes sense, because no shell is started.

Another thing that I noticed is that the build reports two warnings for all projects, saying that the processor architecture of the project is "MSIL", but that it is "AMD64" for the reference "JetBrains.ReSharper.TaskRunner.x64" and "x86" for "JetBrains.ReSharper.Psi.Cpp". All my projects are configured to be "AnyCPU", but even selecting a specific arch would not solve both warnings, because they exclude each other. Not sure if this is at all related to the problem though.

I tried to fix the problem by testing different combinations of x86/x64, by removing/adding the ReSharper NuGet packages again, and even by creating new projects from scratch to make sure that nothing is messed up in the .csproj file... without success. I'm really running out of ideas how to fix this.

Any input or pointers to further material are highly appreciated!

best

Sebastian

[1] http://pastebin.com/1DCq1Cd1

[2] https://github.com/stg-tud/kave/tree/master/feedback-generator/KaVE.RS.Commons

[3] https://github.com/stg-tud/kave/blob/master/feedback-generator/KaVE.RS.Commons.Tests_Integration

[4] https://github.com/stg-tud/kave/blob/master/feedback-generator/KaVE.RS.Commons.Tests_Integration/Properties/AssemblyInfo.cs

[Sebastian P.]

PS: I just upgraded the R#10 SDK to the EAP2 that is available as a prerelease... same error.

[Sebastian P.]

PS-2: I just downgraded the master branch to R#9, I guess I was too optimistic here... I moved my unsuccessful upgrade attempt to an upgrade branch [1]. The question is still valid though, I'd love to upgrade the plugin! :)

[1] https://github.com/stg-tud/kave/tree/resharper-10-upgrade

[Matt Ellis]

Hi Sebastian. Unfortunately, tests are a bit dodgy in ReSharper 10, we will have them working again for 10.0.2, but we're not there yet.

You can sometimes get them to work by making your test project .net 4.5 and x86 only, but there shouldn't actually be a requirement for that. However, this doesn't work in all cases - the issue is with the dependency graph we build to fire up the Component Model, we've got some problems in there.

While it does making testing plugins harder (or impossible!), the plugins themselves aren't affected by this, and still work. So, for now, I'm afraid manual testing is the best we can do. Apologies.

Matt

[Sebastian P.]

Hey Matt, thanks for the honest answer. I tried CLR45 and x86, but it did not make a difference. After reading your post, I had the idea that I could simply downgrade my ReSharper installation to v9.2 to solve the dependency issues for the test runs of my v10 plugin and run a v10 installation in the experimental instance for manually testing the plugin. Unfortunately, the error stays the same... :/

Is there an *approximate* time slot in which we can expect the testing fixes to be included in the EAP version of the SDK?

best

Sebastian

[Matt Ellis]

The SDK errors don't have anything to do with the hosting instance of ReSharper, so downgrading won't make any difference. The tests are run in an external process, and load the assemblies from the output directory - and these are the files from the SDK packages.

I don't have a timeline for the fixes. 10.0.2 is going to be replaced in a couple of weeks, before Christmas, and we should have the fixes in place for that. They might make it into an EAP build before hand, but I don't know when. Here's the issue to track, comment and vote: RSRP-450532.

Matt

[Sebastian P.]

Ok, thanks for the clarification. I just added my vote and I'm looking forward to the fix. :)

best

Sebastian

[Sebastian P.]

Hey,

I have not seen any update on the bug report, yet... is there any news about this issue (or at least a timeline)?

best

Sebastian

[Matt Ellis]

Can you try updating to the latest SDK? It shouldn't have any breaking changes in it, but should hopefully fix the issues you've been seeing (I'm concerned there's another issue, so it would be useful if you could test, please).

Regards

Matt

[Sven Amann]

I updated the SDK and now I'm able to execute the tests. Tests fail now, with one or the other variant of an NPE accessing the "projectFileType". I found a respective issue already in your tracker. Unfortunately it's unresolved and I cannot make much out of the single comment. Can you tell me more about it?

Best,

Sven

[Matt Ellis]

Hmm. I'm not sure, to be honest. There shouldn't be any problems parsing asp files in tests. What does your test environment TestFixtureSetUp look like? Also, try cleaning bin, obj and %TEMP%.

[Sven Amann]

Interesting, now that you mention it... we're not using any ASP. It's a pure C# project.

The setup looks like this:

namespace KaVE.RS.Commons.Tests_Integration

{

    [ZoneDefinition]

    public interface IThisTestZone : ITestsZone, IRequire<PsiFeatureTestZone> {}

    [SetUpFixture]

    public class TestEnvironmentAssembly : ExtensionTestEnvironmentAssembly<IThisTestZone>

    {

        public override void TearDown()

        {

            try

            {

                base.TearDown();

            }

            catch (Exception e)

            {

                Console.WriteLine(e);

            }

        }

    }

    [ZoneMarker]

    public class ZoneMarker {}

}

[Matt Ellis]

Actually, looking at the call stack, it's not necessarily parsing an aspx file, but it is looking to try and parse "secondary" files (an in-memory generated file that can provide mapping to code "islands" in the main file, e.g. a generated C# file for .aspx C# code blocks). Something's failing when it's trying to do this, either because it can't find the file type of the file being added to the project (your test file), or it hasn't initialised the file type for the generated document, which is pretty weird, and I don't know how it can get into this state.

[Sven Amann]

Ok, I'm 100% sure there is no ASP or ASP code block in that project. The test files are all pure C#. May it be that I miss something in the test setup, such that it defaults to ASP where it should be used C#?

[Matt Ellis]

No, the issue is that it's asking each secondary document handler if it supports your file. One of these will be aspx, but that's not necessarily what's causing the issue. It's either the file you're trying to test isn't recognised (which if it's .cs, it really should be), or one of the languages that one of the secondary document handlers is using isn't initialised (which it should be, if it wasn't, then it shouldn't be enumerated). Something's gone weird, which is why I suggested the cleanup of bin, obj and %TEMP%, in case something was hanging around that shouldn't be.

[Sven Amann]

Did that. No change....

[Sven Amann]

I still thinks it must be the setup, rather than anything specific to the test files, as I have over 600 tests with .cs files and one test with a .xaml file, which all fail with the same exception.

All the tests inherit from your CodeCompletionTestBase through our BaseCodeCompletionTest and BaseCSharpCodeCompletionTest. Maybe something in CodeCompletionTestBase changed with RS10?

[Sebastian P.]

I gave it a second shot. This time, I used a minimal example for this that I created back in the day when I was debugging upgrade problems to ReSharper 9.1. The creation of this example really payed off... you guys somehow manage to break our plugin with every major release :P

With this example, I managed to reproduce the bug described by Sven. Unfortunately, deleting "bin", "obj", and "%Temp%" did not help, but upgrading to R#10.1-EAP did. I guess this means that we are not able to ship for R#10.0 and that we have to wait for R#10.1. :/

best

Sebastian

[Matt Ellis]

If you can get a clean test run when in 10.1EAP, then you can ship for 10.0.2. The issue we were hitting there is that the 10.0.2 runner would load its own nunit dlls instead of the ones from your bin folder, which meant the component model couldn't wire everything up properly as it had types from different assemblies, which aren't considered equal. But when run in 10.1, the versions of the dlls are different, so they don't get confused, and you can test again. We'll make sure this is fixed for the 10.1 SDK (that is, 10.1 will be able to test 10.1 plugins).

What you can do is install 10.0.2 into a "custom hive" - a separate VS configuration. Run the installer for 10.0.2, and select Options. You can now set up the name of a custom hive (also known as an experimental instance), e.g. "ReSharper". ReSharper will create this VS instance, and create a separate ReSharper install. You can now run that instance with "devenv.exe /rootSuffix ReSharper" (or whatever name you gave the custom hive). And now you have an instance of 10.0.2 running side by side with 10.1 EAP. You can build and test in 10.1, and deploy and manually test in 10.0.2.

Regards

Matt

[Sebastian P.]

Some time has passed, but 10.1 is still not released. For us, this is really a pain point... :/ Is there any news about when to expect the release?