I recently installed NUnit 2.6 on Windows 7. I have a single assembly
which contains the tests. I use the NUnit GUI. The tests appear to
execute correctly, from the NUnit perspective, if I run the entire
suite of tests. If I run just a subset of the tests some number
always fail with a module load exception which has a
SerializationException at the root of the exception chain.
I can successfully run a subset of the tests if I first run the entire
suite. However, if I exit the GUI and restart and run just a subset I
get the module load exception.
Any idea what's going on?
Also, when I get this module load exception is doesn't show up in the
results pane. Almost like it's white text on a white background. If
I use the horizontal scrollbar to scroll then I see highlighted text
show up in the results pane.
This sounds like it should be a problem with your tests. Some test is required to be run before any others can work. Of course, such a dependency is bad and you would want to root it out.
To verify that this is the problem, first use NUnit's own tests. You should be able to select and run a subset with no problem.
If that works, use something like a binary search to find the problem. Turn on checkboxes in the Gui. Select the first test you know of that fails when run alone. See it fail. Now select in addition half of the previously unselected tests above it. If it passes, the failing test depends on one of those you just selected. If it still fails, then the culprit is one of the tests you did not select. Proceed till you find the test which causes the problem.
Warning: This will only work if the problem is one simple dependency.
On Fri, Mar 30, 2012 at 6:32 PM, nickdu <nic...@msn.com> wrote: > I recently installed NUnit 2.6 on Windows 7. I have a single assembly > which contains the tests. I use the NUnit GUI. The tests appear to > execute correctly, from the NUnit perspective, if I run the entire > suite of tests. If I run just a subset of the tests some number > always fail with a module load exception which has a > SerializationException at the root of the exception chain.
> I can successfully run a subset of the tests if I first run the entire > suite. However, if I exit the GUI and restart and run just a subset I > get the module load exception.
> Any idea what's going on?
> Also, when I get this module load exception is doesn't show up in the > results pane. Almost like it's white text on a white background. If > I use the horizontal scrollbar to scroll then I see highlighted text > show up in the results pane.
> Thanks, > Nick
> -- > You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group. > To post to this group, send email to nunit-discuss@googlegroups.com. > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/nunit-discuss?hl=en.
Thanks. I will try that out but I doubt that is the issue. I'm
almost positive none of the tests are dependent on the others. Also,
in this exception chain somewhere, I guess between the module load
exception and the serialization exception as I believe those are the
first and last respectfully, is an exception indicating it can't find
one of my managed assemblies which is required. The assembly is
there, obviously as it works when I run all tests.
By the way, my NUnit test assembly is targeting the .NET 4.0 runtime.
We also have some unmanaged assemblies brought into the picture by
some third party managed assemblies. Not sure if those are causing
any issues, but as I said everything works as I expect when I run all
tests.
I think it's some sort of assembly loading issue, not sure if it's
caused by NUnit or not. The reason I say this is because of the tests
that fail. They appear to be, but I have to do more work here to
verify, the methods that get past my argument validation code in my
API and then eventually call into this third party library.
Thanks,
Nick
On Mar 30, 10:11 pm, Charlie Poole <nunit...@gmail.com> wrote:
> This sounds like it should be a problem with your tests. Some test is
> required to be run before any others can work. Of course, such a
> dependency is bad and you would want to root it out.
> To verify that this is the problem, first use NUnit's own tests. You
> should be able to select and run a subset with no problem.
> If that works, use something like a binary search to find the problem.
> Turn on checkboxes in the Gui. Select the first test you know of that
> fails when run alone. See it fail. Now select in addition half of the
> previously unselected tests above it. If it passes, the failing test
> depends on one of those you just selected. If it still fails, then the
> culprit is one of the tests you did not select. Proceed till you find
> the test which causes the problem.
> Warning: This will only work if the problem is one simple dependency.
> Charlie
> On Fri, Mar 30, 2012 at 6:32 PM, nickdu <nic...@msn.com> wrote:
> > I recently installed NUnit 2.6 on Windows 7. I have a single assembly
> > which contains the tests. I use the NUnit GUI. The tests appear to
> > execute correctly, from the NUnit perspective, if I run the entire
> > suite of tests. If I run just a subset of the tests some number
> > always fail with a module load exception which has a
> > SerializationException at the root of the exception chain.
> > I can successfully run a subset of the tests if I first run the entire
> > suite. However, if I exit the GUI and restart and run just a subset I
> > get the module load exception.
> > Any idea what's going on?
> > Also, when I get this module load exception is doesn't show up in the
> > results pane. Almost like it's white text on a white background. If
> > I use the horizontal scrollbar to scroll then I see highlighted text
> > show up in the results pane.
> > Thanks,
> > Nick
> > --
> > You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
> > To post to this group, send email to nunit-discuss@googlegroups.com.
> > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/nunit-discuss?hl=en.
It appears my assumption is somewhat correct about which methods are
failing. It appears the .NET runtime is having a problem loading
Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the
same directory as my NUnit test assembly and I have all the other
dependent assemblies there. This directory is c:\data\development
\Redbone\bpf\tests\bin\debug. However, when the fusion loader looks
for the Common assembly it's looking in the app base directory where
the NUnit GUI .exe exists. I thought NUnit somehow changed the app
base (maybe by creating another appdomain and loading the tests from
there) to the directory where the test assembly exists? My main
framework API, which all the tests make use of, gets loaded and that's
in the same directory as the test assembly. Here is the log from the
fusion log viewer:
The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file
specified.
Assembly manager loaded from: C:\Windows\Microsoft.NET
\Framework64\v4.0.30319\clr.dll
Running under executable C:\Program Files (x86)\NUnit 2.6\bin\nunit-
agent.exe
--- A detailed error log follows.
=== Pre-bind state information ===
LOG: User = redbonemobile\nickdu
LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
(Partial)
WRN: Partial binding information was supplied for an assembly:
WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
ID: 1
WRN: A partial bind occurs when only part of the assembly display name
is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity
for the assembly,
WRN: that consists of the simple name, version, culture, and public
key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for
more information and common solutions to this issue.
LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = nunit-agent.exe
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Program Files
(x86)\NUnit 2.6\bin\nunit-agent.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET
\Framework64\v4.0.30319\config\machine.config.
LOG: Policy not being applied to reference at this time (private,
custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/
NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
Redbone.Framework.BusinessProcess.Common.EXE.
LOG: All probing URLs attempted and failed.
Thanks,
Nick
On Mar 31, 9:19 am, nickdu <nic...@msn.com> wrote:
> Thanks. I will try that out but I doubt that is the issue. I'm
> almost positive none of the tests are dependent on the others. Also,
> in this exception chain somewhere, I guess between the module load
> exception and the serialization exception as I believe those are the
> first and last respectfully, is an exception indicating it can't find
> one of my managed assemblies which is required. The assembly is
> there, obviously as it works when I run all tests.
> By the way, my NUnit test assembly is targeting the .NET 4.0 runtime.
> We also have some unmanaged assemblies brought into the picture by
> some third party managed assemblies. Not sure if those are causing
> any issues, but as I said everything works as I expect when I run all
> tests.
> I think it's some sort of assembly loading issue, not sure if it's
> caused by NUnit or not. The reason I say this is because of the tests
> that fail. They appear to be, but I have to do more work here to
> verify, the methods that get past my argument validation code in my
> API and then eventually call into this third party library.
> Thanks,
> Nick
> On Mar 30, 10:11 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > This sounds like it should be a problem with your tests. Some test is
> > required to be run before any others can work. Of course, such a
> > dependency is bad and you would want to root it out.
> > To verify that this is the problem, first use NUnit's own tests. You
> > should be able to select and run a subset with no problem.
> > If that works, use something like a binary search to find the problem.
> > Turn on checkboxes in the Gui. Select the first test you know of that
> > fails when run alone. See it fail. Now select in addition half of the
> > previously unselected tests above it. If it passes, the failing test
> > depends on one of those you just selected. If it still fails, then the
> > culprit is one of the tests you did not select. Proceed till you find
> > the test which causes the problem.
> > Warning: This will only work if the problem is one simple dependency.
> > Charlie
> > On Fri, Mar 30, 2012 at 6:32 PM, nickdu <nic...@msn.com> wrote:
> > > I recently installed NUnit 2.6 on Windows 7. I have a single assembly
> > > which contains the tests. I use the NUnit GUI. The tests appear to
> > > execute correctly, from the NUnit perspective, if I run the entire
> > > suite of tests. If I run just a subset of the tests some number
> > > always fail with a module load exception which has a
> > > SerializationException at the root of the exception chain.
> > > I can successfully run a subset of the tests if I first run the entire
> > > suite. However, if I exit the GUI and restart and run just a subset I
> > > get the module load exception.
> > > Any idea what's going on?
> > > Also, when I get this module load exception is doesn't show up in the
> > > results pane. Almost like it's white text on a white background. If
> > > I use the horizontal scrollbar to scroll then I see highlighted text
> > > show up in the results pane.
> > > Thanks,
> > > Nick
> > > --
> > > You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
> > > To post to this group, send email to nunit-discuss@googlegroups.com.
> > > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com.
> > > For more options, visit this group athttp://groups.google.com/group/nunit-discuss?hl=en.
From the error below, it's clear that NUnit itself - rather than your tests - is trying to load the dll in question. This usually happens when a custom exception, defined in one of your assemblies, is uncaught in the tests. Is that a possibility in this scenario?
If this is the case, then NUnit would be failing in deserializing the exceptioni because it can't load the assembly in which the exception is defined. One way to find out is to temporarily copy the "missing" assembly to the directory containing nunit-agent.exe, along with any dependencies that may be needed.
This issue with custom exceptions may or may not be your problem, but it's a long-standing issue and I suggest looking at it next.
BTW, in NUnit 3.0, the plan is to prevent any exceptions from travelling across the remoting boundary by catching them and transforming them into an xml error report.
On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote: > It appears my assumption is somewhat correct about which methods are > failing. It appears the .NET runtime is having a problem loading > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the > same directory as my NUnit test assembly and I have all the other > dependent assemblies there. This directory is c:\data\development > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks > for the Common assembly it's looking in the app base directory where > the NUnit GUI .exe exists. I thought NUnit somehow changed the app > base (maybe by creating another appdomain and loading the tests from > there) to the directory where the test assembly exists? My main > framework API, which all the tests make use of, gets loaded and that's > in the same directory as the test assembly. Here is the log from the > fusion log viewer:
> === Pre-bind state information === > LOG: User = redbonemobile\nickdu > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common > (Partial) > WRN: Partial binding information was supplied for an assembly: > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain > ID: 1 > WRN: A partial bind occurs when only part of the assembly display name > is provided. > WRN: This might result in the binder loading an incorrect assembly. > WRN: It is recommended to provide a fully specified textual identity > for the assembly, > WRN: that consists of the simple name, version, culture, and public > key token. > WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for > more information and common solutions to this issue. > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/ > LOG: Initial PrivatePath = NULL > LOG: Dynamic Base = NULL > LOG: Cache Base = NULL > LOG: AppName = nunit-agent.exe > Calling assembly : (Unknown). > === > LOG: This bind starts in default load context. > LOG: Using application configuration file: C:\Program Files > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config > LOG: Using host configuration file: > LOG: Using machine configuration file from C:\Windows\Microsoft.NET > \Framework64\v4.0.30319\config\machine.config. > LOG: Policy not being applied to reference at this time (private, > custom, partial, or location-based assembly bind). > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.DLL. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.EXE. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.EXE. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE. > LOG: Attempting download of new URL file:///C:/Program Files (x86)/ > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/ > Redbone.Framework.BusinessProcess.Common.EXE. > LOG: All probing URLs attempted and failed.
> Thanks, > Nick
> On Mar 31, 9:19 am, nickdu <nic...@msn.com> wrote: >> Thanks. I will try that out but I doubt that is the issue. I'm >> almost positive none of the tests are dependent on the others. Also, >> in this exception chain somewhere, I guess between the module load >> exception and the serialization exception as I believe those are the >> first and last respectfully, is an exception indicating it can't find >> one of my managed assemblies which is required. The assembly is >> there, obviously as it works when I run all tests.
>> By the way, my NUnit test assembly is targeting the .NET 4.0 runtime. >> We also have some unmanaged assemblies brought into the picture by >> some third party managed assemblies. Not sure if those are causing >> any issues, but as I said everything works as I expect when I run all >> tests.
>> I think it's some sort of assembly loading issue, not sure if it's >> caused by NUnit or not. The reason I say this is because of the tests >> that fail. They appear to be, but I have to do more work here to >> verify, the methods that get past my argument validation code in my >> API and then eventually call into this third party library.
>> Thanks, >> Nick
>> On Mar 30, 10:11 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> > This sounds like it should be a problem with your tests. Some test is >> > required to be run before any others can work. Of course, such a >> > dependency is bad and you would want to root it out.
>> > To verify that this is the problem, first use NUnit's own tests. You >> > should be able to select and run a subset with no problem.
>> > If that works, use something like a binary search to find the problem. >> > Turn on checkboxes in the Gui. Select the first test you know of that >> > fails when run alone. See it fail. Now select in addition half of the >> > previously unselected tests above it. If it passes, the failing test >> > depends on one of those you just selected. If it still fails, then the >> > culprit is one of the tests you did not select. Proceed till you find >> > the test which causes the problem.
>> > Warning: This will only work if the problem is one simple dependency.
>> > Charlie
>> > On Fri, Mar 30, 2012 at 6:32 PM, nickdu <nic...@msn.com> wrote: >> > > I recently installed NUnit 2.6 on Windows 7. I have a single assembly >> > > which contains the tests. I use the NUnit GUI. The tests appear to >> > > execute correctly, from the NUnit perspective, if I run the entire >> > > suite of tests. If I run just a subset of the tests some number >> > > always fail with a module load exception which has a >> > > SerializationException at the root of the exception chain.
>> > > I can successfully run a subset of the tests if I first run the entire >> > > suite. However, if I exit the GUI and restart and run just a subset I >> > > get the module load exception.
>> > > Any idea what's going on?
>> > > Also, when I get this module load exception is doesn't show up in the >> > > results pane. Almost like it's white text on a white background. If >> > > I use the horizontal scrollbar to scroll then I see highlighted text >> > > show up in the results pane.
>> > > Thanks, >> > > Nick
>> > > -- >> > > You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group. >> > > To post to this group, send email to nunit-discuss@googlegroups.com. >> > > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com. >> > > For more options, visit this group athttp://groups.google.com/group/nunit-discuss?hl=en.
> -- > You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group. > To post to this group, send email to nunit-discuss@googlegroups.com. > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/nunit-discuss?hl=en.
While we do throw some custom exceptions, my initial guess is that
this is not what's going on here because some of the tests that are
failing are positive tests, ones which shouldn't be throwing any
exceptions.
I just ran another scenario to maybe shed some more light on the
issue, though it doesn't shed much more light. I have a class called
Complete which has a bunch of tests which test our Complete method.
These tests always work. I have another class called Lock which tests
our Lock method. Lock doesn't work if run by itself (e.g. only
itself) after just starting the NUnit GUI. I mentioned originally
that if I run all the tests these other ones that fail by themselves
will succeed. That's not entirely true. If I start NUnit GUI up
fresh and then run the Complete tests I can then run the Lock tests,
using the currently running NUnit GUI that ran the Complete tests,
successfully.
Here is what my guess was as to what's going on. It appears the
tests, and I mean the code in the NUnit test methods, which have code
which directly references code from the Common assembly, this is the
assembly that's having the issue of getting loaded, will always work
whether run separately or when running all tests. The tests which
don't reference code from the Common assembly won't work unless the
Common assembly has already been loaded into the process by first
running a test that referenced it. Almost as if the code is getting
jitted with an app base setting which is correct and allows the .NET
assembly loader to find the Common assembly using the usual probing
mechanism, but then executed in a different appdomain which doesn't
have the correct app base setting such that if/when Common needs to be
loaded it cannot be found.
Thanks,
Nick
On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
> From the error below, it's clear that NUnit itself - rather than your
> tests - is trying to load the dll in question. This usually happens
> when a custom exception, defined in one of your assemblies, is
> uncaught in the tests. Is that a possibility in this scenario?
> If this is the case, then NUnit would be failing in deserializing the
> exceptioni because it can't load the assembly in which the exception
> is defined. One way to find out is to temporarily copy the "missing"
> assembly to the directory containing nunit-agent.exe, along with any
> dependencies that may be needed.
> This issue with custom exceptions may or may not be your problem, but
> it's a long-standing issue and I suggest looking at it next.
> BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> travelling across the remoting boundary by catching them and
> transforming them into an xml error report.
> Charlie
> On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote:
> > It appears my assumption is somewhat correct about which methods are
> > failing. It appears the .NET runtime is having a problem loading
> > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the
> > same directory as my NUnit test assembly and I have all the other
> > dependent assemblies there. This directory is c:\data\development
> > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks
> > for the Common assembly it's looking in the app base directory where
> > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> > base (maybe by creating another appdomain and loading the tests from
> > there) to the directory where the test assembly exists? My main
> > framework API, which all the tests make use of, gets loaded and that's
> > in the same directory as the test assembly. Here is the log from the
> > fusion log viewer:
> > === Pre-bind state information ===
> > LOG: User = redbonemobile\nickdu
> > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
> > (Partial)
> > WRN: Partial binding information was supplied for an assembly:
> > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
> > ID: 1
> > WRN: A partial bind occurs when only part of the assembly display name
> > is provided.
> > WRN: This might result in the binder loading an incorrect assembly.
> > WRN: It is recommended to provide a fully specified textual identity
> > for the assembly,
> > WRN: that consists of the simple name, version, culture, and public
> > key token.
> > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for > > more information and common solutions to this issue.
> > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
> > LOG: Initial PrivatePath = NULL
> > LOG: Dynamic Base = NULL
> > LOG: Cache Base = NULL
> > LOG: AppName = nunit-agent.exe
> > Calling assembly : (Unknown).
> > ===
> > LOG: This bind starts in default load context.
> > LOG: Using application configuration file: C:\Program Files
> > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config
> > LOG: Using host configuration file:
> > LOG: Using machine configuration file from C:\Windows\Microsoft.NET
> > \Framework64\v4.0.30319\config\machine.config.
> > LOG: Policy not being applied to reference at this time (private,
> > custom, partial, or location-based assembly bind).
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.DLL.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > Redbone.Framework.BusinessProcess.Common.EXE.
> > LOG: All probing URLs attempted and failed.
> > Thanks,
> > Nick
> > On Mar 31, 9:19 am, nickdu <nic...@msn.com> wrote:
> >> Thanks. I will try that out but I doubt that is the issue. I'm
> >> almost positive none of the tests are dependent on the others. Also,
> >> in this exception chain somewhere, I guess between the module load
> >> exception and the serialization exception as I believe those are the
> >> first and last respectfully, is an exception indicating it can't find
> >> one of my managed assemblies which is required. The assembly is
> >> there, obviously as it works when I run all tests.
> >> By the way, my NUnit test assembly is targeting the .NET 4.0 runtime.
> >> We also have some unmanaged assemblies brought into the picture by
> >> some third party managed assemblies. Not sure if those are causing
> >> any issues, but as I said everything works as I expect when I run all
> >> tests.
> >> I think it's some sort of assembly loading issue, not sure if it's
> >> caused by NUnit or not. The reason I say this is because of the tests
> >> that fail. They appear to be, but I have to do more work here to
> >> verify, the methods that get past my argument validation code in my
> >> API and then eventually call into this third party library.
> >> Thanks,
> >> Nick
> >> On Mar 30, 10:11 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> > This sounds like it should be a problem with your tests. Some test is
> >> > required to be run before any others can work. Of course, such a
> >> > dependency is bad and you would want to root it out.
> >> > To verify that this is the problem, first use NUnit's own tests. You
> >> > should be able to select and run a subset with no problem.
> >> > If that works, use something like a binary search to find the problem.
> >> > Turn on checkboxes in the Gui. Select the first test you know of that
> >> > fails when run alone. See it fail. Now select in addition half of the
> >> > previously unselected tests above it. If it passes, the failing test
> >> > depends on one of those you just selected. If it still fails, then the
> >> > culprit is one of the tests you did not select. Proceed till you find
> >> > the test which causes the problem.
I did some more tests, and while I found out some new behavior I still
don't know what's going on here.
In order to test your custom exception suggestion I figured I would
take one of the tests that failed and catch all exceptions and turn
them into an InvalidOperationException, e.g like the following:
catch(Exception e)
{
throw(new InvalidOperationException(string.Format("Caught an
exception: {0}", e.ToString())));
}
I then ran just that test and much to my surprise it succeeded. So
then I commented out the try/catch I added and rebuilt and retested,
just that test, and again it succeeded. I tested, individually which
a fresh new instance of NUnit GUI, each test that failed and they all
succeeded. For lock I have tests1 - test8. When I run all lock
tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 -
test8 individually with a fresh instance of NUnit GUI they all
succeed. The reason for running fresh instances of NUnit GUI is
because it seems once Common gets loaded then all tests from them on
will work as expected. Also, once loading Common fails it will
continue to fail for that instance of NUnit GUI (looks like a type
initializer, e.g. static constructor, fails and once that happens the
there's no fixing it until the appdomain is unloaded).
> While we do throw some custom exceptions, my initial guess is that
> this is not what's going on here because some of the tests that are
> failing are positive tests, ones which shouldn't be throwing any
> exceptions.
> I just ran another scenario to maybe shed some more light on the
> issue, though it doesn't shed much more light. I have a class called
> Complete which has a bunch of tests which test our Complete method.
> These tests always work. I have another class called Lock which tests
> our Lock method. Lock doesn't work if run by itself (e.g. only
> itself) after just starting the NUnit GUI. I mentioned originally
> that if I run all the tests these other ones that fail by themselves
> will succeed. That's not entirely true. If I start NUnit GUI up
> fresh and then run the Complete tests I can then run the Lock tests,
> using the currently running NUnit GUI that ran the Complete tests,
> successfully.
> Here is what my guess was as to what's going on. It appears the
> tests, and I mean the code in the NUnit test methods, which have code
> which directly references code from the Common assembly, this is the
> assembly that's having the issue of getting loaded, will always work
> whether run separately or when running all tests. The tests which
> don't reference code from the Common assembly won't work unless the
> Common assembly has already been loaded into the process by first
> running a test that referenced it. Almost as if the code is getting
> jitted with an app base setting which is correct and allows the .NET
> assembly loader to find the Common assembly using the usual probing
> mechanism, but then executed in a different appdomain which doesn't
> have the correct app base setting such that if/when Common needs to be
> loaded it cannot be found.
> Thanks,
> Nick
> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > Hi Nick,
> > From the error below, it's clear that NUnit itself - rather than your
> > tests - is trying to load the dll in question. This usually happens
> > when a custom exception, defined in one of your assemblies, is
> > uncaught in the tests. Is that a possibility in this scenario?
> > If this is the case, then NUnit would be failing in deserializing the
> > exceptioni because it can't load the assembly in which the exception
> > is defined. One way to find out is to temporarily copy the "missing"
> > assembly to the directory containing nunit-agent.exe, along with any
> > dependencies that may be needed.
> > This issue with custom exceptions may or may not be your problem, but
> > it's a long-standing issue and I suggest looking at it next.
> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> > travelling across the remoting boundary by catching them and
> > transforming them into an xml error report.
> > Charlie
> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote:
> > > It appears my assumption is somewhat correct about which methods are
> > > failing. It appears the .NET runtime is having a problem loading
> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the
> > > same directory as my NUnit test assembly and I have all the other
> > > dependent assemblies there. This directory is c:\data\development
> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks
> > > for the Common assembly it's looking in the app base directory where
> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> > > base (maybe by creating another appdomain and loading the tests from
> > > there) to the directory where the test assembly exists? My main
> > > framework API, which all the tests make use of, gets loaded and that's
> > > in the same directory as the test assembly. Here is the log from the
> > > fusion log viewer:
> > > === Pre-bind state information ===
> > > LOG: User = redbonemobile\nickdu
> > > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
> > > (Partial)
> > > WRN: Partial binding information was supplied for an assembly:
> > > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
> > > ID: 1
> > > WRN: A partial bind occurs when only part of the assembly display name
> > > is provided.
> > > WRN: This might result in the binder loading an incorrect assembly.
> > > WRN: It is recommended to provide a fully specified textual identity
> > > for the assembly,
> > > WRN: that consists of the simple name, version, culture, and public
> > > key token.
> > > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for > > > more information and common solutions to this issue.
> > > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
> > > LOG: Initial PrivatePath = NULL
> > > LOG: Dynamic Base = NULL
> > > LOG: Cache Base = NULL
> > > LOG: AppName = nunit-agent.exe
> > > Calling assembly : (Unknown).
> > > ===
> > > LOG: This bind starts in default load context.
> > > LOG: Using application configuration file: C:\Program Files
> > > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config
> > > LOG: Using host configuration file:
> > > LOG: Using machine configuration file from C:\Windows\Microsoft.NET
> > > \Framework64\v4.0.30319\config\machine.config.
> > > LOG: Policy not being applied to reference at this time (private,
> > > custom, partial, or location-based assembly bind).
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: All probing URLs attempted and failed.
> > > Thanks,
> > > Nick
> > > On Mar 31, 9:19 am, nickdu <nic...@msn.com> wrote:
> > >> Thanks. I will try that out but I doubt that is the issue. I'm
> > >> almost positive none of the tests are dependent on the others. Also,
> > >> in this exception chain somewhere, I guess between the module load
> > >> exception and the serialization exception as I believe those are the
> > >> first and last respectfully, is an exception indicating it can't find
> > >> one of my managed assemblies which is required. The assembly is
> > >> there, obviously as
Still, this doesn't explain why your tests have anything at all to do with the primary AppDomain. NUnit does run all tests in a separate AppDomain and creates a new one each time you reload the tests.
Have you turned on NUnit's internal trace and looked at whether it tells you anything?
On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote: > I did some more tests, and while I found out some new behavior I still > don't know what's going on here.
> In order to test your custom exception suggestion I figured I would > take one of the tests that failed and catch all exceptions and turn > them into an InvalidOperationException, e.g like the following:
> I then ran just that test and much to my surprise it succeeded. So > then I commented out the try/catch I added and rebuilt and retested, > just that test, and again it succeeded. I tested, individually which > a fresh new instance of NUnit GUI, each test that failed and they all > succeeded. For lock I have tests1 - test8. When I run all lock > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 - > test8 individually with a fresh instance of NUnit GUI they all > succeed. The reason for running fresh instances of NUnit GUI is > because it seems once Common gets loaded then all tests from them on > will work as expected. Also, once loading Common fails it will > continue to fail for that instance of NUnit GUI (looks like a type > initializer, e.g. static constructor, fails and once that happens the > there's no fixing it until the appdomain is unloaded).
> Thanks, > Nick
> On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote: >> While we do throw some custom exceptions, my initial guess is that >> this is not what's going on here because some of the tests that are >> failing are positive tests, ones which shouldn't be throwing any >> exceptions.
>> I just ran another scenario to maybe shed some more light on the >> issue, though it doesn't shed much more light. I have a class called >> Complete which has a bunch of tests which test our Complete method. >> These tests always work. I have another class called Lock which tests >> our Lock method. Lock doesn't work if run by itself (e.g. only >> itself) after just starting the NUnit GUI. I mentioned originally >> that if I run all the tests these other ones that fail by themselves >> will succeed. That's not entirely true. If I start NUnit GUI up >> fresh and then run the Complete tests I can then run the Lock tests, >> using the currently running NUnit GUI that ran the Complete tests, >> successfully.
>> Here is what my guess was as to what's going on. It appears the >> tests, and I mean the code in the NUnit test methods, which have code >> which directly references code from the Common assembly, this is the >> assembly that's having the issue of getting loaded, will always work >> whether run separately or when running all tests. The tests which >> don't reference code from the Common assembly won't work unless the >> Common assembly has already been loaded into the process by first >> running a test that referenced it. Almost as if the code is getting >> jitted with an app base setting which is correct and allows the .NET >> assembly loader to find the Common assembly using the usual probing >> mechanism, but then executed in a different appdomain which doesn't >> have the correct app base setting such that if/when Common needs to be >> loaded it cannot be found.
>> Thanks, >> Nick
>> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> > Hi Nick,
>> > From the error below, it's clear that NUnit itself - rather than your >> > tests - is trying to load the dll in question. This usually happens >> > when a custom exception, defined in one of your assemblies, is >> > uncaught in the tests. Is that a possibility in this scenario?
>> > If this is the case, then NUnit would be failing in deserializing the >> > exceptioni because it can't load the assembly in which the exception >> > is defined. One way to find out is to temporarily copy the "missing" >> > assembly to the directory containing nunit-agent.exe, along with any >> > dependencies that may be needed.
>> > This issue with custom exceptions may or may not be your problem, but >> > it's a long-standing issue and I suggest looking at it next.
>> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from >> > travelling across the remoting boundary by catching them and >> > transforming them into an xml error report.
>> > Charlie
>> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote: >> > > It appears my assumption is somewhat correct about which methods are >> > > failing. It appears the .NET runtime is having a problem loading >> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the >> > > same directory as my NUnit test assembly and I have all the other >> > > dependent assemblies there. This directory is c:\data\development >> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks >> > > for the Common assembly it's looking in the app base directory where >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app >> > > base (maybe by creating another appdomain and loading the tests from >> > > there) to the directory where the test assembly exists? My main >> > > framework API, which all the tests make use of, gets loaded and that's >> > > in the same directory as the test assembly. Here is the log from the >> > > fusion log viewer:
I haven't use NUnit's tracing yet. I just now tried turning on the
most verbose level of tracing. I looked through all the trace files
generated and there didn't seem to be any information in there which
pointed to the problem.
Still doing more testing. As I mentioned before, if I select the Lock
set of tests, and Lock is a class with a bunch of test methods, and
run that set, the first four tests, test1 - test4, succeed and the
last four, test5 - test8, fail. The first four are negative test
cases which don't make it past the first assembly implementing our API
as it validates the parameters and fails. The last four have valid
parameters and thus make it farther into our API and require this
Common assembly be loaded. This is where it fails.
Interestingly, I can execute test1 successfully and then select test5
and execute that successfully in the same instance of the NUnit GUI.
However if I select all Lock test cases it fails. Does NUnit load
things differently if you select individual tests version a set of
tests by choosing the class?
Thanks,
Nick
On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote:
> Still, this doesn't explain why your tests have anything at all to do
> with the primary AppDomain. NUnit does run all tests in a separate
> AppDomain and creates a new one each time you reload the tests.
> Have you turned on NUnit's internal trace and looked at whether it
> tells you anything?
> Charlie
> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote:
> > I did some more tests, and while I found out some new behavior I still
> > don't know what's going on here.
> > In order to test your custom exception suggestion I figured I would
> > take one of the tests that failed and catch all exceptions and turn
> > them into an InvalidOperationException, e.g like the following:
> > I then ran just that test and much to my surprise it succeeded. So
> > then I commented out the try/catch I added and rebuilt and retested,
> > just that test, and again it succeeded. I tested, individually which
> > a fresh new instance of NUnit GUI, each test that failed and they all
> > succeeded. For lock I have tests1 - test8. When I run all lock
> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 -
> > test8 individually with a fresh instance of NUnit GUI they all
> > succeed. The reason for running fresh instances of NUnit GUI is
> > because it seems once Common gets loaded then all tests from them on
> > will work as expected. Also, once loading Common fails it will
> > continue to fail for that instance of NUnit GUI (looks like a type
> > initializer, e.g. static constructor, fails and once that happens the
> > there's no fixing it until the appdomain is unloaded).
> > Thanks,
> > Nick
> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote:
> >> While we do throw some custom exceptions, my initial guess is that
> >> this is not what's going on here because some of the tests that are
> >> failing are positive tests, ones which shouldn't be throwing any
> >> exceptions.
> >> I just ran another scenario to maybe shed some more light on the
> >> issue, though it doesn't shed much more light. I have a class called
> >> Complete which has a bunch of tests which test our Complete method.
> >> These tests always work. I have another class called Lock which tests
> >> our Lock method. Lock doesn't work if run by itself (e.g. only
> >> itself) after just starting the NUnit GUI. I mentioned originally
> >> that if I run all the tests these other ones that fail by themselves
> >> will succeed. That's not entirely true. If I start NUnit GUI up
> >> fresh and then run the Complete tests I can then run the Lock tests,
> >> using the currently running NUnit GUI that ran the Complete tests,
> >> successfully.
> >> Here is what my guess was as to what's going on. It appears the
> >> tests, and I mean the code in the NUnit test methods, which have code
> >> which directly references code from the Common assembly, this is the
> >> assembly that's having the issue of getting loaded, will always work
> >> whether run separately or when running all tests. The tests which
> >> don't reference code from the Common assembly won't work unless the
> >> Common assembly has already been loaded into the process by first
> >> running a test that referenced it. Almost as if the code is getting
> >> jitted with an app base setting which is correct and allows the .NET
> >> assembly loader to find the Common assembly using the usual probing
> >> mechanism, but then executed in a different appdomain which doesn't
> >> have the correct app base setting such that if/when Common needs to be
> >> loaded it cannot be found.
> >> Thanks,
> >> Nick
> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> > Hi Nick,
> >> > From the error below, it's clear that NUnit itself - rather than your
> >> > tests - is trying to load the dll in question. This usually happens
> >> > when a custom exception, defined in one of your assemblies, is
> >> > uncaught in the tests. Is that a possibility in this scenario?
> >> > If this is the case, then NUnit would be failing in deserializing the
> >> > exceptioni because it can't load the assembly in which the exception
> >> > is defined. One way to find out is to temporarily copy the "missing"
> >> > assembly to the directory containing nunit-agent.exe, along with any
> >> > dependencies that may be needed.
> >> > This issue with custom exceptions may or may not be your problem, but
> >> > it's a long-standing issue and I suggest looking at it next.
> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> >> > travelling across the remoting boundary by catching them and
> >> > transforming them into an xml error report.
> >> > Charlie
> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote:
> >> > > It appears my assumption is somewhat correct about which methods are
> >> > > failing. It appears the .NET runtime is having a problem loading
> >> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the
> >> > > same directory as my NUnit test assembly and I have all the other
> >> > > dependent assemblies there. This directory is c:\data\development
> >> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks
> >> > > for the Common assembly it's looking in the app base directory where
> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> >> > > base (maybe by creating another appdomain and loading the tests from
> >> > > there) to the directory where the test assembly exists? My main
> >> > > framework API, which all the tests make use of, gets loaded and that's
> >> > > in the same directory as the test assembly. Here is the log from the
> >> > > fusion log viewer:
Well, the absence of trace info from NUnit at least tells us that NUnit is not aware of any problem. I guess we knew that though.
Regarding loading: NUnit does not reload any tests unless you tell it to. You should probably turn off any automatic reload settings if you have them set. That way you can manually control when a reload takes place.
On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote: > I haven't use NUnit's tracing yet. I just now tried turning on the > most verbose level of tracing. I looked through all the trace files > generated and there didn't seem to be any information in there which > pointed to the problem.
> Still doing more testing. As I mentioned before, if I select the Lock > set of tests, and Lock is a class with a bunch of test methods, and > run that set, the first four tests, test1 - test4, succeed and the > last four, test5 - test8, fail. The first four are negative test > cases which don't make it past the first assembly implementing our API > as it validates the parameters and fails. The last four have valid > parameters and thus make it farther into our API and require this > Common assembly be loaded. This is where it fails.
> Interestingly, I can execute test1 successfully and then select test5 > and execute that successfully in the same instance of the NUnit GUI. > However if I select all Lock test cases it fails. Does NUnit load > things differently if you select individual tests version a set of > tests by choosing the class?
> Thanks, > Nick
> On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote: >> Still, this doesn't explain why your tests have anything at all to do >> with the primary AppDomain. NUnit does run all tests in a separate >> AppDomain and creates a new one each time you reload the tests.
>> Have you turned on NUnit's internal trace and looked at whether it >> tells you anything?
>> Charlie
>> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote: >> > I did some more tests, and while I found out some new behavior I still >> > don't know what's going on here.
>> > In order to test your custom exception suggestion I figured I would >> > take one of the tests that failed and catch all exceptions and turn >> > them into an InvalidOperationException, e.g like the following:
>> > I then ran just that test and much to my surprise it succeeded. So >> > then I commented out the try/catch I added and rebuilt and retested, >> > just that test, and again it succeeded. I tested, individually which >> > a fresh new instance of NUnit GUI, each test that failed and they all >> > succeeded. For lock I have tests1 - test8. When I run all lock >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 - >> > test8 individually with a fresh instance of NUnit GUI they all >> > succeed. The reason for running fresh instances of NUnit GUI is >> > because it seems once Common gets loaded then all tests from them on >> > will work as expected. Also, once loading Common fails it will >> > continue to fail for that instance of NUnit GUI (looks like a type >> > initializer, e.g. static constructor, fails and once that happens the >> > there's no fixing it until the appdomain is unloaded).
>> > Thanks, >> > Nick
>> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote: >> >> While we do throw some custom exceptions, my initial guess is that >> >> this is not what's going on here because some of the tests that are >> >> failing are positive tests, ones which shouldn't be throwing any >> >> exceptions.
>> >> I just ran another scenario to maybe shed some more light on the >> >> issue, though it doesn't shed much more light. I have a class called >> >> Complete which has a bunch of tests which test our Complete method. >> >> These tests always work. I have another class called Lock which tests >> >> our Lock method. Lock doesn't work if run by itself (e.g. only >> >> itself) after just starting the NUnit GUI. I mentioned originally >> >> that if I run all the tests these other ones that fail by themselves >> >> will succeed. That's not entirely true. If I start NUnit GUI up >> >> fresh and then run the Complete tests I can then run the Lock tests, >> >> using the currently running NUnit GUI that ran the Complete tests, >> >> successfully.
>> >> Here is what my guess was as to what's going on. It appears the >> >> tests, and I mean the code in the NUnit test methods, which have code >> >> which directly references code from the Common assembly, this is the >> >> assembly that's having the issue of getting loaded, will always work >> >> whether run separately or when running all tests. The tests which >> >> don't reference code from the Common assembly won't work unless the >> >> Common assembly has already been loaded into the process by first >> >> running a test that referenced it. Almost as if the code is getting >> >> jitted with an app base setting which is correct and allows the .NET >> >> assembly loader to find the Common assembly using the usual probing >> >> mechanism, but then executed in a different appdomain which doesn't >> >> have the correct app base setting such that if/when Common needs to be >> >> loaded it cannot be found.
>> >> Thanks, >> >> Nick
>> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> >> > Hi Nick,
>> >> > From the error below, it's clear that NUnit itself - rather than your >> >> > tests - is trying to load the dll in question. This usually happens >> >> > when a custom exception, defined in one of your assemblies, is >> >> > uncaught in the tests. Is that a possibility in this scenario?
>> >> > If this is the case, then NUnit would be failing in deserializing the >> >> > exceptioni because it can't load the assembly in which the exception >> >> > is defined. One way to find out is to temporarily copy the "missing" >> >> > assembly to the directory containing nunit-agent.exe, along with any >> >> > dependencies that may be needed.
>> >> > This issue with custom exceptions may or may not be your problem, but >> >> > it's a long-standing issue and I suggest looking at it next.
>> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from >> >> > travelling across the remoting boundary by catching them and >> >> > transforming them into an xml error report.
>> >> > Charlie
>> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote: >> >> > > It appears my assumption is somewhat correct about which methods are >> >> > > failing. It appears the .NET runtime is having a problem loading >> >> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the >> >> > > same directory as my NUnit test assembly and I have all the other >> >> > > dependent assemblies there. This directory is c:\data\development >> >> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks >> >> > > for the Common assembly it's looking in the app base directory where >> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app >> >> > > base (maybe by creating another appdomain and loading the tests from >> >> > > there) to the directory where the test assembly exists? My main >> >> > > framework API, which all the tests make use of, gets loaded and that's >> >> > > in the same directory as the test assembly. Here is the log from the >> >> > > fusion log viewer:
There were trace files created with trace information in them, just
none of it seemed to point to a reason for the issue I'm having.
I don't have any automatic reload setting turned on.
When I mentioned loading it was more about how NUnit loads a class and/
or method in order to execute it. It seems odd that if I select the
Lock class and run I get different behavior than when I choose each
Lock test method individually and run them. My guess is that NUnit is
loading the code differently in those two cases.
Thanks,
Nick
On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote:
> Well, the absence of trace info from NUnit at least tells us that NUnit is
> not aware of any problem. I guess we knew that though.
> Regarding loading: NUnit does not reload any tests unless you tell it to.
> You should probably turn off any automatic reload settings if you have
> them set. That way you can manually control when a reload takes place.
> Charlie
> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote:
> > I haven't use NUnit's tracing yet. I just now tried turning on the
> > most verbose level of tracing. I looked through all the trace files
> > generated and there didn't seem to be any information in there which
> > pointed to the problem.
> > Still doing more testing. As I mentioned before, if I select the Lock
> > set of tests, and Lock is a class with a bunch of test methods, and
> > run that set, the first four tests, test1 - test4, succeed and the
> > last four, test5 - test8, fail. The first four are negative test
> > cases which don't make it past the first assembly implementing our API
> > as it validates the parameters and fails. The last four have valid
> > parameters and thus make it farther into our API and require this
> > Common assembly be loaded. This is where it fails.
> > Interestingly, I can execute test1 successfully and then select test5
> > and execute that successfully in the same instance of the NUnit GUI.
> > However if I select all Lock test cases it fails. Does NUnit load
> > things differently if you select individual tests version a set of
> > tests by choosing the class?
> > Thanks,
> > Nick
> > On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> Still, this doesn't explain why your tests have anything at all to do
> >> with the primary AppDomain. NUnit does run all tests in a separate
> >> AppDomain and creates a new one each time you reload the tests.
> >> Have you turned on NUnit's internal trace and looked at whether it
> >> tells you anything?
> >> Charlie
> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote:
> >> > I did some more tests, and while I found out some new behavior I still
> >> > don't know what's going on here.
> >> > In order to test your custom exception suggestion I figured I would
> >> > take one of the tests that failed and catch all exceptions and turn
> >> > them into an InvalidOperationException, e.g like the following:
> >> > I then ran just that test and much to my surprise it succeeded. So
> >> > then I commented out the try/catch I added and rebuilt and retested,
> >> > just that test, and again it succeeded. I tested, individually which
> >> > a fresh new instance of NUnit GUI, each test that failed and they all
> >> > succeeded. For lock I have tests1 - test8. When I run all lock
> >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 -
> >> > test8 individually with a fresh instance of NUnit GUI they all
> >> > succeed. The reason for running fresh instances of NUnit GUI is
> >> > because it seems once Common gets loaded then all tests from them on
> >> > will work as expected. Also, once loading Common fails it will
> >> > continue to fail for that instance of NUnit GUI (looks like a type
> >> > initializer, e.g. static constructor, fails and once that happens the
> >> > there's no fixing it until the appdomain is unloaded).
> >> > Thanks,
> >> > Nick
> >> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote:
> >> >> While we do throw some custom exceptions, my initial guess is that
> >> >> this is not what's going on here because some of the tests that are
> >> >> failing are positive tests, ones which shouldn't be throwing any
> >> >> exceptions.
> >> >> I just ran another scenario to maybe shed some more light on the
> >> >> issue, though it doesn't shed much more light. I have a class called
> >> >> Complete which has a bunch of tests which test our Complete method.
> >> >> These tests always work. I have another class called Lock which tests
> >> >> our Lock method. Lock doesn't work if run by itself (e.g. only
> >> >> itself) after just starting the NUnit GUI. I mentioned originally
> >> >> that if I run all the tests these other ones that fail by themselves
> >> >> will succeed. That's not entirely true. If I start NUnit GUI up
> >> >> fresh and then run the Complete tests I can then run the Lock tests,
> >> >> using the currently running NUnit GUI that ran the Complete tests,
> >> >> successfully.
> >> >> Here is what my guess was as to what's going on. It appears the
> >> >> tests, and I mean the code in the NUnit test methods, which have code
> >> >> which directly references code from the Common assembly, this is the
> >> >> assembly that's having the issue of getting loaded, will always work
> >> >> whether run separately or when running all tests. The tests which
> >> >> don't reference code from the Common assembly won't work unless the
> >> >> Common assembly has already been loaded into the process by first
> >> >> running a test that referenced it. Almost as if the code is getting
> >> >> jitted with an app base setting which is correct and allows the .NET
> >> >> assembly loader to find the Common assembly using the usual probing
> >> >> mechanism, but then executed in a different appdomain which doesn't
> >> >> have the correct app base setting such that if/when Common needs to be
> >> >> loaded it cannot be found.
> >> >> Thanks,
> >> >> Nick
> >> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> >> > Hi Nick,
> >> >> > From the error below, it's clear that NUnit itself - rather than your
> >> >> > tests - is trying to load the dll in question. This usually happens
> >> >> > when a custom exception, defined in one of your assemblies, is
> >> >> > uncaught in the tests. Is that a possibility in this scenario?
> >> >> > If this is the case, then NUnit would be failing in deserializing the
> >> >> > exceptioni because it can't load the assembly in which the exception
> >> >> > is defined. One way to find out is to temporarily copy the "missing"
> >> >> > assembly to the directory containing nunit-agent.exe, along with any
> >> >> > dependencies that may be needed.
> >> >> > This issue with custom exceptions may or may not be your problem, but
> >> >> > it's a long-standing issue and I suggest looking at it next.
> >> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> >> >> > travelling across the remoting boundary by catching them and
> >> >> > transforming them into an xml error report.
> >> >> > Charlie
> >> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote:
> >> >> > > It appears my assumption is somewhat correct about which methods are
> >> >> > > failing. It appears the .NET runtime is having a problem loading
> >> >> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the
> >> >> > > same directory as my NUnit test assembly and I have all the other
> >> >> > > dependent assemblies there. This directory is c:\data\development
> >> >> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks
> >> >> > > for the Common assembly it's looking in the app base directory where
> >> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> >> >> > > base (maybe by creating another appdomain and loading the tests from
> >> >> > > there) to the directory where the test assembly exists? My main
> >> >> > > framework API, which all the tests make use of, gets loaded and that's
> >> >> > > in the same directory as the test assembly. Here is the log from the
> >> >> > > fusion log viewer:
> >> >> > > === Pre-bind state information ===
> >> >> > > LOG: User = redbonemobile\nickdu
> >> >> > > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
> >> >> > > (Partial)
> >> >> > > WRN: Partial binding information was supplied for an assembly:
> >> >> > > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
> >> >> > > ID: 1
> >> >> > > WRN: A partial bind occurs when only part of the assembly display name
> >> >> > > is provided.
> >> >> > > WRN: This might result in the binder loading an incorrect assembly.
> >> >> > > WRN: It is recommended to provide a fully specified textual identity
> >> >> > > for the assembly,
> >> >> > > WRN: that consists of the simple name, version, culture, and public
> >> >> > > key token.
> >> >> > > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for > >> >> > > more information and
On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote: > There were trace files created with trace information in them, just > none of it seemed to point to a reason for the issue I'm having.
Right - that's what I meant. NUnit is simply considering your problem as a test failure, not an NUnit problem worthy of logging.
> I don't have any automatic reload setting turned on.
OK
> When I mentioned loading it was more about how NUnit loads a class and/ > or method in order to execute it. It seems odd that if I select the > Lock class and run I get different behavior than when I choose each > Lock test method individually and run them. My guess is that NUnit is > loading the code differently in those two cases.
NUnit's "Loading" involves only your test assembly. It has to happen before the tests can be displayed in the Gui. That loading is only repeated if you use a reload item on the menu or if you have enabled automatic reload on run or when the assembly changes.
Loading of dependent assemblies happens according to normal .NET rules, as the code is jitted and executed. Again, once a dependent assembly is loaded into the test AppDomain, it stays there.
So, your problem arises when the dependent assembly is first needed by one of the Lock tests. If you have run some other test first, which successfully loads the assembly, then the Lock tests have no problem. This is exactly the sort of inter-test dependency I was originally writing about - not necessarily a dependency that you intentionally created, but one that exists nevertheless.
I guess that gives you two possible attacks on the problem: 1. Trace how and why the Lock class cannot load the assembly. 2. Trace how and why the complete class is able to load the assembly.
What's different in these classes?
Charlie
PS: Feel free to post some code if it doesn't expose any company secrets.
> On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote: >> Well, the absence of trace info from NUnit at least tells us that NUnit is >> not aware of any problem. I guess we knew that though.
>> Regarding loading: NUnit does not reload any tests unless you tell it to. >> You should probably turn off any automatic reload settings if you have >> them set. That way you can manually control when a reload takes place.
>> Charlie
>> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote: >> > I haven't use NUnit's tracing yet. I just now tried turning on the >> > most verbose level of tracing. I looked through all the trace files >> > generated and there didn't seem to be any information in there which >> > pointed to the problem.
>> > Still doing more testing. As I mentioned before, if I select the Lock >> > set of tests, and Lock is a class with a bunch of test methods, and >> > run that set, the first four tests, test1 - test4, succeed and the >> > last four, test5 - test8, fail. The first four are negative test >> > cases which don't make it past the first assembly implementing our API >> > as it validates the parameters and fails. The last four have valid >> > parameters and thus make it farther into our API and require this >> > Common assembly be loaded. This is where it fails.
>> > Interestingly, I can execute test1 successfully and then select test5 >> > and execute that successfully in the same instance of the NUnit GUI. >> > However if I select all Lock test cases it fails. Does NUnit load >> > things differently if you select individual tests version a set of >> > tests by choosing the class?
>> > Thanks, >> > Nick
>> > On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote: >> >> Still, this doesn't explain why your tests have anything at all to do >> >> with the primary AppDomain. NUnit does run all tests in a separate >> >> AppDomain and creates a new one each time you reload the tests.
>> >> Have you turned on NUnit's internal trace and looked at whether it >> >> tells you anything?
>> >> Charlie
>> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote: >> >> > I did some more tests, and while I found out some new behavior I still >> >> > don't know what's going on here.
>> >> > In order to test your custom exception suggestion I figured I would >> >> > take one of the tests that failed and catch all exceptions and turn >> >> > them into an InvalidOperationException, e.g like the following:
>> >> > I then ran just that test and much to my surprise it succeeded. So >> >> > then I commented out the try/catch I added and rebuilt and retested, >> >> > just that test, and again it succeeded. I tested, individually which >> >> > a fresh new instance of NUnit GUI, each test that failed and they all >> >> > succeeded. For lock I have tests1 - test8. When I run all lock >> >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 - >> >> > test8 individually with a fresh instance of NUnit GUI they all >> >> > succeed. The reason for running fresh instances of NUnit GUI is >> >> > because it seems once Common gets loaded then all tests from them on >> >> > will work as expected. Also, once loading Common fails it will >> >> > continue to fail for that instance of NUnit GUI (looks like a type >> >> > initializer, e.g. static constructor, fails and once that happens the >> >> > there's no fixing it until the appdomain is unloaded).
>> >> > Thanks, >> >> > Nick
>> >> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote: >> >> >> While we do throw some custom exceptions, my initial guess is that >> >> >> this is not what's going on here because some of the tests that are >> >> >> failing are positive tests, ones which shouldn't be throwing any >> >> >> exceptions.
>> >> >> I just ran another scenario to maybe shed some more light on the >> >> >> issue, though it doesn't shed much more light. I have a class called >> >> >> Complete which has a bunch of tests which test our Complete method. >> >> >> These tests always work. I have another class called Lock which tests >> >> >> our Lock method. Lock doesn't work if run by itself (e.g. only >> >> >> itself) after just starting the NUnit GUI. I mentioned originally >> >> >> that if I run all the tests these other ones that fail by themselves >> >> >> will succeed. That's not entirely true. If I start NUnit GUI up >> >> >> fresh and then run the Complete tests I can then run the Lock tests, >> >> >> using the currently running NUnit GUI that ran the Complete tests, >> >> >> successfully.
>> >> >> Here is what my guess was as to what's going on. It appears the >> >> >> tests, and I mean the code in the NUnit test methods, which have code >> >> >> which directly references code from the Common assembly, this is the >> >> >> assembly that's having the issue of getting loaded, will always work >> >> >> whether run separately or when running all tests. The tests which >> >> >> don't reference code from the Common assembly won't work unless the >> >> >> Common assembly has already been loaded into the process by first >> >> >> running a test that referenced it. Almost as if the code is getting >> >> >> jitted with an app base setting which is correct and allows the .NET >> >> >> assembly loader to find the Common assembly using the usual probing >> >> >> mechanism, but then executed in a different appdomain which doesn't >> >> >> have the correct app base setting such that if/when Common needs to be >> >> >> loaded it cannot be found.
>> >> >> Thanks, >> >> >> Nick
>> >> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> >> >> > Hi Nick,
>> >> >> > From the error below, it's clear that NUnit itself - rather than your >> >> >> > tests - is trying to load the dll in question. This usually happens >> >> >> > when a custom exception, defined in one of your assemblies, is >> >> >> > uncaught in the tests. Is that a possibility in this scenario?
>> >> >> > If this is the case, then NUnit would be failing in deserializing the >> >> >> > exceptioni because it can't load the assembly in which the exception >> >> >> > is defined. One way to find out is to temporarily copy the "missing" >> >> >> > assembly to the directory containing nunit-agent.exe, along with any >> >> >> > dependencies that may be needed.
>> >> >> > This issue with custom exceptions may or may not be your problem, but >> >> >> > it's a long-standing issue and I suggest looking at it next.
>> >> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from >> >> >> > travelling across the remoting boundary by catching them and >> >> >> > transforming them into an xml error report.
>> >> >> > Charlie
>> >> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <nic...@msn.com> wrote: >> >> >> > > It appears my assumption is somewhat correct about which methods are >> >> >> > > failing. It appears the .NET runtime is having a problem loading >> >> >> > > Redbone.Framework.BusinessProcess.Common.dll. This assembly is in the >> >> >> > > same directory as my NUnit test assembly and I have all the other >> >> >> > > dependent assemblies there. This directory is c:\data\development >> >> >> > > \Redbone\bpf\tests\bin\debug. However, when the fusion loader looks >> >> >> > > for the Common assembly it's looking in the app base directory where >> >> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app >> >> >> > > base (maybe by creating another appdomain and loading the tests from >> >> >> > > there) to the directory where the test assembly exists? My main >> >> >> > > framework API, which all the tests make use of, gets loaded and that's >> >> >> > > in the same directory as the test assembly. Here is the log from the >> >> >> > > fusion log viewer:
Yeah, I'm thinking about spending a little bit of time to create a
skelaton of our framework that has the same dependent assemblies to
see if I can repro and if so send it along or post the code.
With respect to test dependencies, I don't think this could be the
case. As I said, I can start NUnit GUI fresh and run Lock Test5
without a problem, and therefore it's not dependent on running any
other tests. However, if I start NUnit GUI fresh and select the Lock
'set' of tests and run them then Test1 - Test4 succeed and Test5 -
Test8 succeed.
Nick
On Apr 2, 12:54 pm, Charlie Poole <nunit...@gmail.com> wrote:
> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote:
> > There were trace files created with trace information in them, just
> > none of it seemed to point to a reason for the issue I'm having.
> Right - that's what I meant. NUnit is simply considering your
> problem as a test failure, not an NUnit problem worthy of logging.
> > I don't have any automatic reload setting turned on.
> OK
> > When I mentioned loading it was more about how NUnit loads a class and/
> > or method in order to execute it. It seems odd that if I select the
> > Lock class and run I get different behavior than when I choose each
> > Lock test method individually and run them. My guess is that NUnit is
> > loading the code differently in those two cases.
> NUnit's "Loading" involves only your test assembly. It has to happen before
> the tests can be displayed in the Gui. That loading is only repeated if you
> use a reload item on the menu or if you have enabled automatic reload on
> run or when the assembly changes.
> Loading of dependent assemblies happens according to normal .NET rules,
> as the code is jitted and executed. Again, once a dependent assembly is
> loaded into the test AppDomain, it stays there.
> So, your problem arises when the dependent assembly is first needed by
> one of the Lock tests. If you have run some other test first, which successfully
> loads the assembly, then the Lock tests have no problem. This is exactly
> the sort of inter-test dependency I was originally writing about - not
> necessarily
> a dependency that you intentionally created, but one that exists nevertheless.
> I guess that gives you two possible attacks on the problem:
> 1. Trace how and why the Lock class cannot load the assembly.
> 2. Trace how and why the complete class is able to load the assembly.
> What's different in these classes?
> Charlie
> PS: Feel free to post some code if it doesn't expose any company secrets.
> > Thanks,
> > Nick
> > On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> Well, the absence of trace info from NUnit at least tells us that NUnit is
> >> not aware of any problem. I guess we knew that though.
> >> Regarding loading: NUnit does not reload any tests unless you tell it to.
> >> You should probably turn off any automatic reload settings if you have
> >> them set. That way you can manually control when a reload takes place.
> >> Charlie
> >> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote:
> >> > I haven't use NUnit's tracing yet. I just now tried turning on the
> >> > most verbose level of tracing. I looked through all the trace files
> >> > generated and there didn't seem to be any information in there which
> >> > pointed to the problem.
> >> > Still doing more testing. As I mentioned before, if I select the Lock
> >> > set of tests, and Lock is a class with a bunch of test methods, and
> >> > run that set, the first four tests, test1 - test4, succeed and the
> >> > last four, test5 - test8, fail. The first four are negative test
> >> > cases which don't make it past the first assembly implementing our API
> >> > as it validates the parameters and fails. The last four have valid
> >> > parameters and thus make it farther into our API and require this
> >> > Common assembly be loaded. This is where it fails.
> >> > Interestingly, I can execute test1 successfully and then select test5
> >> > and execute that successfully in the same instance of the NUnit GUI.
> >> > However if I select all Lock test cases it fails. Does NUnit load
> >> > things differently if you select individual tests version a set of
> >> > tests by choosing the class?
> >> > Thanks,
> >> > Nick
> >> > On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> >> Still, this doesn't explain why your tests have anything at all to do
> >> >> with the primary AppDomain. NUnit does run all tests in a separate
> >> >> AppDomain and creates a new one each time you reload the tests.
> >> >> Have you turned on NUnit's internal trace and looked at whether it
> >> >> tells you anything?
> >> >> Charlie
> >> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote:
> >> >> > I did some more tests, and while I found out some new behavior I still
> >> >> > don't know what's going on here.
> >> >> > In order to test your custom exception suggestion I figured I would
> >> >> > take one of the tests that failed and catch all exceptions and turn
> >> >> > them into an InvalidOperationException, e.g like the following:
> >> >> > I then ran just that test and much to my surprise it succeeded. So
> >> >> > then I commented out the try/catch I added and rebuilt and retested,
> >> >> > just that test, and again it succeeded. I tested, individually which
> >> >> > a fresh new instance of NUnit GUI, each test that failed and they all
> >> >> > succeeded. For lock I have tests1 - test8. When I run all lock
> >> >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 -
> >> >> > test8 individually with a fresh instance of NUnit GUI they all
> >> >> > succeed. The reason for running fresh instances of NUnit GUI is
> >> >> > because it seems once Common gets loaded then all tests from them on
> >> >> > will work as expected. Also, once loading Common fails it will
> >> >> > continue to fail for that instance of NUnit GUI (looks like a type
> >> >> > initializer, e.g. static constructor, fails and once that happens the
> >> >> > there's no fixing it until the appdomain is unloaded).
> >> >> > Thanks,
> >> >> > Nick
> >> >> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote:
> >> >> >> While we do throw some custom exceptions, my initial guess is that
> >> >> >> this is not what's going on here because some of the tests that are
> >> >> >> failing are positive tests, ones which shouldn't be throwing any
> >> >> >> exceptions.
> >> >> >> I just ran another scenario to maybe shed some more light on the
> >> >> >> issue, though it doesn't shed much more light. I have a class called
> >> >> >> Complete which has a bunch of tests which test our Complete method.
> >> >> >> These tests always work. I have another class called Lock which tests
> >> >> >> our Lock method. Lock doesn't work if run by itself (e.g. only
> >> >> >> itself) after just starting the NUnit GUI. I mentioned originally
> >> >> >> that if I run all the tests these other ones that fail by themselves
> >> >> >> will succeed. That's not entirely true. If I start NUnit GUI up
> >> >> >> fresh and then run the Complete tests I can then run the Lock tests,
> >> >> >> using the currently running NUnit GUI that ran the Complete tests,
> >> >> >> successfully.
> >> >> >> Here is what my guess was as to what's going on. It appears the
> >> >> >> tests, and I mean the code in the NUnit test methods, which have code
> >> >> >> which directly references code from the Common assembly, this is the
> >> >> >> assembly that's having the issue of getting loaded, will always work
> >> >> >> whether run separately or when running all tests. The tests which
> >> >> >> don't reference code from the Common assembly won't work unless the
> >> >> >> Common assembly has already been loaded into the process by first
> >> >> >> running a test that referenced it. Almost as if the code is getting
> >> >> >> jitted with an app base setting which is correct and allows the .NET
> >> >> >> assembly loader to find the Common assembly using the usual probing
> >> >> >> mechanism, but then executed in a different appdomain which doesn't
> >> >> >> have the correct app base setting such that if/when Common needs to be
> >> >> >> loaded it cannot be found.
> >> >> >> Thanks,
> >> >> >> Nick
> >> >> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> >> >> > Hi Nick,
> >> >> >> > From the error below, it's clear that NUnit itself - rather than your
> >> >> >> > tests - is trying to load the dll in question. This usually happens
> >> >> >> > when a custom exception, defined in one of your assemblies, is
> >> >> >> > uncaught in the tests. Is that a possibility in this scenario?
> >> >> >> > If this is the case, then NUnit would be failing in deserializing the
> >> >> >> > exceptioni because it can't load the assembly in which the exception
> >> >> >> > is defined. One way to find out is to temporarily copy the "missing"
> >> >> >> > assembly to the directory containing nunit-agent.exe, along with any
> >> >> >> > dependencies that may be needed.
> >> >> >> > This issue with custom exceptions may or may not be your problem, but
> >> >> >> > it's a long-standing issue and I suggest looking at it next.
> >> >> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> >> >> >> > travelling across the remoting boundary by catching them and
> >> >> >> > transforming them into an xml error report.
Regarding dependencies, I'd say that if running tests 1-4 has an effect on test 5, that's a dependency by definition. It's just one that we don't yet understand and which is probably due to the underlying platform - or NUnit itself - rather than something you coded explicitly.
On Mon, Apr 2, 2012 at 12:43 PM, nickdu <nic...@msn.com> wrote: > Yeah, I'm thinking about spending a little bit of time to create a > skelaton of our framework that has the same dependent assemblies to > see if I can repro and if so send it along or post the code.
> With respect to test dependencies, I don't think this could be the > case. As I said, I can start NUnit GUI fresh and run Lock Test5 > without a problem, and therefore it's not dependent on running any > other tests. However, if I start NUnit GUI fresh and select the Lock > 'set' of tests and run them then Test1 - Test4 succeed and Test5 - > Test8 succeed.
> Nick
> On Apr 2, 12:54 pm, Charlie Poole <nunit...@gmail.com> wrote: >> Hi Nick,
>> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote: >> > There were trace files created with trace information in them, just >> > none of it seemed to point to a reason for the issue I'm having.
>> Right - that's what I meant. NUnit is simply considering your >> problem as a test failure, not an NUnit problem worthy of logging.
>> > I don't have any automatic reload setting turned on.
>> OK
>> > When I mentioned loading it was more about how NUnit loads a class and/ >> > or method in order to execute it. It seems odd that if I select the >> > Lock class and run I get different behavior than when I choose each >> > Lock test method individually and run them. My guess is that NUnit is >> > loading the code differently in those two cases.
>> NUnit's "Loading" involves only your test assembly. It has to happen before >> the tests can be displayed in the Gui. That loading is only repeated if you >> use a reload item on the menu or if you have enabled automatic reload on >> run or when the assembly changes.
>> Loading of dependent assemblies happens according to normal .NET rules, >> as the code is jitted and executed. Again, once a dependent assembly is >> loaded into the test AppDomain, it stays there.
>> So, your problem arises when the dependent assembly is first needed by >> one of the Lock tests. If you have run some other test first, which successfully >> loads the assembly, then the Lock tests have no problem. This is exactly >> the sort of inter-test dependency I was originally writing about - not >> necessarily >> a dependency that you intentionally created, but one that exists nevertheless.
>> I guess that gives you two possible attacks on the problem: >> 1. Trace how and why the Lock class cannot load the assembly. >> 2. Trace how and why the complete class is able to load the assembly.
>> What's different in these classes?
>> Charlie
>> PS: Feel free to post some code if it doesn't expose any company secrets.
>> > Thanks, >> > Nick
>> > On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote: >> >> Well, the absence of trace info from NUnit at least tells us that NUnit is >> >> not aware of any problem. I guess we knew that though.
>> >> Regarding loading: NUnit does not reload any tests unless you tell it to. >> >> You should probably turn off any automatic reload settings if you have >> >> them set. That way you can manually control when a reload takes place.
>> >> Charlie
>> >> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote: >> >> > I haven't use NUnit's tracing yet. I just now tried turning on the >> >> > most verbose level of tracing. I looked through all the trace files >> >> > generated and there didn't seem to be any information in there which >> >> > pointed to the problem.
>> >> > Still doing more testing. As I mentioned before, if I select the Lock >> >> > set of tests, and Lock is a class with a bunch of test methods, and >> >> > run that set, the first four tests, test1 - test4, succeed and the >> >> > last four, test5 - test8, fail. The first four are negative test >> >> > cases which don't make it past the first assembly implementing our API >> >> > as it validates the parameters and fails. The last four have valid >> >> > parameters and thus make it farther into our API and require this >> >> > Common assembly be loaded. This is where it fails.
>> >> > Interestingly, I can execute test1 successfully and then select test5 >> >> > and execute that successfully in the same instance of the NUnit GUI. >> >> > However if I select all Lock test cases it fails. Does NUnit load >> >> > things differently if you select individual tests version a set of >> >> > tests by choosing the class?
>> >> > Thanks, >> >> > Nick
>> >> > On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote: >> >> >> Still, this doesn't explain why your tests have anything at all to do >> >> >> with the primary AppDomain. NUnit does run all tests in a separate >> >> >> AppDomain and creates a new one each time you reload the tests.
>> >> >> Have you turned on NUnit's internal trace and looked at whether it >> >> >> tells you anything?
>> >> >> Charlie
>> >> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote: >> >> >> > I did some more tests, and while I found out some new behavior I still >> >> >> > don't know what's going on here.
>> >> >> > In order to test your custom exception suggestion I figured I would >> >> >> > take one of the tests that failed and catch all exceptions and turn >> >> >> > them into an InvalidOperationException, e.g like the following:
>> >> >> > I then ran just that test and much to my surprise it succeeded. So >> >> >> > then I commented out the try/catch I added and rebuilt and retested, >> >> >> > just that test, and again it succeeded. I tested, individually which >> >> >> > a fresh new instance of NUnit GUI, each test that failed and they all >> >> >> > succeeded. For lock I have tests1 - test8. When I run all lock >> >> >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 - >> >> >> > test8 individually with a fresh instance of NUnit GUI they all >> >> >> > succeed. The reason for running fresh instances of NUnit GUI is >> >> >> > because it seems once Common gets loaded then all tests from them on >> >> >> > will work as expected. Also, once loading Common fails it will >> >> >> > continue to fail for that instance of NUnit GUI (looks like a type >> >> >> > initializer, e.g. static constructor, fails and once that happens the >> >> >> > there's no fixing it until the appdomain is unloaded).
>> >> >> > Thanks, >> >> >> > Nick
>> >> >> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote: >> >> >> >> While we do throw some custom exceptions, my initial guess is that >> >> >> >> this is not what's going on here because some of the tests that are >> >> >> >> failing are positive tests, ones which shouldn't be throwing any >> >> >> >> exceptions.
>> >> >> >> I just ran another scenario to maybe shed some more light on the >> >> >> >> issue, though it doesn't shed much more light. I have a class called >> >> >> >> Complete which has a bunch of tests which test our Complete method. >> >> >> >> These tests always work. I have another class called Lock which tests >> >> >> >> our Lock method. Lock doesn't work if run by itself (e.g. only >> >> >> >> itself) after just starting the NUnit GUI. I mentioned originally >> >> >> >> that if I run all the tests these other ones that fail by themselves >> >> >> >> will succeed. That's not entirely true. If I start NUnit GUI up >> >> >> >> fresh and then run the Complete tests I can then run the Lock tests, >> >> >> >> using the currently running NUnit GUI that ran the Complete tests, >> >> >> >> successfully.
>> >> >> >> Here is what my guess was as to what's going on. It appears the >> >> >> >> tests, and I mean the code in the NUnit test methods, which have code >> >> >> >> which directly references code from the Common assembly, this is the >> >> >> >> assembly that's having the issue of getting loaded, will always work >> >> >> >> whether run separately or when running all tests. The tests which >> >> >> >> don't reference code from the Common assembly won't work unless the >> >> >> >> Common assembly has already been loaded into the process by first >> >> >> >> running a test that referenced it. Almost as if the code is getting >> >> >> >> jitted with an app base setting which is correct and allows the .NET >> >> >> >> assembly loader to find the Common assembly using the usual probing >> >> >> >> mechanism, but then executed in a different appdomain which doesn't >> >> >> >> have the correct app base setting such that if/when Common needs to be >> >> >> >> loaded it cannot be found.
>> >> >> >> Thanks, >> >> >> >> Nick
>> >> >> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> >> >> >> > Hi Nick,
>> >> >> >> > From the error below, it's clear that NUnit itself - rather than your >> >> >> >> > tests - is trying to load the dll in question. This usually happens >> >> >> >> > when a custom exception, defined in one of your assemblies, is >> >> >> >> > uncaught in the tests. Is that a possibility in this scenario?
>> >> >> >> > If this is the case, then NUnit would be failing in deserializing the >> >> >> >> > exceptioni because it can't load the assembly in which the exception >> >> >> >> > is defined. One way to find out is to temporarily copy the "missing" >> >> >> >> > assembly to the directory containing nunit-agent.exe, along with any >> >> >> >> > dependencies that may be needed.
I'm going to put some effort in today to get a simple repro of this.
I hope I'm successful. By the way, I tried 2.5.10 and the same issue
occurs so it's not specific to 2.6.
Thanks,
Nick
On Apr 2, 5:09 pm, Charlie Poole <nunit...@gmail.com> wrote:
> Regarding dependencies, I'd say that if running tests 1-4 has an
> effect on test 5, that's a dependency by definition. It's just one that
> we don't yet understand and which is probably due to the underlying
> platform - or NUnit itself - rather than something you coded explicitly.
> Charlie
> On Mon, Apr 2, 2012 at 12:43 PM, nickdu <nic...@msn.com> wrote:
> > Yeah, I'm thinking about spending a little bit of time to create a
> > skelaton of our framework that has the same dependent assemblies to
> > see if I can repro and if so send it along or post the code.
> > With respect to test dependencies, I don't think this could be the
> > case. As I said, I can start NUnit GUI fresh and run Lock Test5
> > without a problem, and therefore it's not dependent on running any
> > other tests. However, if I start NUnit GUI fresh and select the Lock
> > 'set' of tests and run them then Test1 - Test4 succeed and Test5 -
> > Test8 succeed.
> > Nick
> > On Apr 2, 12:54 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> Hi Nick,
> >> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote:
> >> > There were trace files created with trace information in them, just
> >> > none of it seemed to point to a reason for the issue I'm having.
> >> Right - that's what I meant. NUnit is simply considering your
> >> problem as a test failure, not an NUnit problem worthy of logging.
> >> > I don't have any automatic reload setting turned on.
> >> OK
> >> > When I mentioned loading it was more about how NUnit loads a class and/
> >> > or method in order to execute it. It seems odd that if I select the
> >> > Lock class and run I get different behavior than when I choose each
> >> > Lock test method individually and run them. My guess is that NUnit is
> >> > loading the code differently in those two cases.
> >> NUnit's "Loading" involves only your test assembly. It has to happen before
> >> the tests can be displayed in the Gui. That loading is only repeated if you
> >> use a reload item on the menu or if you have enabled automatic reload on
> >> run or when the assembly changes.
> >> Loading of dependent assemblies happens according to normal .NET rules,
> >> as the code is jitted and executed. Again, once a dependent assembly is
> >> loaded into the test AppDomain, it stays there.
> >> So, your problem arises when the dependent assembly is first needed by
> >> one of the Lock tests. If you have run some other test first, which successfully
> >> loads the assembly, then the Lock tests have no problem. This is exactly
> >> the sort of inter-test dependency I was originally writing about - not
> >> necessarily
> >> a dependency that you intentionally created, but one that exists nevertheless.
> >> I guess that gives you two possible attacks on the problem:
> >> 1. Trace how and why the Lock class cannot load the assembly.
> >> 2. Trace how and why the complete class is able to load the assembly.
> >> What's different in these classes?
> >> Charlie
> >> PS: Feel free to post some code if it doesn't expose any company secrets.
> >> > Thanks,
> >> > Nick
> >> > On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> >> Well, the absence of trace info from NUnit at least tells us that NUnit is
> >> >> not aware of any problem. I guess we knew that though.
> >> >> Regarding loading: NUnit does not reload any tests unless you tell it to.
> >> >> You should probably turn off any automatic reload settings if you have
> >> >> them set. That way you can manually control when a reload takes place.
> >> >> Charlie
> >> >> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <nic...@msn.com> wrote:
> >> >> > I haven't use NUnit's tracing yet. I just now tried turning on the
> >> >> > most verbose level of tracing. I looked through all the trace files
> >> >> > generated and there didn't seem to be any information in there which
> >> >> > pointed to the problem.
> >> >> > Still doing more testing. As I mentioned before, if I select the Lock
> >> >> > set of tests, and Lock is a class with a bunch of test methods, and
> >> >> > run that set, the first four tests, test1 - test4, succeed and the
> >> >> > last four, test5 - test8, fail. The first four are negative test
> >> >> > cases which don't make it past the first assembly implementing our API
> >> >> > as it validates the parameters and fails. The last four have valid
> >> >> > parameters and thus make it farther into our API and require this
> >> >> > Common assembly be loaded. This is where it fails.
> >> >> > Interestingly, I can execute test1 successfully and then select test5
> >> >> > and execute that successfully in the same instance of the NUnit GUI.
> >> >> > However if I select all Lock test cases it fails. Does NUnit load
> >> >> > things differently if you select individual tests version a set of
> >> >> > tests by choosing the class?
> >> >> > Thanks,
> >> >> > Nick
> >> >> > On Apr 1, 1:24 pm, Charlie Poole <nunit...@gmail.com> wrote:
> >> >> >> Still, this doesn't explain why your tests have anything at all to do
> >> >> >> with the primary AppDomain. NUnit does run all tests in a separate
> >> >> >> AppDomain and creates a new one each time you reload the tests.
> >> >> >> Have you turned on NUnit's internal trace and looked at whether it
> >> >> >> tells you anything?
> >> >> >> Charlie
> >> >> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <nic...@msn.com> wrote:
> >> >> >> > I did some more tests, and while I found out some new behavior I still
> >> >> >> > don't know what's going on here.
> >> >> >> > In order to test your custom exception suggestion I figured I would
> >> >> >> > take one of the tests that failed and catch all exceptions and turn
> >> >> >> > them into an InvalidOperationException, e.g like the following:
> >> >> >> > I then ran just that test and much to my surprise it succeeded. So
> >> >> >> > then I commented out the try/catch I added and rebuilt and retested,
> >> >> >> > just that test, and again it succeeded. I tested, individually which
> >> >> >> > a fresh new instance of NUnit GUI, each test that failed and they all
> >> >> >> > succeeded. For lock I have tests1 - test8. When I run all lock
> >> >> >> > tests, test1 - test4 succeed and test5 - test8 fail. If I run test5 -
> >> >> >> > test8 individually with a fresh instance of NUnit GUI they all
> >> >> >> > succeed. The reason for running fresh instances of NUnit GUI is
> >> >> >> > because it seems once Common gets loaded then all tests from them on
> >> >> >> > will work as expected. Also, once loading Common fails it will
> >> >> >> > continue to fail for that instance of NUnit GUI (looks like a type
> >> >> >> > initializer, e.g. static constructor, fails and once that happens the
> >> >> >> > there's no fixing it until the appdomain is unloaded).
> >> >> >> > Thanks,
> >> >> >> > Nick
> >> >> >> > On Apr 1, 8:16 am, nickdu <nic...@msn.com> wrote:
> >> >> >> >> While we do throw some custom exceptions, my initial guess is that
> >> >> >> >> this is not what's going on here because some of the tests that are
> >> >> >> >> failing are positive tests, ones which shouldn't be throwing any
> >> >> >> >> exceptions.
> >> >> >> >> I just ran another scenario to maybe shed some more light on the
> >> >> >> >> issue, though it doesn't shed much more light. I have a class called
> >> >> >> >> Complete which has a bunch of tests which test our Complete method.
> >> >> >> >> These tests always work. I have another class called Lock which tests
> >> >> >> >> our Lock method. Lock doesn't work if run by itself (e.g. only
> >> >> >> >> itself) after just starting the NUnit GUI. I mentioned originally
> >> >> >> >> that if I run all the tests these other ones that fail by themselves
> >> >> >> >> will succeed. That's not entirely true. If I start NUnit GUI up
> >> >> >> >> fresh and then run the Complete tests I can then run the Lock tests,
> >> >> >> >> using the currently running NUnit GUI that ran the Complete tests,
> >> >> >> >> successfully.
> >> >> >> >> Here is what my guess was as to what's going on. It appears the
> >> >> >> >> tests, and I mean the code in the NUnit test methods, which have code
> >> >> >> >> which directly references code from the Common assembly, this is the
> >> >> >> >> assembly that's having the issue of getting loaded, will always work
> >> >> >> >> whether run separately or when running all tests. The tests which
> >> >> >> >> don't reference code from the Common assembly won't work unless the
> >> >> >> >> Common assembly has already been loaded into the process by first
> >> >> >> >> running a test that referenced it. Almost as if the code is getting
> >> >> >> >> jitted with an app base setting which is correct and allows the .NET
> >> >> >> >> assembly loader to find the Common assembly using the usual probing
> >> >> >> >> mechanism, but then executed in a different appdomain which doesn't
> >> >> >> >> have the correct app base setting such that if/when Common needs to be
> >> >> >> >> loaded it cannot be found.
> >> >> >> >> Thanks,
> >> >> >> >> Nick
> >> >> >> >> On Mar 31, 3:15 pm, Charlie Poole <nunit...@gmail.com> wrote:
The first thing I did was to create a console application which
references the assembly that contains the tests and it calls the Lock
Test3 and Test5. These are minimum set of tests I would run from
NUnit GUI from choosing the Lock class that would fail. From my
console app they both ran fine. Here is my console application:
using System;
using Redbone.Framework.BusinessProcess.Tests;
public class Application
{
public static void Main()
{
try
{
Lock @lock = new Lock();
@lock.Test3();
@lock.Test5();
Console.WriteLine("Completed both tests successfully.");
}
catch(Exception e)
{
Console.WriteLine(e.ToString());
}
}
}
Now I'll try to come up with a minimum set of code which fails in
NUnit GUI which I could send out to others.
By the way, here is the details of the exception the debugger gave me
when the exception was thrown in the nunit-agent:
System.TypeInitializationException occurred
Message=The type initializer for '<Module>' threw an exception.
Source=Redbone.Framework.BusinessProcess
TypeName=<Module>
StackTrace:
at
Redbone.Framework.BusinessProcess.ProcessService.Start(XmlQualifiedName
name, String description, Version version, String step, String key,
IRow row)
at Redbone.Framework.BusinessProcess.Tests.Lock.Test5() in C:
\data\development\Redbone\bpf\Tests\Lock.cs:line 76
InnerException: System.TypeInitializationException
Message=The type initializer for '<Module>' threw an exception.
Source=ssoDotNet
TypeName=<Module>
StackTrace:
at
<CrtImplementationDetails>.ThrowModuleLoadException(String ,
Exception )
at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
at .cctor()
InnerException: <CrtImplementationDetails>.ModuleLoadException
Message=The C++ module failed to load while attempting to
initialize the default appdomain.
Source=msvcm90
StackTrace:
at
<CrtImplementationDetails>.ThrowModuleLoadException(String
errorMessage, Exception innerException)
at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
at .cctor()
InnerException:
System.Runtime.Serialization.SerializationException
Message=Unable to find assembly
'Redbone.Framework.BusinessProcess.Common, Version=4.0.0.0,
Culture=neutral, PublicKeyToken=e3fe5053ddfa671d'.
Source=mscorlib
StackTrace:
Server stack trace:
at
System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssemb ly()
at
System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryA ssemblyInfo
assemblyInfo, String name)
at
System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String
objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA,
Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader
objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo,
SizedArray assemIdToAssemblyTable)
at
System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWit hMapTyped(BinaryObjectWithMapTyped
record)
at
System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at
System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(Hea derHandler
handler, __BinaryParser serParser, Boolean fCheck, Boolean
isCrossAppDomain, IMethodCallMessage methodCallMessage)
at
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize( Stream
serializationStream, HeaderHandler handler, Boolean fCheck, Boolean
isCrossAppDomain, IMethodCallMessage methodCallMessage)
at
System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject (MemoryStream
stm)
at
System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage.FixupForNewAppD omain()
at
System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[]
reqStmBuff, SmuggledMethodCallMessage smuggledMcm,
SmuggledMethodReturnMessage& smuggledMrm)
at
System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCal lback(Object[]
args)
Exception rethrown at [0]:
at
System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage
reqMsg, IMessage retMsg)
at
System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
msgData, Int32 type)
at System.AppDomain.get_Id()
at
<CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr function,
Void* cookie)
at
<CrtImplementationDetails>.DefaultDomain.Initialize()
at
<CrtImplementationDetails>.LanguageSupport.InitializeDefaultAppDomain(Langu ageSupport* )
at
<CrtImplementationDetails>.LanguageSupport._Initialize(LanguageSupport* )
at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
InnerException:
Our framework makes use of SSO, some managed TIBCO assemblies. The
SSO assemblies in turn have implicit links to unmanaged assemblies.
These assemblies are in a directory which are in the path. I had a
breakpoint on ProcessService.Start() but it never hit that. It shows
Start() at the top of the exception stack. My guess is that when the
call was attempted it started to jit the code for Start() and that's
where it ran into a problem.
Thanks,
Nick
On Apr 4, 11:47 am, nickdu <nic...@msn.com> wrote:
> I'm going to put some effort in today to get a simple repro of this.
> I hope I'm successful. By the way, I tried 2.5.10 and the same issue
> occurs so it's not specific to 2.6.
> Thanks,
> Nick
> On Apr 2, 5:09 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > That would help. I'd definitely like to see it.
> > Regarding dependencies, I'd say that if running tests 1-4 has an
> > effect on test 5, that's a dependency by definition. It's just one that
> > we don't yet understand and which is probably due to the underlying
> > platform - or NUnit itself - rather than something you coded explicitly.
> > Charlie
> > On Mon, Apr 2, 2012 at 12:43 PM, nickdu <nic...@msn.com> wrote:
> > > Yeah, I'm thinking about spending a little bit of time to create a
> > > skelaton of our framework that has the same dependent assemblies to
> > > see if I can repro and if so send it along or post the code.
> > > With respect to test dependencies, I don't think this could be the
> > > case. As I said, I can start NUnit GUI fresh and run Lock Test5
> > > without a problem, and therefore it's not dependent on running any
> > > other tests. However, if I start NUnit GUI fresh and select the Lock
> > > 'set' of tests and run them then Test1 - Test4 succeed and Test5 -
> > > Test8 succeed.
> > > Nick
> > > On Apr 2, 12:54 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > >> Hi Nick,
> > >> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote:
> > >> > There were trace files created with trace information in them, just
> > >> > none of it seemed to point to a reason for the issue I'm having.
> > >> Right - that's what I meant. NUnit is simply considering your
> > >> problem as a test failure, not an NUnit problem worthy of logging.
> > >> > I don't have any automatic reload setting turned on.
> > >> OK
> > >> > When I mentioned loading it was more about how NUnit loads a class and/
> > >> > or method in order to execute it. It seems odd that if I select the
> > >> > Lock class and run I get different behavior than when I choose each
> > >> > Lock test method individually and run them. My guess is that NUnit is
> > >> > loading the code differently in those two cases.
> > >> NUnit's "Loading" involves only your test assembly. It has to happen before
> > >> the tests can be displayed in the Gui. That loading is only repeated if you
> > >> use a reload item on the menu or if you have enabled automatic reload on
> > >> run or when the assembly changes.
> > >> Loading of dependent assemblies happens according to normal .NET rules,
> > >> as the code is jitted and executed. Again, once a dependent assembly is
> > >> loaded into the test AppDomain, it stays there.
> > >> So, your problem arises when the dependent assembly is first needed by
> > >> one of the Lock tests. If you have run some other test first, which successfully
> > >> loads the assembly, then the Lock tests have no problem. This is exactly
> > >> the sort of inter-test dependency I was originally writing about - not
> > >> necessarily
> > >> a dependency that you intentionally created, but one that exists nevertheless.
> > >> I guess that gives you two possible attacks on the problem:
> > >> 1. Trace how and why the Lock class cannot load the assembly.
> > >> 2. Trace how and why the complete class is able to load the assembly.
> > >> What's different in these classes?
> > >> Charlie
> > >> PS: Feel free to post some code if it doesn't expose any company secrets.
> > >> > Thanks,
> > >> > Nick
> > >> > On Apr 1, 6:08 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > >> >> Well, the absence of trace info from NUnit at least tells us that NUnit is
> > >> >> not aware of any problem. I guess we knew that though.
> > >> >> Regarding loading: NUnit does not reload any tests unless you tell it to.
> > >> >> You should probably turn off any automatic reload settings if you have
> > >> >> them set. That way you can manually control when a reload takes place.
I believe I found the cause of the error. I have a class defined in
the Common assembly (the assembly that the appdomain says it can't
load which is in the same directory as my NUnit test assembly) which
derives from ILogicalThreadAffinitive. I have this class deriving
from ILogicalThreadAffinitive so that the class can flow across
threads in a call chain. The problem is that in NUnit we have a
process with multiple appdomains with different application base
directories so that when a call flows from the appdomain that the
NUnit test assembly is loaded calls into the other appdomain which
doesn't have the application directory set to the directory where the
NUnit test assembly exists, it can't load the Common assembly.
> The first thing I did was to create a console application which
> references the assembly that contains the tests and it calls the Lock
> Test3 and Test5. These are minimum set of tests I would run from
> NUnit GUI from choosing the Lock class that would fail. From my
> console app they both ran fine. Here is my console application:
> using System;
> using Redbone.Framework.BusinessProcess.Tests;
> public class Application
> {
> public static void Main()
> {
> try
> {
> Lock @lock = new Lock();
> @lock.Test3();
> @lock.Test5();
> Console.WriteLine("Completed both tests successfully.");
> }
> catch(Exception e)
> {
> Console.WriteLine(e.ToString());
> }
> }
> }
> Now I'll try to come up with a minimum set of code which fails in
> NUnit GUI which I could send out to others.
> By the way, here is the details of the exception the debugger gave me
> when the exception was thrown in the nunit-agent:
> System.TypeInitializationException occurred
> Message=The type initializer for '<Module>' threw an exception.
> Source=Redbone.Framework.BusinessProcess
> TypeName=<Module>
> StackTrace:
> at
> Redbone.Framework.BusinessProcess.ProcessService.Start(XmlQualifiedName
> name, String description, Version version, String step, String key,
> IRow row)
> at Redbone.Framework.BusinessProcess.Tests.Lock.Test5() in C:
> \data\development\Redbone\bpf\Tests\Lock.cs:line 76
> InnerException: System.TypeInitializationException
> Message=The type initializer for '<Module>' threw an exception.
> Source=ssoDotNet
> TypeName=<Module>
> StackTrace:
> at
> <CrtImplementationDetails>.ThrowModuleLoadException(String ,
> Exception )
> at
> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
> at .cctor()
> InnerException: <CrtImplementationDetails>.ModuleLoadException
> Message=The C++ module failed to load while attempting to
> initialize the default appdomain.
> Source=msvcm90
> StackTrace:
> at
> <CrtImplementationDetails>.ThrowModuleLoadException(String
> errorMessage, Exception innerException)
> at
> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
> at .cctor()
> InnerException:
> System.Runtime.Serialization.SerializationException
> Message=Unable to find assembly
> 'Redbone.Framework.BusinessProcess.Common, Version=4.0.0.0,
> Culture=neutral, PublicKeyToken=e3fe5053ddfa671d'.
> Source=mscorlib
> StackTrace:
> Server stack trace:
> at
> System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssemb ly()
> at
> System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryA ssemblyInfo
> assemblyInfo, String name)
> at
> System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String
> objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA,
> Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader
> objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo,
> SizedArray assemIdToAssemblyTable)
> at
> System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWit hMapTyped(BinaryObjectWithMapTyped
> record)
> at
> System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
> at
> System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(Hea derHandler
> handler, __BinaryParser serParser, Boolean fCheck, Boolean
> isCrossAppDomain, IMethodCallMessage methodCallMessage)
> at
> System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize( Stream
> serializationStream, HeaderHandler handler, Boolean fCheck, Boolean
> isCrossAppDomain, IMethodCallMessage methodCallMessage)
> at
> System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject (MemoryStream
> stm)
> at
> System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage.FixupForNewAppD omain()
> at
> System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[]
> reqStmBuff, SmuggledMethodCallMessage smuggledMcm,
> SmuggledMethodReturnMessage& smuggledMrm)
> at
> System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCal lback(Object[]
> args)
> Exception rethrown at [0]:
> at
> System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage
> reqMsg, IMessage retMsg)
> at
> System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
> msgData, Int32 type)
> at System.AppDomain.get_Id()
> at
> <CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr function,
> Void* cookie)
> at
> <CrtImplementationDetails>.DefaultDomain.Initialize()
> at
> <CrtImplementationDetails>.LanguageSupport.InitializeDefaultAppDomain(Langu ageSupport* )
> at
> <CrtImplementationDetails>.LanguageSupport._Initialize(LanguageSupport* )
> at
> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
> InnerException:
> Our framework makes use of SSO, some managed TIBCO assemblies. The
> SSO assemblies in turn have implicit links to unmanaged assemblies.
> These assemblies are in a directory which are in the path. I had a
> breakpoint on ProcessService.Start() but it never hit that. It shows
> Start() at the top of the exception stack. My guess is that when the
> call was attempted it started to jit the code for Start() and that's
> where it ran into a problem.
> Thanks,
> Nick
> On Apr 4, 11:47 am, nickdu <nic...@msn.com> wrote:
> > I'm going to put some effort in today to get a simple repro of this.
> > I hope I'm successful. By the way, I tried 2.5.10 and the same issue
> > occurs so it's not specific to 2.6.
> > Thanks,
> > Nick
> > On Apr 2, 5:09 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > > That would help. I'd definitely like to see it.
> > > Regarding dependencies, I'd say that if running tests 1-4 has an
> > > effect on test 5, that's a dependency by definition. It's just one that
> > > we don't yet understand and which is probably due to the underlying
> > > platform - or NUnit itself - rather than something you coded explicitly.
> > > Charlie
> > > On Mon, Apr 2, 2012 at 12:43 PM, nickdu <nic...@msn.com> wrote:
> > > > Yeah, I'm thinking about spending a little bit of time to create a
> > > > skelaton of our framework that has the same dependent assemblies to
> > > > see if I can repro and if so send it along or post the code.
> > > > With respect to test dependencies, I don't think this could be the
> > > > case. As I said, I can start NUnit GUI fresh and run Lock Test5
> > > > without a problem, and therefore it's not dependent on running any
> > > > other tests. However, if I start NUnit GUI fresh and select the Lock
> > > > 'set' of tests and run them then Test1 - Test4 succeed and Test5 -
> > > > Test8 succeed.
> > > > Nick
> > > > On Apr 2, 12:54 pm, Charlie Poole <nunit...@gmail.com> wrote:
> > > >> Hi Nick,
> > > >> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <nic...@msn.com> wrote:
> > > >> > There were trace files created with trace information in them, just
> > > >> > none of it seemed to point to a reason for the issue I'm having.
> > > >> Right - that's what I meant. NUnit is simply considering your
> > > >> problem as a test failure, not an NUnit problem worthy of logging.
> > > >> > I don't have any automatic reload setting turned on.
> > > >> OK
> > > >> > When I mentioned loading it was more about how NUnit loads a class and/
> > > >> > or method in order to execute it. It seems odd that if I select the
> > > >> > Lock class and run I get different behavior than when I choose each
> > > >> > Lock test method individually
Sounds like you may have found an obscure NUnit bug.
Nunit actually has code that's supposed to prevent this from happening and even some tests to make sure it doesn't happen. But it could be that we have missed some particular situation.
All the calls back to the gui or console runner go through an event queue, which is written by the test thread and read by a thread that is used only to return events to the NUnit AppDomain. So, in theory, this should prevent any part of the thread context from flowing back to the NUnit application.
The only place where I know this can be violated is if an exception is thrown, but maybe there are others.
If you have time and are able to create a simple repro of this problem, I hope you'll submit a bug.
On Tue, Apr 10, 2012 at 5:27 PM, nickdu <nic...@msn.com> wrote: > I believe I found the cause of the error. I have a class defined in > the Common assembly (the assembly that the appdomain says it can't > load which is in the same directory as my NUnit test assembly) which > derives from ILogicalThreadAffinitive. I have this class deriving > from ILogicalThreadAffinitive so that the class can flow across > threads in a call chain. The problem is that in NUnit we have a > process with multiple appdomains with different application base > directories so that when a call flows from the appdomain that the > NUnit test assembly is loaded calls into the other appdomain which > doesn't have the application directory set to the directory where the > NUnit test assembly exists, it can't load the Common assembly.
> Thanks, > Nick
> On Apr 5, 3:41 pm, nickdu <nic...@msn.com> wrote: >> The first thing I did was to create a console application which >> references the assembly that contains the tests and it calls the Lock >> Test3 and Test5. These are minimum set of tests I would run from >> NUnit GUI from choosing the Lock class that would fail. From my >> console app they both ran fine. Here is my console application:
>> using System; >> using Redbone.Framework.BusinessProcess.Tests;
>> public class Application >> { >> public static void Main() >> { >> try >> { >> Lock @lock = new Lock(); >> @lock.Test3(); >> @lock.Test5(); >> Console.WriteLine("Completed both tests successfully."); >> } >> catch(Exception e) >> { >> Console.WriteLine(e.ToString()); >> } >> }
>> }
>> Now I'll try to come up with a minimum set of code which fails in >> NUnit GUI which I could send out to others.
>> By the way, here is the details of the exception the debugger gave me >> when the exception was thrown in the nunit-agent:
>> System.TypeInitializationException occurred >> Message=The type initializer for '<Module>' threw an exception. >> Source=Redbone.Framework.BusinessProcess >> TypeName=<Module> >> StackTrace: >> at >> Redbone.Framework.BusinessProcess.ProcessService.Start(XmlQualifiedName >> name, String description, Version version, String step, String key, >> IRow row) >> at Redbone.Framework.BusinessProcess.Tests.Lock.Test5() in C: >> \data\development\Redbone\bpf\Tests\Lock.cs:line 76 >> InnerException: System.TypeInitializationException >> Message=The type initializer for '<Module>' threw an exception. >> Source=ssoDotNet >> TypeName=<Module> >> StackTrace: >> at >> <CrtImplementationDetails>.ThrowModuleLoadException(String , >> Exception ) >> at >> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* ) >> at .cctor() >> InnerException: <CrtImplementationDetails>.ModuleLoadException >> Message=The C++ module failed to load while attempting to >> initialize the default appdomain.
>> Source=msvcm90 >> StackTrace: >> at >> <CrtImplementationDetails>.ThrowModuleLoadException(String >> errorMessage, Exception innerException) >> at >> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* ) >> at .cctor() >> InnerException: >> System.Runtime.Serialization.SerializationException >> Message=Unable to find assembly >> 'Redbone.Framework.BusinessProcess.Common, Version=4.0.0.0, >> Culture=neutral, PublicKeyToken=e3fe5053ddfa671d'. >> Source=mscorlib >> StackTrace: >> Server stack trace: >> at >> System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssemb ly() >> at >> System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryA ssemblyInfo >> assemblyInfo, String name) >> at >> System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String >> objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, >> Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader >> objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, >> SizedArray assemIdToAssemblyTable) >> at >> System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWit hMapTyped(BinaryObjectWithMapTyped >> record) >> at >> System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run() >> at >> System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(Hea derHandler >> handler, __BinaryParser serParser, Boolean fCheck, Boolean >> isCrossAppDomain, IMethodCallMessage methodCallMessage) >> at >> System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize( Stream >> serializationStream, HeaderHandler handler, Boolean fCheck, Boolean >> isCrossAppDomain, IMethodCallMessage methodCallMessage) >> at >> System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject (MemoryStream >> stm) >> at >> System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage.FixupForNewAppD omain() >> at >> System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[] >> reqStmBuff, SmuggledMethodCallMessage smuggledMcm, >> SmuggledMethodReturnMessage& smuggledMrm) >> at >> System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCal lback(Object[] >> args) >> Exception rethrown at [0]: >> at >> System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage >> reqMsg, IMessage retMsg) >> at >> System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& >> msgData, Int32 type) >> at System.AppDomain.get_Id() >> at >> <CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr function, >> Void* cookie) >> at >> <CrtImplementationDetails>.DefaultDomain.Initialize() >> at >> <CrtImplementationDetails>.LanguageSupport.InitializeDefaultAppDomain(Langu ageSupport* ) >> at >> <CrtImplementationDetails>.LanguageSupport._Initialize(LanguageSupport* ) >> at >> <CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* ) >> InnerException:
>> Our framework makes use of SSO, some managed TIBCO assemblies. The >> SSO assemblies in turn have implicit links to unmanaged assemblies. >> These assemblies are in a directory which are in the path. I had a >> breakpoint on ProcessService.Start() but it never hit that. It shows >> Start() at the top of the exception stack. My guess is that when the >> call was attempted it started to jit the code for Start() and that's >> where it ran into a problem.
>> Thanks, >> Nick
>> On Apr 4, 11:47 am, nickdu <nic...@msn.com> wrote:
>> > I'm going to put some effort in today to get a simple repro of this. >> > I hope I'm successful. By the way, I tried 2.5.10 and the same issue >> > occurs so it's not specific to 2.6.
>> > Thanks, >> > Nick
>> > On Apr 2, 5:09 pm, Charlie Poole <nunit...@gmail.com> wrote:
>> > > That would help. I'd definitely like to see it.
>> > > Regarding dependencies, I'd say that if running tests 1-4 has an >> > > effect on test 5, that's a dependency by definition. It's just one that >> > > we don't yet understand and which is probably due to the underlying >> > > platform - or NUnit itself - rather than something you coded explicitly.
>> > > Charlie
>> > > On Mon, Apr 2, 2012 at 12:43 PM, nickdu <nic...@msn.com> wrote: >> > > > Yeah, I'm thinking about spending a little bit of time to create a >> > > > skelaton of our framework that has the same dependent assemblies to >> > > > see if I can repro and if so send it along or post the code.
>> > > > With respect to test dependencies, I don't think this could be the >> > > > case. As I said, I can start NUnit GUI fresh and run Lock Test5 >> > > > without a problem, and therefore it's not dependent on running any >> > > > other tests. However, if I start NUnit GUI fresh and select the Lock >> > > > 'set' of tests and run them then Test1 - Test4 succeed and Test5 - >> > > > Test8 succeed.
