Is there an XSD available for the Gallio XML format report?

[jasonint]

I'm working on a way to import our Gallio test report data into a .trx
format so it can be published to TFS. Is there an XSD available for
the Gallio XML formatted report? Has anybody dealt with this issue
successfully yet? I'd appreciate any suggestions on the best way to
do this.

[Jeff Brown]

Yes, there is an XSD available.

In v3.1, you'll find it in bin\Gallio.xsd.
In v3.2, you'll find it in sdk\Gallio.xsd.

I seem to recall some people have tried to do this already.  Here are a couple of different approaches:

1. You could use XSLT to transform the Gallio XML report into TRX.  This is going to be non-trivial because Gallio and TFS have rather different notions of what tests look like.  That said, an XSL template would be good because you could add it to Gallio as an extension.  Look at Gallio.Reports.plugin for examples.

2. You could write an ITestRunnerExtension that captures Gallio test runner events and writes out a TRX or uploads the information directly to TFS some other way.  Look at TeamCity.plugin for examples.

3. You could actually run tests with Gallio via MSTest and have MSTest do its magic for you.  For that to work, you'll need to install the Gallio Visual Studio add-in.  Then MSTest.exe will be able to run MbUnit tests, among other things.

Jeff.

--
You received this message because you are subscribed to the Google Groups "gallio-dev" group.
To post to this group, send email to galli...@googlegroups.com.
To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gallio-dev?hl=en.

[jasonint]

I should have mentioned that we are using VSTS and TeamBuild to do
automated builds. So, I'm executing the tests using the Gallio
MSBuild task in my TfsBuild.proj file. In that context does option #3
still offer any possible resolution or is that strictly for testing
through the VS GUI? Can I execute MbUnit via MsUnit from a build
script?

If that cannot work I'll probably look at option #2 to programatically
write out the trx xml. I've been hunting around for more info about
writing ITestRunnerExtensions but I haven't hit upon much yet. Can you
direct me to a good reference source?

Thanks!

- Jay

> > gallio-dev+...@googlegroups.com<gallio-dev%2Bunsubscribe@googlegrou­ps.com>

> > .
> > For more options, visit this group at

> >http://groups.google.com/group/gallio-dev?hl=en.- Hide quoted text -
>
> - Show quoted text -

[jasonint]

I see the TeamCity extension. Checking that out now.

> > >http://groups.google.com/group/gallio-dev?hl=en.-Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -

[Jeff Brown]

Just use the <Exec> task to run MSTest.exe and provide it with the path of an MbUnit dll.  It should work (although there were some regressions in Gallio v3.1).

Personally I think the test runner extension option is the way to go, though.  It's much more flexible.

Jeff.

To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.

[jasonint]

Thanks Jeff.

I experimented yesterday with the msunit.exe /publish commands to
publish a couple of desktop generated trx files to TFS. One file was
a straight up MSTest trx file from a dummy MSTest project I made. The
other was a file generated by running my real MBUnit tests via the VS
context menu "Run Tests" menu item which I am assuming is wired into
the Gallio/VS Integration.

I examined the trx files created from both approaches. I see that the
file produced by MbUnit via MsTest has Gallio specific markup in it
such as "GallioTestResult". When I tried to publish that file to TFS
it spit back some errors related to those unrecognized elements and
produced no visible change in the build report, - statistics were not
updated, no test results were associated with it. The publish
operation appears to have completed though, despite the errors,
because it would not let me re-run the publish command for the same
build number. However, no useful test data was actually persisted.

So, publishing test summary statistics and individual test results
back to TFS being the primary objective, if the markup produced by the
MbUnit via MsTest method is only fully understood by the GUI with the
help of a plugin, but not by TFS, then that approach will not work.

It seems like an XSLT approach would be tricky because the
transformation has to generate and keep track of several GUIDs that
the trx needs injected. I'm not sure how that would be done from a
pure transform operation, though I'm not an expert on such things.
Maybe its possible.

From what I can tell so far, writing an extension that composes the
xml for the trx based on the Visual Studio vstst.vsd schema seems like
the only way to generate xml that TFS will accept. The only special
operation would be to generate and inject the Guids.

Does that sound about accurate? Am I missing anything?

Thanks again for your time Jeff!

- Jay

> > > >http://groups.google.com/group/gallio-dev?hl=en.-Hide quoted text -

[Jeff Brown]

Oh right, Gallio has its own TestResult type which will appear in TRX files.  That's just how MSTest extensions are supposed to work.  You could pretty easily "downgrade" the results to a common type that TFS will understand on its own, maybe spoof an MSTest test result type.  Kind of annoying that TFS requires extensions to be installed into all layers of the system.

Anyways, an extension sounds like the way to do.  You have a few choices when it comes to this.  You can implement ITestRunnerExtension (subclass TestRunnerExtension) or perhaps implement IReportFormatter.  If you want to think of the process as publishing to TFS, then the first approach is better and will give you more control.  If you want to think of the process as generating a TRX that can be further manipulated by TFS tools separately from the test run, then the latter approach may work better.

Generating GUIDs that MSTest will accept isn't too hard.  You could construct them from a hash of "TestData.Id" and "TestRunData.Id" fields in Gallio reports depending on whether you need a unique identifier for the test or for a particular test run.

Jeff.

To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.

[jasonint]

Jeff,

In your earlier post you said I should be able to run MSTest with an
EXEC task and pass an MbUnit dll to it. I tried this but am getting
"Exception has been thrown by the target of an invocation". I'm doing
something like the following:

mstest /testcontainer:"C:\MyUnitTests.dll"

Any ideas what I'm doing wrong?

Also, I see that from the VSTS 2008 IDE
I can publish a TRX generated from the Gallio/VS integration to TFS
without a problem. Works great. I'm trying to understand if its
possible to
do the same thing from the command line. If I could do that, I'd be
all set without the need for further customization.

Thanks,

Jay

> > > > > >http://groups.google.com/group/gallio-dev?hl=en.-Hidequoted text -

[Jeff Brown]

Doh.  Known bug that I fixed a couple weeks ago.
Use a recent build of Gallio v3.1 or v3.2.

http://ccnet.gallio.org/Distributables/

(This is precisely why we will be releasing Gallio v3.1 Update 3 sometime soonish.)

Jeff.

To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.

[jasonint]

Sweet! I'll update and give it another go. I will be very happy if
this works.

Thanks once again.

- Jay

[jasonint]

I tried to run a dll containing mbunit tests with msunit from the
command line with no success.

First I ran it on my desktop where I have a side-by-side installation
of VS2008 SP1 and VS2010 RC2.

I ran it against the 2008 version of MSTEST and received an exception
with the message, "MS TEST HAS HIT A BREAKPOINT".

mstest /testcontainer:MyMbUnitTests.dll

I guessed this might be due to a conflict with 2010. I see in the
issue logs that there was an issue last year relating to side-by-side
2008/2005 installations. Maybe this is the same type of issue?

I then tried running it against the 2010 MSTest and received the
following:

Starting execution...
No tests to execute.

I then tried the same on my development VM which only has VS 2008 SP1
installed. In that environment I did not see any exceptions but again
received:

Starting execution...
No tests to execute.

Now, not knowing any better, this is the output I would expect. After
all, why would MSTest be able to recognize and process MBUnit
attributes? However, you stated that this should work, so I'm
assuming there is some magic that goes on between MSTest and Gallio
that makes it work.

Any suggestions?

I owe you a box of donuts.

- Jay

> > > > > > >http://groups.google.com/group/gallio-dev?hl=en.-Hidequotedtext -

>
> > > > > > - Show quoted text -
>
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "gallio-dev" group.
> > > > > To post to this group, send email to galli...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > > > > gallio-dev+...@googlegroups.com<gallio-dev%2Bunsubscribe@googlegrou­­ps.com>
> > > <gallio-dev%2Bunsubscribe@googlegrou­ps.com>
> > > > > .
> > > > > For more options, visit this group at
>

> ...
>
> read more »- Hide quoted text -

[jasonint]

I should have also mentioned those last attempts were made with:

GallioBundle-3.1.494.0-Setup-x86.msi on my local system and

GallioBundle-3.1.494.0-Setup-x64.msi on my VM system

Maybe I should look for a 3.2 build instead?

[jasonint]

Hold the phone. I see now that I totally failed to read the dates on
the dev build page and downloaded an older build. I'll try the newest
one and post the results after testing with that.

On Mar 19, 10:41 am, jasonint <jason...@gmail.com> wrote:

[jasonint]

OK, ran it with :

GallioBundle-3.2.372.0-Setup-x86.msi

and

GallioBundle-3.2.372.0-Setup-x64.msi

but same exact results as described in earlier post.

Are these bugs?

Thanks,

Jay

[jasonint]

Do I need to create a vsmdi or testrunconfig file to pass an MbUnit
dll as TestContainer to MsTest command line?

[jasonint]

I actually had to uninstall the 64bit dev build from my VM because
something in it broke the VS Integration. I rolled back to the latest
official 64bit release.

I could not run any MbUnit tests from the Test List Editor or with the
Run Tests context menuitem. Every test would just error out without
any specific exception details and without executing the test.

- Jay

[Jeff Brown]

Ok, I've reproduced the problem on VS 2010 RC.
I'll get back to you.

Jeff.

--
You received this message because you are subscribed to the Google Groups "gallio-dev" group.
To post to this group, send email to galli...@googlegroups.com.

To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.

[Jeff Brown]

Fixed in v3.2 build 375 or newer.

You should be able to use MSTest to run MbUnit tests when Gallio is installed.

Jeff.

[jasonint]

Very good. I'll update and take another whack at it. Gracias. Merci.
Danka. Sheh-Sheh. Right-awn.

On Mar 20, 4:48 am, Jeff Brown <jeff.br...@gmail.com> wrote:
> Fixed in v3.2 build 375 or newer.
>
> You should be able to use MSTest to run MbUnit tests when Gallio is
> installed.
>
> Jeff.
>
>
>
> On Fri, Mar 19, 2010 at 2:52 PM, Jeff Brown <jeff.br...@gmail.com> wrote:
> > Ok, I've reproduced the problem on VS 2010 RC.
> > I'll get back to you.
>
> > Jeff.
>

> ...
>
> read more »

[jasonint]

I uninstalled previous version and did new install of

Tuesday, March 23, 2010 4:44 AM 41215356 GallioBundle-3.2.381.0-
Setup-x64.msi

I updated all my project references to point to the new bin
assemblies, rebuilt and attepted new test run.

All tests errored out as before without actually running. I unzipped
and ran the MbUnit.Samples.vs2008 project and got the same results.

I will post this as an issue to the issues board.

- Jay

[jasonint]

I've submitted Issue 642 for this issue.

Also, I tried running my test dll against the MSTest command line and
it worked! Hazah!

Unfortunately it also has the same bug as through the IDE and is not
actually executing the tests.

It is attempting to though and producing the trx file which is very
encouraging. If y'all can just sort out that
execution bug, I think we'll be back in business.

Thanks.

[Jeff Brown]

What was the error?

--

[jasonint]

It doesn't say. The result of every test comes back as "ERROR" without
any additional detail. In the IDE test run panel the result column
displays a warning icon and the word "ERROR". At the command line the
echos all return "ERROR" for every test. If you look at the issue I
opened for this I included a screenshot of the IDE.

[Jeff Brown]

Hmm.  I'll check it out.

Jeff.

[jasonint]

Any news regarding this issue? I've managed to get all the other
pieces working. I can run MbUnit tests from MsTest command line and
then transform the trx format output by that to a purely Microsoft
schema format using Linq-To-Xml and then publish to TFS. I just can't
get any valid results from any tests. So close!

Thanks!

[Jeff Brown]

Interesting. I fixed things in VS 2010 and broke them in VS 2008. Joy.

(Time to write more integration tests.)

Jeff.

[jasonint]

Sorry to be the bringer of bad news!
That's just the way it goes sometimes. I understand. : )

Please post back to this thread when you find a resolution.
I'll be pacing the floor chain smoking and chugging coffee like an
anxious father in the maternity ward until I hear from you!
(No pressure)

Thanks

- Jay

> > For more options, visit this group athttp://groups.google.com/group/gallio-dev?hl=en.- Hide quoted text -

[Jeff Brown]

I have checked in a tentative fix but I haven't tested it yet. It
will be in build 385.

Jeff.

[jasonint]

I just uninstalled 3.2.381, rebooted and then installed 3.2.388.0

I opened VSTS 2008 and loaded my unit test projecvt, then clicked on
the Test-->Windows-->Test List Editor

which caused VSTS to crash with the following error:

Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: devenv.exe
Problem Signature 02: 9.0.30729.1
Problem Signature 03: 488f2b50
Problem Signature 04: Gallio
Problem Signature 05: 3.2.0.0
Problem Signature 06: 4baeb944
Problem Signature 07: 3e21
Problem Signature 08: 0
Problem Signature 09: System.NullReferenceException
OS Version: 6.0.6002.2.2.0.272.7
Locale ID: 1033

Additional information about the problem:
LCID: 1033

I've restarted VS 3 or 4 times, but keeps crashing with same error.

- Jay

[Jeff Brown]

Yikes.
Well it should be pretty straightforward for me to find.

Jeff.

[jasonint]

I'm rooting for you. I'm thinking it wouldn't be a bad idea if I
figured out how to get setup as a Gallio developer over here so I can
help debug some of this stuff myself. Just not quite sure where to
start.

[Jeff Brown]

Here's one starting point. :)

http://www.gallio.org/wiki/doku.php?id=development

Jeff.

[Jeff Brown]

Hmm, I can't reproduce this particular crash with build 389 in VS
2008. Can you get more information about the NPE? Just start another
instance of Visual Studio, configure it to load Gallio debug symbols,
attach it to the first instance, and make the bug happen.

How to configure symbols and source code debugging:
http://www.gallio.org/wiki/doku.php?id=gallio:debugging_with_symbols_and_source_code

That said, I can see that tests still aren't running correctly with VS
2008. It seems that AssemblyResolve events aren't getting fired
correctly in the vstesthost. No clue why yet.

Jeff.

[Philip Japikse]

Ok, so not know where to turn *first*, I thought I'd throw this up here and
see if it rings any bells...

My MSpec Specification runs fine if I have a simple Assert.IsTrue(true)
It Should_Return_The_Correct_Value_From_Setup = () =>
{
Assert.IsTrue(true);
};

Once I add in a more complex Assert (like Assert.AreEqual) then I get the
following errors (the first is using the R# test runner, the second is using
the TD.Net test runner)

Gallio 3.2.416
R# 5.0 RTM
MSpec 0.3 (4/7/2010 release)

The MSpec CI server and GIT sites are currently down, otherwise I would have
tried to get the latest bits for MSpec...

Any thoughts?

============================================================================
================================================
ReSharper Test runner Error

Should Return The Correct Value From Setup : FailedAn exception occurred
while verifying an assertion.

System.InvalidOperationException: The runtime has not been initialized.
at Gallio.Runtime.RuntimeAccessor.get_Instance() in
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Runtime\RuntimeAccessor.cs:line 41
at Gallio.Framework.ComparisonSemantics.get_Default() in
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\ComparisonSemantics.cs:line 59

at MSpecSandbox.When_Working_With_Rhino_Mocks_Stubs.<.ctor>b__5() in
E:\Projects\2010\Presentations\BDD\MSpecSandbox\0-MockingConcern.cs:line 27
at Machine.Specifications.Model.Specification.InvokeSpecificationField()
in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs:line 76
at Machine.Specifications.Model.Specification.Verify() in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs:line 53
at
Machine.Specifications.Runner.Impl.SpecificationRunner.Run(Specification
specification) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\SpecificationRunner.cs:line 25
at
Machine.Specifications.Runner.Impl.SetupOnceContextRunner.RunSpecifications(
Context context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 resultSupplementers) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs:line 57
at Machine.Specifications.Runner.Impl.SetupOnceContextRunner.Run(Context
context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 globalCleanups, IEnumerable`1 resultSupplementers) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs:line 28
at Machine.Specifications.Runner.Impl.AssemblyRunner.Run(Assembly
assembly, IEnumerable`1 contexts) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\AssemblyRunner.cs:line 46
at
Machine.Specifications.Runner.Impl.DefaultRunner.StartRun(IDictionary`2
contextMap) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 94
at Machine.Specifications.Runner.Impl.DefaultRunner.RunClass(MemberInfo
member, Assembly assembly) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 81
at Machine.Specifications.Runner.Impl.DefaultRunner.RunMember(Assembly
assembly, MemberInfo member) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 60
at
Machine.Specifications.ReSharperRunner.Runners.RecursiveMSpecTaskRunner.Fini
sh(TaskExecutionNode node) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications.ReSharpe
rRunner.4.1\Runners\RecursiveMSpecTaskRunner.cs:line 66
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.Execute(TaskExe
cutionNode node)
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.DoExecuteNodes(
IEnumerable`1 nodes)
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.ExecuteNodes(IL
ist`1 nodes)
at
JetBrains.ReSharper.TaskRunnerFramework.TaskRunnerHostBase.ExecuteSerialized
Tree(String serializedTree)

Gallio.Framework.Assertions.AssertionFailureException: An exception occurred
while verifying an assertion.

System.InvalidOperationException: The runtime has not been initialized.
at Gallio.Runtime.RuntimeAccessor.get_Instance() in
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Runtime\RuntimeAccessor.cs:line 41
at Gallio.Framework.ComparisonSemantics.get_Default() in
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\ComparisonSemantics.cs:line 59

at MSpecSandbox.When_Working_With_Rhino_Mocks_Stubs.<.ctor>b__5() in
E:\Projects\2010\Presentations\BDD\MSpecSandbox\0-MockingConcern.cs:line 27
at Machine.Specifications.Model.Specification.InvokeSpecificationField()
in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs:line 76
at Machine.Specifications.Model.Specification.Verify() in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs:line 53
at
Machine.Specifications.Runner.Impl.SpecificationRunner.Run(Specification
specification) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\SpecificationRunner.cs:line 25
at
Machine.Specifications.Runner.Impl.SetupOnceContextRunner.RunSpecifications(
Context context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 resultSupplementers) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs:line 57
at Machine.Specifications.Runner.Impl.SetupOnceContextRunner.Run(Context
context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 globalCleanups, IEnumerable`1 resultSupplementers) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs:line 28
at Machine.Specifications.Runner.Impl.AssemblyRunner.Run(Assembly
assembly, IEnumerable`1 contexts) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\AssemblyRunner.cs:line 46
at
Machine.Specifications.Runner.Impl.DefaultRunner.StartRun(IDictionary`2
contextMap) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 94
at Machine.Specifications.Runner.Impl.DefaultRunner.RunClass(MemberInfo
member, Assembly assembly) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 81
at Machine.Specifications.Runner.Impl.DefaultRunner.RunMember(Assembly
assembly, MemberInfo member) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs:line 60
at
Machine.Specifications.ReSharperRunner.Runners.RecursiveMSpecTaskRunner.Fini
sh(TaskExecutionNode node) in
d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications.ReSharpe
rRunner.4.1\Runners\RecursiveMSpecTaskRunner.cs:line 66
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.Execute(TaskExe
cutionNode node)
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.DoExecuteNodes(
IEnumerable`1 nodes)
at
JetBrains.ReSharper.TaskRunnerFramework.CurrentAppDomainHost.ExecuteNodes(IL
ist`1 nodes)
at
JetBrains.ReSharper.TaskRunnerFramework.TaskRunnerHostBase.ExecuteSerialized
Tree(String serializedTree)

at
Gallio.Framework.Assertions.AssertionContext.Scope.ThrowFailureAccordingToBe
havior(AssertionFailure failure) in c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionContext.cs: line
287
at
Gallio.Framework.Assertions.AssertionContext.SubmitFailure(AssertionFailure
failure) in c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionContext.cs: line
108
at Gallio.Framework.Assertions.AssertionHelper.Fail(AssertionFailure
failure) in c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionHelper.cs: line
107
at Gallio.Framework.Assertions.AssertionHelper.Verify(Func`1 assertionFunc)
in c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionHelper.cs: line 93
at MbUnit.Framework.Assert.AreEqual(T expectedValue, T actualValue,
EqualityComparison`1 comparer, String messageFormat, Object[] messageArgs)
in c:\Server\Projects\MbUnit
v3.2\Work\src\MbUnit\MbUnit\Framework\Assert.Comparisons.cs: line 108
at MbUnit.Framework.Assert.AreEqual(T expectedValue, T actualValue) in
c:\Server\Projects\MbUnit
v3.2\Work\src\MbUnit\MbUnit\Framework\Assert.Comparisons.cs: line 38
at MSpecSandbox.When_Working_With_Rhino_Mocks_Stubs.<.ctor>b__5() in
0-MockingConcern.cs: line 27

============================================================================
====================================================================
TD.Net Test Runner
------ Test started: Assembly: MSpecSandbox.dll ------

Mocking, When Working With Rhino Mocks Stubs
» Should Return The Correct Value From Setup (FAIL)

TestCase 'Should Return The Correct Value From Setup' failed:
Gallio.Framework.Assertions.AssertionFailureException: An exception
occurred while verifying an assertion.

System.InvalidOperationException: The runtime has not been
initialized.
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Runtime\RuntimeAccessor.cs(41,0): at
Gallio.Runtime.RuntimeAccessor.get_Instance()
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\ComparisonSemantics.cs(59,0): at
Gallio.Framework.ComparisonSemantics.get_Default()

0-MockingConcern.cs(27,0): at
MSpecSandbox.When_Working_With_Rhino_Mocks_Stubs.<.ctor>b__5()

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs(76,0): at
Machine.Specifications.Model.Specification.InvokeSpecificationField()

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs(53,0): at Machine.Specifications.Model.Specification.Verify()

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\SpecificationRunner.cs(25,0): at
Machine.Specifications.Runner.Impl.SpecificationRunner.Run(Specification
specification)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs(57,0): at
Machine.Specifications.Runner.Impl.SetupOnceContextRunner.RunSpecifications(
Context context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 resultSupplementers)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\ContextRunner.cs(28,0): at
Machine.Specifications.Runner.Impl.SetupOnceContextRunner.Run(Context
context, ISpecificationRunListener listener, RunOptions options,
IEnumerable`1 globalCleanups, IEnumerable`1 resultSupplementers)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\AssemblyRunner.cs(46,0): at
Machine.Specifications.Runner.Impl.AssemblyRunner.Run(Assembly assembly,
IEnumerable`1 contexts)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs(94,0): at
Machine.Specifications.Runner.Impl.DefaultRunner.StartRun(IDictionary`2
contextMap)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs(68,0): at
Machine.Specifications.Runner.Impl.DefaultRunner.RunField(MemberInfo member,
Assembly assembly)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\DefaultRunner.cs(60,0): at
Machine.Specifications.Runner.Impl.DefaultRunner.RunMember(Assembly
assembly, MemberInfo member)

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Runner\I
mpl\AppDomainRunner.cs(175,0): at
Machine.Specifications.Runner.Impl.AppDomainRunner.MemberRunner..ctor(ISpeci
ficationRunListener listener, Assembly assembly, RunOptions options,
MemberInfo memberInfo)
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr,
Binder binder, Object[] args, CultureInfo culture, Object[]
activationAttributes)
at System.Activator.CreateInstance(Type type, BindingFlags
bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[]
activationAttributes)
at System.Activator.CreateInstance(String assemblyName, String
typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder,
Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence
securityInfo, StackCrawlMark& stackMark)
at System.Activator.CreateInstance(String assemblyName, String
typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder,
Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence
securityInfo)
at System.AppDomain.CreateInstance(String assemblyName, String
typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder,
Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence
securityAttributes)
at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName,
String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder
binder, Object[] args, CultureInfo culture, Object[] activationAttributes,
Evidence securityAttributes)

c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionContext.cs(287,0):
at
Gallio.Framework.Assertions.AssertionContext.Scope.ThrowFailureAccordingToBe
havior(AssertionFailure failure)
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionContext.cs(108,0):
at
Gallio.Framework.Assertions.AssertionContext.SubmitFailure(AssertionFailure
failure)
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionHelper.cs(107,0):
at Gallio.Framework.Assertions.AssertionHelper.Fail(AssertionFailure
failure)
c:\Server\Projects\MbUnit
v3.2\Work\src\Gallio\Gallio\Framework\Assertions\AssertionHelper.cs(93,0):
at Gallio.Framework.Assertions.AssertionHelper.Verify(Func`1 assertionFunc)
c:\Server\Projects\MbUnit
v3.2\Work\src\MbUnit\MbUnit\Framework\Assert.Comparisons.cs(108,0): at
MbUnit.Framework.Assert.AreEqual[T](T expectedValue, T actualValue,
EqualityComparison`1 comparer, String messageFormat, Object[] messageArgs)
c:\Server\Projects\MbUnit
v3.2\Work\src\MbUnit\MbUnit\Framework\Assert.Comparisons.cs(38,0): at
MbUnit.Framework.Assert.AreEqual[T](T expectedValue, T actualValue)
0-MockingConcern.cs(27,0): at
MSpecSandbox.When_Working_With_Rhino_Mocks_Stubs.<.ctor>b__5()

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs(75,0): at
Machine.Specifications.Model.Specification.InvokeSpecificationField()

d:\BuildAgent-01\work\340c36596c29db8\Source\Machine.Specifications\Model\Sp
ecification.cs(53,0): at Machine.Specifications.Model.Specification.Verify()

0 passed, 1 failed, 0 skipped, took 3.07 seconds (MSpec).
============================================================================
=======================================



Phil


Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
Principal Consultant, Pinnacle Solutions Group, Inc.
Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664
https://mvp.support.microsoft.com/profile/Japikse 
www.skimedic.com/blog

[Philip Japikse]

It looks like it's an MSpec issue:

E:\Projects\2010\Presentations\BDD\MSpecSandbox\bin\Debug>E:\Downloads\tdd\M
achine.Specifications-release\mspec.exe mspecsandbox.dll
Could not load file or assembly
'file:///E:\Projects\2010\Presentations\BDD\MSpecSandbox\bin\Debug\mspecsand
box.dll' or one of its dependencies. This assembly is built by a runtime
newer than the currently loaded runtime and cannot be loaded.

Their servers are still down, so I can't even see if this issue has been
posted yet...

Phil


Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
Principal Consultant, Pinnacle Solutions Group, Inc.
Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664
https://mvp.support.microsoft.com/profile/Japikse 
www.skimedic.com/blog

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.814 / Virus Database: 271.1.1/2835 - Release Date: 04/25/10
14:31:00

[Jeff Brown]

Hmm...
The basic problem is that MbUnit supports extension via plugins.  For example, adding new comparison semantics can alter how types are compared for equality.

Unfortunately it looks like two things are happening.

1. The Gallio runtime isn't getting initialized prior to running the tests.  That may have something to do with the issue you identified.
2. There seems to be a minor regression in the assertion framework since it is supposed to be able to work even if the runtime isn't initialized -- it'll just revert to some basic default behavior.  That should be easy to fix.  (Yann, can you look into how the ComparisonSemantics is using the runtime?  It might need to check whether the runtime is initialized before doing some stuff.)

Jeff.

--
You received this message because you are subscribed to the Google Groups
"gallio-dev" group.
To post to this group, send email to galli...@googlegroups.com.
To unsubscribe from this group, send email to
gallio-dev+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gallio-dev?hl=en.

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.814 / Virus Database: 271.1.1/2835 - Release Date: 04/25/10
14:31:00

--
You received this message because you are subscribed to the Google Groups "gallio-dev" group.
To post to this group, send email to galli...@googlegroups.com.
To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gallio-dev?hl=en.

[Philip Japikse]

The error comes on this line:

 

                    instance = (IComparisonSemantics)Runtime.RuntimeAccessor.ServiceLocator.ResolveByComponentId("Gallio.ComparisonSemantics");

 

because this never gets called:

        public static void SetRuntime(IRuntime runtime)

        {

            EventHandler instanceChangedHandlers = InstanceChanged;

            instance = runtime;

 

            EventHandlerPolicy.SafeInvoke(instanceChangedHandlers, null, EventArgs.Empty);

        }

 

 

This is when running a test via MSpec using either the R# or TD.Net runners.  I tried setting the runtime to the defaul gallio runtime (gallio.Runtime.DefaultRuntime), but then got a host of other errors.

 

So, pardon my ignorance, but is this something in MSpec or Gallio? Or Both?

 

Thanks!

 

 

Phil

 

[Yann Trevin]

I modified how the comparison semantics service is instantiated
accordingly.
A default instance is now returned when the runtime is not
initialized.

Sorry for the delay. I had completely missed that discussion until
now.

Regards,
Yann

On Apr 26, 5:34 am, Jeff Brown <jeff.br...@gmail.com> wrote:
> Hmm...
> The basic problem is that MbUnit supports extension via plugins.  For
> example, adding new comparison semantics can alter how types are compared
> for equality.
>
> Unfortunately it looks like two things are happening.
>
> 1. The Gallio runtime isn't getting initialized prior to running the tests.
> That may have something to do with the issue you identified.
> 2. There seems to be a minor regression in the assertion framework since it
> is supposed to be able to work even if the runtime isn't initialized --
> it'll just revert to some basic default behavior.  That should be easy to
> fix.  (Yann, can you look into how the ComparisonSemantics is using the
> runtime?  It might need to check whether the runtime is initialized before
> doing some stuff.)
>
> Jeff.
>

> On Sun, Apr 25, 2010 at 4:48 PM, Philip Japikse <phil.japi...@pinnsg.com>wrote:
>
>
>
> > It looks like it's an MSpec issue:
>
> > E:\Projects\2010\Presentations\BDD\MSpecSandbox\bin\Debug>E:\Downloads\tdd\ M
> > achine.Specifications-release\mspec.exe mspecsandbox.dll
> > Could not load file or assembly
>
> > 'file:///E:\Projects\2010\Presentations\BDD\MSpecSandbox\bin\Debug\mspecsan d
> > box.dll' or one of its dependencies. This assembly is built by a runtime
> > newer than the currently loaded runtime and cannot be loaded.
>
> > Their servers are still down, so I can't even see if this issue has been
> > posted yet...
>
> > Phil
>
> > Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
> > Principal Consultant, Pinnacle Solutions Group, Inc.

> > Phil.japi...@pinnsg.com |www.twitter.com/skimedic| (513) 312-5664

> > at
>
> > Gallio.Framework.Assertions.AssertionContext.Scope.ThrowFailureAccordingToB e
> > havior(AssertionFailure failure)
>
> ...
>
> read more »

[Philip Japikse]

Awesome! What rev will the patch be in?

Phil


Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
Principal Consultant, Pinnacle Solutions Group, Inc.

Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664

No virus found in this incoming message.
Checked by AVG - www.avg.com

Version: 9.0.814 / Virus Database: 271.1.1/2842 - Release Date: 04/29/10
02:27:00

[Yann Trevin]

I just committed it.

It should be in the build v3.2.424.

The error was due to a recent change I made. Basically it exposes a way to register custom equality/comparison points for MbUnit (and possibly other test frameworks). The implementation did require to move the comparison semantics in the plugin system; which was the root of all evil :)

Regards,

Yann

2010/4/29 Philip Japikse <phil.j...@pinnsg.com>

Awesome!  What rev will the patch be in?

Phil


Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
Principal Consultant, Pinnacle Solutions Group, Inc.

Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664

https://mvp.support.microsoft.com/profile/Japikse 
www.skimedic.com/blog


-----Original Message-----
From: galli...@googlegroups.com [mailto:galli...@googlegroups.com] On

Behalf Of Yann Trevin
Sent: Thursday, April 29, 2010 2:45 PM
To: gallio-dev

No virus found in this incoming message.
Checked by AVG - www.avg.com

Version: 9.0.814 / Virus Database: 271.1.1/2842 - Release Date: 04/29/10
02:27:00

--
You received this message because you are subscribed to the Google Groups "gallio-dev" group.
To post to this group, send email to galli...@googlegroups.com.
To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gallio-dev?hl=en.

[Yann Trevin]

I mean the move, not the plugin system itself :)

On Apr 29, 8:56 pm, Yann Trevin <yann.tre...@gmail.com> wrote:
> I just committed it.
> It should be in the build v3.2.424.
>
> The error was due to a recent change I made. Basically it exposes a way to
> register custom equality/comparison points for MbUnit (and possibly other
> test frameworks). The implementation did require to move the comparison
> semantics in the plugin system; which was the root of all evil :)
>
> Regards,
> Yann
>

> 2010/4/29 Philip Japikse <phil.japi...@pinnsg.com>

>
>
>
> > Awesome!  What rev will the patch be in?
>
> > Phil
>
> > Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
> > Principal Consultant, Pinnacle Solutions Group, Inc.

[Jeff Brown]

Ohh, the plug-in system is a little bit evil...  Maybe not as evil as MEF, or maybe more, depending on how you look at it.  :-)

(just kidding folks!)

Jeff.

[Philip Japikse]

Did the build fail?  I don't see it...holding my breath...trying...to...be....patient....

 

;-)

 

Phil

 

Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP Principal Consultant, Pinnacle Solutions Group, Inc. Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664 https://mvp.support.microsoft.com/profile/Japikse  www.skimedic.com/blog

[Yann Trevin]

Well. I don't know. It seems that it is due to a failing test (http://ccnet.gallio.org/server/local/project/MbUnit%20v3.2/build/log20100430043306.xml/GallioTestReportCondensed.aspx)

Jeff, it's like "cmd.exe" cannot be found in the system path of the VM! It's very odd. Do I miss something as obvious as an elephant in a corridor? Or is it due to some corruption in the VM?

Yann.

2010/4/30 Philip Japikse <phil.j...@pinnsg.com>

Did the build fail?  I don't see it...holding my breath...trying...to...be....patient....

 

;-)

 

Phil

 

Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP

Principal Consultant, Pinnacle Solutions Group, Inc. Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664 https://mvp.support.microsoft.com/profile/Japikse  www.skimedic.com/blog

 

 

From: galli...@googlegroups.com [mailto:galli...@googlegroups.com] On Behalf Of Yann Trevin
Sent: Thursday, April 29, 2010 2:57 PM

To: galli...@googlegroups.com

[Jeff Brown]

With my luck lately it is probably corrupt again or something. *sigh*

Makes me wonder if I should have kept the virtual private server.

Jeff

On Apr 30, 2010 1:41 PM, "Yann Trevin" <yann....@gmail.com> wrote:
> Well. I don't know. It seems that it is due to a failing test (
> http://ccnet.gallio.org/server/local/project/MbUnit%20v3.2/build/log20100430043306.xml/GallioTestReportCondensed.aspx
> )
>
> Jeff, it's like "cmd.exe" cannot be found in the system path of the VM! It's
> very odd. Do I miss something as obvious as an elephant in a corridor? Or is
> it due to some corruption in the VM?
>
> Yann.
>
>
> 2010/4/30 Philip Japikse <phil.j...@pinnsg.com>
>
>> Did the build fail? I don't see it...holding my
>> breath...trying...to...be....patient....
>>
>>
>>
>> ;-)
>>
>>
>>

>> *Phil***
>>
>>
>>
>> [image: cid:image0...@01CA1042.AC744F30]

>>
>> Philip Japikse, MVP, MCSD, MCDBA, CSM, CSP
>>
>> Principal Consultant, Pinnacle Solutions Group, Inc.
>>
>> Phil.j...@pinnsg.com | www.twitter.com/skimedic | (513) 312-5664
>>
>> https://mvp.support.microsoft.com/profile/Japikse
>>
>> www.skimedic.com/blog
>>
>>
>>
>>
>>

>> *From:* galli...@googlegroups.com [mailto:galli...@googlegroups.com] *On
>> Behalf Of *Yann Trevin
>> *Sent:* Thursday, April 29, 2010 2:57 PM
>>
>> *To:* galli...@googlegroups.com
>> *Subject:* Re: MbUnit, .NET 4, R# 5, MSpec, TD.Net

>> > > Phil.japi...@pinnsg.com |www.twitter.com/skimedic|<http://www.twitter.com/skimedic%7C>(513) 312-5664

>> gallio-dev+...@googlegroups.com<gallio-dev%2Bunsu...@googlegroups.com>

>> .
>> For more options, visit this group at
>> http://groups.google.com/group/gallio-dev?hl=en.
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com
>>
>> Version: 9.0.814 / Virus Database: 271.1.1/2842 - Release Date: 04/29/10
>> 02:27:00
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "gallio-dev" group.
>> To post to this group, send email to galli...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> gallio-dev+...@googlegroups.com<gallio-dev%2Bunsu...@googlegroups.com>

>> .
>> For more options, visit this group at
>> http://groups.google.com/group/gallio-dev?hl=en.
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "gallio-dev" group.
>> To post to this group, send email to galli...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> gallio-dev+...@googlegroups.com<gallio-dev%2Bunsu...@googlegroups.com>

>> .
>> For more options, visit this group at
>> http://groups.google.com/group/gallio-dev?hl=en.
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com
>> Version: 9.0.814 / Virus Database: 271.1.1/2842 - Release Date: 04/29/10
>> 02:27:00
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "gallio-dev" group.
>> To post to this group, send email to galli...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> gallio-dev+...@googlegroups.com<gallio-dev%2Bunsu...@googlegroups.com>

>> .
>> For more options, visit this group at
>> http://groups.google.com/group/gallio-dev?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups "gallio-dev" group.
> To post to this group, send email to galli...@googlegroups.com.
> To unsubscribe from this group, send email to gallio-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/gallio-dev?hl=en.
>

--