Error: "Cannot find a variable with the name 'TestDrive'"

[Jamie Thomson]

Hi folks,

Day 1 of using pester and I've had a successful day so far (already got it integrated into our CI build) but I've encountered problem.

I'm using Pester within a build process in which we're using psake. That psake build is being called from a TeamCity CI build configuration.

My tests are passing successfully:

Describing PowershellMajorVersion

 [+] is not v1 333ms

 [+] is not v2 61ms

 [+] is not v3 4ms

Tests completed in 400ms

Passed: 3 Failed: 0

and the psake build succeeds however I get some errors "pop" out of the bottom after the psake build is complete, and these errors cause my CI build to fail. the errors are:

Cannot find a variable with the name 'TestDrive'. Cannot find drive. A drive with the name 'TestDrive' does not exist.

I know from another topic on this group that this error has got something to do with Pester however even after reading that topic I'm flummoxed as to what is going on here. 

I am using Pester 3.0. Ca anyone perhaps advise as to what the problem might be?

TIA

Jamie

P.S. Not that I suspect its significant but in case it is, here's the definition of my tests:

$here = Split-Path -Parent $MyInvocation.MyCommand.Path
$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path).Replace(".Tests.", ".")
. "$here\$sut"


Describe "PowershellMajorVersion" {
    It "is not v1" {
        $PSVersionTable.PSVersion.Major | Should Not Be 1
    }
    It "is not v2" {
        $PSVersionTable.PSVersion.Major | Should Not Be 2
    }
    It "is not v3" {
        $PSVersionTable.PSVersion.Major | Should Not Be 3
    }
}

As you can see they're very simple. These are just "canary" tests to get the whole end-to-end working before I start writing my real tests.

[Dave Wyatt]

Well, that's annoying.  Those are non-terminating errors that were suppressed from showing up at the console via -ErrorAction SilentlyContinue, but they still show up in the error stream, which I assume is what your CI process is reading.  We could use -ErrorAction Ignore, except that only exists in PowerShell 3.0 and later.

 

I can't upload this to a new branch until this evening, but you can try replacing the New-TestDrive function (found at the beginning of the Functions\TestDrive.ps1 in the Pester module) with this code instead.  It's less efficient, but avoids polluting the error stream with garbage:

 

function New-TestDrive ([Switch]$PassThru) {
    $Path = New-RandomTempDirectory
    $DriveName = "TestDrive"

 

    if (-not (Microsoft.PowerShell.Management\Test-Path -Path $Path))
    {
        New-Item -ItemType Container -Path $Path | Out-Null
    }

 

    #setup the test drive
    $drive = Get-PSDrive | Where-Object { $_.Name -eq $DriveName }
    if (-not $drive)
    {
        New-PSDrive -Name $DriveName -PSProvider FileSystem -Root $Path -Scope Global -Description "Pester test drive" | Out-Null
    }

 

    #publish the global TestDrive variable used in few places within the module
    $variable = Get-Variable -Scope Global | Where-Object { $_.Name -eq $DriveName }
    if (-not $variable)
    {
        New-Variable -Name $DriveName -Scope Global -Value $Path
    }

 

    if ( $PassThru ) { Get-PSDrive -Name $DriveName }
}

[Dave Wyatt]

Oh, yuck.  When I run Pester's internal suite of tests, even though they all pass, the $Error variable goes from zero to 81 records.  These are all errors that were handled and suppressed either by Pester or by the test code.  This is just an annoying bit of PowerShell behavior; even when you handle errors with a try/catch or trap construct, or when they're suppressed with -ErrorAction SilentlyContinue, they still wind up in the $Error variable.  (They actually go there before the Catch and Trap blocks are even evaluated.)

I could fix the TestDrive code, but that would just be sticking a band-aid on the errors you know about so far.  Even if I managed to make Pester's own tests completely clean, your own code under test could add to the $Error variable for the same reasons.

Based on what I'm seeing here, it would be better if the CI code didn't look at $Error at all, but instead just focused on the actual Pester output (either using its -PassThru parameter, or -OutputXml, and checking for any failed tests.)

[Dave Wyatt]

BTW, that's 81 errors in Pester after I changed the TestDrive and "Should Throw" code to get rid of some of them.  Using the current master branch, it's 255.  :\

[Jamie Thomson]

Hi Dave,

Sounds as though changing CI is the way to go.

Can I clarify, you're alluding that the underlying cause is simply that this is the way that Powershell works, correct? In that case, I shouldn't expect that this will ever be "fixed" in pester ("fixed is the wrong word, but I use it in the loosest sense), correct?

I must say, I'm surprised no-one else has stumbled across this. I can't be the only person using Pester within CI.

thanks

Jamie

[Jamie Thomson]

Extra info. As i said earlier, our build process is built in psake but on examining the code I don't think that's significant. The only significant thing is that the build process is built in Powershell, the fact that its using psake within there is irrelevant.

I'll clarify. Our build process is kicked off using a .bat file, here's the significant portion of code:

:BUILD
echo Starting build
powershell.exe -NoProfile -command ".\Start-Build.ps1 -task %TASK% -configMode %CONFIGURATION% -buildCounter %BUILDCOUNTER% -updateNuGetPackages %UPDATENUGETPACKAGES% -webdeploypackage %WEBDEPLOYPACKAGE%;exit $LASTEXITCODE"
echo. %ERRORLEVEL%

if %ERRORLEVEL% == 0 goto OK
echo ##teamcity[buildStatus status='FAILURE' text='{build.status.text} in execution']
exit /b %ERRORLEVEL%

When I call that .bat file here is the output:

Cannot find a variable with the name 'TestDrive'. Cannot find drive. A drive with the name 'TestDrive' does not exist.

 2
##teamcity[buildStatus status='FAILURE' text='{build.status.text} in execution']

Every test that I include/exclude causes the ERRORLEVEL (shown in output above as 2) to increment/decrement by 1. In other words, the ERRORLEVEL returned from calling powershell.exe appears to be the number of errors that occurred.

I suspect this isn't news to you Dave as its simply confirming what you had already assumed, but I thought it might be useful for you to read this anyway.

Regards

Jamie

[Jamie Thomson]

[Last bit of feedback today (hopefully :) )]

Hello again Dave,

I replaced the definition of function New-TestDrive with that which you provided below and am happy to report that it eliminated the errors relating to 'TestDrive'. Hence, if I use this new version of New-TestDrive and I don't use any "Should Throw" assertions then I can Pester in my CI build as intended.

Not being able to use "Should Throw" is annoying though -  that is a very very useful assertion. 

Regards

JT

[Dave Wyatt]

Can I see the Start-Build.ps1 file that this batch file is calling?  If you'd like, you can email it directly to me at dlwya...@gmail.com .

[Jamie Thomson]

Sent via email

[Dave Wyatt]

I haven't received it.  PS1 attachments may be blocked by some mail servers, though; try renaming it to .txt.

[Jamie Thomson]

Hmmm, I renamed it to ."ps_". let me try again.

[Dave Wyatt]

Here we go.  Just after your call to Invoke-psake, you have this block:

if ($error -ne '')
{

    Write-Host "$error"

    exit $error.Count

}

Even if Pester reports that the tests were all passed and psake reports that the build was successful, as we've discussed, there will still be some ErrorRecord objects in the $error variable which have been suppressed as part of normal operation.

Instead of checking for $error, I'd recommend that you just look at the $psake.build_success property to determine whether to abort the CI process or not.  You can still output the contents of $error if you like (though some of those contents might be misleading):

if (-not $psake.build_success)
{

    Write-Host "$error"

    exit $error.Count

[Dave Wyatt]

Here's a demonstration of the annoying try/catch behavior I was describing:

$error.Clear()

try
{
    throw 'Oops, something went wrong!'
}
catch
{
    Write-Host -ForegroundColor Green 'No problem, I got this handled.'
}

$error

Because of this, I don't think it's likely that we can fully police the $error variable during test script execution.  There may be another option, however.  If Pester reports that all tests were successful, we could just include a step at the end which clears out any errors that were added since the beginning of the call to Invoke-Pester.  That would even cover errors that originated from the code under test instead of from within Pester.

[Jamie Thomson]

DOH! never even bothered to look inside StartBuild.ps1 (assumed it was fairly dumb). Thanks for making me feel dumb in public Dave :)

That script is centrally maintained so I need to go and persuade someone else that it needs changing. Off to work i go.

Thank you very very much Dave. Very much appreciated.

[Dave Wyatt]

I make myself feel dumb in public all the time.  You get used to it after a while.  :)

[Jamie Thomson]

Sounds like a good option to me. I'll let you smart folks figure out if that's the correct thing to do or not. Me, I'm just grateful for all your help thus far. Thanks again.