Re: [specflow] UI Automation using Remote PC and SpecFlow
Jonas Bandi
Basically SpecFlow is only responsible for the "literal automation" of
your test steps.
If you have a programmatic api for driving you UI on the remote PC,
you can implement your steps with that.
For instance for driving web-apps there are also different
possibilities to drive your application: ie. through selenium or
through watri etc...
However I don't know what remoting api you would use in your case.
Why can't you deploy the specflow-tests and the ui on the same machine
for testing?
Regards
jonas
On Thu, Jan 13, 2011 at 10:22 AM, psn <knka...@gmail.com> wrote:
> I tried SpecFlow and was successful in automating our UI application
> which is in WPF. Here I installed the application also in the same PC
> where SpecFlow Test environment was setup and got the UI Automation to
> work.
>
> Buf if the UI application is in a remote PC and SpecFlow Test System
> is in the Test PC, is there any way that I can automate the UI
> application remotely. Is there any way to automate the UI application
> and send the UI event clicks using some server/client configuration.
>
> Thanks,
> psn
>
--
mail: jonas...@gmail.com
web: www.jonasbandi.net
blog: blog.jonasbandi.net
twitter: twitter.com/jbandi
Jonas Bandi
If you are automating a winforms/wpf application with White, then I
would argue that the you should run the steps on the same PC as as the
application, since the steps are driving White ...
If this is a question about running the SpecFlow scenarios on a build
server, without VisualStudio, then of course you can and should do
this. There is a MSBuild task for SpecFlow that you can use to
generate the code behind.
Since the scenarios are pure NUnit/MSTests at runtime you can automate
them on your build-server as any other tests with the given framework.
But this has nothing to do with running steps and app on defferent PCs...
We think it is a reduction of accidental complexity that you dont need
any server and communication (wire protocol) compared to Cuke4Nuke.
On Fri, Jan 14, 2011 at 7:13 AM, psn <knka...@gmail.com> wrote:
> Initially we tried to do UI automation using Cuke4Nuke and white
> server.
> Here the Feature Files are written in Ruby and the StepDefinitions are
> written in C# on VS environment.Cucumber.wire was used to communicate
> to the Target Instrument using Wireprotocol over the network.Hence the
> application was present in the instrument while the Test System was
> present in the PC. Specific dll's of the compiled Step Definition
> files were loaded in the Target instrument to communicate with the
> test PC.
>
> After coming across SpecFlow where in the Feature Files and Step
> Definitions are in VS environment we are in the process of discussions
> to finalize as to whether we need to migrate to VS environment.Our
> intention is have something similar to Cuke4Nuke server and retain the
> Test environment in the PC and perform UI Automation.
>
> Hence I am looking out for similar support in SpecFlow
> environment.Please let me know if there is any method which I can
> explore in SpecFlow Setup
>
> Thanks,
> psn
>
> On Jan 13, 5:39 pm, Jonas Bandi <jonas.ba...@gmail.com> wrote:
>> This is not a typical usage scenario.
>> Basically SpecFlow is only responsible for the "literal automation" of
>> your test steps.
>> If you have a programmatic api for driving you UI on the remote PC,
>> you can implement your steps with that.
>>
>> For instance for driving web-apps there are also different
>> possibilities to drive your application: ie. through selenium or
>> through watri etc...
>>
>> However I don't know what remoting api you would use in your case.
>>
>> Why can't you deploy the specflow-tests and the ui on the same machine
>> for testing?
>>
>> Regards
>> jonas
>>
>> On Thu, Jan 13, 2011 at 10:22 AM, psn <knkavi...@gmail.com> wrote:
>> > I tried SpecFlow and was successful in automating our UI application
>> > which is in WPF. Here I installed the application also in the same PC
>> > where SpecFlow Test environment was setup and got the UI Automation to
>> > work.
>>
>> > Buf if the UI application is in a remote PC and SpecFlow Test System
>> > is in the Test PC, is there any way that I can automate the UI
>> > application remotely. Is there any way to automate the UI application
>> > and send the UI event clicks using some server/client configuration.
>>
>> > Thanks,
>> > psn
>>
>> --
>> mail: jonas.ba...@gmail.com
psn
that contains the app.I will
use MSBuild to run SpecFlow scenarios on a build without VS
environment.
PSN
>
> - Show quoted text -
PerryHunter
work with "psn").
We currently use the Ruby/Cuke4Nuke environment to drive our
application on a remote "PC". This is necessary, because the "PC" is
used as a front end on other, integrated instrumentation, and we need
to keep it as close to the as-shipped configuration as possible. The
wire protocol support in the Cuke4Nuke server gives us this, but we
really want to move to an all-C# implementation for the test
framework, because it makes doing BDD that much easier if my
developers are not having to context-switch back and forth between
Ruby and C#.
This sort of need is common among those of us who test applications
that run on PC platforms, but which are not used as PC's in the
traditional sense. Examples include various types of electronic and
life sciences instrumentation, avionics systems, telecom applications
(like emergency call centers) etc.
Perry
On Jan 13, 4:39 am, Jonas Bandi <jonas.ba...@gmail.com> wrote:
> This is not a typical usage scenario.
> Basically SpecFlow is only responsible for the "literal automation" of
> your test steps.
> If you have a programmatic api for driving you UI on the remote PC,
> you can implement your steps with that.
>
> For instance for driving web-apps there are also different
> possibilities to drive your application: ie. through selenium or
> through watri etc...
>
> However I don't know what remoting api you would use in your case.
>
> Why can't you deploy the specflow-tests and the ui on the same machine
> for testing?
>
> Regards
> jonas
>
> > I tried SpecFlow and was successful in automating our UI application
> > which is in WPF. Here I installed the application also in the same PC
> > where SpecFlow Test environment was setup and got the UI Automation to
> > work.
>
> > Buf if the UI application is in a remote PC and SpecFlow Test System
> > is in the Test PC, is there any way that I can automate the UI
> > application remotely. Is there any way to automate the UI application
> > and send the UI event clicks using some server/client configuration.
>
> > Thanks,
> > psn
>
> --
Jonas Bandi
Or am I missing something?
When you have to install Cucumber/Ruby, you could as well install .NET
and MSTest... would that not be less painful?
--
mail: jonas...@gmail.com
PerryHunter
Cuke4Nuke.Server.exe -a steps\AwgGuiSteps.dll
No Ruby, Cucumber or anything else gets installed to the remote PC. Literally, I can plug in a USB stick with the needed files on it to the test platform and it's ready to go.
Perry
Jonas Bandi
Still, you have to deploy the steps and white to the remote PC ... so
you could also deploy mstest (or Nunit) along with it and run the
specflow tests (which could be generatd and compiled on another
machine), could you not?
--
PerryHunter
don't need to change the system by installing anything, or change the
operating environment in any way. The only impact on the system is the
cuke4nuke.server.exe and the comparatively tiny amount of wire
traffic.
Regards,
Perry
"PSN" is looking into deploying MSTest or NUnit - we'll let you know
how this turns out.
On Jan 17, 4:08 pm, Jonas Bandi <jonas.ba...@gmail.com> wrote:
> Ok, I got it the wrong way with Cuke4Nuke calling the steps on the remote PC.
>
> Still, you have to deploy the steps and white to the remote PC ... so
> you could also deploy mstest (or Nunit) along with it and run the
> specflow tests (which could be generatd and compiled on another
> machine), could you not?
>
> > We only have to copy the White server support files and the compiled DLL of
> > UI Step definitions, then run the Cuke4Nuke server on the remote PC like
> > this:
>
> > Cuke4Nuke.Server.exe -a steps\AwgGuiSteps.dll
>
> > No Ruby, Cucumber or anything else gets installed to the remote PC.
> > Literally, I can plug in a USB stick with the needed files on it to the test
> > platform and it's ready to go.
>
> > Perry
>
> --
psn
I was just browsing to see how we can run the scripts using MStest and
without Visual Studio IDE installed. I found this info in one of the
blogs of MSDN
http://blogs.msdn.com/b/anutthara/archive/2009/12/16/running-tests-in-mstest-without-installing-the-vs-ide.aspx
Here they say,
Yes, you CAN run your unit tests or any other VS test type on a
machine without having Visual Studio IDE installed on that machine.
You need the Visual Studio Team Agent to be installed on your test
machine to run your tests. This is a 15 min lightweight install and is
pretty easy to configure.
Those of you familiar with remote testing from our previous versions,
will already know that you can trigger a test run on the client to run
tests on a remote machine with the test agent installed. Just use the
test settings to point to a test agent where your tests can run
remotely. With the VS Team Agent installed on machine A and the client
(VS Ultimate or Pro or Test Elements) installed on machine B, you can
run tests remotely on A by triggering the run from client B.
Here they say that we need to install Visual Studio Team Agent. Please
let me know Is Visual Studio Team Agent the only way that we can run
SpecFlow tests using MStest or is there any other method.
Thanks,
Kavitha
Jonas Bandi
Frameworks. MSTest is the default, but you can also use NUnit.
With NUnit you can run the SpecFlow Tests on any machine, you only
have to deploy NUnit itself (standalone dlls and exe), the test-dlls
and the application under test (of course).
You then can run the SpecFlow tests with NUnit via simple execution of
the exe (or any automation of that, ie. via build-server)
I actually expect this should be equally simple with MSTest. MSTest
also gives you the MSTest.exe. I expect that to work standalone too,
but I never actually checked.
I never heard of the Visual Studio Team Agent. But judging from the
name I think this is only needed if you want to run the remotely
deployed tests from your local Visual Studio... which I think is not
your initial scenario, or is it?
--
mail: jonas...@gmail.com
psn
inbuilt in VS environment. If we have to use NUnit then it becomes yet
another installer.
I am exploring ways to find out how MStest can be used in remote
machine.
By googling I found out that if we have to use MStest.exe(standalone)
then some configurations/settings needs to be done in the Test
machine.
If we use MStest in combination with VS team agent then the various
configuration/settings can be avoided is the information I gathered.
Hence I am trying to find out how we can achieve this. If MStest and
VS team agent work in combination, then executing the scripts in the
remote machine from the local machine will become easy and this will
also help in debugging.
At this point in time I am exploring the various ways we can execute
tests in the remote machine.
Thanks,
PSN
On Jan 26, 2:55 pm, Jonas Bandi <jonas.ba...@gmail.com> wrote:
> As you might know, SpecFlow supports different Unit-Testing
> Frameworks. MSTest is the default, but you can also use NUnit.
>
> With NUnit you can run the SpecFlow Tests on any machine, you only
> have to deploy NUnit itself (standalone dlls and exe), the test-dlls
> and the application under test (of course).
> You then can run the SpecFlow tests with NUnit via simple execution of
> the exe (or any automation of that, ie. via build-server)
>
> I actually expect this should be equally simple with MSTest. MSTest
> also gives you the MSTest.exe. I expect that to work standalone too,
> but I never actually checked.
>
> I never heard of the Visual Studio Team Agent. But judging from the
> name I think this is only needed if you want to run the remotely
> deployed tests from your local Visual Studio... which I think is not
> your initial scenario, or is it?
>
>
>
>
>
> On Mon, Jan 24, 2011 at 4:56 PM,psn<knkavi...@gmail.com> wrote:
> > Hi,
>
> > I was just browsing to see how we can run the scripts using MStest and
> > without Visual Studio IDE installed. I found this info in one of the
> > blogs of MSDN
>
psn
Controller we are able to run tests on the Remote PC.
From the command prompt also MStest works fine with Visual Studio team
Agent and Controller.
Can anyone please let me know if there is way to generate the
feature.cs file automatically when I place the feature file in the
specified folder i.e without using the VS IDE. My intention is to
create a .feature file place it in the desired folder and compile it
from command prompt using MSTest without using VSIDE. The only problem
I see here is the generation of feature.cs file. Which file should be
modified so that the once the feature file is present and when
Specflow is used to compile the feature.cs file gets generated.
Thanks,
psn
Jonas Bandi
feature files.
Like that:
specflow.exe generateall BookShop.AcceptanceTests.csproj
However you need the whole project for this to work, not only
stand-alone feature files.
There is also a MSBuild Task in the specflow distribution that lets
you automate this task.
--
mail: jonas...@gmail.com
psn
working fine. For the feature file I had, I could get the
corresponding .feature.cs file. This will help us a lot in our
Automation. This satisfies our requirement to generate
the .feature.cs file from the .csproj file and from command prompt.
I am yet to try with MSBuild Task. I will try even that and update the
results here.
Regards
psn
On Feb 16, 4:09 pm, Jonas Bandi <jonas.ba...@gmail.com> wrote:
> You can use specflow.exe to generate the code-behind files from the
> feature files.
> Like that:
>
> specflow.exe generateall BookShop.AcceptanceTests.csproj
>
> However you need the whole project for this to work, not only
> stand-alone feature files.
>
> There is also a MSBuild Task in the specflow distribution that lets
> you automate this task.
>
>
>
>
>
> On Wed, Feb 16, 2011 at 11:52 AM,psn<knkavi...@gmail.com> wrote:
> > With SpecFlow and using MStest and Visual Studio team Agent and
> > Controller we are able to run tests on the Remote PC.
> > From the command prompt also MStest works fine with Visual Studio team
> > Agent and Controller.
>
> > Can anyone please let me know if there is way to generate the
> > feature.cs file automatically when I place the feature file in the
> > specified folder i.e without using the VS IDE. My intention is to
> > create a .feature file place it in the desired folder and compile it
> > from command prompt using MSTest without using VSIDE. The only problem
> > I see here is the generation of feature.cs file. Which file should be
> > modified so that the once the feature file is present and when
> > Specflow is used to compile the feature.cs file gets generated.
>
> > Thanks,
> >psn
>