Can we do pit mutation test for Android project?

[Subrata Mandal]

following is the process we followed for pit.
1 : we tested pitest with Java project, junit for the Java projects works fine as well as coverage.

2 : while trying the same with android project we found the following issue.

- after Configuring android with PIT, we are successfully running the pit tools, but coverage for the same is not done.
- while analyzing the log I have found that, pit is not able to cover the Android junit. PIT requires a support tool to run the Android based code coverage.

- for the support tool, I have chosen "Robolectric". (ie) I am able to run a standalone android project with "Robolectric" tool.

- as per current status, i am trying to integrate the pit and robolectric, so that I can get coverage for Android project.

- we have created one android project and one Java project. From Java project I am testing junit with help of robolectric tools. As a result inside android project a mirror test class create, from Java project with same pkg.

- from robolectric I am able to do junit test for Android project, but if I am trying to do pit mutation testing, then not getting expected results.

My Eclipse piclipse version is 0.31
Eclipse : Mars 2
Support : http://stackoverflow.com/questions/20241134/android-how-to-run-pit-mutation-testing-with-robolectric

From this link, only Sulai and Henry told they are successful with android sample project. I humbly request to both of you, please provide some doc or tutorial, how to set up for Android.

From this link, Henry confirmed, 0.32-snapeshot will work with robolectric, but I am unable to get Eclipse plug-in of 0.32-snapeshot, only jar files are there, but how to use this jar. Please help me.

[henry]

I'm not entirely clear on what the issue you are experiencing is.

Perhaps you could create an example project on github that reproduces the issue?

[Subrata Mandal]

Failure coverage report of android project through PIT Mutation testing

To get this REPORT WE FOLLOWED 2 approaches, these are below

Setup Eclipse with Pitclipse:

- opern Eclipse

- Click on Help

- Click on Eclipse Markectplace

- Type "PIT" and search

- Install pitclipse..031.4.201401081810

- Restart Eclipse

A: With ActivityInstrumentation test

CASE 1: creating Android Application Project (CoverageNew.prj)

- open Eclipse

- Create a new Android Application project with package name "cpm.example.coveragenew".

and class MainActivity.java

- MainActivity.java extends Activity Class, Now

- There is only one function added, named Addition(int , int)

CASE 2: createing Android test project(CoverageNewTest.prj)

- File --> New --> Other --> Android --> Android Test Project--> Next

- Give project name CoverageNewTest.prj

- press Next

- Select existing project(CoverageNew.prj) --> Next -->Finish

- Create one class of name MainActivityTest inside the package (com.example.coveragenew.test)

- MainActivityTest extends ActivityInstrumentationTestCase2<>

- Do setup() 

- call the Addition()through mActivity=getActivity(), mActivity.Addition(2,3)

CASE 3: "run as" Android Junit Test

- Add <Use Library /> and the permissin for ActivityInstrumentationTestcase2<>, in Main project Manifest file

<Uses-library android:name="android.test.runner" />

<instrumentation android:name="android.test.instrumentation.testrunner"

android:targetPackage="com.example.coveragenewtest" />

- Right click on test project

- Select "run as" Android junit test

- Junit run successfully

CASE 4: "run as" "Pit Mutation Test", and not able to run

- Now, right click on testproject

- Select "run as" Pit Mutation Test.

- Errors came. not able to run (Invalid layout of java.lang.String at value Error occurred during initialization of VM

Invalid layout of preloaded class: use -XX:+TraceClassLoading to see the origin of the problem class)

CASE 5: able to run as Pit Mutation test, solve the error by setting the classpath, but , not expected coverage report

- Right click on test project 

- Select "run as" , and click run configuration 

- Select Classpath, inside thisclasspath, delete Android 5.1.1 from Bootstrap, now, click Advance-->Add library--> JRE System Library, and 

again add,

Advance-->Add Library-->Junit 4--> apply and run

[henry]

As I noted in response to your question on stack overflow, pitest requires unit tests where the tests can code both run in a  normal JVM.

http://stackoverflow.com/questions/35916322/android-class-not-covered-even-after-successfully-pit-mutation-test-need-sugges#comment59542199_35916322

In practice this means that there are two situations in which pitest can be used in an Android project

1. To test pure Java code within the project with a normal unit test
2. To test code with dependencies on android using a technology such as Roboelectric that runs things in a normal JVM

The code and test you have posted cannot work with pitest as the logic you are trying to test is within a class tied to Android and the test case is a standard android one.

To mutation test you need to either design your code so that the logic you wish to test is not tied to Android, or use Roboelectric to isolate it.

Henry

[Subrata Mandal]

Hi Henry,

 sub: failed with robolectric also.

Thank you for your suggation. As you suggested me to work with Robolectric to get Pit Mutation Coverage for Android Project, I have set up with Robolectric and Pit with Android Project, but the output result is same. not getting expected coverage report.

In both way 1: ActivityInstrumentation and 2: with Robolectric getting same coverage report.

Please let me know whether I am doing wrong in a wrong way or not.

I have followed the below youtube link to setup robolectric,

1 : "https://www.youtube.com/watch?v=3mjcg5gfDLE"

2 : "https://www.youtube.com/watch?v=Y3nbdxDZyMY"

Steps followed to set up with robolectric are below:

Step 1: Creating Android Project

- Create an android Project in Eclipse

- Select project and --> right click-->click one folder named "test"

- Select "libs" folder --> right click --> create one folder inside libs folder, named "test'

- copy all into libs/test folder downloaded jar files

Steps 2: Create Java Project and link with above Android project through robolectric

- Eclipse-->file-->new-->java project--> give name--next

- Source menu

- Select android project

- From below -->click link--> select "test" folder from Android projet

- Project menu

- Select Android project

- Libraries menu

- Add jars--> select all jars from Android-Project/libs/test

- Add External jar--> android jar(sdk/platforms/android.jar)

- Order & Export menu

- arrange all jars by order

Steps 3: Create test class into java project with RobolectricTestRunner.class

- right click on "test"--> new class--> give package name and classname

- Writhe test case

Steps 4: Configure the "Run Configuration"

- Select java project-->'Run' -->Run configuration--> set junit-4-->in argument menu--select 'other"--> give android project path

- Change manifest file of android project, like below

 <uses-sdk

        android:minSdkVersion="17"

        android:targetSdkVersion="17" />

Steps 5: Select Java project and run as junit(eclipse)

- Successfully run

Steps 6: Now configure with Pit Mutation test

- Set up the Run Configuration by adding "add library"

- Junit-4 and

- JRE System

Steps 7: Run As Pit Mutation Test

- run as Pit Mutation Test

- Not getting the expected report.

Please find the attachment 

Thanks & Regards

Subrata Mandal

[Subrata Mandal]

Hi Henry 

Please let me know whether I am in wrong way or not.

Regards

Subrata

[Henry Coles]

Hi Subrata,

Its not instantly clear what the issue is. I won't be able to look at this for the next two weeks, but I'll try and recreate the issue when I'm able.

--
You received this message because you are subscribed to the Google Groups "PIT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pitusers+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Subrata Mandal]

Thank you Henry, I have been trying to get solution from last few weeks, but I am stuck. If I get any help from your side , it will be very pleasure and helpful for me.

[Subrata Mandal]

Hi Henry,

I am stuck here. can you please help to get solution. I spent lot time, but still unable to find out, the reason, why PIT is not working with robolectric also. Even I am not getting any error, only coverage report I am getting with no line and mutation coverage. At least if I get the cause, I can work to fix this issue. please help me to get solution.

Regards

Subrata

[henry]

Hi Subrata,

It is difficult to debug what your problem is based on textual descriptions, screenshots and individual files.

As I suggested earlier in this thread, it would be easiest to create a project on github that recreates the issue.

I'm not an android developer myself, but I've just done a little research in what's involved in setting up a project and created an example of using pitest with roboelectric. It is available here

https://github.com/hcoles/pitest-roboeletric-sample

It uses maven as a build tool as it appears it is not currently possible to use the pitest gradle plugin for android projects due to this issue.

https://github.com/szpak/gradle-pitest-plugin/issues/31

It is based on this sample from the roboelectric team

https://github.com/robolectric/deckard

I suggest you create a fork of this project on github and update it so that it recreates the issue you are experiencing.

Regards

Henry

[Subrata Mandal]

Thank you for his information. I'll create and update my issue

Regards

Subrata

--
You received this message because you are subscribed to a topic in the Google Groups "PIT Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pitusers/GHgonTYti-w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pitusers+u...@googlegroups.com.