Ability to run unit tests

[Nathen Drees]

I'm kicking the tires with this plugin, and so far things have gone well. I noticed that currently there doesn't appear to be a way to run the unit tests from within the IDE, is that on the roadmap for a future feature?

Thanks for the hard work so far! Still poking around the system but so far things look promising.

[Scott]

Hi, Nathen.  I'm glad to hear your experience so far has been positive!  As for unit test integration, that's absolutely on the roadmap.  I don't have a concrete schedule, but I've posted my thoughts on what comes next here:

http://www.illuminatedcloud.com/about

And I've opened enhancement issues in the bug tracker for each of these to start organizing a feature backlog:

https://bitbucket.org/RoseSilverSoftware/illuminatedcloud/issues

Once I knock out some of the bugs that other early adopters are seeing (look for a patch release that addresses some with deployment error integration and a few other items later today or tomorrow), I plan to partition these enhancements into subsequent release buckets.  I've been thinking about one or more of the following as the highest priority:

* Anonymous Apex execution with the full editor features

* SQL query execution with the full editor features

* Unit test execution, results review, and code coverage overlay

* Visualforce completion/navigation for controller class/fields/methods and global variables/expression functions

I'd definitely love to hear feedback from you and other users about prioritization of additional features!

Thanks for checking in and please let me know your thoughts, positive or otherwise, about the plugin!

[Nathen Drees]

Scott -

I think for us our top priority would be the unit test coverage, results and code coverage overlay. So far it looks like the plugin does everything else we're used to in the Force.com IDE or the parts of Mavens Mate we use regularly. The other parts would be nice, but we're used to no IDE doing them well so we're not really regressing there if we don't have them. However, Mavens does do unit testing well so that would be a definite pain point for us.

[Scott]

That's very useful feedback, Nathen.  Thanks!  I agree that being able to execute unit tests easily from the IDE, review results with simple navigation through stack traces in failing tests, and a code coverage overlay in the editor itself would round out the initial feature set for end-to-end work nicely.

Honestly the high prioritization of anonymous Apex and SOQL query execution is because I've become so accustomed to Illuminated Cloud's editor that it's annoying to have to type Apex or SOQL without it, e.g., in the Developer Console or Workbench.  I've heard similar from some of my beta testers over the past few months, but there's definitely a self-serving component to that priority call!

Keep sending feedback, both here and in the bug tracker!

[Scott Wells]

Just a quick update.  I'm working on unit test and code coverage integration for the 1.1 release of the plugin and am making solid progress.  As of the other night you can create and edit run configurations where you select the unit test classes to include in a particular run, and as of about ten minutes ago you can actually execute those configurations in IDEA's standard unit test window and watch the results come in, appropriately marked as pass or fail and, in the case of a failure, displaying the corresponding failure message and Apex stack trace.

Still plenty to do, but I'm hoping to have this out in a rock solid form in the next couple of weeks...

[Nathen Drees]

Excellent, we look forward to trying it out once you have it available for release.

[Scott Wells]

I just finished uploading version 1.1.0 of the plugin that includes integrated Apex unit testing:

http://www.illuminatedcloud.com/home/unittest

Right now it just test selection at the class level and not the method level, and there aren't yet any context menu items to create a new run configuration from the test class/method under the caret.  It also doesn't yet support code coverage.

Having said that, it does support creation of standard IDEA run configurations for Apex Unit Tests.  Each run config allows the selection of any subset of unit tests associated with a particular Force.com connection as well as whether debug logs should be generated and at what level.  When you execute tests, progress is displayed in IDEA's standard test runner.  Stack traces for failed test methods are hyperlinked (as are any Apex stack traces, e.g., those pasted into Analyze>Analyze Stacktrace...), and you can re-run failed tests with a single button click.  You can easily navigate back and forth between product classes and test classes using Navigate>Related Symbol... (CTRL+ALT+HOME) and Navigate>Test Subject (CTRL+SHIFT+T).

Let me know what you think, and of course if you have any issues with it, please let me know about those as well!

[Nathen Drees]

This is excellent! It gives us a nice way to run our unit tests without having to leave the IDE, and it seems to work pretty quickly.

Some initial feedback: when a test fails, the ! is orangish, which looks very similar to the currently running tests. Not sure if this is under your control or not, but I would prefer that its red to help it stand out.

How do I view the log files that were generated? I see the hyperlinks in the unit test window, but I don't see the full debug log anywhere?

I think that adding some instructions on how to run unit tests would be beneficial. I don't have a ton of experience with IDEA, so it took me a bit to figure out that I needed to set up a new run configuration to run the tests. Makes sense now, but I would bet there will be as many people as not who are like me and don't know where to start.

Lastly, a few convenience shortcuts would be nice at some point in the future:
1 - Right click in the current editor and have a 'run tests' command that runs all tests in the current class file.
2 - Right click in the Project window and have a 'run tests' command that runs tests for all the selected class files, or the entire classes folder if selected.

As you said, code coverage isn't there yet, and would be nice to have at some point.

Overall I think this is a huge step forward, and makes my life easier for sure.

[Scott Wells]

Thanks for the quick feedback, Nathen!

As for the orangish color, I'm just bridging IDEA's unit test framework to Force.com's Tooling API.  That's the standard presentation for tests reported as failed.  Given how configurable IDEA is, I thought perhaps there might be a way to change this from Settings, but I'm not seeing anything obvious.

As for seeing generated logs, I'm assuming you're already selecting that in the run configuration and setting log levels as appropriate.  Once that's done, it should display the logs for each executed test method in its results console (with rollup at the class/suite and run level).  However, I'm finding that even though I set a trace flag for the test classes, sometimes I don't get back an apex log ID in the results.  If I go into the Developer Console (which is precisely what I'm trying to keep anyone from having to do!) and actually observe things, the logs start generating properly and showing in Illuminated Cloud.  Needless to say, not what I'm wanting.  There must be something I'm missing.  I was going to follow up with my contacts at Salesforce today and see if they can share any thoughts on why this would be happening.

Agreed on instructions.  I plan to create another short video showing how to get this going and what all it can do.  Just didn't have time to do so over the weekend or last night.  Expect that in the next day or two.

As for the convenience shortcuts, totally agreed.  I plan to model this after IDEA's Java unit test functionality where you can right-click on a test method and run just that method, a test class and run all of its methods, or a node in the Project view and run all tests under that node.  It would create a run configuration for you based on the selection and present that first, then run the config.

And yes, once I knock out some of these small-ish things (vs. getting this going at all!), I plan to move on to code coverage.  That one might be a little tricky because of how I support IDEA 12, 13, and 14.  Evidently code coverage will only work in IDEA 12 and 13 Ultimate Edition but in IDEA 14+ Community Edition, so I'm sure there's some wonky conditional stuff I'll have to do to support my full test matrix.

Again, thanks for the quick feedback!

[Scott Wells]

Nathen, I uploaded 1.1.0b last night that hopefully resolves the issue with seeing debug logs when so configured.  Please let me know if you're not seeing proper behavior there now.

[law...@gmail.com]

Hi, do I need to do anything 'special' to get the unit tests to show up? In the run/debug configuration I just have the 'All' check-box in the test classes but no tests underneath. All my test files are in the src->classes folder with a suffix of test. ie I have AccountReader file and then AccountReaderTest file containing in the unit tests.

Also if I go to the test file, I can select the 'Test Subject' from the 'Go To' menu, but from the production file I the 'Test' menu option is greyed out.

I have the lates v15 of IntelliJ and v1.6.5.5 of Illuminated Cloud
Thanks! Paul

[Scott]

Hi, Paul.  Sorry for the delayed response.  I've had a long outage today and am just now catching up on things.  Just to clarify, you're saying that when you create a new run/debug configuration of type "Apex Unit Tests", you see the "All" checkbox but the tests in your local project aren't listed?  It's looking for classes that are annotated as @IsTest, not just with names ending in "Test".  Can you verify that your classes are properly annotated?

As for the second issue, I think I've reproduced it successfully.  I can use Navigate>Related Symbol... to navigate back and forth between test classes and corresponding tested classes, but Navigate>Test Subject is only enabled when I'm in a test class and not when I'm in a project class.  I'll look into that, but you should be able to use the Related Symbol navigation to go back and forth reliably.

Regards,

Scott

[law...@gmail.com]

Thank you for the quick response! You need to forgive me. I spent all day yesterday changing configurations, changing my src folder (as I  am using a symlink in the project to my actual folder which is linked to Git) and could not see them. I now just tried again and they appeared. Only thing I did differently was I did a File->Synchronize as I updating files.
Run them and all works.

Anyway, thanks again for getting back to me, and love it so far!

Cheers,
Paul

[Scott]

No need to apologize, Paul!  IntelliJ IDEA's caching virtual file system can sometimes produce unexpected results, particularly when you've been moving things around "outside" of its purview.  As you discovered, File>Synchronize is the way to catch things up when you do that.  Glad it's working well for you!

[eric.alex...@gmail.com]

Many times when writing test methods it would be helpful to simply run the current class without having to set up any configurations. In Mavensmate and other IDE's you can simply press a shortcut sequence to runt he current apex test. Is this feature on the roadmap? It would be nice to have all the subsequent features when running a test as well but the ability to simply run the current test would be a big plus.

[Scott]

Yes, this is definitely on the roadmap.  Right now it's captured in the following enhancement request:

https://bitbucket.org/RoseSilverSoftware/illuminatedcloud/issues/85/

That request is for both method-level execution and ad hoc/transient run configurations.