Unit testing on iPhone fails

[Vladimir Pouzanov]

Hello,

I'm using beta4 sdk and try to debug an iPhone project. While running
the tests I get a message:

Tests[1334:4303] SBSetAccelerometerClientEventsEnabled failed: (ipc/
send) invalid destination port

After which the script seem to be frozen, until I kill it. Any hints?

[Thomas Van Lenten]

Without more info, it's sorta hard to tell, that's something from your code, so what happens after/around that is up in the air.  I'd suggest putting the test under the debugger, to see what happens and where you're really getting stuck.

TVL

--
You received this message because you are subscribed to the Google Groups "google-toolbox-for-mac" group.
To post to this group, send email to google-tool...@googlegroups.com.
To unsubscribe from this group, send email to google-toolbox-fo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-toolbox-for-mac?hl=en.

[Vladimir Pouzanov]

On Mar 27, 5:24 pm, Thomas Van Lenten <thoma...@gmail.com> wrote:
> Without more info, it's sorta hard to tell, that's something from your code,
> so what happens after/around that is up in the air.  I'd suggest putting the
> test under the debugger, to see what happens and where you're really getting
> stuck.

It's somewhat hard to set all the required environment variables to
run the app's binary directly from shell, so I can be sure only about
the following steps:
1. Shell script runs
"$TARGET_BUILD_DIR/$EXECUTABLE_PATH" -RegisterForSystemEvents
Starting the actual application binary.

2. Application makes some activity:
Tests(6354) malloc: protecting edges
Tests(6354) malloc: recording malloc stacks to disk using standard
recorder
Tests(6354) malloc: enabling scribbling to detect mods to free blocks
Tests(6354) malloc: stack logs being written into /tmp/stack-logs.
6354.Tests.msbKrs.index
objc[6354]: REPLACED: -[NSSet initWithCoder:] by category NSSet (IMP
was 0x1c4a950 (/Developer/Platforms/iPhoneSimulator.platform/Developer/
SDKs/iPhoneSimulator3.1.3.sdk/System/Library/Frameworks/
CoreFoundation.framework/CoreFoundation), now 0x20e50 (/Developer/
Platforms/iPhoneSimulator.platform/Developer/SDKs/
iPhoneSimulator3.1.3.sdk/System/Library/Frameworks/
Foundation.framework/Foundation))
...
objc[6354]: REPLACED: -[NSString drawInRect:withFont:] by category
UIStringDrawing (IMP was 0x44baf4 (/Developer/Platforms/
iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.3.sdk/
System/Library/Frameworks/UIKit.framework/UIKit), now 0x2abbae (/
Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/
iPhoneSimulator3.1.3.sdk/System/Library/Frameworks/UIKit.framework/
UIKit))
2010-03-27 21:47:38.337 Tests[6354:4303]
SBSetAccelerometerClientEventsEnabled failed: (ipc/send) invalid
destination port

After that is sits in run loop forever.

I have an older iPhone project that uses GTM for its unit testing.
Tests run flawlessly there. They fail only on newly created projects.

[Thomas Van Lenten]

Nothing is jumping out at me there, if you've got some that work and some that don't, I'd suggest comparing them for config difference and/or the differences in what of your own code your pulling in, bisection that might help you figure out what's triggering it.  Otherwise, putting things under a debugger might also shed some light.

TVL

--

[vincent daubry]

I have exactly the same problem. I solved it by removing the
mainWindow from the test target.
This is weird...

On 6 avr, 20:16, Thomas Van Lenten <thoma...@gmail.com> wrote:
> Nothing is jumping out at me there, if you've got some that work and some
> that don't, I'd suggest comparing them for config difference and/or the
> differences in what of your own code your pulling in, bisection that might
> help you figure out what's triggering it.  Otherwise, putting things under a
> debugger might also shed some light.
>
> TVL
>

> > google-toolbox-fo...@googlegroups.com<google-toolbox-for-mac% 2Bunsu...@googlegroups.com>

[vincent daubry]

MainWidow.xib i meant

[Dave MacLachlan]

Can you hook a debugger up to it? Does the
applicationDidFinishLaunching: in GTMIPhoneUnitTestDelegate ever get
called?

Cheers,
Dave

[vincent daubry]

Hi,

Sorry for answering so late.
I'm not sure to understand what you mean : the run script build phase
never ends so i cannot use xcode debugger (it only stop on breakpoint
at runtime, not at compile time, or am i missing something ?)

Should i use some external debugger ?

Thanks,
Vincent.

To unsubscribe from this group, send email to google-toolbox-fo...@googlegroups.com.

[Dave MacLachlan]

Once the script is being run your project has actually been built. You
should be able to set up a custom executable and debug it.

BTW if you have trouble setting up a custom executable, can I
recommend installing the GTM Xcode plugin from here:

http://code.google.com/p/google-toolbox-for-mac/downloads/list

It gives you a menu option for creating custom executables which you
could then use to debug your problem.

Limited docs are here:
http://code.google.com/p/google-toolbox-for-mac/wiki/GTMXcodePlugin

Cheers,
Dave

> For more options, visit this group at http://groups.google.com/group/google-toolbox-for-mac?hl=en

[Meredith]

The relevant line is:

UnitTest[44140:903] SBSetAccelerometerClientEventsEnabled failed: (ipc/
send) invalid destination port

So the problem has something to do with how the plugin is handling the
Accelerometer, probably related to GTMObjC2Runtime.

[Thomas Van Lenten]

On Mon, May 31, 2010 at 2:29 PM, Meredith <bitc...@gmail.com> wrote:

The relevant line is:

UnitTest[44140:903] SBSetAccelerometerClientEventsEnabled failed: (ipc/
send) invalid destination port

So the problem has something to do with how the plugin is handling the
Accelerometer, probably related to GTMObjC2Runtime.

The file basically compiles away on the iPhone into nothing.

Do you already have a copy of the Simulator running when you are trying to run the unit tests?

TVL

[Justin DeWind]

I am also having this exact same problem when attempting to run tests
against the iOS4 SDK. The tests were running fine previous to the
upgrade

On May 31, 6:50 pm, Thomas Van Lenten <thoma...@gmail.com> wrote:

> > google-toolbox-fo...@googlegroups.com<google-toolbox-for-mac% 2Bunsu...@googlegroups.com>

> > > >>> .
> > > >>> For more options, visit this group athttp://
> > groups.google.com/group/google-toolbox-for-mac?hl=en

> > > >>> .
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "google-toolbox-for-mac" group.
> > > > To post to this group, send email to
> > google-tool...@googlegroups.com
> > > > .
> > > > To unsubscribe from this group, send email to

> > google-toolbox-fo...@googlegroups.com<google-toolbox-for-mac% 2Bunsu...@googlegroups.com>

> > > > .
> > > > For more options, visit this group athttp://
> > groups.google.com/group/google-toolbox-for-mac?hl=en

> > > > .
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "google-toolbox-for-mac" group.
> > > To post to this group, send email to
> > google-tool...@googlegroups.com.
> > > To unsubscribe from this group, send email to

> > google-toolbox-fo...@googlegroups.com<google-toolbox-for-mac% 2Bunsu...@googlegroups.com>
> > .

> > > For more options, visit this group athttp://
> > groups.google.com/group/google-toolbox-for-mac?hl=en.
>
> > --

> > You received this message because you are subscribed to the Google Groups
> > "google-toolbox-for-mac" group.
> > To post to this group, send email to
> > google-tool...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-toolbox-fo...@googlegroups.com<google-toolbox-for-mac% 2Bunsu...@googlegroups.com>

[Thomas Van Lenten]

Is the simulator still running?

TVL

To unsubscribe from this group, send email to google-toolbox-fo...@googlegroups.com.

[Justin DeWind]

No, it is not. I declared open war on all processes that could even remotely be associated with the simulator; including the simulator itself. I also made sure to reboot in order to clear any potential zombie processes that I could not find.

Is it possible that the socket/ipc communication has changed in the new simulator?

Justin DeWind

[Thomas Van Lenten]

On Wed, Jun 30, 2010 at 10:11 AM, Justin DeWind <dew...@gmail.com> wrote:

No, it is not. I declared open war on all processes that could even remotely be associated with the simulator; including the simulator itself. I also made sure to reboot in order to clear any potential zombie processes that I could not find.

Is it possible that the socket/ipc communication has changed in the new simulator?

Almost possible, I don't think Dave or I have had enough cycles to poke around and specific things like that.

TVL

[Justin DeWind]

Yikes,

Well, I'll have to spend sometime drilling down into the codebase in order to determine why this might be happening.

Justin DeWind

[Dave MacLachlan]

Justin, can you send me some sample code that shows a failure that I can play with?

Cheers,

Dave

[Justin DeWind]

Yes,

I will put together a project that reflects the structure of the ones I am working on.

Sent from my iPhone

[Justin DeWind]

Alright,

I have created a project that exhibits the same symptoms that my other iOS4 projects have. The tests run infinitely because the application run loop never terminates and the test application is never launched (headless). It look as if it it could be a spring board socket communication issue that prevents the script from every properly invoking the tests?

A a couple of things to note:

• This project depends on two static libraries -- OCMock and OCHamcrest
• OCMock and OCHamcrest cannot be linked against anymore in iOS4 -- yay
• The project has direct dependencies on the project
• It links in the static libraries via the library and header search paths under the build preferences for that target
• I was forced into include all of my XIB resources
• I was getting exceptions because certain iOS 4 (UINib) were explicitly looking for MainWindow and related dependencies within that XIB file 

The git repo is located at http://github.com/dewind/iOS4GoogleToolbox and you can clone it at g...@github.com:dewind/iOS4GoogleToolbox.git

Justin DeWind

[Justin DeWind]

I should amend my first paragraph. The test application is "launched", but the tests never run.

Justin DeWind

[Dave MacLachlan]

The tests aren't running because of a problem with the delegate. Unfortunately this is a very easy mistake to make.

We need GTMIPhoneUnitTestDelegate to be the application delegate. We were doing this by starting up with UIApplicationMain(argc, argv, nil, @"GTMIPhoneUnitTestDelegate") in GTMIPhoneUnitTestMain. Your app starts up fine, but when it loads MainWindow.xib it reassigns the app delegate from GTMiPhoneUnitTestDelegate to your iOS4GoogleToolboxAppDelegate. Therefore our delegate never gets the applicationDidFinishLaunching: notification.

I'm not sure if this behavior changed in iOS4 or has always been there, but I've got a fix I can think of, so I'll try and put something together for you. Assuming my partner in crime thinks it's worthwhile, I should have it posted today.

Thanks for the project.

Cheers,

Dave

[Justin DeWind]

That makes complete sense. It seems that the errors led me down a path that hosed the test target. The fact that I was forced to include XIB files -- which I am not happy with.

Justin DeWind

[Dave MacLachlan]

Fix for you is in the TOT for GTM. While I was playing I figured out a solution to another iOS4 issue that's been plaguing me, and a work around for it should be in soon... around 1100 AM PST.

Problem is detailed here:

http://openradar.appspot.com/8081169

Cheers,

Dave

[Justin DeWind]

Dave,

Thanks, the tests now run. However, and maybe this is related to the other issue you are addressing. It looks like any failure exceptions are not being caught by the code that is executing tests. But instead, it is bubbling to the top and causing the application to quit with an exception. Which means that once a test fails the rest will not run. 

Justin DeWind

[Dave MacLachlan]

That would be the issue that is now fixed in TOT.

Cheers,

Dave

[Justin DeWind]

Thanks Dave,

It looks like we are almost there. It seems that normal test failures are caught appropriately now. However, other exceptions are making there way up there. Alas, OCMock checks and Hamcrest assertions are bubbling to the top still -- this was working before.

Hamcrest: 2010-07-01 15:36:15.458 Tests[30200:903] *** Terminating app due to uncaught exception 'SenTestFailureException', reason: 'Expected "world", got "hello"'

OCMock: 2010-07-01 15:39:30.206 Tests[30312:903] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'OCMockObject[NSString]: unexpected method invoked: stringByAppendingString:@"hello" '

I imagine this is the result of some underlying changes in iOS4 in how exceptions are handled via dynamic invocation. I.E., you can't catch some of them. Frustrating, indeed. I have added these as tests cases to the project.

On another front. Removing the Main nib file base name entry in the tests info property list avoids the problem of having to include all of your resources.

You have been very helpful. I have 3-4 projects that went from being unable to run any tests to at least being able to work through them.

Justin DeWind

[Justin DeWind]

Oddly enough,

I have a test that rigs a mock to throw an exception. The target of the test is setup to catch the exception and handle it appropriately (shows an alert to the user). In the test, the exception is not caught by the target and it bubbles up to the top and fails. When it is running in the simulator and device the exception is caught -- as expected -- and the alert is shown.

It appears to be *how* exceptions are being thrown that makes a difference.

I have updated our little sandbox project to exercise this particular issue; which I think is happening when exceptions are raised via Objective C's NSProxy forward invocation mechanism. I created a proxy class that wraps a real object and forwards all calls to that object -- that object throws an exception. If the proxy class is used the exception bubbles to the top (the test runner cannot catch it). However, if the underlying strategy class is invoked directly (outside of forwardInvocation) the exception is caught.

OCMock *and* Hamcrest both use forwardInvocation heavily which would explain why the test runner cannot catch the exceptions. This feels like a egregious bug in iOS3.2 and iOS4 or they changed it on purpose and we need to find some obscure compiler setting to bring it back.

Justin DeWind

[Justin DeWind]

I should clarify. This does not feel like an SDK issue, but rather a compiler issue.

Justin DeWind

[Justin DeWind]

Sorry for pelting the group.

It does appear to be an NSInvocation issue, I ran into a red herring that said otherwise. Exception handling does not work at all when "invoke" is used. The only way to get around it is to use objc_msgSend.

It means these libraries have to be changed to do this.

Justin DeWind

[Justin DeWind]

Clarification, it is both an NSInvocation and NSProxy issue.

Exceptions directly raised in NSProxy are not catchable along with invoking NSInvocation directly.