Corner case in ExcelDnaUtiil.IsInFunctionWizard

[Harley Blumenfeld]

I came across a weird bug in ExcelDnaUtil.IsInFunctionWizard. 

It appears that if you have that call in your UDF there are certain conditions that can cause ArithmeticOverflowException. 

I got this to happen when I had a UDF that was an RTD call in a spread sheet that used IsInFunctionWizard.

The spread sheet was an xlsm file and I opened it by double clicking an attachment on an Email. 

When it opened in Excel it opened with editing disabled and macros disabled where you have to hit a button on the yellow bar on the top to enable editing and another one to enable macros. Under those conditions the exception was thrown. Here is the stack trace:

StackTrace " at System.IntPtr.op_Explicit(Int64 value)
at ExcelDna.Integration.ExcelDnaUtil.get_WindowHandle()
at ExcelDna.Integration.ExcelDnaUtil.IsInFunctionWizard()

When I saved the file locally and opened it by going to File->Open in a fresh instance of Excel it behaved normally and did not throw the exception.

If I have time I'll see if I can make it happen outside my project and if so provide a patch to the project, but wanted to raise it as a potential problem and see if anyone else has seen this.

This happened under Excel 2010 (14.0.7128.5000) and Excel DNA 0.30.22.1.

[Govert van Drimmelen]

Hi Harley,

As a first step, you might like to try with Excel-DNA version 0.32. I did quite a lot of work to improve the Application and WindowHandle stuff, and you might find the problem already fixed.

If not, please let me know.

-Govert