Float vs integer constant

[Eric Fournier]

Hi,

I am trying to create a new node column based on another column. The column should be TRUE if the existing numeric column is 0, or FALSE otherwise.

However, using =$columnName=0 always return the following error:

#ERR: 14: incompatible operands for "=". (lhs=IdentNode:  distanceToTSS:class.java.lang.Integer, rhs=...

I tried using another column (Degree) to test a possible solution, but then I get error #7, with the same message, but I can see that the rhs is a float.

Is there a way to cast the 0 constant from a float to an integer? Or am I going about this the wrong way?

Thanks,
-Eric Fournier

[Mike Kucera]

Hi Eric,

You are running into a bug. The formula evaluator should automatically convert the integer value in $columnName to a float before comparing, but that's not working. Because of the bug the conversion only works for "Long Integer" columns and not "Integer" columns.

You can't cast the 0 constant to an integer. All numeric constants are treated as floats. But you can cast $columnName to a float using the VALUE function. The formula looks like this:

=VALUE($columnName)=0

[Barry Demchak]

This concerns me …

 

Doing an “=” comparison on integers is always exact and correct. It gets much dicier when doing “=” comparisons on floats, even those created via up-cast. True, the up-cast is deterministic, so up-casting two values “10” would result in identical values. However, calculating 10.0 as 100.0/10 would not be expected to compare equally to an up-cast “10”.

 

Do you have a perspective on this? … or maybe some accuracy-preserving compare strategy??

--
You received this message because you are subscribed to the Google Groups "cytoscape-helpdesk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cytoscape-helpd...@googlegroups.com.
To post to this group, send email to cytoscape...@googlegroups.com.
Visit this group at https://groups.google.com/group/cytoscape-helpdesk.
For more options, visit https://groups.google.com/d/optout.

[Mike Kucera]

This concerns me as well. All of the available arithmetic and comparison operators auto-convert their operands to floats. We absolutely need to fix this for 3.5. But maintaining backwards compatibility might be tricky.

[Barry Demchak]

Yeah … if we can get a defensible comparison strategy and it sometimes produces different results than the existing comparisons, we’ll need to have a Properties setting to allow a user to fall back onto an outcome consistent with previous results.

 

Glad you’re on this.