floor() outputs wrong result

[Ognen]

Hello,

for example floor(35563.082031) outputs 35564 instead of 35563. Can someone tell me why?

Thanks

Ognen

[Ognen]

Now I notice that sometimes, when using printf("%f", floor( 35563.082031 )) to check the output value, floor() acts as ceil(), and sometimes it outputs the correct result.

Is this an issue related only at the preview state and does not affect the final output when OK button is pressed?

[paul simoens]

Hi Ognen,

For an attempt to help find the cause of this anomaly, it’s necessary to see also the surrounding code where this is embedded. I guess it’s not just that insolated function on its own. I’m thinking in the direction of probably an asynchronous display of a series? A value that runs one sequence behind?

 

Paul

--
You received this message because you are subscribed to the Google Groups "FilterMeister Mailing List (FMML)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to filtermeiste...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/filtermeister/e2d92da3-dc9e-4506-9aae-9cc83026059dn%40googlegroups.com.

[Ognen]

Hi Paul,

It is indeed on its own because it is just for testing purposes:

/////////////////////////////////////////////////////

%fml

ForEveryTile:

{
        for (y = 0; y < Y; y++) {
                for (x = 0; x < X; x++) {
                        if (x == 1 & y == 1) {

                                printf("%f",  floor(35563.082031));

                        };
                }
        }
        return true;
}

/////////////////////////////////////////////////////

It even might show the problem without being wrapped in any loops too.

This happens when using the zoom on the preview screen, at each zoom value it outputs different number, not even close to the integer 35563. At 100% zoom it outputs 35563, at 33.3% zoom it outputs 35564, at 25% it outputs 29561 and so on.

I guess this problem is only related to the preview screen.

Ognen

[Ognen]

One more thing what I have missed to mention is that the problem with the preview screen happens after some time working in Photoshop. It is strange that it is not present at the beginning of working session in PS.

[paul simoens]

Hi Ognen,

You’re using a bitwise AND operator ‘&’ instead of the logical AND ‘&&’.

That’s reason enough to obtain irregularities…

Anyway, I tried the wrong code but couldn’t reconstruct your varying results.
In case this couldn’t resolve your problem, compare the results with an alternative method:
float fq000 =35563.082031, fq001, fq002;

int iq002;

fq001 =floor( fq000);
fq002 =iq002 =iq000;


Regards,
Paul

To view this discussion on the web visit https://groups.google.com/d/msgid/filtermeister/5c3f98fc-8ad4-43ef-b92f-cb0ae10f569fn%40googlegroups.com.