Varying processing speed - BoxUnits

[paul simoens]

Hi All,

1.  Huge differences in processing speed for the same task
I want to make a catalogue of how much time it takes for a specific type of calculation or algorithm to be processed, in relation to each other. Just for comparison. This can be very helpfull in the construction of more complex filters, in order to choose for the right option and strategy. Therefore a made a tool in which this can be measured inside a simple loop. The loop runs typically about 40 million times, to create enough acccuracy. At my surprise I notice huge differences between the same measurement in the order of 1 to 2. Thus a specific calculation could give e.g. 250 ticks, but repeated immediately after it could also give up to 480 ticks – and back again!

I’m aware that the internal management of a CPU can differ between operations, but I can hardly believe the dimension of this. Is this really the way a computer works? If it should, it will be hard to obtain reliable data. Ideas to improve this or what could be done?

- - - - - - - - - - - - - - - - - - - - - - -
2.  How to deal with BoxUnits and Pixels in the construct of a filter ?
I understand the idea to stabilize the GUI between different platforms with the introduction of BoxUnits. But once you want to apply this automatic adaptation, you collide quickly with an unmanageable amount of variables that even won’t assure a reasonable result. Even Horizontal and Vertical Units can differ. In such case do you choose the largest one?

Therefore my question: what’s the right order of steps to be followed to make a more or less flexible GUI?
Another guess: or do you simply build 2 or 3 versions around the most actual used resolutions, so the actual user can pick the best choice?

 

Best Regards,
Paul

 

[Ognen]

On Thursday, January 20, 2022 at 3:28:18 PM UTC+1 paul simoens wrote:

>1.  Huge differences in processing speed for the same task
>I want to make a catalogue of how much time it takes for a specific type of calculation or algorithm to be processed, in relation to each other. Just for comparison. This can be very helpfull in the construction of more complex filters, in order to >choose for the right option and strategy. Therefore a made a tool in which this can be measured inside a simple loop. The loop runs typically about 40 million times, to create enough acccuracy. At my surprise I notice huge differences between the >same measurement in the order of 1 to 2. Thus a specific calculation could give e.g. 250 ticks, but repeated immediately after it could also give up to 480 ticks – and back again!

>I’m aware that the internal management of a CPU can differ between operations, but I can hardly believe the dimension of this. Is this really the way a computer works? If it should, it will be hard to obtain reliable data. Ideas to improve this or what >could be done?

High speeds of calculation for complex filters to a degree are only suitable to be created in a language specialized for GPU programming. CPU performs tasks either synchronous or asynchronous, but the limits are very high in comparison to the GPU execution times even if the GPU is old enough. GPU shaders vs CPU is light years difference. FM should be made for GPU shader programming, the CPU is not anymore popular for graphics programming. 

Regards,

Ognen

 

[paul simoens]

Thanks for responding, but this is besides the issue. I just wonder why a similar calculation can differ relatively so much in processing speed from one moment to another.

Paul

--
You received this message because you are subscribed to the Google Groups "FilterMeister Mailing List (FMML)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to filtermeiste...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/filtermeister/f82c397f-ad4a-4ab3-b5b5-e95983151367n%40googlegroups.com.

[Ognen]

On Friday, January 21, 2022 at 7:41:53 PM UTC+1 paul simoens wrote:

     >  Thus a specific calculation could give e.g. 250 ticks, but repeated immediately after it could also give up to 480 ticks – and back again!

> I just wonder why a similar calculation can differ relatively so much in processing speed from one moment to another.

 The answer is simply maybe FM does not cope very good with the HEAP memory. Maybe PS is the reason. But definitely, if a loop is slower after the previous one, then certainly this shows as the FM or PS issue.

Regards,

Ognen

[Roberto]

The answer is 'simply maybe' ...

Seems at the moment nothing is simple at all if you are still in guessing mode :-)

Oh well, I'm sure the actual answer is more simple than the ability to write down the contents of your email ;-)

What are ticks? Number of individual steps/chunks or are those a measure of time?

The same routine taking different amount of steps is not expected when a single CPU is processing it. The process taking different amounts of time to be finished is not expected at all considering the thousands of times a CPU might be interrupted by other processes.

Hard to share thoughts anyway without knowing what 'tools' are used in the test loop.

Roberto

-

Op Sat, 22 Jan 2022 03:18:46 +0100 schreef Ognen <gog...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "FilterMeister Mailing List (FMML)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to filtermeiste...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/filtermeister/dda5634c-74e2-4101-ab0c-655ae20516ffn%40googlegroups.com.

--

Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/

[paul simoens]

Hi Roberto,

The ‘ticks’ are the time/value differences between two measurements with clock( ) (=FM function). One is placed just before and the second just after the execution of an algorithm/calculation. It’s of no fundamental importance how it exactly behaves to

real micro or nanoseconds, what’s important is the ratio between different types of algorithm/calculations, since the circumstances of measurement are equal (same amount of loops, etc.), only the type of algorithm/calculation differs.

Unexpected are the huge differences in repeatability of the same measurements, notwithstanding a high choosen number of loops (e.g. a rounded 40 million times, about the size of a 12 Mp image). That it differs: yes. That it (can) differ up to 1 to 2 ?... Difficult to understand. Thus what causes this huge deviations in processing speed? Is this a typical internal management of ROM or something? Are there probably tricks to purge unnecessary filled memory to keep the speed always on it shortest? If we can understand why this happens, we could to a significant extent optimize execution times of filter processing.


Greetings,
Paul

 

To view this discussion on the web visit https://groups.google.com/d/msgid/filtermeister/op.1geb7fgvncx852%40roberto-asus-g11.home.

[Roberto]

Hi Paul,

Did you test a 'naked' clock() function test without additional redundancy? I know from e.g. cheap measuring tools or Android phone apps everything time related (clock, speed, frequency) can behave very inconsistent. In general cheap = containing microcomputer = deals with interrupts and that is taken for granted. Same for e.g. processors with on-board measuring devices (voltage, temperature, etc.): often not accurate and individually inconsistent; mostly 'ball park' gadgets.

An on-board clock-counter is one thing (whether running accurate or not) but 'lost time' between request and actual moment of delivery of data is a totally different other thing.

If there is significant spread in time between request of clock data and the moment of getting the actual data, can this in your type of test loop cumulatively add up error? I mean: assuming the clock data requested is correct at the moment of reading but the number is not passed on to the requester immediately.

Do you clock once each image is processed or once each pixel or row/column?

Perhaps it helps to request clock data several times in one pass; this might average out random delays.

Or putting clock requests once each pixel, row and image is processed: this might help getting insight in which process parts are more consistent than the other.

I mean, FM is a higher programming language; I would not be surprised if it suffers from timing issues on a machine that works like (is) a computer since computers deal with many interrupts.

On the other hand, 40 million loops would statistically average out everything very well to a constant value without much spread. If thát number (processing a whole image) would vary between 250 and 400 ticks it is indeed remarkable.

If individual loops differ only from 250 to 500 ticks each in processing time I think it is a good number actually: if a processor is 'waiting' while running an other process or if it can finish an other process without waiting it certainly can affect the duration of the interrupt. Sometimes computers can work hard on the background like when virus scanners, backup tools or update processes are running.

Then there is a modern issue also: modern processors can change clock speed on the run: when not very busy they run on lower (energy saving) clock speed while at higher demand they can boost clock speed to the max. The difference in clock speed can be significant so if you are only looking at time values you easily could be fooled. However, this would show up in test results as group-wise all taking few or many ticks instead of randomly.

Roberto

Op Sat, 22 Jan 2022 14:14:10 +0100 schreef paul simoens <paul.s...@telenet.be>:

[Ognen]

HI Paul,

If ticks result is showing different values each time of execution, then this is CPU related thing and as I said in my first answer, CPU performs either synchronous or asynchronous tasks and that would be complex to explain because there are many processes that are happening without you knowing what exactly executed at the moment, and also these processes have either sync or async commands related to their nature for what they are programmed to do. Hence, every time you execute your code, it gives you random tick values. Take this as an example:

var now= performance.now();

for(var i=0; i<1000; i++) {
    console.log(i);
}

var after = performance.now();
console.log("This execution took" + (after - now) + " milliseconds.");

This is a JavaScript example on how to measure ticks in ms. Since JS is slower in performing tasks, I have used 1000 counts in a for loop to be printed on the screen and shows the ms that are passed after its execution. On every execution, the time passed is random(which can also be measured in ticks), but not in constant rise as I did understand it wrong before.

Regards,

Ognen