Cloudlet Execution Time Accuracy

[Marios Avgeris]

Greetings to the community,

I have spotted that the functions which return a Cloudlet's start/execution/finished time, return results that are always around "x.1" seconds. For example, this is what a simple

Log.printFormatted("Finish Time: " + cloudlet.getFinishTime() + "\n") prints for some Cloudlets:

...

Finish Time: 1.1

Finish Time: 2.0900000000000007

Finish Time: 3.1

Finish Time: 4.1

Finish Time: 5.0975

...

Is this an issue concerning the way CloudSim & eventually CloudSim Plus handles the execution simulation? If so, is there a way to get more accurate results? 

Thanks in advance,

Marios

[Manoel Campos]

I've seen these values since I started to use CloudSim. When CloudSim Plus was initiated, I didn't worry about such values. 

I think it happens because the first event is processed at time 0.1.

However, if you check the start time and the actual CPU time, you'll see the values are in fact accurate.

Maybe it's just a matter or formatting the values to be printed.

Values such as 2.0900000000000007 occur due Java double precision issues.

Double values are implemented in Java following the IEEE 754 standard

.

Since they  are represented as approximations

, sometimes you get values such as  2.0900000000000007 instead of 2.1.

More examples can be seen here.

If you see the examples, the decimal places are not shown.

Manoel Campos da Silva Filho Software Engineer

Computer Science and Engineering Ph.D. Student at University of Beira Interior (Portugal)

Professor at Federal Institute of Education, Science and Technology of Tocantins (Brazil)

http://manoelcampos.com

 about.me

--
http://cloudsimplus.org
---
You received this message because you are subscribed to the Google Groups "CloudSim Plus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudsim-plus+unsubscribe@googlegroups.com.
To post to this group, send email to clouds...@googlegroups.com.
Visit this group at https://groups.google.com/group/cloudsim-plus.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloudsim-plus/73e3737e-26c1-47fd-86fe-b550f8c7c86e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marios Avgeris]

Hi Manoel, 

thanks for the answer but probably I didn't make myself clear. What I meant was that I find peculiar that every Cloudlet finishes exactly at "x.1" time units. For example, in the real life, finish times times like 1.320, 2.102, 4.569, 5.734 would seem more reasonable. 

But then I dug a little inside the simulation implementation and the whole event processing machine and, to my understanding, this happens because CloudSim aggregates the events and processes them all together on every "tick", which coincides with 1 time unit (or 1 sec? I'm not sure).

This results in integer-like execution times, meaning that a Cloudlet taking 600ms to execute will probably appear with 1s execution time, another taking 1700ms appears like 2s etc.

As you can understand, such rounding hampers the creation of precise models for the experiment simulation. Is there anything that can be done to make the "tick" more precise for instance? Or can I get more accurate results following another path?

Thanks again for your time,

Marios

To unsubscribe from this group and stop receiving emails from it, send an email to cloudsim-plu...@googlegroups.com.

[Manoel Campos]

I see your point now.

 

thanks for the answer but probably I didn't make myself clear. What I meant was that I find peculiar that every Cloudlet finishes exactly at "x.1" time units. For example, in the real life, finish times times like 1.320, 2.102, 4.569, 5.734 would seem more reasonable. 

You are right, but since cloudlet length is defined in MI (Million Instructions) and PEs capacity are defined in MIPS (Million Instructions per Second) using only integer values as defined in all examples will result in interger execution times.

 

But then I dug a little inside the simulation implementation and the whole event processing machine and, to my understanding, this happens because CloudSim aggregates the events and processes them all together on every "tick", which coincides with 1 time unit (or 1 sec? I'm not sure).

It aggregates only the events happening at the same time. The time unit is in seconds, but the tick doesn't happen at each second by default.

Consider an example of a static simulation where there are 2 Cloudlets running into 1 VM: 1 cloudlet takes 10 seconds to finish and the other one 20 seconds. After the cloudlets start executing, the clock usually ticks 4 times: one time for when each cloudlet starts and one for when each cloudlet finishes. Other tick events aren't related to cloudlets processing in this example.

You can confirm this by adding a listener using the CloudSim.addOnClockTickListener method.

The clock only ticks in static interval if the Datacenter.schedulingInterval is set to a value greater than zero.

However, this drastically impacts simulation performance.

 

This results in integer-like execution times, meaning that a Cloudlet taking 600ms to execute will probably appear with 1s execution time, another taking 1700ms appears like 2s etc.

If you want to get more precise cloudlet finish times, you may try to define the length of the cloudlets to values which aren't multiple of the PEs MIPS capacity. For instance, define MIPS as 1000 and cloudlet length as 600. Using such values, the cloudlet is expected to finish in 600ms.

However, I created a small example and realized there is a bug in the CloudletSchedulerAbstract.timeSpan method. It is rouding the computed time span, making the execution time be like you mentioned above (1 second instead of 0.6s).

I've just created and fixed the issue #121

. The only change was in the timeSpan method.

Use the attached example to test.

Please let me know if everyting is OK and thanks for reporting the issue.

 

To unsubscribe from this group and stop receiving emails from it, send an email to cloudsim-plus+unsubscribe@googlegroups.com.

To post to this group, send email to clouds...@googlegroups.com.
Visit this group at https://groups.google.com/group/cloudsim-plus.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudsim-plus/24f5b98d-2b63-43e3-b3b4-8b5cf4328c7e%40googlegroups.com.

[Marios Avgeris]

That's incredible! now it works as expected. However, now I'm experiencing another bug, probably occurring due to the changes you made in the timeSpan method;

Sometimes, when running a simulation, although cloudlets continue to submit to the respected Vm for execution, the Vm stops executing them until the end of the simulation. For example, this is a log printing every 30sec, showing info about Vm0: 

        Time 1350.1: | Vm 0 | Average Interval CPU Usage:  17.39% | Average Interval Ram Usage:   4.21% | Average Interval Response Time:   1.00 s 

Time 1380.1: | Vm 0 | Average Interval CPU Usage:  34.35% | Average Interval Ram Usage:   4.81% | Average Interval Response Time:   1.31 s 

Time 1410.1: | Vm 0 | Average Interval CPU Usage:  30.43% | Average Interval Ram Usage:   4.56% | Average Interval Response Time:   1.30 s 

Time 1440.1: | Vm 0 | Average Interval CPU Usage:   0.00% | Average Interval Ram Usage:   1.95% | Average Interval Response Time:   0.00 s 

Time 1470.1: | Vm 0 | Average Interval CPU Usage:   0.00% | Average Interval Ram Usage:   1.95% | Average Interval Response Time:   0.00 s 

and from that point on, the CPU Usage stays 0.00% until the simulation exits. Two things to clear out though:

1. This does not happen every time I perform a simulation, but clearly most of the times. In the above scenario, I create cloudlets with random rate, length, PES, as well as, every 30s the VM is randomly scaled up or down. Maybe a combination of these configurations triggers it?

2. This never happens when I use the old timeSpan method. 

Have you got any insight on this? Thanks again for the quick responses and addressing of the issues.

[Manoel Campos]

That's incredible! now it works as expected. However, now I'm experiencing another bug, probably occurring due to the changes you made in the timeSpan method;

The test suits didn't get any issue, but despite test coverage has increased a lot, it is still low.

 

Sometimes, when running a simulation, although cloudlets continue to submit to the respected Vm for execution, the Vm stops executing them until the end of the simulation. For example, this is a log printing every 30sec, showing info about Vm0: 

        Time 1350.1: | Vm 0 | Average Interval CPU Usage:  17.39% | Average Interval Ram Usage:   4.21% | Average Interval Response Time:   1.00 s 

Time 1380.1: | Vm 0 | Average Interval CPU Usage:  34.35% | Average Interval Ram Usage:   4.81% | Average Interval Response Time:   1.31 s 

Time 1410.1: | Vm 0 | Average Interval CPU Usage:  30.43% | Average Interval Ram Usage:   4.56% | Average Interval Response Time:   1.30 s 

Time 1440.1: | Vm 0 | Average Interval CPU Usage:   0.00% | Average Interval Ram Usage:   1.95% | Average Interval Response Time:   0.00 s 

Time 1470.1: | Vm 0 | Average Interval CPU Usage:   0.00% | Average Interval Ram Usage:   1.95% | Average Interval Response Time:   0.00 s 

and from that point on, the CPU Usage stays 0.00% until the simulation exits. Two things to clear out though:

1. This does not happen every time I perform a simulation, but clearly most of the times. In the above scenario, I create cloudlets with random rate, length, PES, as well as, every 30s the VM is randomly scaled up or down. Maybe a combination of these configurations triggers it?

2. This never happens when I use the old timeSpan method. 

The issue is not clear for me. Could you provide a minimalist example, describing what the expected results are? 

Have you got any insight on this?

I have to see an example to try figuring out.

 

Thanks again for the quick responses and addressing of the issues.

You're welcome. 

 

To unsubscribe from this group and stop receiving emails from it, send an email to cloudsim-plus+unsubscribe@googlegroups.com.

To post to this group, send email to clouds...@googlegroups.com.
Visit this group at https://groups.google.com/group/cloudsim-plus.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudsim-plus/aa5398a8-0108-44f0-a026-e2fff23447f9%40googlegroups.com.

[Marios Avgeris]

Hi Manoel,

sorry for the delay but it took me too long to actually narrow down the problem and provide the minimum example. So in the attached example, what we're trying to do is construct a linear model for the simulation, although that's irrelevant to the bug. 

In order to inspect how the system reacts to various inputs and states, we randomly scale a Vm's CPU (up/down between some defined thresholds) every 30sec and throw some cloudlets to it, measuring the corresponding CPU usage and average

response time for the entire interval.

Here is what I've come down to:

1) If you run the example as is (at least at my system) the average CPU usage and average response time becomes zero after the second interval (cloudlet execution stops), although cloudlets are submitted normally.

2) If you run the example with a lot bigger cloudlet length (e.g 6000 to 9000) it executes without the above issue.

3) The "random Vm scaling process" takes place in the following lines:

int numberOfPesForScaling = ThreadLocalRandom.current().nextInt(VM_PES_LOWER_BOUND, VM_PES_UPPER_BOUND);
Vm vm = vmList.get(0);
vm.getHost().getVmScheduler().deallocatePesFromVm(vm);
List<Double> PEsList = new ArrayList<>();
PEsList.addAll(Collections.nCopies(numberOfPesForScaling, (double) VM_MIPS_CAPACITY));
vm.getHost().getVmScheduler().allocatePesForVm(vm, PEsList);
vm.getProcessor().setCapacity(numberOfPesForScaling);

If the line:

vm.getHost().getVmScheduler().deallocatePesFromVm(vm);

gets removed, it executes without the above issue. 

4) In the timeSpan method, if I revert the change you made, meaning change this line:

return currentTime - cl.getLastProcessingTime();

to this line:

return Math.floor(currentTime) - Math.floor(cl.getLastProcessingTime());

the average response time become at least twice as much (I believe this is normal, due to the rounding errors), but the average CPU usage appears up to 4-5 times as much (for the exact same configuration).

So what is this timeSpan method actually affecting in the simulation and is the above behaviour a bug or is there something wrong with the experiment we're setting up?

Sorry for the long answer and the many issues raised at once, but I've reached a dead end.

[Manoel Campos]

Hello Marios,

Sorry for the long message, but there are some points to be made clear.

I hope it helps you and other users to understand better how CloudSim Plus (and also CloudSim) works.

I don't think the issue is with CloudSim Plus. I have noticed two issues in your code:

1) You are creating cloudlets with random lengths between 600 and 900. Accordingly, your cloudlets take less than 1 second to finish. 

After I fixed the issue #121, cloudlet's execution times are very accurate. 

However, if you use a CloudletTableBuilder to print the results, it rounds values to 1. 

Since you are defining the datacenter scheduling interval as 1, all your cloudlets will have finished before the first second. That explains why you are getting 0% CPU usage at the second clock tick. 

You can confirm that by seeing that all messages "DatacenterBrokerSimple2: CloudletSimple XX finished and returned to broker" are being printed before the print inside the scaleResourcesInInterval method.

Since you are defining all intervals as integer values and other variables such as the howManySubmitted are also storing integers, you have issues related to this. Inside the onClockTickCollectListener you are also checking if the event time is multiple of the SAMPLING_INTERVAL. Since your cloudlets finish in less than one second, the SAMPLING_INTERVAL cannot be int anymore.

You can see that the attached example works fine with a scheduling interval equals to 0.2 second (see the onClockTickListener method and the logs).

2) The other issue is inside your scaleResourcesInInterval method.

You are manually calling vm.getHost().getVmScheduler().deallocatePesFromVm and vm.getHost().getVmScheduler().allocatePesFromVm methods to perform vertical VM CPU scaling. That is not the way you are supposed to do that. Vertical VM Scaling should be performed using a VerticalVmScaling implementation.

This is a mistake that I've seen a lot of CloudSim users to make for different purposes. It's understandable because the API just lets users call some methods that shouldn't be public. 

The point is: allocation or deallocation of Vm resources should be made internally by a Host. 

You should use methods such as the mentioned ones only if you are extending framework classes. 

That is not the case for your example. You are just creating a simulation. 

Thus, you should follow the regular process: 

1) create a datacenter with some VmAllocationPolicy;

2) create some Hosts for that datacenter and define a VmScheduler and ResourceProvioners;

3) create brokers representing the customers;

4) create VMs defining their CloudletScheduler;

5) create Cloudlets;

6) and finally submit VMs and Cloudlets to brokers and start the simulation.

Other steps not included in this flow are, for instance, the creation of additional objects such as VmScaling. However, usually you just have to call setters to define the object's configuration and relationships with other objects.

Why am I saying that it isn't supposed to call methods such as the allocatePesFromVm from you simulation?

Because sometimes they have to be called followed by other methods to ensure the correct simulation state.

For instance, the VmAllocationPolicyAbstract.scaleVmPesUpOrDown calls the deallocatePesFromVm and then performs some additional checks to verify if the VM is under or overloaded. If the VM is in either of these states, some additional actions are performed. If you call these methods directly in these situations, you are bypassing these checks and may end up with inconsistent results.

In your example, I don't see any side effects. But you have to take care when calling some of those methods directly.

In summary, you have to fix the first issue I mentioned to make your code work.

If you have any questions, just keep in touch.

Manoel Campos da Silva Filho Software Engineer

Computer Science and Engineering Ph.D. Student at University of Beira Interior (Portugal)

Professor at Federal Institute of Education, Science and Technology of Tocantins (Brazil)

http://manoelcampos.com

 about.me

To unsubscribe from this group and stop receiving emails from it, send an email to cloudsim-plus+unsubscribe@googlegroups.com.

To post to this group, send email to clouds...@googlegroups.com.
Visit this group at https://groups.google.com/group/cloudsim-plus.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudsim-plus/91dedcc3-d206-4c27-a5ed-e116ef301a49%40googlegroups.com.

[Marios Avgeris]

Hi Manoel, 

redefining the datacenter scheduling interval actually seems to do the trick. As for the scaling process, I understand the simulation flow you described, however I find it lacking the freedom needed for this kind of work (model building); what I mean, for example, is that we want to scale a Vm up/down randomly, rather than defining an overload/underload threshold. 

However, we might consider fitting more into your your flow due to the possible inconsistencies you mentioned (or at least create custom functions in order to perform the scaling according to our needs).

Nonetheless I will continue stretching and testing the capabilities of CloudSim Plus and report back anytime something comes up! 

Once more, thanks for your time and patience going through my code and doing some explaining. 

[Manoel Campos]

redefining the datacenter scheduling interval actually seems to do the trick.

Good.

 

As for the scaling process, I understand the simulation flow you described, however I find it lacking the freedom needed for this kind of work (model building); what I mean, for example, is that we want to scale a Vm up/down randomly, rather than defining an overload/underload threshold. 

I see.

 

Nonetheless I will continue stretching and testing the capabilities of CloudSim Plus and report back anytime something comes up! 

I appreciate that.

 

Once more, thanks for your time and patience going through my code and doing some explaining. 

You're welcome.