SyncIQ file throttling

[dynamox]

Hello guys,

how do i view current settings for SyncIQ, how many files/sec is it set to copy. I am looking at throttling that value down to reduce CPU utilization during SyncIQ schedules. Current workers/per node is at 3 (default) and we have 7 nodes in the cluster.

OneFS 6.5.5.12
7 x 108NL

Thank you

[Peter Serocka]

One needs to create specific rules for this kind of throttling,
quite straightforward in the WebGUI under SyncIQ->Performance.

SyncIQ Best Practices whitepapers (for 6.5 / 7.0) are highly
recommended, they have some essential do's/don'ts especially for
throttling.

Peter

> --
> You received this message because you are subscribed to the Google
> Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to isilon-user-gr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[LinuxRox]

Peter, my question is what the default value is set to ..i know how to change it in the WebUI. I need to know what the default value is in case i need to reset it back.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

[Saker Klippsten]

You cannot control the amount of files per /second. You can control the workers per node as you stated above its 3w per node. Note once the job is started and you want to change the amount of workers you will have to restart the job .
Another area to reduce the CPU usage is limiting the network bandwidth of SyncIQ under the performance tab -- > network rules and then set the limit of how much you want to transfer durring the day and then let if fly on off hours..

-Saker

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.

[Peter Serocka]

BEST PRACTICES FOR DATA REPLICATION
WITH EMC ISILON SYNCIQ
(April 2011, SynIQ 3.0)

"By default SyncIQ will take all idle
system and network resources. For additional flexibility, you can
limit SyncIQ system
or network resource utilization and schedule synchronization jobs for
off-peak hours."

I would interprete this is in the way that each worker (where there
are 3 per node by default as we know) is not limited in resource
consumption by default. So just by setting performance rules the
workers can be deliberately restricted. But the defaults are: unlimited.

Peter

> send an email to isilon-user-gr...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it,

> send an email to isilon-user-gr...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it,

> send an email to isilon-user-gr...@googlegroups.com.

[Peter Serocka]

Sure?

From the 6.5 admin guide:

View SyncIQ performance rules:

"Limit: Indicates the maximum network usage allowed (in bits per
second), or CPU and drive usage allowed for file operations (****in
files per second****) for SyncIQ processes."

(in total we have three areas: 1. Workers 2. Performance->Network 3.
Performance->File Ops.)

Peter

[Cory Snavely]

Also, any given worker will not handle more than one file at a time and
any one file will not be handled by more than one worker, though I hear
that will be improving in an upcoming release to better parallelize on
large files.

That gets to the original question, although the original question may
be irrelevant at this point. :)

[Peter Serocka]

Correct. The Best Practice papers give all the background and derive
the practical tips from that foundation!

Peter

[Saker Klippsten]

Looking at the gui yes peter is correct.

"(in total we have three areas: 1. Workers 2. Performance->Network 3. Performance->File Ops)

We only set the workers and network I never mess with the file ops but I am sure it works ;)
On our 20 node 5000s series we use 20 workers per node and throttle network to 3 gbit during the day and let it run full speed which gets up to 10gbit but that's only limited by our connection to our DR site.


-s

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.



--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.




--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.




--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-group+unsubscribe@googlegroups.com.

[LinuxRox]

looking in SyncIQ > Performance > File Operations performance  ..on the graph i see that each node is doing around 2k files/sec. I am thinking of adding a file operations rule and set it to 1k and see if it helps with CPU utilization.

To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.