How do I shuffle/randomize long lists of hosts please?

[Eric Wedaa]

How do I shuffle/randomize long lists of hosts?  That is, without writing an executable hosts file that randomizes them and then spits them out, and then dealing with all the groupings and variables and other wonderfulness that I can include in the hosts file?

My inventory file is collected semi-dynamically and comprises long lists of servers that are all running on the same blade on a long list of blades.  I run upto 40 servers per blade, and I have about a 80 blades (or more).  The FIRST time I run an "update bash" (for instance) I'll wind up hitting all of one blade first, effectively flat-lineing it.  If I can shuffle my hosts list, I can have a larger fork/serial value and won't  flatline my blades and I can shorten my effective runtime.

I was hoping for a "--shuffle-hosts" variable but I couldn't find it.

>>>Ericw

[Michael DeHaan]

There's no such setting.

However, I don't understand why changing the order wouldn't smakc your *other* blade, so this question seems orthogonal. 

Sounds like you need to take it out of rotation if it's already nearing 100% CPU, such as using our load balancing modules.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/679fde26-6b71-49ab-a836-f0806d545d3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eric Wedaa]

Changing the setting to random and the fork value to 40 gives me a reasonable chance that all 40 forks will be more or less equally distributed.  With more than 40 blades, it's a good bet only one fork per blade.  Of course, being random, I could someday get all 40 forks going to one blade.  It's not that the blade is already at 100% CPU,  it's that hitting a single blade with 40 updates at the same time will not only flatline my cpu, but the ethernet sockets for the same blade as well.  And on some of my blades, the system may try and move LPARs to different blades to re-equalize the load, which just then makes it take even longer.

This is a semi-common option, at least in the network scanning tools I have used.  I was hoping it would already be here and I just wasn't seeing it.

For instance,

 Nmap 6.47SVN ( http://nmap.org )
 Usage: nmap [Scan Type(s)] [Options] {target specification}
 TARGET SPECIFICATION:
   -iR <num hosts>: Choose random targets

And for Nessus
  http://static.tenable.com/documentation/nessus_5.0_user_guide.pdf  page 11

 Avoid Sequential Scans 

 By default, Nessus scans a list of IP addresses in sequential order. If checked, Nessus
 will scan the list of hosts in a random order. This is typically useful in helping to
 distribute the network traffic directed at a particular subnet during large scans

 
We have to be careful when we scan our Z series boxes for instance with Nessus.  Thank goodness for random host selection.

This is a scalability issue as far as I am concerned.

>>>Ericw

[Strahinja Kustudić]

If you are collecting the inventory file semi-dynamically, why not randomize the order while you are collecting the hosts?

[Eric Wedaa]

That's what we're apparently going to have to do :-(

Like I said though, the option exists in other tools that access a large number of hosts.  I was hoping it would be in Ansible, just hidden where I couldn't find it.

And the problem with doing ourselves is that then we lose order in the hosts file itself.  I could write another script to shuffle the hosts and I'm going to have to do that I guess.

Can somebody tell me where can I submit this as a future enhancement request?

>>>>Ericw

[Strahinja Kustudić]

You can on github https://github.com/ansible/ansible

[Tomasz Kontusz]

Eric Wedaa <csgsurfe...@gmail.com> napisał:

>That's what we're apparently going to have to do :-(
>
>Like I said though, the option exists in other tools that access a
>large
>number of hosts. I was hoping it would be in Ansible, just hidden
>where I
>couldn't find it.
>
>And the problem with doing ourselves is that then we lose order in the
>hosts file itself. I could write another script to shuffle the hosts
>and
>I'm going to have to do that I guess.

You can look at vars_plugins, IIRC they have access to inventory but are separate from it - you wouldn't even have to touch your inventory scripts then.

>Can somebody tell me where can I submit this as a future enhancement
>request?
>
>>>>>Ericw

--
Wysłane za pomocą K-9 Mail.

[Michael DeHaan]

"Changing the setting to random and the fork value to 40 gives me a reasonable chance that all 40 forks will be more or less equally distributed.  With more than 40 blades, it's a good bet only one fork per blade"

So this seems to imply you have more than one VM running on a given blade (and not that you are talking about a blade chassis).   

Ok I don't think it's appropriate for ansible to resolve to random chance to avoid too much activity on a device, and I'm not sure what the best way to express that in your configuration would be.

I'm interested in the discussion but don't believe we're ready for a feature proposal just yet - what you propose could still fail - what you really seem to want is a way to say "only target one of these set of machines at a time" inside some sort of micro groups.

Honestly, I'm really curious why your blade can't handle the load and should explore that first.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/fdbc7eec-8b19-4d02-a0bf-f519950e2583%40email.android.com.

[Eric Wedaa]

We can have multiple VMs on a blade, for instance MANY guests on an esxi blade, or multiple LPARs on a p-blade. Each of these VMs don't consume a lot of CPU/Networking. 

We have tested 100 VMs on a single blade successfully.  Not that they were doing a lot, but were available.  Reality is more like 5-20, depending on what they are doing.

If I run a yum update all on all of those at the same time, it will flatline my blade.

I would take ""only target one of these set of machines at a time" inside some sort of micro groups." if that was easy enough to do, but I'd even prefer "only target N of these set..." since the blades CAN take more than one yum update at a time.  For instance, I have 16 cores on some of my blades, so I'd take 8 at a time and see how that goes.

I really thought --randomize-hosts would have been easier.

>>>Ericw

[Brian Coca]

So I'v been thinking of a new play parameter for a while, sort:, which
will sort the host selection according to
inventory(default)|r_inventory|alpha|r_alpha|numeric|r_numeric|random

This would sort hosts for processing depending on this parameter.


--
Brian Coca
Stultorum infinitus est numerus
0110000101110010011001010110111000100111011101000010000001111001011011110111010100100000011100110110110101100001011100100111010000100001
Pedo mellon a minno

[Eric Wedaa]

YES!!!!!!!!!

I'll even beta test the darn thing, that's how important this is to us :-)

Of course, now I need to learn while loops...

>>>>>>Ericw