how exactly to use rsync to speed up Vagrant?

[Ian Robert Douglas]

Hi,

I use Vagrant to develop Wordpress builds locally and then get them online. I use Scotchbox. I'm seeing latency of 7-12 seconds in wp-admin, and it's driving me nuts.

I've read around the issue, and it seems that rsync can help get speeds of under 1s.

This is about all I know. I can comment out lines in a Vagrantfile and I know how to reload Vagrant. Beyond that, I know nothing.

I don't understand anything about what rsync is for where it lives or how to enable it or what the pitfalls of using it are.

I need someone to explain in baby steps what to do.

I'm really sorry. I know that much be frustrating for devs. I'm a designer. My brain just can't process code, beyond absolute basics. I try, but I'm lost.

Can someone help?

Basically I want to use rsync instead of NFS, to try to speed up Wordpress on Vagrant to a level that is bearable.

Thanks in advance for any help,

ian

[Ian Robert Douglas]

Attempting to answer my own question ………….. I commented out my previous NFS shared folder line in my Vagrantfile and added this:

    config.vm.synced_folder ".", "/var/www", type: "rsync", rsync__exclude: ".git/", rsync__auto: true

I also installed the vagrant-gatling-rsync plugin because someone in a comment on a page said it helped.

I notice that when the action happens, it seems to happen faster now. But I still have about a 7s latency.

All this is under the latest Virtualbox, with KVM paravirtualisation, SSD drive, low number of processors (apparently it’s better that way than the logic of maxing it out), etc.

Here is my waterfall for wp-admin:

You can see clearly the very long (7s) lag and then everything happens superfast.

Life would be perfect if I could get rid of the lag.

But I don’t know if it’s possible.

Anyone?

best wishes,

ian

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/5c10459b-6155-4273-aef2-557ce6117925%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ian Robert Douglas]

I still have this problem if anyone can jump in. Developing Wordpress is now painful locally. In fact, it's faster to develop from a live server! The latency on clicks is around 8-9 seconds consistently when working under Vagrant. My life would change significantly for the better if I could get it to 1 second.

To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.

[Ian Robert Douglas]

1. I started to think about latency in the host machine. Looking for "slow DNS” and OSX I find information on IPV6 and how its best to turn it off. In System Preferences / Network / Advanced / TCP/IP, you chose this and Apply.

This cut latency in Wordpress under Vagrant from 8s to 4s.

2. I then went looking for IPV6 in the VM. Executing these commands in SSH for the VM cut latency from 4s to 2.8s:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1

sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

3. I’m not sure if this change survives a reload. So I’m looking for a permanent fix. Alvaro Miranda Aguilera mentioned in a 2015 thread here a solution to this, but I don’t understand it. It involves editing /etc/sysctl.conf but I don’t know how to do that. I’ll try to Google that.

4. Changing natdns settings to “off” in my Vagrantfile seemed to help:

v.customize ["modifyvm", :id, "--natdnsproxy1", "off" ]

v.customize ["modifyvm", :id, "--natdnshostresolver1", "off" ]

5. I saw something on Github that seems related:

try adding options single-request-reopen to your /etc/resolv.conf

RH tries both ipv4 and ipv6 resolve queries over the same socket, and the ipv6 can take some time to time out. The above option makes your system open a new connection for each request.

Trying to figure out how to do that.

To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/64576a63-6c74-45bc-a4c4-9789b7dac4b1%40googlegroups.com.

[Alvaro Miranda Aguilera]

Hello

You can domething like this in a script.

#!/usr/bin/env bash

mv /etc/sysctl.conf /etc/sysctl.conf.ori

grep -v net.ipv6.conf /etc/sysctl.conf.ori | tee /etc/sysctl.conf

echo "net.ipv6.conf.all.disable_ipv6=1" | tee -a /etc/sysctl.conf

echo "net.ipv6.conf.default.disable_ipv6=1" | tee -a /etc/sysctl.conf

sysctl -p

Alvaro.

[Ian Robert Douglas]

Thank you so much for that, Alvaro.

If I understand correctly, this script disables ipv6 in the virtual machine. This helps with latency as it cuts out one thread that will ultimately fail. Turning off ipv6 on my machine has sped up normal URLs significantly.

And indeed, it does help with latency, if I’m using it correctly.

But I’m not sure how to actually use this code and have it always in force on my virtual machines.

What I have done is place your code in an .sh file and add the following to my Vagrantfile

   config.trigger.before [:up, :suspend, :halt, :destroy] do

   info "Turning off ipv6"

   run_remote  "bash /var/www/ipv6.sh"

 end

Maybe this is wrong, but it’s the only way I know how.

Anyway, Vagrant runs okay on vagrant up.

I don’t know how to verify if it turns off ipv6 on the VM or not.

Latency at this point is down to 4s, more or less. It’s not entirely consistent, but it’s better than it was.

Is this about as good as I can expect?

I want to try to enable rsync next, to see if this helps.

But to be honest, the 4s seems mostly a network delay. When action starts, it’s VERY fast.

If I could get rid of this 4s of latency I’d have a local dev environment for Wordpress that would be practically instantaneous.

best,

ian

[Alvaro Miranda Aguilera]

Hello

Check if this helps on the VM or not

mv /etc/sysctl.conf /etc/sysctl.conf.ori

grep -v net.ipv6.conf /etc/sysctl.conf.ori | tee /etc/sysctl.conf

echo "net.ipv6.conf.all.disable_ipv6=1" | tee -a /etc/sysctl.conf

echo "net.ipv6.conf.default.disable_ipv6=1" | tee -a /etc/sysctl.conf

If does then the best is put it in a script that run always:

$script = <<SCRIPT

mv /etc/sysctl.conf /etc/sysctl.conf.ori
grep -v net.ipv6.conf /etc/sysctl.conf.ori | tee /etc/sysctl.conf
echo "net.ipv6.conf.all.disable_ipv6=1" | tee -a /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6=1" | tee -a /etc/sysctl.conf

SCRIPT


  config.vm.provision "shell", , run: "always", inline: $script

Alvaro

[Ian Robert Douglas]

I take it you mean to run the following on SSH on the VM.

I tried and can run the first line with sudo.

The next three lines generate “Permisssion denied” even with sudo.

[Alvaro Miranda Aguilera]

hello

try like this

ssh into the vm

then sudo su - 

with that you became root

my command to connect to the vm was

vagrant ssh -c 'sudo su -'  

if you run that you log in and became root.

so the commands should work

but to on safe side you can run the commands like this

sudo mv /etc/sysctl.conf /etc/sysctl.conf.ori

grep -v net.ipv6.conf /etc/sysctl.conf.ori | sudo tee /etc/sysctl.conf

echo "net.ipv6.conf.all.disable_ipv6=1" | sudo tee -a /etc/sysctl.conf

echo "net.ipv6.conf.default.disable_ipv6=1" | sudo tee -a /etc/sysctl.conf

Alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/1E81FE7E-70C5-4A00-9495-8AD832E19FCB%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Alvaro

[Torben Knerr]

Hi Robert,

interesting thread on improving vagrant performance in general! +1

On you sudo problem: I assume you tried 

sudo e​

cho "net.ipv6.conf.all.disable_ipv6=1" | tee -a /etc/sysctl.conf

Where it should be:

echo "net.ipv6.conf.all.disable_ipv6=1" |

​ sudo​

tee -a /etc/sysctl.conf

Does that work?

Cheers,

Torben

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/1E81FE7E-70C5-4A00-9495-8AD832E19FCB%40gmail.com.

[Ian Robert Douglas]

Thank you so much, Alvaro.

That worked. At least didn’t generate errors (except the first command, which had worked earlier, so there was nothing to remove).

In terms of performance, I don’t see any improvement, so perhaps this is simply repeating something already in effect.

Perhaps my cowboy script based on your earlier code had worked. Or perhaps the VM just doesn’t try to use ipv6 as it’s disabled on the host anyway.

In other news, I really tried to figure out how to enable rsync instead of NFS, and in particular auto-rsync, but failed so far.

So the best I can get is between 2s and 7s latency in Wordpress wp-admin.

I think there’s nothing else to try at this point. It’s just the best I can manage.

[Ian Robert Douglas]

On Jul 25, 2017, at 3:48 PM, Torben Knerr <torben...@gmail.com> wrote:

On you sudo problem: I assume you tried 

sudo e​

cho "net.ipv6.conf.all.disable_ipv6=1" | tee -a /etc/sysctl.conf

Where it should be:

echo "net.ipv6.conf.all.disable_ipv6=1" |

​ sudo​

 tee -a /etc/sysctl.conf

Does that work?

:) Thank you Torben!

Yes, I had tried the first. I had no clue where to put sudo.

I’ll hold off trying the second, because Alvaro’s other suggestion worked and if I mess it up somehow I’ll never be able to fix it! But I’m sure your code is correct!

best,

ian