Result of 1 year of work improving free VPN for Linux users

[VPN user]

To give back to the team ...

This efficient VPN process, which took about a year to come up with,
is mostly thanks to help from *Marek Novotny* although others pitched
in such as William Unruh, Aitch, Bit Twister, J.O. Aho, Eef Hartman,
JG Miller & some whom I may not remember, mainly from the two Usenet
newsgroups "alt.os.linux" & "alt.os.linux.ubuntu".

These vpngate reviews tell you about the free service:
https://www.bestvpn.com/blog/9987/vpn-gate-review
https://vpncreative.net/2013/06/11/vpn-gate
https://vpncritic.com/vpn-gate

Here's a youtube video about the vpngate service:
https://www.youtube.com/watch?v=Ydqtk-fFmv8

See also source code at:
https://github.com/marek-novotny/vpntools
https://github.com/marek-novotny?tab=repositories
(Do we have a definitive list of related tools?)

Here's a 10-step summary of the current process, honed over a year's time:

1. Download the latest vpngate ovpn files (bear in mind they go stale over time):
$ mkdir 20160304
$ cd 20160304
$ vpnget.sh
Where the operative line is:
wget --wait=10 --random-wait -r -I /en,/common -A do_openvpn.aspx,openvpn_download.aspx\* -nc -nd http://www.vpngate.net/en/ -e robots=off --user-agent "$USERAGENT"

Note: You can get a list of user-agent strings here:
http://www.krazyworks.com/wp-content/uploads/2009/06/user_agents.zip

Typical file names:
openvpn_download.aspx?sid=1457115491218&tcp=1&host=31.133.92.97&port=995&hid=3909726&%2Fvpngate_31.133.92.97_tcp_995.ovpn
openvpn_download.aspx?sid=1457115491218&udp=1&host=106.0.176.61&port=1194&hid=460215&%2Fvpngate_106.0.176.61_udp_1194.ovpn
openvpn_download.aspx?sid=1457115491218&udp=1&host=fyredeath.opengw.net&port=1194&hid=3677786&%2Fvpngate_fyredeath.opengw.net_udp_1194.ovpn
openvpn_download.aspx?sid=1457115491218&tcp=1&host=windflower.opengw.net&port=443&hid=803929&%2Fvpngate_windflower.opengw.net_tcp_443.ovpn

Typical statistics:
$ ls|wc -l
340 <== 340 ovpn files were downloaded

2. Remove any useless HTML files that *sometimes* contaminate this download:
$ grep DOCTYPE * | wc -l
9 <== for some reason, 9 files are HTML files ending with the ovpn extension

$ grep ^remote * | wc -l
329 <== 329 files are probably ovpn files

Note that 2 files are neither obviously HTML nor obviously openvpn config files.

Get rid of the HTML files:
$ rm $(grep DOCTYPE *.ovpn | awk -F: '{print $1}')
$ ls|wc -l
330 <== Now you have 330 ovpn files.

3. Remove special characters by greatly shortening (truncating) the file names:
$ vpncut.sh
Where the operative line is:
for i in *.ovpn;do mv $i $(echo $i|cut -d'F' -f2);done

The result is file names of the form:
vpngate_31.133.92.97_tcp_995.ovpn
vpngate_106.0.176.61_tcp_992.ovpn
vpngate_fyredeath.opengw.net_udp_1194.ovpn

$ ls|wc -l
330

4. Batch test the ovpn files kicking out the bad ones & renaming the good ones:
$ kickorkeep.sh
Where there are too many operative lines to list here but the final line is.
mv $x ${VPN_WINNERS}${VPN_PREFIX}_${GEO_COUNTRY}_${GEO_STATE}_${GEO_LONGSTATE}_${GEO_CITY}_${IP}-${SERVER}_${PROTOCOL}${PORT}_${DATE}.ovpn

Note: These files go bad & then good again over time, so always save the
bad ones for use again later.

Typical vpn_winners renamed file names will be of the form:
$ ls vpn_winners
vpngate_UA_07_Kharkivs-kaOblast-_Kharkiv_31.133.92.97-31.133.92.97_tcp995_20160304.ovpn
vpngate_TH_40_KrungThep_Bangkok_106.0.176.61-106.0.176.61_tcp992_20160304.ovpn
vpngate_US_NY_NewYork_Buffalo_198.23.197.184-fyredeath.opengw.net_udp1194_20160304.ovpn

At the end of the kickorkeep process, you'll have something like:
$ ls vpn_winners | wc -l
171

$ ls vpn_losers | wc -l
158

https://github.com/marek-novotny/vpntools

5. Clean up the winners directory by putting files into folders by country:
$ vpnmove.sh
Where the operative lines are:
for FNAME in $(ls *.ovpn) ; do
COUNTRY=$(echo $FNAME|awk -F_ '{print $2}')
mkdir -p $COUNTRY
echo "Moving $FNAME into $COUNTRY subfolder"
mv $FNAME $COUNTRY
done

At the end of this cleanup, you might have these folders:
$ ls
AR BY DE FR GB HK IP IS JP KR MM RO RU TH TR UA US VE VN

$ for i in *;do ls *i|wc -l;done
AR BY DE FR GB HK IP IS JP KR MM RO RU TH TR UA US VE VN
4 2 6 3 2 7 1 4 47 37 2 2 7 5 4 7 24 3 4

Note: Folder "IP" is for those files which had geolookup errors of IP not found.
Note: If you use gmail, it's a good idea to keep geographically similar files
together because Google hates when you change IP addresses often.

6. Run vpn but continue to kickorkeep ovpn files based on whether they still work:
$ vpnrun.sh
Where there are many operative lines, but the command to run VPN is:
sudo openvpn --config "$x" --script-security 2 --up /etc/openvpn/update-resolv-conf --down /etc/openvpn/update-resolv-conf
Note: These files go bad and then go good again over time, so save the bad ones.
Note: Without those additional lines, you *will* have DNS leaks.

7. In another window, keep watch over sensitive apps to immediately kill:
$ vpnwatch.sh
Where there are too many operative lines but the key operative line is:
kill -9 $procID

8. Disable apps that shouldn't run when you're NOT on your home IP address:
$ tbird.sh <== you can make this script the starting script for your app.
Where the operative line is:
approvedIPs=($(cat $HOME/.tbird))

https://raw.githubusercontent.com/marek-novotny/tbird/master/tbirdb.sh

9. Enable apps that run ONLY when you're NOT on your home IP address:
$ mlaunch.sh <== you can make this script the starting script for your app.
Where the operative line is:
approvedIPs=($(cat $HOME/.mlaunch))

10. Reflect upon what you have accomplished!
a. You've downloaded vpngate openvpn confuration files en masse
b. You've batch tested & batch renamed them by their test results
c. You've started vpn and allowed further testing (as their status changes)
d. You've disabled apps that you don't want running when you are on VPN
e. You've enabled apps that you want to only run when you are on VPN
f. And you put in place a watchdog process that kills apps if VPN drops

Many thanks to Marek Novotny, William Unruh, Aitch, Bit Twister J.O. Aho,
Eef Hartman, & JG Miller & others for help to get past stumbling blocks.

All the scripts have been previously posted to a.o.l or a.o.l.u but if you
want any particular script, please check Marek's repository (or just ask).

See also this script (which I'm not sure whether it's useful or not):
https://gist.github.com/Lazza/bbc15561b65c16db8ca8

[/less]

On 03/05/2016 09:43 AM, VPN user wrote:
> To give back to the team ...
> This efficient VPN process, which took about a year to come up with,

[cut]

"Why should someone switch to Linux?"
"Specifically why?"
"Security?"
SIMPLICITY???? --- ridiculous!

[Carlos E. R.]

On 2016-03-05 06:01, /less wrote:

>
> "Why should someone switch to Linux?"
> "Specifically why?"
> "Security?"
> SIMPLICITY???? --- ridiculous!

Don't feed the troll, anyone, please.

--
Cheers,
Carlos E.R.

[Dirk T. Verbeek]

Op 05-03-16 om 03:43 schreef VPN user:

<SNIP>

> 10. Reflect upon what you have accomplished!
> a. You've downloaded vpngate openvpn confuration files en masse
> b. You've batch tested & batch renamed them by their test results
> c. You've started vpn and allowed further testing (as their status changes)
> d. You've disabled apps that you don't want running when you are on VPN
> e. You've enabled apps that you want to only run when you are on VPN
> f. And you put in place a watchdog process that kills apps if VPN drops
>
> Many thanks to Marek Novotny, William Unruh, Aitch, Bit Twister J.O. Aho,
> Eef Hartman, & JG Miller & others for help to get past stumbling blocks.
>
> All the scripts have been previously posted to a.o.l or a.o.l.u but if you
> want any particular script, please check Marek's repository (or just ask).
>
> See also this script (which I'm not sure whether it's useful or not):
> https://gist.github.com/Lazza/bbc15561b65c16db8ca8

Nice, the power of open source and cooperation at work via a
non-moderated newsgroup!

The success can simply be measure by the jealous reaction of a new troll :)

This is not just about security, it is also about being in control of
your own equipment and in the end your life, a 'burden' MS etc. would
love to take away :)

[VPN user]

On Sat, 05 Mar 2016 11:42:55 +0100, Dirk T. Verbeek wrote:

> Nice, the power of open source and cooperation at work via a
> non-moderated newsgroup!

It has been fantastic the help and collaboration from all of you!

This quest started in January of 2015 (actually Christmas 2014) and
has progressed steadily ever since, with you wonderful guys getting me
past the stumbling blocks, one by one, surmounting them all!

I'm not a good coder but I had the burning desire to come up with an
efficient mechanism for myself and for anyone to use a free public
VPN efficiently & freely on Linux - and you all made it happen!

It goes without repeating that Marek helped everywhere, except when
(heaven forbid!) he got sick suddenly on us in the middle of the
dns-leaks crusade! (Glad to see that he healed quickly & is better!).

Together, we initially came up with lousy free VPN solutions such as
VpnOneClick, SecurityKiss, & ThreatSpike and later we came up with
more just-as-lousy free VPN solutions such as FreeVpnMe, VPN.me,
and VpnBook where, in those early days, I had to edify most of you
that there was the concept of a free VPN public server solution
(e.g., Bahnhof) which didn't require you to roll-your-own VPN-at-home
server! (since you all initially thought of only one kind of VPN).

Then we struggled between the various *types* of VPN to choose,
where Melzzzzz & mrr &

I don't remember who hit upon first the mofolinux mirror and then
the vpngate web site, but it was Marek for sure who taught me how
to disregard all the super-confusing "softEther" talk in the vpngate
site to just download an ovpn config file to execute simply with the
"sudo openvpn --config file.ovpn" command!

That was the first hurdle breakthrough, little as it seems now!

In those early days, we were downloading the openvpn configuration files
manually, and then we were downloading *everything* and then deleting
the garbage until both J.O. Aho & Aitch showed us how to obtain just
the ovpn files of interest.

The downloaded ovpn file names contained special characters and were
too meaninglessly long until Marrgol, Marek Novotny, & William Unruh
all suggested different ways of cutting the file names gracefully.

Eef Hartman and others helped tremendously in pointing us in the right
direction for the command-line geotagging solution, as did Mike Easter
and John Hasler for the command-line speedtest solution.

I keep having to thank Marek for writing the safety solutions, i.e.,
the scripts to prevent certain programs from running at the same time
as the scripts to only allow others to run while on VPN, along with
the critical safety valve of killing everything sensitive when the
vpn suddenly drops (which triggers about once a week or so).

There were some frustrations, for example when Caver1 & Kirk_Von_Rockstein
tried to help on the dns leaks but it was essentially J. G. Miller
who made the tests which proved *why* the DNS leaks were occurring
and *how* to fix them (which, in the end, turned out to be very easy
to resolve once we got past the initial frustration of dns-leak semantics).

Wildman helped us with the command-line IP address resolution
using inxi as did Chris Alstrom with his lynx example and
Jason Betts with wget and Bit Twister suggested a few more
curl examples (uncharacteristically, even "I" came up with a
new method, which was simply to use "route -n" with grep") and
Richard Kettlewell helped answer some of the lingering questions.

And, of course, Marek strung it all together into seamless batch
scripts in almost every step of the way, but where "vpnkok" (aka
vpnkickorkeep) was the fundamental method of navigation among
a sea of config files filled with bad & good files all mixed up.

There were, of course, a few dead ends (such as the wrong approach
of killing the wifi NIC instead of just killing the sensitive jobs)
where we messed with the sudoers file and the network manager where
Pascal Hambourg, David W. Hodgins & Lew Pitcher all pitched in with
helpful advice.

Even lil' ole me, who shall remain anonymous, wrote a few of the
(ugly) minor scripts such as the country-code movers and testers
but mostly I was the main *beneficiary* of all your wonderful help
and advice!

Thanks!

[VPN user]

On Sat, 05 Mar 2016 18:55:41 +0000, VPN user wrote:

> Even lil' ole me, who shall remain anonymous, wrote a few of the
> (ugly) minor scripts such as the country-code movers and testers
> but mostly I was the main *beneficiary* of all your wonderful help
> and advice!

After running vpnkok (kick-or-keep) on a few thousand ovpn files from
vpngate this morning, we get roughly 1/3 good & 2/3 bad (bearing in
mind some of these files are up to a few weeks old by now):

$ ls vpn_winners/* | wc -l (& removing the 40 directories from the count)
677
$ ls vpn_losers/* | wc -l
1201

While we have only obtained about half of the four thousand existing
server files on vpngate (they don't publish them all at once so as
to foil censors), of the good files we do have, the set of ISO-3166
countries involved are currently the following 40 countries:

$ ls vpn_winners
AE BR CA DE EG FR HK IS KR MM MX NZ PL QA RU SE TR TW US VN
AR BY CN DO ES GB ID IQ JP MA MO NL PH PT RO SA TH TT UA VE

With the numbers of files skewed greatly toward Japan, Korea, & USA:
(AE 5) (AR 5) (BR 3) (BY 2) (CA 13) (CN 2) (DE 13) (DO 1) (EG 2)
(ES 2) (FR 19) (GB 11) (HK 13) (ID 6) (IQ 1) (IS 4) (JP 136)
(KR 228) (MA 1) (MM 2) (MO 1) (MX 2) (NL 4) (NZ 4) (PH 2) (PL 3)
(PT 3) (QA 1) (RO 9) (RU 20) (SA 1) (SE 3) (TH 15) (TR 11) (TT 1)
(TW 1) (UA 7) (US 67) (VE 37) (VN 14)

As reported by the code below that I wrote (yes, I know it's f'ugly):
#!/bin/bash
# vpnmove.sh moves renamed VPN files into iso 3166 country folders
# AE BR CA CN DE DO FR HK IP JP KR NL PL QA RO RU SA SE TH TR US VE VN
# Also reports the number of ovpn files per country.
# Note that there are three types of output to geoiplookup:
# $ geoiplookup -f /usr/share/GeoIP/GeoLiteCity.dat 8.8.8.8
# GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94040, 37.384499, -122.088097, 807, 650
# $ geoiplookup -f /usr/share/GeoIP/GeoLiteCity.dat 0.8.8.8
# GeoIP City Edition, Rev 1: IP Address not found
# $ geoiplookup -f /usr/share/GeoIP/GeoLiteCity.dat 0.0.0.0
# GeoIP City Edition, Rev 1: can't resolve hostname ( 0.0.0.0 )
# So a country code of "IP" or "ca" are due to geoiplookup errors!
##
# Move ovpn files into country-code folders:

for FNAME in $(ls *.ovpn) ; do
COUNTRY=$(echo $FNAME|awk -F_ '{print $2}')
mkdir -p $COUNTRY
echo "Moving $FNAME into $COUNTRY subfolder"
mv $FNAME $COUNTRY
done

# Count the number of ovpn files in each country-code folder:
for COUNTRY_CODE in *;do
NUMBER=$(ls $COUNTRY_CODE|wc -l)
echo $COUNTRY_CODE $NUMBER
done

exit 0

## Lookup:
# !/bin/bash
# country.sh will output the country for a given country code
# Use: $ country US
# https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
# COUNTRY_FILE=/usr/local/bin/iso3166.txt
# egrep -i ^${1} ${COUNTRY_FILE}

[Marek Novotny]

On 2016-03-05, VPN user <vpn...@example.com> wrote:

I've updated the vpnstatus script to make it simpler. It now executes an
ovpn config and checks status all rolled into one smaller, simpler
script.

While the app is running you can have set apps that cannot run. And of
the vpn tunnel fails a set group of apps will terminate as always.

You no longer have to setup a vpn connection in one terminal and then
run status in another. It's just one screen. So you may wish to modify
my launch method for the vpn.

It should be much easier to read and understand and kick out better
status as you run it. It will not send you a message every few seconds
to tell you it is running. It will tell you if the sessions disconnects
and it will terminate everything including the session fairly instantly
should that happen.

https://github.com/marek-novotny/vpntools

--
Marek Novotny
https://github.com/marek-novotny

[VPN user]

On Sat, 05 Mar 2016 11:23:57 -0800, Marek Novotny wrote:

> I've updated the vpnstatus script to make it simpler. It now executes an
> ovpn config and checks status all rolled into one smaller, simpler
> script.
>
> While the app is running you can have set apps that cannot run. And of
> the vpn tunnel fails a set group of apps will terminate as always.
>
> You no longer have to setup a vpn connection in one terminal and then
> run status in another. It's just one screen. So you may wish to modify
> my launch method for the vpn.
>
> It should be much easier to read and understand and kick out better
> status as you run it. It will not send you a message every few seconds
> to tell you it is running. It will tell you if the sessions disconnects
> and it will terminate everything including the session fairly instantly
> should that happen.
>
> https://github.com/marek-novotny/vpntools

Just saw this! Thanks Marek. I will test today or tomorrow (I have to
go to a wedding pretty soon).

It would be nice NOT to have to have two windows running (although you
just reminded me of *another* breakthrough which was when the a.o.l
team helped me run multiple tabs in the konsole, each with a different
color and icon, one for running what I call 'vpnrun' and the other for
running what I call 'vpnwatch' (which are, respectively your kickorkeep
and vpnstatus scripts, modified for my needs).

So, for more thanks, Eef Hartman helped me understand how to adjust
the default profile ($HOME/.kde/share/apps/konsole) while William Unruh
showed me manually how to set konsole > settings > Edit Current Profile
> Appearance > Edit, etc. so that I could keep track (by color) of
which window was running vpn and which window was watching processes.

In that thread J.G. Miller and J.O. Aho also helped, while Bit Twister
suggested alternatives using xterm and even you, Marek, suggested both
tmux and screen.

The whole point was to keep the vpn actions in order, to keep one window
running the vpn (which disables use of that window) and another watching
over the processes (which also disables use of that second window),
while subsequent windows were for performing actions while on VPN.

That you proactively recognize this as wasteful is wonderfully astute
of you, although with the help above, I have it down to a science where
the "red" window is for running VPN while the "yellow" window is for
watching over processes, and only the subsequent "green" windows are
for performing tasks while on VPN.

I must run to the wedding (I don't even "fit" in my suits anymore!)
so I will test at my first opportunity and let you know the results,
as always!

[VPN user]

On Sat, 05 Mar 2016 19:46:05 +0000, VPN user wrote:

> That you proactively recognize this as wasteful is wonderfully astute
> of you, although with the help above, I have it down to a science where
> the "red" window is for running VPN while the "yellow" window is for
> watching over processes, and only the subsequent "green" windows are
> for performing tasks while on VPN.

And again, by way of thanks, somewhere in the unsearchable a.o.l.u archives
is the help from all you great guys on the team for changing and saving the
modified icons for each type of vpn-related konsole window so that when
the windows are backgrounded, one can tell on the desktop *which* one is which.

NOTE: It's sad that google groups search doesn't work for the ubuntu
sibling of this a.o.l Usenet newsgroup.

[/less]

On 03/05/2016 05:42 PM, Dirk T. Verbeek wrote:
> Op 05-03-16 om 03:43 schreef VPN user:

[cut]

> The success can simply be measure by the jealous reaction of a new troll :)

Jealousy? Your response is so typical...

Remarking on the obviously stated embrangled "improved free VPN" is a
far cry from being envious.

This entire procedure for applying the "improved free VPN for Linux
users" is way too complicated for an average Linux user.

I am a Linux user and use an 'one-click-working' paid version of VPN.
It's inexpensive, most reliable and I can afford it!

Heck, as there are virtually no uncomplicated VPN's available for Linux
and since the esteemed authors of this complexified VPN application seem
to have all the time in world why don't they putting in the effort for
developing a simplified version which is overdue by a long shot and
would be most appreciated by the 'ordinary' Linux user. But, alas, why
make it simple if there is an complex way... I am 'moved'!

[VPN user]

On Sat, 05 Mar 2016 11:23:57 -0800, Marek Novotny wrote:

> I've updated the vpnstatus script to make it simpler.

Thanks Marek for the updated vpnstatus.sh script!

I had to go to a wedding but I tested it as soon as I returned and
it worked nicely. I agree the new vpnstatus (which I'll call "vpnrun")
is simpler in that it does two jobs (running vpn & watching over apps)
in one window.

Here are my results where the only unexpected result was that
vpnstatus didn't kill chromium (maybe I have the syntax wrong?)

Here's a line-by-line log of what happened in my first test so far:

1. Start a bunch of apps to see what happens:
$ chromium-browser &
$ firefox &
$ start-tor-browser &
$ pan &
$ transmission &

2. Edit the new vpnstatus.sh to blacklist & whitelist apps:
$ sudo vi $(which vpnstatus.sh)
Change from:
# apps allowed to run under vpn. These terminate if vpn fails...
vpnApps=(transmission)
Change to:
vpnApps=(transmission pan firefox)

Change from:
# apps not allowed to run when vpn is up.
# If launched or running these will terminate when the vpn is up.
nonVpnApps=(thunderbird)
Change to:
nonVpnApps=(thunderbird chromium-browser)

3. Start the new vpnstatus.sh on a known-good VPN config file:
$ vpnstatus.sh vpn_AU_Victoria_Port-Melbourne_124.188.36.71.ovpn
vpnstatus.sh Status: vpn_AU_Victoria_Port-Melbourne_124.188.36.71.ovpn accepted!
vpnstatus.sh Status: sudo validation.
vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection
vpnstatus.sh Status: OpenVPN Process ID: 4178.
#####
vpnstatus.sh Status: Obtained VPN: tun0.
vpnstatus.sh Status: Task thunderbird is running...
vpnstatus.sh Status: Task thunderbird has been terminated.

4. Interestingly, notice Thunderbird died, but the Chromium-browser remained.
$ ps -elf |grep -i chrome
4 S vpnuser 19505 19496 0 80 0 - 1624 wait 19:56 ? 00:00:00 /usr/lib/chromium-browser/chrome-sandbox /usr/lib/chromium-browser/chromium-browser --type=zygote

5. See what happens if I try to *start* Thunderbird:
$ thunderbird &
[1] 12336
[1]+ Killed thunderbird

6. That makes sense that Thunderbird isn't allowed to start! :)

7. Now kill the vpn and see what happens:
^C (in the vpnstatus window)

vpnstatus.sh Status: Task transmission is running...
vpnstatus.sh Status: Task transmission has been terminated.
vpnstatus.sh Status: Task pan is running...
vpnstatus.sh Status: Task pan has been terminated.
vpnstatus.sh Status: Task firefox is running...
vpnstatus.sh Status: Task firefox has been terminated.
vpnstatus.sh Status: BREAK!
vpnstatus.sh Status: OpenVPN PID: 4178 terminated...
vpnstatus.sh Status: Device ID - wlan0
vpnstatus.sh Status: vpnstatus.sh Terminated...

8. Immediately, Transmission, Pan, Firefox and the Tor Browser Bundle died.
Notice that the Tor Browser Bundle dies with 'firefox'! <=== interesting

9. The Chromium-browser was the only thing left running:
$ ps -elf|grep -i chrome
4 S vpnuser 19505 19496 0 80 0 - 1624 wait 19:56 ? 00:00:00 /usr/lib/chromium-browser/chrome-sandbox /usr/lib/chromium-browser/chromium-browser --type=zygote

In summary:
a. The good news is that only one konsole window is occupied which does
both the starting of the VPN file and the blacklisting & whitelisting
of apps.

b. The only bad news is that I have to figure out how to get the syntax
for the chromium browser to die.

That's only the first test so there will be more, but I figured I'd
let you know right away as you might be waiting for results.

Thanks for helping everyone whenever you help me!

[Marek Novotny]

Okay, I think I know what the issue would be with Chrome. Working on my
NFS right now. Then I'll come back to this once install Chrome so I can
test a solution. Chrome likely has several tasks running. So I'll have
to write it in to kill the tree.

I'll come back to it.

Also, updated kickorkeep as well. It also should trap a ctrl-c as you
can see vpnstatus now does so you can quit the loops and get your
network back without having to figure out PIDs or know how to use pkill.

[VPN user]

On Sat, 05 Mar 2016 20:29:49 -0800, Marek Novotny wrote:

> Okay, I think I know what the issue would be with Chrome. Working on my
> NFS right now. Then I'll come back to this once install Chrome so I can
> test a solution. Chrome likely has several tasks running. So I'll have
> to write it in to kill the tree.
>
> I'll come back to it.
>
> Also, updated kickorkeep as well. It also should trap a ctrl-c as you
> can see vpnstatus now does so you can quit the loops and get your
> network back without having to figure out PIDs or know how to use pkill.

Hi Marek,

I just saw your note above as I came back to report what else I changed
in the vpnstatus.sh script that I originally downloaded (version 3.0).

So I will test the new version after I post this report on one other change
that people need to make to vpnstatus.sh in order to eliminate dns leaks.

1. When one installs openvpn using "app get", automatically a directory is
created and populated containing "/etc/openvpn/update-resolv-conf".

2. Unless you want dns leaks, you generally want to run the openvpn command
not as "sudo openvpn --config file.ovpn" but as the command below.

3. So I changed this line in vpnstatus.sh:
$priv openvpn --config $configFile &> /dev/null &

To this line (which eliminates dns leaks):
$priv openvpn --config $configFile --script-security 2 --up /etc/openvpn/update-resolv-conf --down /etc/openvpn/update-resolv-conf &> /dev/null &

I realize you might not know if your users will have the /etc/openvpn directory
(which they will have if they used apt-get to install openvpn but maybe not if
they compiled openvpn themselves), so you might just want to include this line
in your script, but perhaps commented out.

Just a suggestion to help others.

PS: I see you eliminated the ubuntu xpost so I will stay on a.o.l for replies.

[Marek Novotny]

On 2016-03-06, VPN user <vpn...@example.com> wrote:

> On Sat, 05 Mar 2016 20:29:49 -0800, Marek Novotny wrote:
>
>> Okay, I think I know what the issue would be with Chrome. Working on my
>> NFS right now. Then I'll come back to this once install Chrome so I can
>> test a solution. Chrome likely has several tasks running. So I'll have
>> to write it in to kill the tree.
>>
>> I'll come back to it.
>>
>> Also, updated kickorkeep as well. It also should trap a ctrl-c as you
>> can see vpnstatus now does so you can quit the loops and get your
>> network back without having to figure out PIDs or know how to use pkill.
>
> Hi Marek,
>
> I just saw your note above as I came back to report what else I changed
> in the vpnstatus.sh script that I originally downloaded (version 3.0).
>
> So I will test the new version after I post this report on one other change
> that people need to make to vpnstatus.sh in order to eliminate dns leaks.
>
> 1. When one installs openvpn using "app get", automatically a directory is
> created and populated containing "/etc/openvpn/update-resolv-conf".
>
> 2. Unless you want dns leaks, you generally want to run the openvpn command
> not as "sudo openvpn --config file.ovpn" but as the command below.
>
> 3. So I changed this line in vpnstatus.sh:
> $priv openvpn --config $configFile &> /dev/null &
>
> To this line (which eliminates dns leaks):
> $priv openvpn --config $configFile --script-security 2 --up /etc/openvpn/update-resolv-conf --down /etc/openvpn/update-resolv-conf &> /dev/null &

I think this is something with your vpn provider. I don't see leaks with
mine so I have not done that. I'll leave that up to you and those who'd
modify the script for their needs.

> I realize you might not know if your users will have the /etc/openvpn directory
> (which they will have if they used apt-get to install openvpn but maybe not if
> they compiled openvpn themselves), so you might just want to include this line
> in your script, but perhaps commented out.
>
> Just a suggestion to help others.
>
> PS: I see you eliminated the ubuntu xpost so I will stay on a.o.l for replies.

I tested pkill for chrome and it works fine for me.

Are you running chrome or chromium? Maybe you have it not typed in
correctly in the vpnApps. Seems to be fine here even with lots of tabs.

[Dirk T. Verbeek]

Op 06-03-16 om 03:44 schreef /less:

> On 03/05/2016 05:42 PM, Dirk T. Verbeek wrote:
>> Op 05-03-16 om 03:43 schreef VPN user:
>
> [cut]
>
>> The success can simply be measure by the jealous reaction of a new
>> troll :)
>
> Jealousy? Your response is so typical...
>
> Remarking on the obviously stated embrangled "improved free VPN" is a
> far cry from being envious.
>
> This entire procedure for applying the "improved free VPN for Linux
> users" is way too complicated for an average Linux user.

It would be even more complicated for an avergage Windows or Mac user.

>
> I am a Linux user and use an 'one-click-working' paid version of VPN.
> It's inexpensive, most reliable and I can afford it!

But how do you think it was developed?

That's exactly what you saw happening here except this product is
available for all and it's gratis.

>
> Heck, as there are virtually no uncomplicated VPN's available for Linux
> and since the esteemed authors of this complexified VPN application seem
> to have all the time in world why don't they putting in the effort for
> developing a simplified version which is overdue by a long shot and
> would be most appreciated by the 'ordinary' Linux user. But, alas, why
> make it simple if there is an complex way... I am 'moved'!

This product does make the whole affair much less complicated.
And it's likely better than your paid service as it has a very large
pool of ever changing IP addresses, something you need once outside that
comfortable world of democratic countries.

[Richard Kettlewell]

"Dirk T. Verbeek" <dver...@xs4all.nl> writes:
> This product does make the whole affair much less complicated.
> And it's likely better than your paid service as it has a very large
> pool of ever changing IP addresses, something you need once outside
> that comfortable world of democratic countries.

What’s the use case here? If you’re just trying to frustrate tracking
by advertizers and whatnot then whatever, but if you’re trying to hide
from state actors then:

| wget [...] http://www.vpngate.net/en/ [...]

...downloads the configuration files over a completely unauthenticated
channel. Even the feeblest of tinpot dictatorships will have no trouble
at all in compromising this.

--
http://www.greenend.org.uk/rjk/

[VPN user]

On Sun, 06 Mar 2016 09:54:41 +0100, Dirk T. Verbeek wrote:

> This product does make the whole affair much less complicated.
> And it's likely better than your paid service as it has a very large
> pool of ever changing IP addresses, something you need once outside that
> comfortable world of democratic countries.

I will back up this observation, although I don't know the payware service
that is being discussed.

It seemed that when I used the trialware servers from VpnOneClick,
SecurityKiss, ThreatSpike, FreeVpnMe, VPN.me, and VpnBook, they were often
blocked by certain web sites, which *knew* they were VPNs (and which may
have been spammed by those IP addresses).

However, almost never, if ever, has one of the vpngate nodes been blocked
so, that's another advantage of an ever-changing pool of VPN IP addresses
(expressly designed to circumvent state censorship).

BTW, I have a few thousand openvpn files, and one on by one, vpnkickorkeep
is going through them, so here are the latest statistics which still show
about 1/3 good to 2/3 bad (although many files are weeks old):

$ ls vpn_losers/ | wc -l
1962

$ ls vpn_winners/* | wc -l

907

$ ls vpn_winners
AE AU BY CN DO ES FR HK IE IQ IT KR MM MX NZ PL QA RU SE TR TW US VN
AR BR CA DE EG FI GB ID IP IS JP MA MO NL PH PT RO SA TH TT UA VE

With the number of ovpn files reverse sorted for each country listed as shown below:
258 KR (korea)
159 JP (japan)
93 US (usa)
40 VE (venezuela)
26 FR (france)
26 RU (russia)
19 DE (germany)
19 TH (thailand)
16 VN (viet nam)
15 CA (canada)
14 GB (uk)
13 HK (hong kong)
13 TR (turkey)
12 RO (romania)
9 NL (netherlands)
8 ID (indonesia)
8 UA (ukraine)
7 AE (uae)
6 PT (portugal)
5 AR (argentina)
5 BR (brazil)
5 NZ (new zealand)
4 IS (iceland)
3 PL (poland)
3 SE (sweden)
2 BY (belarus)
2 CN (china)
2 EG (egypt)
2 ES (spain)
2 IQ (iraq)
2 MA (moracco)
2 MM (myanmar)
2 MO (macao)
2 MX (mexico)
2 PH (philippines)
2 TW (taiwan)
1 AU (australia)
1 DO (dominican republic)
1 FI (finland)
1 IE (ireland)
1 IT (italy)
1 QA (qatar)
1 SA (saudi arabia)
1 TT (trinidad and tobago)

[VPN user]

On Sun, 06 Mar 2016 11:09:22 +0000, Richard Kettlewell wrote:

> What’s the use case here? If you’re just trying to frustrate tracking
> by advertizers and whatnot then whatever, but if you’re trying to hide
> from state actors then:

For "me", the use case is privacy, and for *that* purpose, vpngate works
just fine (especially since the vpngate.net servers are almost never
blocked by web sites (whereas, by way of contrast, Tor exit nodes are
often blocked).

> ...downloads the configuration files over a completely unauthenticated
> channel. Even the feeblest of tinpot dictatorships will have no trouble
> at all in compromising this.

I'm not the one to ask, but I would tend to agree with you that even
"we" could come up with a way to block these 4,000 IP addresses, so,
it should be trivial for a tin-pot dictatorship to do the same.

However, if you read what they say on the vpngate.net web page, they
*do* seem to expect users in repressive countries to use vpngate to
circumvent state-sponsored censorship.

How that works in practice I would not have a clue. We'd need someone
who lives in a (more) repressive regime than the US (where I am) to
tell us their experiences.

[VPN user]

On Sat, 05 Mar 2016 11:23:57 -0800, Marek Novotny wrote:

> I've updated the vpnstatus script to make it simpler. It now executes an
> ovpn config and checks status all rolled into one smaller, simpler
> script.

Hi Marek,

I think I may have found a bug, but I'm not sure of the expected use model
when a control-C is used to break off the vpn tunnel.

When I execute the latest vpnstatus.sh on a bad file, all works as
expected in that vpnstatus times out and returns me to the command
prompt.

$ vpnstatus.sh
vpnstatus.sh Status: vpngate_US_TX_Texas_Irving_108.206.180.66-vpn817573606.opengw.net_udp1195_20160303.ovpn accepted!

vpnstatus.sh Status: sudo validation.
vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection

vpnstatus.sh Status: OpenVPN Process ID: 15089.
#########################
vpnstatus.sh Error: vpngate_US_TX_Texas_Irving_108.206.180.66-vpn817573606.opengw.net_udp1195_20160303.ovpn hung.
$

However, if I control-C, I end up getting asked to log in as
sudo *before* I'm returned back to the command line!

$ vpnstatus.sh vpngate_US_OK_Oklahoma_Hinton_199.73.99.253-vpn264880674.opengw.net_udp1195_20160305.ovpn
vpnstatus.sh Status: vpngate_US_OK_Oklahoma_Hinton_199.73.99.253-vpn264880674.opengw.net_udp1195_20160305.ovpn accepted!

vpnstatus.sh Status: sudo validation.
vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection

vpnstatus.sh Status: OpenVPN Process ID: 7244.

#####
vpnstatus.sh Status: Obtained VPN: tun0.

^C

vpnstatus.sh Status: Task pan is running...
vpnstatus.sh Status: Task pan has been terminated.
vpnstatus.sh Status: Task firefox is running...
vpnstatus.sh Status: Task firefox has been terminated.

[sudo] password for vpnuser:

Only *after* I log in as sudo, does the rest of the lines occur
and does the prompt re-appear:
<enter password for sudo>
vpnstatus.sh Status: BREAK!
vpnstatus.sh Status: OpenVPN PID: 7244 terminated...

vpnstatus.sh Status: Device ID - wlan0
vpnstatus.sh Status: vpnstatus.sh Terminated...

$

Is that requirement for sudo before the prompt reappears built in?

[VPN user]

On Sat, 05 Mar 2016 21:35:50 -0800, Marek Novotny wrote:

> I tested pkill for chrome and it works fine for me.
>
> Are you running chrome or chromium? Maybe you have it not typed in
> correctly in the vpnApps. Seems to be fine here even with lots of tabs.

Hmmmmmmm..... Actually I just click on the icon, so, let me look...

The icon properties say: chromium-browser %U

When I run *that* from the command line, I get:

$ chromium-browser %U
$ chromium-browser %U
[26375:26540:0306/041024:ERROR:logging.h(775)] Failed to call method: org.kde.KWallet.isEnabled: object_path= /modules/kwalletd: org.freedesktop.DBus.Error.ServiceUnknown: The name org.kde.kwalletd was not provided by any .service files
[26375:26540:0306/041024:ERROR:native_backend_kwallet_x.cc(230)] Error contacting kwalletd (isEnabled)
[26375:26540:0306/041024:ERROR:logging.h(775)] Failed to call method: org.kde.KWallet.isEnabled: object_path= /modules/kwalletd: org.freedesktop.DBus.Error.NoReply: Message did not receive a reply (timeout by message bus)
[26375:26540:0306/041024:ERROR:native_backend_kwallet_x.cc(230)] Error contacting kwalletd (isEnabled)

I have no idea what those errors indicate, but, moving on, I run ps:

$ ps -auxww|grep -i chrome
vpnuser 10730 0.0 0.0 16188 948 pts/16 S+ 04:12 0:00 grep --color=auto -i chrome
vpnuser 26442 0.0 0.0 6500 396 pts/18 S+ 04:10 0:00 /usr/lib/chromium-browser/chrome-sandbox /usr/lib/chromium-browser/chromium-browser --type=zygote

So I am flummoxed, but, don't worry because I won't use Chromium when I'm
on VPN so it won't matter to me. I just wanted you to know about it.

[Richard Kettlewell]

VPN user <vpn...@example.com> writes:
> Richard Kettlewell wrote:
>> What’s the use case here? If you’re just trying to frustrate tracking
>> by advertizers and whatnot then whatever, but if you’re trying to hide
>> from state actors then:
>
> For "me", the use case is privacy, and for *that* purpose, vpngate works
> just fine (especially since the vpngate.net servers are almost never
> blocked by web sites (whereas, by way of contrast, Tor exit nodes are
> often blocked).
>
>> ...downloads the configuration files over a completely unauthenticated
>> channel. Even the feeblest of tinpot dictatorships will have no trouble
>> at all in compromising this.
>
> I'm not the one to ask, but I would tend to agree with you that even
> "we" could come up with a way to block these 4,000 IP addresses, so,
> it should be trivial for a tin-pot dictatorship to do the same.

The concern is not (just) blocking but also tampering. A user expecting
confidentiality from their government would be given a false sense of
security. In the worst case that would be a fatal mistake.

--
http://www.greenend.org.uk/rjk/

[VPN user]

Marek Novotny wrote:
> Are you running chrome or chromium? Maybe you have it not typed in
> correctly in the vpnApps. Seems to be fine here even with lots of tabs.

Hi Marek,
Here's a test I ran of the google browser (whatever it's called).
In the end, I gave up on using the darn thing. I'll just avoid it.
But here's the test result (some of which I don't understand).

1. I added "chrome-sandbox" & "chrome" to the vpn blacklist.

2. I started the Chromium Browser from the icon.
3. I also started Firefox, the Tor Browser Bundle, Thunderbird, and Transmission.

4. I grepped for the vpn whitelist and vpn blacklist:
$ grep pnApps= $(which vpnstatus.sh)
# vpnApps=(transmission)
vpnApps=(transmission pan firefox)
# nonVpnApps=(thunderbird)
nonVpnApps=(thunderbird chromium-browser chrome-sandbox chrome)

5. I then started vpnstatus.sh using the syntax below:
$ vpnstatus.sh vpngate_KR_10_Pusan-jikhalsi_Pusan_164.125.68.90-vpn164414121.opengw.net_udp1195_20160305.ovpn

6. Bad things happened because of the addition of "chrome"!
$ vpnstatus.sh vpngate_KR_10_Pusan-jikhalsi_Pusan_164.125.68.90-vpn164414121.opengw.net_udp1195_20160305.ovpn
vpnstatus.sh Status: sudo validation.
[sudo] password for vpnuser:

vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection

vpnstatus.sh Status: OpenVPN Process ID: 25890.

#####
vpnstatus.sh Status: Obtained VPN: tun0.
vpnstatus.sh Status: Task thunderbird is running...
vpnstatus.sh Status: Task thunderbird has been terminated.

vpnstatus.sh Status: Task chrome-sandbox is running...
vpnstatus.sh Status: Task chrome-sandbox has been terminated.
vpnstatus.sh Status: Task chrome is running...
vpnstatus.sh Status: Task chrome has been terminated.
... ... ... this goes on forever and will never stop ... ... ...
vpnstatus.sh Status: Task chrome-sandbox is running...
vpnstatus.sh Status: Task chrome-sandbox has been terminated.
vpnstatus.sh Status: Task chrome is running...
vpnstatus.sh Status: Task chrome has been terminated.

7. ... ... so at some point I hit the control-C to stop it ... ...
^C

vpnstatus.sh Status: Task transmission is running...
vpnstatus.sh Status: Task transmission has been terminated.

vpnstatus.sh Status: Task firefox is running...
vpnstatus.sh Status: Task firefox has been terminated.
vpnstatus.sh Status: BREAK!

vpnstatus.sh Status: OpenVPN PID: 25890 terminated...

vpnstatus.sh Status: Device ID - wlan0
vpnstatus.sh Status: vpnstatus.sh Terminated...

$

8. The interesting thing is that I can't seem to kill chrome!

$ ps -elf |grep chrome
0 S vpnuser 5193 5031 0 80 0 - 4047 pipe_w 04:56 pts/16 00:00:00 grep --color=auto chrome
4 Z vpnuser 23946 23894 0 80 0 - 0 exit 04:17 ? 00:00:00 [chrome-sandbox] <defunct>

$ killall chrome-sandbox
$ ps -elf |grep chrome
0 S vpnuser 5249 5031 0 80 0 - 4047 pipe_w 04:57 pts/16 00:00:00 grep --color=auto chrome
4 Z vpnuser 23946 23894 0 80 0 - 0 exit 04:17 ? 00:00:00 [chrome-sandbox] <defunct>

$ kill -9 23946
$ ps -elf |grep chrome
0 S vpnuser 5257 5031 0 80 0 - 4047 pipe_w 04:57 pts/16 00:00:00 grep --color=auto chrome
4 Z vpnuser 23946 23894 0 80 0 - 0 exit 04:17 ? 00:00:00 [chrome-sandbox] <defunct>

$ sudo kill -9 23946
$ ps -elf |grep chrome
0 S vpnuser 5271 5031 0 80 0 - 4047 pipe_w 04:57 pts/16 00:00:00 grep --color=auto chrome
4 Z vpnuser 23946 23894 0 80 0 - 0 exit 04:17 ? 00:00:00 [chrome-sandbox] <defunct>

9. This mysterious "chrome" thing causes a problem when I run vpnstatus.sh
$ grep pnApps= $(which vpnstatus.sh)
# vpnApps=(transmission)
vpnApps=(transmission pan firefox)
# nonVpnApps=(thunderbird)
nonVpnApps=(thunderbird chromium-browser chrome-sandbox chrome)

10. Here's what happens when I run vpn with "chrome-anything" in the blacklist:
$ vpnstatus.sh vpngate_PL_77_Malopolskie_Kraków_83.27.207.198-vpn805768177.opengw.net_1841_20160302.ovpn

vpnstatus.sh Status: vpngate_PL_77_Malopolskie_Kraków_83.27.207.198-vpn805768177.opengw.net_1841_20160302.ovpn accepted!
vpnstatus.sh Status: sudo validation.
[sudo] password for vpnuser:

vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection

vpnstatus.sh Status: OpenVPN Process ID: 6182.
############

vpnstatus.sh Status: Obtained VPN: tun0.

vpnstatus.sh Status: Task chrome-sandbox is running...
vpnstatus.sh Status: Task chrome-sandbox has been terminated.
vpnstatus.sh Status: Task chrome is running...
vpnstatus.sh Status: Task chrome has been terminated.
vpnstatus.sh Status: Task chrome-sandbox is running...
vpnstatus.sh Status: Task chrome-sandbox has been terminated.
.... this goes on forever unless I control-C out of it ....
vpnstatus.sh Status: Task chrome is running...
vpnstatus.sh Status: Task chrome has been terminated.
vpnstatus.sh Status: Task chrome-sandbox is running...
vpnstatus.sh Status: Task chrome-sandbox has been terminated.
vpnstatus.sh Status: Task chrome is running...
vpnstatus.sh Status: Task chrome has been terminated.
^C
vpnstatus.sh Status: BREAK!
vpnstatus.sh Status: OpenVPN PID: 6182 terminated...

vpnstatus.sh Status: Device ID - wlan0
vpnstatus.sh Status: vpnstatus.sh Terminated...

11. So, if I *remove* chrome anything from the blacklist, it's back to working:
$ grep pnApps= $(which vpnstatus.sh)
# vpnApps=(transmission)
vpnApps=(transmission pan firefox)
# nonVpnApps=(thunderbird)
# nonVpnApps=(thunderbird chromium-browser chrome-sandbox chrome)
nonVpnApps=(thunderbird)

12. Without chrome anything in the blacklist, all works fine:
$ vpnstatus.sh vpngate_PL_77_Malopolskie_Kraków_83.27.207.198-vpn805768177.opengw.net_1841_20160302.ovpn
vpnstatus.sh Status: vpngate_PL_77_Malopolskie_Kraków_83.27.207.198-vpn805768177.opengw.net_1841_20160302.ovpn accepted!
vpnstatus.sh Status: sudo validation.
[sudo] password for vpnuser:

vpnstatus.sh Status: Obtaining Device ID
vpnstatus.sh Status: Device ID Set: wlan0.
vpnstatus.sh Status: Obtaining VPN Connection

vpnstatus.sh Status: OpenVPN Process ID: 6922.
###########

vpnstatus.sh Status: Obtained VPN: tun0.

... which is what I'm using to type this ...

In summary, the Google browser (whatever it's called) is a PITA so I'll just
avoid using it.

[VPN user]

Richard Kettlewell wrote in message 87r3fn9...@mantic.terraraq.uk

> The concern is not (just) blocking but also tampering. A user expecting
> confidentiality from their government would be given a false sense of
> security. In the worst case that would be a fatal mistake.

I agree that, literally, the mistake can be fatal.
I don't envy anyone hiding from a state-sponsored adversary.

It's hard enough to hide from my own government, which "says" it doesn't
spy on us (but does).

But, using VPN has to be (slightly) better than not using VPN,
I guess, for those who live in repressive regimes.

[Richard Kettlewell]

Ineffective security can be *worse* than nothing - partly because of the
false sense of security (i.e. if you think that e.g. Syrian intelligence
is not spying on you, you’ll risk saying things that you wouldn’t
otherwise) and, in some cases, because it draws the attention of the
adversary.

--
http://www.greenend.org.uk/rjk/

[VPN user]

Richard Kettlewell wrote in message 87io0z8...@mantic.terraraq.uk

> Ineffective security can be *worse* than nothing - partly because of the
> false sense of security (i.e. if you think that e.g. Syrian intelligence
> is not spying on you, you’ll risk saying things that you wouldn’t
> otherwise) and, in some cases, because it draws the attention of the
> adversary.

I understand.

But the first three lines of the vpngate.net site are:
- You can get through your government's firewall to browse restricted websites. (e.g. YouTube.)
- You can disguise your IP address to hide your identity while surfing the Internet.
- You can protect yourself by utilizing the strong encryption while using public Wi-Fi

I think they work off of "obfuscation", since they say this at the bottom:
- The prior list is partial.
- In order to prevent DoS attacks we show only some portion of available servers.

I'm not so sure this big red "warning" at the very bottom is gonna scare big brother though:
- Warning for Censorship Firewall Authorities
- Using the VPN Server List of VPN Gate Service as the IP Blocking List of your
country's Censorship Firewall is prohibited by us.
- The VPN Server List sometimes contains wrong IP addresses.
- If you enter the IP address list into your Censorship Firewall,
unexpected accidents will occur on the firewall.
- Therefore you must not use the VPN Server List for managing your
Censorship Firewall's IP blocking list.

[VPN user]

VPN user wrote in message nbhhpr$upb$1...@news.mixmin.net

> I'm not so sure this big red "warning" at the very bottom
> is gonna scare big brother though:

Here's more about the vpngate philosophy...
http://www.vpngate.net/en/about_overview.aspx

Out motivation is to solve the following problems:
Problem #1. Governmental Firewall Blocks Accesses to Certain Websites
Problem #2. It is possible to identify an individual by tracing the
IP address which is found in the logs of the server.
Problem #3. Public wireless networks are vulnerable to packet sniffing.

[VPN user]

VPN user wrote in message nbhiha$upb$2...@news.mixmin.net

> Here's more about the vpngate philosophy...
> http://www.vpngate.net/en/about_overview.aspx

BTW, they always plug their "SoftEther VPN Software" which,
originally, made me think their config files wouldn't work
on Linux (because SoftEther isn't ported to Linux).

But they're just ovpn files, so, they work fine with OpenVPN.

What I don't get is why would anyone use SoftEther when they
can just use OpenVPN?

Is there any real advantage to SoftEther that any of you know of?
https://www.softether.org/

"SoftEther VPN ("SoftEther" means "Software Ethernet") is one of
the world's most powerful and easy-to-use multi-protocol VPN
software. It runs on Windows, Linux, Mac, FreeBSD and Solaris.
SoftEther VPN is open source. You can use SoftEther for any
personal or commercial use for free charge."

Are any of you using SoftEther instead of OpenVPN?
What are the "real" advantages, if any?

[Marek Novotny]

On 2016-03-06, VPN user <vpn...@example.com> wrote:

Okay, tricky little bastard... I have updated it. Not sure why it would
do what you described but I rewrote a large part of the trap. While
testing it chrome got in my face... Starting to think chrome runs all by
itself or something... I never even started it. Going to get rid of that
evil app shortly.

Anyway, posted an update. Try that out. Oddly the sudo validation
happens very quickly so unless it has expired I'm not sure why it would
do that. It doesn't last forever. After so many minutes sudo has to be
re-validated. But it sounds like you did all this within a minute of
running the script so I doubt that's the issue.

[Dirk T. Verbeek]

Op 06-03-16 om 15:51 schreef Richard Kettlewell:

It isn't only about security from snooping, it can be something simple
as using voip and Skype from the Arab Gulf states.

[William Unruh]

On 2016-03-06, VPN user <vpn...@example.com> wrote:

> Marek Novotny wrote:
>> Are you running chrome or chromium? Maybe you have it not typed in
>> correctly in the vpnApps. Seems to be fine here even with lots of tabs.

The below looks like a bug in your program. From later it seems that
chrome-sandbox is defunct. That means it is dead, but is still in the
process table so that its parent or child knows what its parent or
child is. YOu program should not continue to try to kill it after it has
gone defunct. You can either try to kill the parent process as well, or
just ignore it.

>
> 6. Bad things happened because of the addition of "chrome"!
> $ vpnstatus.sh vpngate_KR_10_Pusan-jikhalsi_Pusan_164.125.68.90-vpn164414121.opengw.net_udp1195_20160305.ovpn
> vpnstatus.sh Status: sudo validation.
> [sudo] password for vpnuser:
> vpnstatus.sh Status: Obtaining Device ID
> vpnstatus.sh Status: Device ID Set: wlan0.
> vpnstatus.sh Status: Obtaining VPN Connection
> vpnstatus.sh Status: OpenVPN Process ID: 25890.
> #####
> vpnstatus.sh Status: Obtained VPN: tun0.
> vpnstatus.sh Status: Task thunderbird is running...
> vpnstatus.sh Status: Task thunderbird has been terminated.
> vpnstatus.sh Status: Task chrome-sandbox is running...
> vpnstatus.sh Status: Task chrome-sandbox has been terminated.
> vpnstatus.sh Status: Task chrome is running...
> vpnstatus.sh Status: Task chrome has been terminated.
> ... ... ... this goes on forever and will never stop ... ... ...

The bad thing happening is probably that your program keeps trying to
flog a dead horse.

>
> 8. The interesting thing is that I can't seem to kill chrome!
>
> $ ps -elf |grep chrome
> 0 S vpnuser 5193 5031 0 80 0 - 4047 pipe_w 04:56 pts/16 00:00:00 grep --color=auto chrome
> 4 Z vpnuser 23946 23894 0 80 0 - 0 exit 04:17 ? 00:00:00 [chrome-sandbox] <defunct>

chrome sandbox is what is continuing to run, not chrome.

>
> $ killall chrome-sandbox

And doing this will not help. The process is dead. Just not burried. The
corpse hangs around but cannot do anything.

http://askubuntu.com/questions/201303/what-is-defunct-for-a-process-and-why-it-doesnt-get-killed

>
> In summary, the Google browser (whatever it's called) is a PITA so I'll just
> avoid using it.

OR learn what "defunct" means.

[Andy Ho]

Dirk T. Verbeek wrote:

> It isn't only about security from snooping, it can be something simple
> as using voip and Skype from the Arab Gulf states.

Aren't most VOIP such as Skype already encrypted?

[Richard Kettlewell]

"Dirk T. Verbeek" <dver...@xs4all.nl> writes:

Yes. But this subthread is about the case where confidentiality is
required, not just availability.

--
http://www.greenend.org.uk/rjk/

[VPN user]

Marek Novotny wrote:

> Okay, tricky little bastard... I have updated it. Not sure why it would
> do what you described but I rewrote a large part of the trap. While
> testing it chrome got in my face... Starting to think chrome runs all by
> itself or something... I never even started it. Going to get rid of that
> evil app shortly.
>
> Anyway, posted an update. Try that out. Oddly the sudo validation
> happens very quickly so unless it has expired I'm not sure why it would
> do that. It doesn't last forever. After so many minutes sudo has to be
> re-validated. But it sounds like you did all this within a minute of
> running the script so I doubt that's the issue.
>
> https://github.com/marek-novotny/vpntools

Hi Marek,
Thanks. I'll take a look. I suspect, but am not sure, that the sudo
timed out, just like you said, so that's why it only comes up sometimes
after the control-C but not always after the control-C. (I think.)

When I log my tests, it isn't always in the direct sequence of the
logs 'cuz I make mistakes and the baby cries and the wife needs
attention in between computer tests! :)

I'll try the update and let you know what happens.

Thanks.

[Andy Ho]

Richard Kettlewell wrote:

> Yes. But this subthread is about the case where confidentiality is
> required, not just availability.

I think a key advantage if you use VPN day in and day out
is that anyone snooping on your ISP feed (government, ISP,
nosy neighbor, wife, kids, librarian, starbucks cashier,
whatever) can't tell *what* you're doing.

That is, without VPN, they can tell that you're using Tor
directory nodes, for example. So they know you're using Tor.

Without VPN, they can tell, I guess, that you're using VOIP
(dunno - does VOIP have a telltale signature)?

I'm *assuming* that with VPN, the snoopers can't tell you're
using VOIP.

Is that right?

[VPN user]

William Unruh wrote in message nbhqeq$uis$1...@dont-email.me

> And doing this will not help. The process is dead. Just not burried. The
> corpse hangs around but cannot do anything.
>
> http://askubuntu.com/questions/201303/what-is-defunct-for-a-process-and-why-it-doesnt-get-killed

I tried to kill it using that method of using *both* PIDs that showed
up in a ps of defunct, and the machine crashed.

Go figure.

Anyway, Chrome-whatever is an anathema.
I'll just stay away from any browser from Google.

[VPN user]

Marek Novotny wrote in message
FrCdncSgJ7klXEbL...@giganews.com

> I tested pkill for chrome and it works fine for me.
>
> Are you running chrome or chromium? Maybe you have it not typed in
> correctly in the vpnApps. Seems to be fine here even with lots of tabs.

Hi Marek,
I'm gonna give up on Chrome-anything because Google puts in
pernicious apps and updates and stuff that goes defunct.

So, don't worry for me about chrome-stuff.
I won't test chrome any more ... I don't even use it.
I was just testing stuff for you.

[Dirk T. Verbeek]

Op 06-03-16 om 20:26 schreef Andy Ho:

Not the point, the point is these countries ban those services.

[VPN user]

/less wrote in message nbg5j8$1kc8$1...@gioia.aioe.org

> I am a Linux user and use an 'one-click-working' paid version of VPN.
> It's inexpensive, most reliable and I can afford it!

What VPN provider do you use?

What's the cost per year?

And what level of security does it provide?
L2TP over IPSec ?
PPTP ?
Cisco IPSec ?
SSL/TLS ?

[Andy Ho]

Dirk T. Verbeek wrote:

> Not the point, the point is these countries ban those services.

I don't live under an oppressive regime so pardon this question.

Doesn't VPN /hide/ the fact that you're using VOIP?

[/less]

On 03/07/2016 06:12 AM, VPN user wrote:
> /less wrote in message nbg5j8$1kc8$1...@gioia.aioe.org
>
>> I am a Linux user and use an 'one-click-working' paid version of VPN.
>> It's inexpensive, most reliable and I can afford it!
>
> What VPN provider do you use?

BolehVPN

> What's the cost per year?

$80.-

> And what level of security does it provide?

Changeable; Pending on type of connection one wishes to use.

[Marek Novotny]

On 2016-03-06, VPN user <vpn...@example.com> wrote:

I actually use WiTopia. And I have made a script specifically for quite
a few of their end points. So what I did is made a menu of end points
and I choose the one I want. Then the script dynamically creates the
ovpn file for that end point. So I don't actually store any ovpn files.
They are created on the fly as I need one. I pay about $69 to $99 per
year for their service. They offer several services but I stick with
ovpn as it is simple to use, open and easily available on Linux and
FreeBSD.

[William Unruh]

On 2016-03-06, VPN user <vpn...@example.com> wrote:

> William Unruh wrote in message nbhqeq$uis$1...@dont-email.me
>
>> And doing this will not help. The process is dead. Just not burried. The
>> corpse hangs around but cannot do anything.
>>
>> http://askubuntu.com/questions/201303/what-is-defunct-for-a-process-and-why-it-doesnt-get-killed
>
> I tried to kill it using that method of using *both* PIDs that showed
> up in a ps of defunct, and the machine crashed.

Not sure what you mean. If you kill PID 1 yes, you will crash.

>
> Go figure.
>
> Anyway, Chrome-whatever is an anathema.
> I'll just stay away from any browser from Google.

Since flash development for Linux has stopped, the only browser able to
deliver movies etc via flask is Chrome.
Once everyone uses HTML5 ( which should be probably within 20 years I
would guess) then it will be irrelevant.

[Marek Novotny]

On 2016-03-07, William Unruh <un...@invalid.ca> wrote:
> Since flash development for Linux has stopped, the only browser able to
> deliver movies etc via flask is Chrome.
> Once everyone uses HTML5 ( which should be probably within 20 years I
> would guess) then it will be irrelevant.

LOL... Yeah, it won't take that long. Got me laughing though. There is
so much hate out there for Flash that it will go much more quickly than
that. 19 years... ;)

[Dirk T. Verbeek]

Op 07-03-16 om 00:18 schreef Andy Ho:

That's exactly the reason why VPN is so valuable in these places.

And because some of the worst countries will try to ban every VPN IP
they can find it is important to have a large pool of changing numbers.

A script as shown in the OP is a great help to get your hands on these
numbers.
Till the local services also hear about the script...

[VPN user]

/less wrote in message nbihon$aji$1...@gioia.aioe.org

>> What VPN provider do you use?
> BolehVPN
>> What's the cost per year?

Thanks.
https://bolehvpn.net/

They have servers in Canada, France, Germany, Italy, Japan,
Luxembourg, Malaysia, Netherlands, Singapore, Sweden,
Switzerland, United Kingdom and USA.

Do you find that they get blocked?


> $80.-
Thanks.
I see that's the yearly rate.
I will use that as my new datapoint as to how much VPN
should cost.

>> And what level of security does it provide?
> Changeable; Pending on type of connection one wishes to use.
>> L2TP over IPSec ?
>> PPTP ?
>> Cisco IPSec ?
>> SSL/TLS ?

With the free openvpn, I can *only* use SSL/TLS.
Do you find the others useful?

[VPN user]

Marek Novotny wrote:

> I actually use WiTopia. And I have made a script specifically for quite
> a few of their end points. So what I did is made a menu of end points
> and I choose the one I want. Then the script dynamically creates the
> ovpn file for that end point. So I don't actually store any ovpn files.
> They are created on the fly as I need one. I pay about $69 to $99 per
> year for their service. They offer several services but I stick with
> ovpn as it is simple to use, open and easily available on Linux and
> FreeBSD.

Thanks for that input Marek.

It seems that about a US quarter a day (i.e., twenty or twenty five
cents a day), one can get a decent VPN.

I see no value (yet anyway) in anything other than SSL/TLS and
OpenVPN seems to work just fine (with the exception of the DNS
leaks).

You might want to check that you're not getting DNS leaks "if"
you're running openvpn from the command line.

https://ipleak.net

[VPN user]

Marek Novotny wrote in message

a9ydnTjPgthp_0HL...@giganews.com

> Anyway, posted an update. Try that out.

I've been using the update and so far, when I control-C
out of it, it doesn't ask for the sudo password.

I'll keep testing though, as I use all your scripts
daily (although I've renamed many of them which confuses
me over time as you use your names and I use mine). :)

a. vpnkok (this is vpnkickorkeep which I plan on running
daily on the thousands of existing config files
so that each day I have a vpn_winners directory
that contains files tested that day to be good)
b. vpnrun (this runs vpn and does a kick or keep as needed)
c. vpnwatch (this watches over the processes but it was just
replaced by your new vpnstatus today but your
new vpnstatus doesn't do a kickorkeep so I have
to merge that since the kickorkeep is very useful
given that the ovpn files' goodness is ephemeral)
d. vpnspeed (this is your vpnit scipt which renames files based
on speedtest results - but speedtest-cli is really
slow unfortunately and I can't figure out how to
speed it up)
e. vpngeo (this renames the files based on geolocation data
and it uses your trick of set -- $(geolocation).
f. vpnmove (this moves the vpn_winners into country directories)
g. country (this just greps the country file for two-letter codes)
h. vpnget (this downloads about 300 or so new ovpn files)
i. vpncut (this shortens the downloaded file names)
j. vpnroute (this outputs the VPN IP address from the route command)
k. mlaunch (this whitelists programs to run when on vpn, but it
was superceded this week by the new vpnstatus)
l. tbird (this blacklists programs to NOT run when on vpn, but it
was superceded this week by the new vpnstatus)
m. vpninit (I think this is superceded by vpnkickorkeep?)

[Marek Novotny]

I posted a README on the vpntools git which explains what each of the
four are. One is a simple test script so you can easily check your
status.

The other three, vpnit, vpnstatus and kickorkeep do similar but
different things.

kickorkeep is faster than vpnit and sorts through piles of ovpn files
sorting them into good and bad.

vpnit should be run on just the good ones you find and it will rename
them based on ping time, upload and download speeds plus country so you
can easily sort through them when you want to connect later with the
fastest one or from a particular country.

vpnstatus is used to execute openvpn and monitor your connection status
to kill off apps that should not run should you become disconnected.

[VPN user]

Marek Novotny wrote:

> I posted a README on the vpntools git which explains what each of the
> four are. One is a simple test script so you can easily check your
> status.

Thanks for writing that. I often rename them and change them so
I get confused myself which is which.

BTW, I finished running vpnkickorkeep on the 3,000 files that I currently
have of the supposed 4,000 on vpngate with the to-be-expected result
of a 1:2 ratio of 1/3 good files and 2/3 bad at any one moment.
$ ls -l vpn_winners/* | wc -l
1019
$ ls -l vpn_losers/* |wc -l
2181

Given that they rotate these files, they may even statistically
keep to that ratio (bearing in mind that the entire site is part
of a research project so I'm sure they have their math figured
out well ahead of time).

The number of unique countries only grew slightly to 44 in toto:
$ ls vpn_winners | grep -v IP
AE AU BY CN DO ES FR HK IE IQ IT KR MM MX NZ PL QA RU SE TR TW US
AR BR CA DE EG FI GB ID IS JP MA MO NL PH PT RO SA TH TT UA VE VN

The vast majority are in Korea, Japan, and the US (in that order).

An example of the hundred or so good US files today are:
vpngate_US_AP_ArmedForcesPacific_Apo_153.143.49.253-vpn112244944.opengw.net_tcp1365.ovpn
vpngate_US_AZ_Arizona_Phoenix_70.190.153.23-70.190.153.23_tcp1863.ovpn
vpngate_US_AZ_Arizona_Phoenix_70.190.153.23-70.190.153.23_udp1260.ovpn
vpngate_US_AZ_Arizona_Phoenix_70.190.153.23-vpn271785661.opengw.net_tcp1863.ovpn
vpngate_US_AZ_Arizona_Phoenix_70.190.153.23-vpn271785661.opengw.net_udp1260.ovpn
vpngate_US_AZ_Arizona_Tucson_69.244.59.185-69.244.59.185_tcp1992.ovpn
vpngate_US_AZ_Arizona_Tucson_69.244.59.185-69.244.59.185_udp1557.ovpn
vpngate_US_AZ_Arizona_Tucson_69.244.59.185-vpn641633069.opengw.net_tcp1992.ovpn
vpngate_US_CA_California_SanDiego_99.75.20.224-99.75.20.224_udp1195.ovpn
vpngate_US_FL_Florida_FortLauderdale_99.102.164.134-99.102.164.134_udp1195.ovpn
vpngate_US_FL_Florida_FortLauderdale_99.117.129.116-99.117.129.116_udp1195.ovpn
vpngate_US_FL_Florida_Hollywood_107.141.185.58-107.141.185.58_udp1195.ovpn
vpngate_US_FL_Florida_Hollywood_73.139.234.47-73.139.234.47_tcp995.ovpn
vpngate_US_FL_Florida_Hollywood_73.139.234.47-vpn809774788.opengw.net_tcp995.ovpn
vpngate_US_FL_Florida_Hollywood_73.139.234.47-vpn809774788.opengw.net_udp1195.ovpn
vpngate_US_FL_Florida_Miami_71.196.72.62-71.196.72.62_udp1374.ovpn
vpngate_US_FL_Florida_Miami_71.196.72.62-vpn394528625.opengw.net_udp1374.ovpn
vpngate_US_HI_Hawaii_Honolulu_72.234.241.15-72.234.241.15_udp1195.ovpn
vpngate_US_IA_Iowa_CouncilBluffs_68.227.161.96-68.227.161.96_tcp1914.ovpn
vpngate_US_IA_Iowa_CouncilBluffs_68.227.161.96-68.227.161.96_udp1653.ovpn
vpngate_US_IA_Iowa_CouncilBluffs_68.227.161.96-vpn251664221.opengw.net_tcp1914.ovpn
vpngate_US_IA_Iowa_CouncilBluffs_68.227.161.96-vpn251664221.opengw.net_udp1653.ovpn
vpngate_US_ID_Idaho_Boise_72.24.210.71-72.24.210.71_udp1195.ovpn
vpngate_US_IL_Illinois_Chicago_98.227.80.11-98.227.80.11_udp1841.ovpn
vpngate_US_IL_Illinois_Chicago_98.227.80.11-vpn964460031.opengw.net_udp1841.ovpn
vpngate_US_IL_Illinois_Waukegan_99.182.214.55-99.182.214.55_udp1195.ovpn
vpngate_US_MA_Massachusetts_JamaicaPlain_24.63.106.142-24.63.106.142_udp1195.ovpn
vpngate_US_MD_Maryland_Columbia_73.132.241.133-73.132.241.133_udp1379.ovpn
vpngate_US_MD_Maryland_Gaithersburg_69.243.96.174-69.243.96.174_tcp1490.ovpn
vpngate_US_MD_Maryland_Gaithersburg_69.243.96.174-vpn687432006.opengw.net_tcp1490.ovpn
vpngate_US_MD_Maryland_Gaithersburg_69.243.96.174-vpn687432006.opengw.net_udp1590.ovpn
vpngate_US_MD_Maryland_Hurlock_73.133.137.219-73.133.137.219_tcp1912.ovpn
vpngate_US_MD_Maryland_Hurlock_73.133.137.219-73.133.137.219_udp1572.ovpn
vpngate_US_MN_Minnesota_Minneapolis_107.2.93.150-107.2.93.150_udp1354.ovpn
vpngate_US_na_na_na_184.89.18.244-184.89.18.244_udp1879.ovpn
vpngate_US_na_na_na_45.36.182.35-45.36.182.35_tcp1615.ovpn
vpngate_US_na_na_na_45.36.182.35-45.36.182.35_udp1394.ovpn
vpngate_US_na_na_na_45.36.182.35-vpn248685690.opengw.net_tcp1615.ovpn
vpngate_US_na_na_na_73.226.208.209-73.226.208.209_tcp1306.ovpn
vpngate_US_na_na_na_73.226.208.209-73.226.208.209_udp1332.ovpn
vpngate_US_na_na_na_73.226.208.209-vpn648988082.opengw.net_tcp1306.ovpn
vpngate_US_na_na_na_73.226.208.209-vpn648988082.opengw.net_udp1332.ovpn
vpngate_US_na_na_na_73.246.184.173-73.246.184.173_tcp1462.ovpn
vpngate_US_na_na_na_73.246.184.173-vpn364200989.opengw.net_tcp1462.ovpn
vpngate_US_NC_NorthCarolina_FortBragg_75.178.28.124-75.178.28.124_tcp1668.ovpn
vpngate_US_NC_NorthCarolina_FortBragg_75.178.28.124-75.178.28.124_udp1991.ovpn
vpngate_US_NC_NorthCarolina_FortBragg_75.178.28.124-vpn773294077.opengw.net_tcp1668.ovpn
vpngate_US_NC_NorthCarolina_FortBragg_75.178.28.124-vpn773294077.opengw.net_udp1991.ovpn
vpngate_US_NH_NewHampshire_Canterbury_73.219.10.53-73.219.10.53_udp1503.ovpn
vpngate_US_NH_NewHampshire_Canterbury_73.219.10.53-vpn176133331.opengw.net_tcp1687.ovpn
vpngate_US_NH_NewHampshire_Canterbury_73.219.10.53-vpn176133331.opengw.net_udp1503.ovpn
vpngate_US_NJ_NewJersey_FortLee_68.172.253.96-68.172.253.96_udp1953.ovpn
vpngate_US_NJ_NewJersey_Swedesboro_73.10.20.96-73.10.20.96_udp1195.ovpn
vpngate_US_NY_NewYork_Brooklyn_47.18.164.224-vpn909280342.opengw.net_tcp1812.ovpn
vpngate_US_NY_NewYork_Brooklyn_47.18.164.224-vpn909280342.opengw.net_udp1444.ovpn
vpngate_US_NY_NewYork_Buffalo_192.3.226.12-192.3.226.12_udp1194.ovpn
vpngate_US_NY_NewYork_Buffalo_192.3.226.12-vpn175001405.opengw.net_udp1194.ovpn
vpngate_US_NY_NewYork_Buffalo_198.23.197.184-198.23.197.184_tcp443.ovpn
vpngate_US_NY_NewYork_Buffalo_198.23.197.184-198.23.197.184_udp1194.ovpn
vpngate_US_NY_NewYork_Buffalo_198.23.197.184-fyredeath.opengw.net_tcp443.ovpn
vpngate_US_NY_NewYork_Buffalo_198.23.197.184-fyredeath.opengw.net_udp1194.ovpn
vpngate_US_NY_NewYork_Jamaica_67.244.90.194-67.244.90.194_tcp1407.ovpn
vpngate_US_NY_NewYork_Jamaica_67.244.90.194-67.244.90.194_udp1894.ovpn
vpngate_US_NY_NewYork_Jamaica_67.244.90.194-vpn329303477.opengw.net_tcp1407.ovpn
vpngate_US_NY_NewYork_Jamaica_67.244.90.194-vpn329303477.opengw.net_udp1894.ovpn
vpngate_US_OH_Ohio_Columbus_99.47.249.37-99.47.249.37_udp1195.ovpn
vpngate_US_OH_Ohio_Delaware_74.140.64.17-74.140.64.17_udp1912.ovpn
vpngate_US_OH_Ohio_Painesville_104.231.101.67-104.231.101.67_udp1550.ovpn
vpngate_US_OH_Ohio_Toledo_72.241.201.8-vpn889661762.opengw.net_tcp1691.ovpn
vpngate_US_OH_Ohio_Toledo_72.241.201.8-vpn889661762.opengw.net_udp1262.ovpn
vpngate_US_OH_Ohio_Xenia_98.28.205.34-98.28.205.34_tcp1560.ovpn
vpngate_US_OH_Ohio_Xenia_98.28.205.34-98.28.205.34_udp1681.ovpn
vpngate_US_OK_Oklahoma_ElkCity_198.71.101.26-198.71.101.26_udp1708.ovpn
vpngate_US_OK_Oklahoma_Hinton_199.73.99.253-vpn264880674.opengw.net_udp1195.ovpn
vpngate_US_OK_Oklahoma_Norman_162.242.30.62-vpn304468933.opengw.net_tcp995.ovpn
vpngate_US_OK_Oklahoma_Norman_162.242.30.62-vpn304468933.opengw.net_udp1195.ovpn
vpngate_US_OR_Oregon_Boardman_54.201.110.154-sctcable.opengw.net_tcp443.ovpn
vpngate_US_OR_Oregon_Boardman_54.201.110.154-sctcable.opengw.net_udp1194.ovpn
vpngate_US_PA_Pennsylvania_Springfield_69.242.67.150-69.242.67.150_udp1195.ovpn
vpngate_US_TN_Tennessee_Helenwood_207.144.215.24-207.144.215.24_udp1293.ovpn
vpngate_US_TX_Texas_Austin_70.114.247.222-70.114.247.222_tcp1817.ovpn
vpngate_US_TX_Texas_Austin_70.114.247.222-vpn610305241.opengw.net_tcp1817.ovpn
vpngate_US_TX_Texas_Carrollton_70.119.165.125-70.119.165.125_udp1654.ovpn
vpngate_US_TX_Texas_Dallas_97.99.75.27-97.99.75.27_udp1402.ovpn
vpngate_US_TX_Texas_Irving_108.206.180.66-vpn817573606.opengw.net_udp1195.ovpn
vpngate_US_TX_Texas_Magnolia_97.93.177.140-97.93.177.140_udp1893.ovpn
vpngate_US_TX_Texas_Magnolia_97.93.177.140-vpn300474190.opengw.net_udp1893.ovpn
vpngate_US_TX_Texas_Rowlett_71.123.228.241-71.123.228.241_udp1751.ovpn
vpngate_US_TX_Texas_Rowlett_71.123.228.241-vpn458870419.opengw.net_tcp1959.ovpn
vpngate_US_TX_Texas_Rowlett_71.123.228.241-vpn458870419.opengw.net_udp1751.ovpn
vpngate_US_VA_Virginia_Hampton_108.17.157.208-108.17.157.208_udp1422.ovpn
vpngate_US_VA_Virginia_Ruckersville_73.171.118.32-73.171.118.32_udp1710.ovpn
vpngate_US_VA_Virginia_VirginiaBeach_70.184.164.12-70.184.164.12_udp1217.ovpn
vpngate_US_VA_Virginia_VirginiaBeach_70.184.164.12-vpn987190871.opengw.net_udp1217.ovpn

[Marek Novotny]

On 2016-03-07, VPN user <vpn...@example.com> wrote:

> Marek Novotny wrote:
>
>> I posted a README on the vpntools git which explains what each of the
>> four are. One is a simple test script so you can easily check your
>> status.
>
> Thanks for writing that. I often rename them and change them so
> I get confused myself which is which.
>
> BTW, I finished running vpnkickorkeep on the 3,000 files that I currently
> have of the supposed 4,000 on vpngate with the to-be-expected result
> of a 1:2 ratio of 1/3 good files and 2/3 bad at any one moment.
> $ ls -l vpn_winners/* | wc -l
> 1019
> $ ls -l vpn_losers/* |wc -l
> 2181
>
> Given that they rotate these files, they may even statistically
> keep to that ratio (bearing in mind that the entire site is part
> of a research project so I'm sure they have their math figured
> out well ahead of time).
>
> The number of unique countries only grew slightly to 44 in toto:
> $ ls vpn_winners | grep -v IP
> AE AU BY CN DO ES FR HK IE IQ IT KR MM MX NZ PL QA RU SE TR TW US
> AR BR CA DE EG FI GB ID IS JP MA MO NL PH PT RO SA TH TT UA VE VN
>
> The vast majority are in Korea, Japan, and the US (in that order).

That's interesting. I like data like that.

// snip

[/less]

On 03/07/2016 09:45 AM, VPN user wrote:
> /less wrote in message nbihon$aji$1...@gioia.aioe.org
>>> What VPN provider do you use?
>> BolehVPN
>>> What's the cost per year?
> Thanks.
> https://bolehvpn.net/

Yep, that's them.

> They have servers in Canada, France, Germany, Italy, Japan,
> Luxembourg, Malaysia, Netherlands, Singapore, Sweden,
> Switzerland, United Kingdom and USA.
> Do you find that they get blocked?

I mainly utilize 'Fully Routed' Canada for one specific service, and use
some other servers for Surfing/Streaming.
I recall that either 'United Kingdom' or 'Sweden' was very briefly out
of service a few months ago (didn't bother to time). Other than this
outage I never experienced any 'blockages' or other disruptions. It's so
easy switching to another server so server disruptions do not concern me
at all.

>> $80.-
> Thanks.
> I see that's the yearly rate.

Yes it is, I find annual payments most convenient.

> I will use that as my new datapoint as to how much VPN
> should cost.

They are much less costly VPN services available probably providing
outstanding services too... but I can't be bothered changing because
BolehVPN just works for me (I am only a happy customer and am not
affiliated in any way).

>>> And what level of security does it provide?
>> Changeable; Pending on type of connection one wishes to use.
>>> L2TP over IPSec ?
>>> PPTP ?
>>> Cisco IPSec ?
>>> SSL/TLS ?
>
> With the free openvpn, I can *only* use SSL/TLS.
> Do you find the others useful?

I use Mint on desktop and Netrunner on Laptop and never tried anything
else but 'openvpn'. Because of the technical complexities I am steering
away from investigated/experimenting the usefulness of alternatives.

Look, my curiosity was aroused by your original post and the (almost
unfathomable) time it took for coming up with an 'efficient' VPN
process; I expected to read something which would make sense to me alas
this is imho is not meant for the average utilizer. I am a very
unsophisticated user which sadly is in Linux groups almost always
automatically or purposefully related to a troll activity, oh well - I
am much too old for this!

As long as the DNS leak result is negative, the 'kill' switch is
working, my ISP is kept in the dark of my 'activities' and no
significant reduction in browsing/download speed I am very happy
spending 22 cent per day which does include really good professional
support which in my experience beats the hell out of any Linux groups
excluding Linux fora.

[/less]

fwiw...
https://www.bestvpn.com/blog/31750/a-complete-guide-to-ip-leaks/
https://diafygi.github.io/webrtc-ips/
https://www.browserleaks.com/
http://whatismyipaddress.com/
https://www.dnsleaktest.com/
http://www.canyouseeme.org/
http://test-ipv6.com/
https://ipleak.net/
http://myip.dk/

[jjb]

https://airvpn.org/ works for me.

[Dirk T. Verbeek]

Op 07-03-16 om 07:42 schreef /less:

>
> I use Mint on desktop and Netrunner on Laptop and never tried anything
> else but 'openvpn'. Because of the technical complexities I am steering
> away from investigated/experimenting the usefulness of alternatives.
>
> Look, my curiosity was aroused by your original post and the (almost
> unfathomable) time it took for coming up with an 'efficient' VPN
> process; I expected to read something which would make sense to me alas
> this is imho is not meant for the average utilizer. I am a very
> unsophisticated user which sadly is in Linux groups almost always
> automatically or purposefully related to a troll activity, oh well - I
> am much too old for this!
>
> As long as the DNS leak result is negative, the 'kill' switch is
> working, my ISP is kept in the dark of my 'activities' and no
> significant reduction in browsing/download speed I am very happy
> spending 22 cent per day which does include really good professional
> support which in my experience beats the hell out of any Linux groups
> excluding Linux fora.

Contrary to your first reaction these are valid points.

My troll remark was about the statements you made insinuating Linux
(use) is inherently complicated thereby implying for solutions you
should go non-Linux.

Many Linux users would like to see a wider adaptation of their favourite
system but anyone should realise until then the average Linux user is
more tech savy than the average PC user.
Here we saw some of these tech savy people supporting each other to come
up with an easier solution for all.

Derision was totally out of order.

[VPN user]

/less wrote in message nbj7si$1e0p$1...@gioia.aioe.org

> Look, my curiosity was aroused by your original post and the (almost
> unfathomable) time it took for coming up with an 'efficient' VPN
> process;

Bear in mind Marek did most of the heavy lifting, and that the
"unfathomable time" was *elapsed* time. I'd suspect that Marek
spent a total of a couple of 8-hour days in toto on this "project",
where he leveraged his result to everyone in the world who knows
about it and wants it, including (himself) and me.

> I expected to read something which would make sense to me alas
> this is imho is not meant for the average utilizer. I am a very
> unsophisticated user which sadly is in Linux groups almost always
> automatically or purposefully related to a troll activity, oh well - I
> am much too old for this!

I am also very unsophisticated.
I just "string commands" together.

For example, I improved the vpncut script, which, as you can
see from my stringing of commands together, is 'fugly.

#!/bin/bash
echo "This many ovpn files before removing HTML ovpn files"
ls *.ovpn|wc -l
echo "Shortening file names"
for i in *.ovpn;do mv $i $(echo $i|cut -d'F' -f2);done
grep DOCTYPE * | awk -F: '{print $1}'| xargs rm
echo "This many ovpn files after removing HTML ovpn files"
ls *.ovpn|wc -l
echo "This many files not ending with ovpn"
ls --ignore='*.ovpn' | wc -l
exit 0

> As long as the DNS leak result is negative, the 'kill' switch is
> working, my ISP is kept in the dark of my 'activities' and no
> significant reduction in browsing/download speed I am very happy
> spending 22 cent per day which does include really good professional
> support which in my experience beats the hell out of any Linux groups
> excluding Linux fora.

The huge advantage you have over what I'm doing is *speed*.

The penalty in speed that I get is probably great, but, it's enough
for nntp and browsing - but it's noticeable when watching youtube.

I have *two* advantages over you, one of which is cost (duh), and
the other of which is that the exit nodes are never blocked.

I suspect your exit nodes are often blocked by third parties
(I don't think you understood the original question but I won't
press you on that) and I also think that by paying, you lose
a little bit of anonymity (not much because I have to give the
VPN server my IP address which is me, even if it's not literally
me).

But it's much more *fun* for me to use VPN than you because I
*like* using the scripts and the power of Linux to do stuff for
free that you have to pay for! :)

[VPN user]

Dirk T. Verbeek wrote in message 56dd4470$0$24036$e4fe...@news.xs4all.nl

> My troll remark was about the statements you made insinuating Linux
> (use) is inherently complicated thereby implying for solutions you
> should go non-Linux.

To support Dirk T. Verbeek, the "complexity" would have been the same
on Windows, Mac, Android, and iOS in that the *problems* we resolved
exist on all the platforms.

So, the supposed complexity had *nothing* to do with Linux anyway.

Bear in mind that the *approach* that vpngate uses *purposefully*
puts bad files in the mix (at the ratio of 2:1).

This is to foil censorship (that's what they say on their web site).

The *advantage* to me is that USA web sites don't *block* them
like they block Tor exit nodes.

So, while government censorship isn't "my" problem, I like that
with Marek's kickorkeep-style scripts, I can easily *find* the
good ovpn files, none of which (to date) have been blocked by
any web sites!

That's one advantage I have over "/less" whose VPN IP addresses
are (most likely) static, and hence *easily* blocked by third
party web sites.

He also presumably has *less* anonymity than I do, since I can
"afford" to use a different IP address for every NNTP post,
for example, while he probably has far fewer IP addresses to
choose from than I do.

[VPN user]

jjb wrote in message 56dd3219$0$24086$e4fe...@news.xs4all.nl

> https://airvpn.org/ works for me.

Thanks for that input.

It's 54 Euros a year, which is less expensive than the others
(but in the range of about 20 to 25 cents a day which the others were).

I see that https://airvpn.org supports openvpn (SSL/TLS).

They say "No monitoring nor logging of your online activities", but
almost nobody *believes* that (certainly at least they save your
"connection logs" (since *everyone* does that, usually for at
least two weeks).

When it says "Make it impossible to identify the type of traffic or
protocol you are using, even for your ISP", I believe that; however
I suspect that our ISPs know we are using VPN based on what the
gibberish looks like.

Given that, I'm not sure what they mean when they say "your government
will see only TCP or UDP traffic on a unique port".

[Marek Novotny]

On 2016-03-07, VPN user <vpn...@example.com> wrote:

> /less wrote in message nbj7si$1e0p$1...@gioia.aioe.org
>
>
>> Look, my curiosity was aroused by your original post and the (almost
>> unfathomable) time it took for coming up with an 'efficient' VPN
>> process;
>
> Bear in mind Marek did most of the heavy lifting, and that the
> "unfathomable time" was *elapsed* time. I'd suspect that Marek
> spent a total of a couple of 8-hour days in toto on this "project",
> where he leveraged his result to everyone in the world who knows
> about it and wants it, including (himself) and me.

It's fun to pick a pet project and work on it regardless of the time
involved because it serves many purposes. Yes, it may have taken you a
year to get here and it indeed took me a few hours. But I had already
written vpn based scripts for witopia when I first saw this. So I had
some experience already in dealing with ovpn files, which is why I
already knew to suggest using them instead of the method proposed by
your providers.

Regarding the time investment:

I've been writing my linfo script for over two years. The initial script
took one day and then every day since that day have been improvements or
changes in coding style and technique. It's been a learning project
for me.

I've even threw it away completely and started over from scratch once.
I am sure anyone experienced can write such a script in a day or so
including debugging it. But when I was started I didn't have a clue as
to what I was doing. I had to google everything and it really showed.
Absolutely no consistency at all. Every single datapoint I gathered were
the result of looking through the web and finding how someone else
suggested a particular piece of info be obtained.

I chose to rewrite it from scratch the day I felt I knew and understood
enough to gather the data using my own technique. It was a happy day for
me and the items it gathered were shorter at first. I only included that
which I could do myself. And as time moved forward and I learned more, I
added more. As I was told my coding style sucked, I listened, learned and
improved it where I could. And every month I'd look back at it and
realize how bad it is and I'd improve it. And this goes on to this day.

Pick a project you enjoy, like your vpn issue and just keep playing
with it. You can teach yourself a lot simply by having a small project
you're interested in and reading and writing code for that project every
other day or so. Pretty soon the code you read from others starts to
make sense and you learn. Once you start learning you begin to write
more and more of the code yourself without help and you begin to be able
to read code as easily as reading English. Once that happens the whole
thing starts to open up to you.

Don't worry about that time. Just go at your own pace. Learn slowly,
learn quickly, it doesn't matter. What matters as that you stick with
it. Those that don't never learn. Those that stick with it learn it well
and begin to realize that they have this amazing tool at their
fingertips and it's fun, too.

[VPN user]

/less wrote in message nbj9dd$1gbh$1...@gioia.aioe.org

> fwiw...
> https://www.bestvpn.com/blog/31750/a-complete-guide-to-ip-leaks/
> https://diafygi.github.io/webrtc-ips/
> https://www.browserleaks.com/
> http://whatismyipaddress.com/
> https://www.dnsleaktest.com/
> http://www.canyouseeme.org/
> http://test-ipv6.com/
> https://ipleak.net/
> http://myip.dk/

Just as a warning, from my research, *all* command-line ovpn solutions
on Linux have DNS leaks because command-line openvpn does not mess with
the /etc/resolv.conf file.

Only a GUI-based solution messes with the DNS servers according to what
I read online.

[VPN user]

Marek Novotny wrote:

> That's interesting. I like data like that.

>> BTW, I finished running vpnkickorkeep on the 3,000 files that I currently
>> have of the supposed 4,000 on vpngate with the to-be-expected result
>> of a 1:2 ratio of 1/3 good files and 2/3 bad at any one moment.

> That's interesting. I like data like that.

I think I made a minor assumption error when I said that I have 3,000
of the 4,000 servers.

I have 3,000 *files* but any one *server* can commonly have as many
as 4 files:
1. TCP and an IP address so as to foil DNS-based censorship
2. UDP and an IP address so as to foil DNS-based censorship
3. TCP with a DNS lookup (so they can change IP addresses to foil censorship)
4. UDP with a DNS lookup (so they can change IP address to foil censorship)

For example, Marek's kickorkeep script found these 4 files to be good today:
1. vpngate_US_NY_NewYork_Buffalo_198.23.197.184-198.23.197.184_tcp443.ovpn
2. vpngate_US_NY_NewYork_Buffalo_198.23.197.184-198.23.197.184_udp1194.ovpn
3. vpngate_US_NY_NewYork_Buffalo_198.23.197.184-fyredeath.opengw.net_tcp443.ovpn
4. vpngate_US_NY_NewYork_Buffalo_198.23.197.184-fyredeath.opengw.net_udp1194.ovpn

I think what /less/ and others need to realize is that the main point
of vpngate is to get around censorship while still being free.

The way they get around censorship, apparently, is to throw a lot of
bad data in with the good data, which, without Marek's scripts, means
that finding a good files takes, on average, three tries (since I've
found, overall, about 2:1 bad files to good files).

It's pretty *easy* to do this manually, but it's even *better* now with
Marek's scripts.

Someone made the point that we might have made it *easier* for the
censors to block them, so, we should keep that in mind here I guess.

[William Unruh]

On 2016-03-07, VPN user <vpn...@example.com> wrote:

As a general statement this is nonsense. A command line script is just
as capable of messing with /etc/resolv.conf files as is a GUI. Now it
may be that the person who wrote the command line did not bother to
change resolv.conf, but that was a decision which had nothing to do with
it being command line or gui.

[VPN user]

William Unruh wrote in message nbl1ol$tr4$1...@dont-email.me

> As a general statement this is nonsense. A command line script is just
> as capable of messing with /etc/resolv.conf files as is a GUI. Now it
> may be that the person who wrote the command line did not bother to
> change resolv.conf, but that was a decision which had nothing to do with
> it being command line or gui.

This makes sense, so I stand corrected.

Here is a more accurately stated sentence:
https://forum.vpn.ac/discussion/13/running-openvpn-in-linux-terminal-with-no-dns-leaks
"OpenVPN won't assign server's DNS resolver(s) if you run it from terminal.
This is a known behavior and you have to run an external script to assign
the DNS resolvers once the tunnel is connected, and revert when disconnected.
This should work on Ubuntu, Debian, Mint and any Debian based distros
(most likely others, too - like Fedora)."

Luckily, when you install openvpn using apt-get:
$ sudo apt-get install openvpn

There are files which come with that openvpn installation
that *do* mess with the /etc/resolv.conf such as:
/etc/openvpn/update-resolv-conf.sh

They just don't mess with resolv.conf by default!

So, to start openvpn and mess with the resolv.conf, use this:
$ sudo openvpn --config file.ovpn --script-security 2 --up /etc/openvpn/update-resolv-conf.sh --down /etc/openvpn/update-resolv-conf.sh

Or, to put those three lines in all your thousands of ovpn files
just run this command on all your ovpn files:

#!/bin/bash
# fixovpn.sh adds three lines to all ovpn files in the directory to stop dns leaks
# https://forum.vpn.ac/discussion/13/running-openvpn-in-linux-terminal-with-no-dns-leaks
# script-security 2
# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf
for filename in *.ovpn; do echo -e "script-security 2\nup /etc/openvpn/update-resolv-conf\ndown /etc/openvpn/update-resolv-conf" >> $filename; done

Is this post more accurate?
Is there anything else we need to add?

[VPN user]

VPN user wrote in message nbl4jc$47a$1...@news.mixmin.net

> So, to start openvpn and mess with the resolv.conf, use this:
> $ sudo openvpn --config file.ovpn --script-security 2 --up /etc/openvpn/update-resolv-conf.sh --down /etc/openvpn/update-resolv-conf.sh

Here is what "sudo apt-get install openvpn" put in
/etc/openvpn/update-resolv-conf

#!/bin/bash
#
# Parses DHCP options from openvpn to update resolv.conf
# To use set as 'up' and 'down' script in your openvpn *.conf:

# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf

#
# Used snippets of resolvconf script by Thomas Hood and Chris Hanson.
# Licensed under the GNU GPL. See /usr/share/common-licenses/GPL.
#
# Example envs set from openvpn:
#
# foreign_option_1='dhcp-option DNS 193.43.27.132'
# foreign_option_2='dhcp-option DNS 193.43.27.133'
# foreign_option_3='dhcp-option DOMAIN be.bnc.ch'
#


# echo $1 $2 $3 $4 $5 $6
[ -x /sbin/resolvconf ] || exit 0
[ "$script_type" ] || exit 0
[ "$dev" ] || exit 0

split_into_parts()
{
part1="$1"
part2="$2"
part3="$3"
}

case "$script_type" in
up)
NMSRVRS=""
SRCHS=""
for optionvarname in ${!foreign_option_*} ; do
option="${!optionvarname}"
echo "$option"
split_into_parts $option
if [ "$part1" = "dhcp-option" ] ; then
if [ "$part2" = "DNS" ] ; then
NMSRVRS="${NMSRVRS:+$NMSRVRS }$part3"
elif [ "$part2" = "DOMAIN" ] ; then
SRCHS="${SRCHS:+$SRCHS }$part3"
fi
fi
done
R=""
[ "$SRCHS" ] && R="search $SRCHS
"
for NS in $NMSRVRS ; do
R="${R}nameserver $NS
"
done
echo -n "$R" | /sbin/resolvconf -a "${dev}.openvpn"
;;
down)
/sbin/resolvconf -d "${dev}.openvpn"
;;
esac

[VPN user]

Just by way of statistics, here is a run today using Marek's scripts:

1. First I ran "vpnget" to obtain all the ovpn files:
$ cd 20160307
$ vpnget
.... (this took some time) ....
$ ls -l *.ovpn | wc -l
352

2. Then I cleaned up the files & ran Marek's vpnkickorkeep script:
$ vpncut
$ vpnkickorkeep
.... (this took some time) ....

3. The result was the following in the two respective directories:
$ ls -l vpn_winners/vpngate* | wc -l
182

$ ls -l vpn_losers | wc -l
170

So, given this was an en masse download of the vpngate site, roughly
about half the files downloaded today worked after downloading them.

I think *that* is part of their method they use to foil censorship.

[jjb]

On 07-03-2016 14:34, VPN user wrote:
> jjb wrote in message 56dd3219$0$24086$e4fe...@news.xs4all.nl
>
>> https://airvpn.org/ works for me.
>
> Thanks for that input.
>
> It's 54 Euros a year, which is less expensive than the others
> (but in the range of about 20 to 25 cents a day which the others were).
>
> I see that https://airvpn.org supports openvpn (SSL/TLS).
>
> They say "No monitoring nor logging of your online activities", but
> almost nobody *believes* that (certainly at least they save your
> "connection logs" (since *everyone* does that, usually for at
> least two weeks).

They say they don't. Concerning who are behind this organization and
the reports from independent observers on the web, I tend to believe
them. THEIR service providers (datacenters) of course can and probably
will log connections, but that is unavoidable.

>
> When it says "Make it impossible to identify the type of traffic or
> protocol you are using, even for your ISP", I believe that; however
> I suspect that our ISPs know we are using VPN based on what the
> gibberish looks like.
>
> Given that, I'm not sure what they mean when they say "your government
> will see only TCP or UDP traffic on a unique port".
>

Quote (from their website):
"We offer OpenVPN on ports 80 TCP / UDP, 443 TCP / UDP and 53 TCP / UDP.
Additionally, every Air server supports directly OpenVPN over SSH,
OpenVPN over SSL and OpenVPN over Tor."

If you use port 443, your ISP will see traffic just like a HTTPS
connection. Of course, without using TOR they will see your destination
IP address, and a deeper analysis of traffic amount and timing will show
that you are not using a normal HTTPS connection.

[VPN user]

Marek Novotny wrote in message

xtKdnVf3uMPKGUDL...@giganews.com

> It's fun to pick a pet project and work on it regardless of the time
> involved because it serves many purposes.

I agree with you Marek.

Pretty much, this "pet" project was to make using vpngate as easy
as possible, given their propensity to feed fake files and to
rotate good files with bad files and back.

At this point, with all your wonderful help, it's pretty much a
fait accompli.

About the only thing I have left is to work around the command-line
speedtest deficiencies.

While my "vpngeo.sh" script has already been inserted into your
kickorkeep.sh test on recently downloaded good/bad vpn files, that
geoiplookup operation only renames the vpn_winners based on geo
location.

What I still need is to rename the vpn_winners with speedtest
results.

While your vpnit.sh script renames the files based on the speedtest,
the results take too long and even so, the naming conventions turn
out to be flaky simply because the results of speedtest-cli is flaky.

So, what I still need to figure out how to do is to re-rename the
existing geotagged vpn_winners in a batch process that *adds* speedtest
results *only* if they're valid numerical speeds.

So, the use model for this to-be-written "vpnspeed" script is:

1. Obtain (using vpnget) & truncate new config file names (vpncut).
2. Test config files (using vpnkickorkeep, putting winners into
geo-tagged vpn_winners country directories, sorted by country.
3. (As a separate batch run!) re-rename vpn_winners by *appending*
only valid speedtests using a modified version of your vpnit script.

In the end, the file name structure for the winners would be something like:
./vpn_winners/US/
vpngate_US_VA_Virginia_VirginiaBeach_70.184.164.12_udp1217_20ms_10dn_4up.ovpn

Where the desired fields are:
a. prefix (e.g., vpngate)
b. country (e.g., US)
c. region (e.g., VA)
d. state (e.g., Virginia)
e. city (e.g., VirginiaBeach)
f. IP (e.g., 70.184.164.12)
g. protocol (e.g., udp)
h. port (e.g., 1217)
i. ping times (e.g., 20ms)
j. download speed (e.g., 10Mbps)
k. upload speed (e.g., 4Mbps)
l. extension (e.g., ovpn)

Note that the speeds *must* be appended *last* because they are the
flakiest of all the tests (geo tagging isn't nearly as flaky, and
is very fast anyway so it's easier accomplished earlier in the process).

While I realize the vpnit script already does most of this, I need to
modify it a bit, which requires a rewrite because I never could get
the hang of the way you use "bash shell operations".

[Richard Kettlewell]

William Unruh <un...@invalid.ca> writes:
> On 2016-03-06, VPN user <vpn...@example.com> wrote:
>> William Unruh wrote in message nbhqeq$uis$1...@dont-email.me
>>

>>> corpse hangs around but cannot do anything.
>>>
>>> http://askubuntu.com/questions/201303/what-is-defunct-for-a-process-and-why-it-doesnt-get-killed
>>
>> I tried to kill it using that method of using *both* PIDs that showed
>> up in a ps of defunct, and the machine crashed.
>
> Not sure what you mean. If you kill PID 1 yes, you will crash.

No, killing pid 1 will not cause a crash.

--
http://www.greenend.org.uk/rjk/

[Marek Novotny]

On 2016-03-08, VPN user <vpn...@example.com> wrote:

// snip

> While I realize the vpnit script already does most of this, I need to
> modify it a bit, which requires a rewrite because I never could get
> the hang of the way you use "bash shell operations".

When I first got started there was plenty of help that went over my
head. I would just save it and look at it once a month or so. As you go
forward it will start to make more and more sense.

What helped me a lot is that I wrote lots of really tiny scripts that
did one simple thing. Often just two or three lines of code. And then
later when I wanted to accomplish a task I would use those as references
for how to do one particular thing.

Keep doing things like that. Try to break things down into something
simple. That way it becomes less confusing. And then one day the light
bulb goes off and it becomes more clear. The clearer it becomes the more
silly it becomes. Pretty soon a lot of it starts to make sense.

[VPN user]

Marek Novotny wrote in message

qp-dnSfcDJ7QzULL...@giganews.com

> When I first got started there was plenty of help that went over my
> head. I would just save it and look at it once a month or so. As you go
> forward it will start to make more and more sense.
>
> What helped me a lot is that I wrote lots of really tiny scripts that
> did one simple thing. Often just two or three lines of code. And then
> later when I wanted to accomplish a task I would use those as references
> for how to do one particular thing.
>
> Keep doing things like that. Try to break things down into something
> simple. That way it becomes less confusing. And then one day the light
> bulb goes off and it becomes more clear. The clearer it becomes the more
> silly it becomes. Pretty soon a lot of it starts to make sense.

Hi Marek,
What I love is that you seem to *understand* what it's like to get
help that is over my head. Thanks for understanding.

Also, I like your advice to make small simple scripts, which is
what my vpnmove is, for example, which simply moves vpn_winners
files into country directories based on geotagged country names:

#!/bin/bash
# vpnmove.sh moves geotagged config files into country directories
for FNAME in $(ls *.ovpn) ; do
COUNTRY=$(echo $FNAME|awk -F_ '{print $2}')
mkdir -p $COUNTRY
echo "Moving $FNAME into $COUNTRY subfolder"
mv $FNAME $COUNTRY
done

BTW, I ran your "vpnkickorkeep" on a thousand previously *bad* files,
where I found something interesting (to me anyway).

When I ran vpnkickorkeep on old vpn_losers directories, the ratio
of now-good previously-bad files was 1:4 (about two hundred out
of a thousand were suddenly good again).

So, I guess the moral of that story is to both keep the old files
around, and to run vpnkickorkeep periodically on those old files.

At the moment I have at least 1200 "good" files in the vpn_winners
directory, which, if we divide by an average of 4 files per server,
means that I have about 300 good VPN servers to choose from, which,
one would think, should be enough. :)

Your scripts make it so easy to handle thousands of files!
Thanks.

[Marek Novotny]

On 2016-03-09, VPN user <vpn...@example.com> wrote:
> Also, I like your advice to make small simple scripts, which is
> what my vpnmove is, for example, which simply moves vpn_winners
> files into country directories based on geotagged country names:
>
> #!/bin/bash
> # vpnmove.sh moves geotagged config files into country directories
> for FNAME in $(ls *.ovpn) ; do
> COUNTRY=$(echo $FNAME|awk -F_ '{print $2}')
> mkdir -p $COUNTRY
> echo "Moving $FNAME into $COUNTRY subfolder"
> mv $FNAME $COUNTRY
> done

Well, the first thing is that you know what the source file looks like
but I do not. So when you post something, always show the details.

Don't use all uppercase variables. Your Environment variables are all
upper case. You don't want to confuse the two. There are a few styles
for naming your variables.

var_name
_var_name
varName

I use what is called, camelCase, which can be CamelCase or camelCase. I
picked up a book on C and the author used camelCase and I found it
simple to read so I adopted it. Choose something you like. You want your
variables to be descriptive and easy to read while at the same time not
being confused with environment variables.

I'd also suggest better spacing which makes it easier to read.

for fileName in $(ls *.ovpn) ; do

That's easy to read and everyone knows I'm looking at the file name.

country="$(echo $fileName | cut -d'_' -f2)"

I'd space the pipe so it is easy to read what I've done. Not everyone is
familiar with awk beyond the default delimiter. I personally prefer awk
but cut is very descriptive. cut meaning to cut something into parts and
-d meaning set the delimiter and '_' meaning use an underscore as that
delimiter. Lastly we can -f2 which you might guess means take the second
field assuming once the underscore has been used to define the field
delimiter.

Your use of makedir is always going to place files into directories at
the current working directory. Is that what you want? What would be a
better idea?

This is good for a small simple script. It will do fine for something
you're doing for yourself. When you start to think about others using
it, things will need to change. But for now focus on simple tools for
yourself.

Now, it's also a good idea to know why you do things. Did someone tell
you to use makedir -p or did you come up with that yourself. What is the
benefit of makedir -p over makedir? Using the man page reply back with
why it is useful here.

[VPN user]

Marek Novotny wrote in message

dMudnRYQNaGKEULL...@giganews.com

> Well, the first thing is that you know what the source file looks like
> but I do not. So when you post something, always show the details.

Hi Marek,
This is a good point that I didn't mention what the source files
looked like.

With your modified vpnkickorkeep script, all the "good" source files
have been renamed with their geotag, so they all have as a file name
the country as the second field, as in these real examples:
vpngate_JP_40_Tokyo_Tokyo_60.239.253.106-60.239.253.106_udp1516_20160307.ovpn
vpngate_GB_V8_NorthLanarkshire_Motherwell_82.32.75.42-vpn969287065.opengw.net_tcp1843_20160305.ovpn
vpngate_US_OH_Ohio_GroveCity_204.210.244.81-204.210.244.81_udp1548_20160307.ovpn

> Don't use all uppercase variables. Your Environment variables are all
> upper case. You don't want to confuse the two. There are a few styles
> for naming your variables.
>
> var_name
> _var_name
> varName

That is a very *interesting* observation because I have been using
all-uppercase variables ever since I first started writing shell scripts
maybe 10 years ago or so. I always thought that was the right way to
do things.

I realize there are coding styles, but I don't actually use environment
variables. Well, I guess I use $HOME, but I don't think I use any other
shell environment variables than $HOME.

Googling for how to list all my bash shell environment variables, I see
this command. Egads! There are a ton of them! (87 to be exact).

$ printenv | wc -l
87

Of those 87 existing shell environment variables, I use exactly 1, which
is $HOME.

> I use what is called, camelCase, which can be CamelCase or camelCase. I
> picked up a book on C and the author used camelCase and I found it
> simple to read so I adopted it. Choose something you like. You want your
> variables to be descriptive and easy to read while at the same time not
> being confused with environment variables.

Thanks for the suggestion for camel-case variables. I think I *like*
all-uppercase, so, I think I'll stick with that, but I will ensure that
I don't clash with the existing 87 ones (looking at them, I don't think
I will since they are mostly crazy names like DBUS_SESSION_BUS_ADDRESS,
GTK_IM_MODULE, TEXTDOMAIN, SESSIONTYPE, SHLVL, and TEXTDOMAINDIR).

> I'd also suggest better spacing which makes it easier to read.
> for fileName in $(ls *.ovpn) ; do
> That's easy to read and everyone knows I'm looking at the file name.

This is a good idea as it does make it easier to read.

> country="$(echo $fileName | cut -d'_' -f2)"
>
> I'd space the pipe so it is easy to read what I've done.

Again, adding whitespace does make it more readable, I agree.

> Not everyone is familiar with awk beyond the default delimiter.
> I personally prefer awk but cut is very descriptive.
> cut meaning to cut something into parts and -d meaning set the delimiter
> and '_' meaning use an underscore as that delimiter.

Thanks for the advice on cut.

I see these are equivalent:
country=$(echo $fileName | awk -F_ '{print $2}')

country="$(echo $fileName | cut -d'_' -f2)"

I've made the change.

> Lastly we can -f2 which you might guess means take the second
> field assuming once the underscore has been used to define the field
> delimiter.

I'm not sure which is better but I do notice you added double quotes
where I didn't have any.

> Your use of makedir is always going to place files into directories at
> the current working directory. Is that what you want? What would be a
> better idea?

Yes. I always run the vpnmove.sh script when I'm in the vpn_winners
directory. The files get into the vpn_winners directory from your
kickorkeep script (which I modified to rename them as it moves the
files into the vpn_winners directory, based on geolocation).

I guess a better idea would be to use a shell environment variable
for the vpn_winners directory, since the country codes will always
be in the vpn_winners directory and the vpn_winners directory is
used in *multiple* scripts.

> This is good for a small simple script. It will do fine for something
> you're doing for yourself. When you start to think about others using
> it, things will need to change. But for now focus on simple tools for
> yourself.

Oh my! I do agree that others would have a LOT of trouble reading
and using "my" shell scripts. It must be ten times harder to write
something for general use than for specific use. That's why I see
a lot of checks in your scripts, e.g., checks for the geolocation
databases and for the file names and for the available of sudo, etc.

> Now, it's also a good idea to know why you do things. Did someone tell
> you to use makedir -p or did you come up with that yourself.

I used the "-p" option to mkdir because I first didn't use it and
the command complained that the directory already existed, so I
used the manpage and figured out that "-p" would allow the mkdir
to not complain if the directory already existed (which, most of
the time, it does).

> What is the benefit of makedir -p over makedir?
> Using the man page reply back with why it is useful here.

In the manpage, the two-line entry for "-p" is the following:
-p, --parents
no error if existing, make parent directories as needed

That means that the "-p" option does two things:
1. It will make *all* the directories if necessary, e.g.,
mkdir -p /a/b/c/d/e/f/g/vpn_winners

2. It won't complain if the directory already exists.

The reason "I" used the "-p" option was #2 above, as I *knew* that
most of the time the two-letter country-code directories would
already exist in the vpn_winners directory.

[VPN user]

VPN user wrote in message nbo4c0$2cg$1...@news.mixmin.net

>> Your use of makedir is always going to place files into directories at
>> the current working directory. Is that what you want? What would be a
>> better idea?
>
> Yes. I always run the vpnmove.sh script when I'm in the vpn_winners
> directory. The files get into the vpn_winners directory from your
> kickorkeep script (which I modified to rename them as it moves the
> files into the vpn_winners directory, based on geolocation).
>
> I guess a better idea would be to use a shell environment variable
> for the vpn_winners directory, since the country codes will always
> be in the vpn_winners directory and the vpn_winners directory is
> used in *multiple* scripts.

I guess a better idea would also be a *check* to see if I'm
accidentally *not* in the vpn_winners directory.

[VPN user]

VPN user wrote in message nbo4c0$2cg$1...@news.mixmin.net

> Thanks for the suggestion for camel-case variables. I think I *like*
> all-uppercase

Thinking about it some more, I *do* understand that your suggestion
makes more sense because if other people use my scripts, we can never
be sure what shell environment variables *they* have.

So, it does make sense to use:

a. All uppercase for shell environment variables, and,
b. Camelcase for script environment variables.

[Marek Novotny]

On 2016-03-09, VPN user <vpn...@example.com> wrote:

> Marek Novotny wrote in message

// snip

>> What is the benefit of makedir -p over makedir?
>> Using the man page reply back with why it is useful here.
>
> In the manpage, the two-line entry for "-p" is the following:
> -p, --parents
> no error if existing, make parent directories as needed
>
> That means that the "-p" option does two things:
> 1. It will make *all* the directories if necessary, e.g.,
> mkdir -p /a/b/c/d/e/f/g/vpn_winners
>
> 2. It won't complain if the directory already exists.
>
> The reason "I" used the "-p" option was #2 above, as I *knew* that
> most of the time the two-letter country-code directories would
> already exist in the vpn_winners directory.

Good. Now, go back to the man page for mkdir. What does MKDIR(1) mean
all the way at the top left corner?

[Marek Novotny]

On 2016-03-09, VPN user <vpn...@example.com> wrote:

You can use underscores as well. My way is my way. If you like that
method then use it. If you like underscores, choose that. Just be
consistent and glad to hear you won't be using all UPPERS.

[VPN user]

Marek Novotny wrote in message

4cidnULoZ6jdBULL...@giganews.com

> Good. Now, go back to the man page for mkdir. What does MKDIR(1) mean
> all the way at the top left corner?

I never use "man" per se, because I use "manvi", as in:
$ manvi man

Where "manvi" is the following script:
$ which manvi
/usr/local/bin/manvi

And manvi is:
$ cat $(which manvi)
#!/bin/bash
# Usage: manvi cmd
man $1 | col -b > /tmp/$1.txt
vi /tmp/$1.txt

I had never noticed the "MAN(1)" at top left, but there it is.
In that page it says: info coreutils 'mkdir invocation', so:

$ info coreutils 'mkdir invocation'
...
`mkdir' creates each directory NAME in the order given. It reports
an error if NAME already exists, unless the `-p' option is given and
NAME is a directory.
...
`--parents'
Make any missing parent directories for each argument, setting
their file permission bits to the umask modified by `u+wx'. Ignore
existing parent directories, and do not change their file
permission bits.

To set the file permission bits of any newly-created parent
directories to a value that includes `u+wx', you can set the umask
before invoking `mkdir'. For example, if the shell command
`(umask u=rwx,go=rx; mkdir -p P/Q)' creates the parent `P' it sets
the parent's permission bits to `u=rwx,go=rx'. To set a parent's
special mode bits as well, you can invoke `chmod' after `mkdir'.
*Note Directory Setuid and Setgid::, for how the set-user-ID and
set-group-ID bits of newly-created parent directories are
inherited.

So I had to look up the "umask":
$ umask
0002

Then, I guessed the syntax of the *second* level of the manpage:
$ man 2 mkdir
Holy cow. It worked! :)

So I created a new "manvi2" command:
$ cat /usr/local/bin/manvi2
#!/bin/bash
# Usage: manvi2 cmd
man 2 $1 | col -b > /tmp/$1.txt
vi /tmp/$1.txt

Now it says "MKDIR(2)" at the top left.
...
mkdir() returns zero on success, or -1 if an error occurred (in which case,
errno is set appropriately).

So, I guess you're suggesting I "trap" the error code.
Googling, I find:
http://www.linuxjournal.com/article/10844
Work the Shell - Understanding Exit Codes

After reading the first few lines, I tried this quick test on my own:
$ mkdir /tmp/foo
$ echo $?
0
$ mkdir /tmp/foo
mkdir: cannot create directory ‘/tmp/foo’: File exists
$ echo $?
1

OK. So the exit code tells me what happened with the mkdir command.

Reading more, I found this script:
#!/bin/sh
mkdir /usr
error=$?
if [ $error -ne 0 ] ; then
echo "mkdir /usr failed: we have an exit code of $error"
exit 1
fi

And, they also had this command:
$ mkdir /usr >& /dev/null

So I created a test directory:
$ mkdir /tmp/vpn_winners
$ cd !$
$ alias xxx=`for i in 1 2 3; do for j in AA BB; do touch vpngate_${j}_${i}.ovpn; done ; done`
$ xxx
$ ls
vpngate_AA_1.ovpn vpngate_AA_2.ovpn vpngate_AA_3.ovpn
vpngate_BB_1.ovpn vpngate_BB_2.ovpn vpngate_BB_3.ovpn

And I started with the old vpnmove command:
$ cat $(which vpnmove.sh)
#!/bin/bash
# moves renamed VPN files into iso 3166 country folders

for fileName in $(ls *.ovpn) ; do

# isoCountry=$(echo $fileName|awk -F_ '{print $2}')
isoCountry="$(echo $fileName | cut -d'_' -f2)"
mkdir -p $isoCountry
echo "Moving $fileName into $isoCountry subfolder"
mv $fileName $isoCountry
done

I was gonna change it to check the "mkdir" exit code, but, then I realized
I should really just check to see if the directory already exists.

So, googling for how to check if a directory exists, I found this (among others):
https://stackoverflow.com/questions/59838/check-if-a-directory-exists-in-a-shell-script

Which had this snippet:
if [ ! -d "$DIRECTORY" ]; then
# Control will enter here if $DIRECTORY doesn't exist.
fi

So, taking advantage of that snippet, I changed the vpnmove script to:
$ cat $(which vpnmove.sh)
#!/bin/bash
# moves renamed VPN files into iso 3166 country folders

for fileName in $(ls *.ovpn) ; do

isoCountry="$(echo $fileName | cut -d'_' -f2)"
if [ ! -d "$isoCountry" ]; then
mkdir $isoCountry
fi
echo "Moving $fileName into $isoCountry subfolder"
mv $fileName $isoCountry
done

I just realized that the files clobbered existing files in the ISO country
directories, but, for now, that's OK. (I should probably print an error
when they clobber an existing file).

But, anyway, I think you wanted me to get rid of the "-p" and now it's gone!

[VPN user]

VPN user wrote in message nboje2$pd8$1...@news.mixmin.net

> I just realized that the files clobbered existing files in the ISO country
> directories, but, for now, that's OK. (I should probably print an error
> when they clobber an existing file).

Feeling guilty for writing f'ugly code, and getting a bit adventurous,
I decided to add my first "file check" to go along with my first "directory check"
ever....

#!/bin/bash
# moves renamed VPN files into iso 3166 country folders
for fileName in $(ls *.ovpn) ; do
isoCountry="$(echo $fileName | cut -d'_' -f2)"
if [ ! -d "$isoCountry" ]; then
mkdir $isoCountry
fi
echo "Moving $fileName into $isoCountry subfolder"

if [ ! -f ${isoCountry}/${fileName} ]; then
mv $fileName $isoCountry
else
echo "...Ooops: ${isoCountry}/${fileName} already exists; not moving"
fi
done

[Marek Novotny]

On 2016-03-09, VPN user <vpn...@example.com> wrote:

> Marek Novotny wrote in message
> 4cidnULoZ6jdBULL...@giganews.com
>
>> Good. Now, go back to the man page for mkdir. What does MKDIR(1) mean
>> all the way at the top left corner?
>
> I never use "man" per se, because I use "manvi", as in:
> $ manvi man

Okay, so I'd suggest not doing that for a while. Instead I'd focus on
understanding man and how to use it. Go back to basics so you can build
a better foundation from which to go forward on.

At the top of the man page for mkdir, the mkdir(1) means this:

1. Executable programs or shell commands

If you type man mkdir you're getting man 1 mkdir because that is the
first man page available for mkdir.

Now do this:

$ which mkdir
/bin/mkdir

Now do this...

$ which shutdown
/sbin/shutdown

Now get the man page for shutdown. Do not use your method, use mine.
Just use the man page using man. Tell me what's at the top and why it is
different from what's at the top for the man page of mkdir.

What you have learned about using the man pages as a result?

The purpose of this is to start using man pages effectively so that when
you see other code you can effectively use the man page to understand
what the code means and why some of the surrounding code exists.

Here is a simple shell script. I used the man page to write it and it
will teach you a little more about man pages and builtins. Read the
script and use the man page to tell me how it works and what it does.

#!/bin/bash

#########################################################
#
# script: manit
# version: .01
# date: 2016-03-09
# written by: marek novotny
# git: https://github.com/marek-novotny/
# license: GPL v2 (only)
# dependencies: man pages and built in help
# purpose: Show man page or help page
# usage: man {argument} as in man page
# : lookup tool.
#
#########################################################

appName="$(basename $0)"

sendMessage () {
if [ $1 -ge 1 ] ; then
echo "$2" >&2 && exit $1
else
echo "$2"
fi
}

if [ $# -ne 1 ] ; then
sendMessage 1 "$appName usage error: Use $appName {argument}"
else
argument="$1"
fi

man $argument &> /dev/null
case $? in
0 ) sendMessage 0 "Successful program execution."
sendMessage 0 "Using the man page to argument..."
man $argument
;;
1 ) sendMessage 0 "Usage, Syntax or configuration file error."
;;
2 ) sendMesaage 0 "Operational Error"
;;
3 ) sendMessage 0 "A child process returned a non-zero exit status"
;;
16 ) sendMessage 0 "At least one of the pages/files/keywords didn't exist or wasn't matched."
sendMessage 0 "Using help $argument for this built in..."
help $argument | less
;;
esac

## END ##

[VPN user]

Marek Novotny wrote in message

eoOdnQHUluHIzX3L...@giganews.com

> Now do this:

Hi Marek,

I just saw this; but I have been writing the "vpnspeed.sh" script
so I didn't see this manpage test until now.

I'll look at the manpage test, but, first, I came here just now to
post the rudimentary vpnspeed.sh renaming script that I hacked out.

It's an early first pass, so it's f'ugly; but it works (so far).
I'm still testing it so it's not ready to clean up just yet.

This vpnspeed is intended to be run *during* a successful VPN run,
where it's only intended to *add* information to the ovpn file name.

#!/bin/bash
# vpnspeed.sh appends speedtest results to the currently running ovpn file.
# Thanks to Marek Novotny for the syntax to capture speedtest-cli results.
# WIP: I need to add a test to abort if openvpn is not running.
# WIP: I need to streamline the overall flow.

# Obtain the currently running ovpn file name:
fileName=$(ps -elfww|grep "sudo openvpn" | grep -v grep | head -1| awk -F'--config ' '{print $2}'| cut -d' ' -f1)
shortFileName=$(echo $fileName|sed -e 's/.ovpn//')

# Test if the currently running ovpn file is in the current directory:
if [ ! -f $fileName ]; then
echo "Ooops: ${fileName} is not in the current directory; aborting"
exit 1
else
echo "Good: ${fileName} is in the current directory; proceeding..."
fi

# Run the speedtest (courtesy of Marek Novotny):
echo "$(basename $0) message: collecting speed test data..."
IFS=$'\n'
set -- $(speedtest-cli --simple --secure --timeout 5)
pingResult="${1/*: /}"
downResult="${2/*: /}"
upResult="${3/*: /}"
IFS=$'\t\n '
pingR=${pingResult/.*/}
downR=${downResult/' Mbit/s'/}
upR=${upResult/' Mbit/s'/}

# Determine if each of the 3 speedtest results are numerical:
isnum() { awk -v a="$1" 'BEGIN {print (a == a + 0)}'; }
isNumPingR=`isnum "$pingR"`
isNumUpR=`isnum "$upR"`
isNumDownR=`isnum "$downR"`

# Rename the ovpn file *only* when all 3 speedtest results are numerical:
if [ "$isNumPingR" == "1" ] && [ "$isNumUpR" == "1" ] && [ "$isNumDownR" == "1" ]; then
echo "All 3 speedtest-cli results are numerical; rename $fileName..."
if [ -f $fileName ]; then
echo "Renaming $fileName to ${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn"
mv $fileName ${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn
else
echo "Ooops: ${fileName} is not in the current directory; not renaming"
fi
else
echo "Speedtest results are not numerical; can't rename $fileName"
fi

# PS: Yes, I know that I tested for the file twice (that's an historical artifact).
# PS: Yes, I know the "isNumPingR" is using the old syntax (for now, that's fine).
exit 0
## END ##

[Marek Novotny]

On 2016-03-10, VPN user <vpn...@example.com> wrote:
> Marek Novotny wrote in message
> eoOdnQHUluHIzX3L...@giganews.com
>
>> Now do this:
>
> Hi Marek,
>
> I just saw this; but I have been writing the "vpnspeed.sh" script
> so I didn't see this manpage test until now.
>
> I'll look at the manpage test, but, first, I came here just now to
> post the rudimentary vpnspeed.sh renaming script that I hacked out.
>
> It's an early first pass, so it's f'ugly; but it works (so far).
> I'm still testing it so it's not ready to clean up just yet.
>
> This vpnspeed is intended to be run *during* a successful VPN run,
> where it's only intended to *add* information to the ovpn file name.

Okay, how about adding a test to see if openvpn is running and if not,
then exit. Try to add that.

And here are two short cuts for you to look into and learn from...

fileName=$(basename $(cat /proc/$(pgrep openvpn)/cmdline))
shortFileName="${fileName%%.*}"

Also in my original script I wanted to show two different pieces of info
and parameter expansion worked for those two stages. Consider a shorter,
single method to get what you want rather than combining my two stages
for two purposes to get the end result you want.

[VPN user]

Marek Novotny wrote in message

U6udnbKMop7TZX3L...@giganews.com

> Okay, how about adding a test to see if openvpn is running and if not,
> then exit. Try to add that.
>
> And here are two short cuts for you to look into and learn from...
>
> fileName=$(basename $(cat /proc/$(pgrep openvpn)/cmdline))
> shortFileName="${fileName%%.*}"
>
> Also in my original script I wanted to show two different pieces of info
> and parameter expansion worked for those two stages. Consider a shorter,
> single method to get what you want rather than combining my two stages
> for two purposes to get the end result you want.

I just came here to post my update to the vpnspeed, so I haven't implemented
your cat proc pgrep but I think you may have solved my problem of obtaining
the *full filespec* to the config file when the vpnspeed script is *not*
run in the same directory as was the openvpn command!

Here's my "debug" script *before* taking into account what you wrote above.
I will now test out what you wrote above...

#!/bin/bash
# vpnspeed.sh appends speedtest results to the currently running ovpn file.

# WIP1: Need a better way to guess the full filespec if the ovpn file is not in the current directory!
# WIP2: This is a debug version so many of the "echo" statements will be removed.
# WIP3: The entire flow needs to be streamlined better.

# Obtain the currently running ovpn file name:
fileName=$(ps -elfww|grep "sudo openvpn" | grep -v grep | head -1| awk -F'--config ' '{print $2}'| cut -d' ' -f1)

# Test if the variable is empty which probably means openvpn is not running:
if [ -z "$fileName" ]; then
echo "Oops: \$fileName is empty which means openvpn is probably not running; aborting"
exit 1
else
echo "Good: openvpn is currently running"

shortFileName=$(echo $fileName|sed -e 's/.ovpn//')

fi

# Create the full filespec to the currently running ovpn file:
# WIP1: Is there a better way to obtain the *full* file spec?
if [ -f $fileName ]; then

echo "Good: ${fileName} is in the current directory"

fullFileDir=$(pwd)/
fullFileSpec=$(echo "${fullFileDir}${fileName}")
echo "Good: \$fullFileSpec is $fullFileSpec"
fullFilePrefix=$(echo $fullFileSpec|sed -e 's/.ovpn$//')
echo "Good: \$fullFilePrefix is $fullFilePrefix"
echo "Good: \$fullFileDir is $fullFileDir"
else
echo "Oops: ${fileName} is not in the current directory; guessing the filespec"
sudo /usr/bin/updatedb
fullFileSpec=$(/usr/bin/locate $fileName|head -1)
echo "Good: Assuming \$fullFileSpec is $fullFileSpec"
fullFilePrefix=$(echo $fullFileSpec|sed -e 's/.ovpn$//')
echo "Good: Assuming \$fullFilePrefix is $fullFilePrefix"
fullFileDir=$(echo $fullFileSpec|sed 's%/[^/]*$%/%')
echo "Good: Assuming \$fullFileDir is $fullFileDir"
fi

# Capture speedtest into variables (courtesy of Marek Novotny):
echo "Good: $(basename $0) is now collecting speed test data"

IFS=$'\n'
set -- $(speedtest-cli --simple --secure --timeout 5)
pingResult="${1/*: /}"
downResult="${2/*: /}"
upResult="${3/*: /}"
IFS=$'\t\n '

# WIP2: This is a debug version; the echo statements below will be removed later.
echo "\$pingResult is $pingResult"
pingR=${pingResult/.*/}
echo "\$pingR is $pingR"
echo " "

echo "\$downResult is $downResult"
downR=${downResult/' Mbit/s'/}
echo "\$downR is $downR"
echo " "

echo "\$upResult is $upResult"
upR=${upResult/' Mbit/s'/}
echo "\$upR is $upR"
echo " "

# Determine if each of the speedtest-cli results are numerical:

isnum() { awk -v a="$1" 'BEGIN {print (a == a + 0)}'; }

# isNumPingR=`isnum "$pingR"`
isNumPingR=$(isnum "$pingR")
echo "isNumPingR is $isNumPingR"

# isNumUpR=`isnum "$upR"`
isNumUpR=$(isnum "$upR")
echo "isNumUpR is $isNumUpR"

# isNumDownR=`isnum "$downR"`
isNumDownR=$(isnum "$downR")
echo "isNumDownR is $isNumDownR"

# Rename the file only if all 3 speedtest-cli results are numerical:
# WIP3: There is no need to check the full file spec again (remove that test when streamlining)

if [ "$isNumPingR" == "1" ] && [ "$isNumUpR" == "1" ] && [ "$isNumDownR" == "1" ]; then

echo "Good: All 3 speedtest-cli results are numerical; rename $fileName..."
if [ -f $fullFileSpec ]; then
echo "Good: Moving $fullFileSpec to ${fullFileDir}renamed/${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn"
if [ ! -d "${fullFileDir}renamed" ]; then
echo "Making ${fullFileDir}renamed"
mkdir ${fullFileDir}renamed
fi
mv $fullFileSpec ${fullFileDir}renamed/${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn
else
echo "Oops: ${fullFileSpec} does not exist; not renaming"
exit 1
fi
else
echo "Oops: Speedtest results are not all numerical; can't rename $fileName"
exit 1
fi

exit 0
## End ##

[/less]

On 03/09/2016 04:05 AM, VPN user wrote:
> Marek Novotny wrote in message
> xtKdnVf3uMPKGUDL...@giganews.com
>> It's fun to pick a pet project and work on it regardless of the time
>> involved because it serves many purposes.
> I agree with you Marek.
>

[snip]
You guys obviously are having the time, expertness and seem to have a
lot fun for these kind of 'hobby' projects.

Existing free VPNs are for the proficient ones (there are evidently only
a few participating here, others seem just steer away from this
particular thread).
Why is it so difficult developing or working purposefully on a VPN
application (similar to the paid ones) which can be applied by the
'broader' community i.e.technical challenged users?
Has it something to do with accessibility of world-wide servers and/or
an inevitably commitment for providing continued services?

[VPN user]

VPN user wrote in message nbqveo$r2k$1...@news.mixmin.net

> # WIP1: Need a better way to guess the full filespec if the ovpn
> file is not in the current directory!

My current "big" problem is finding the full filespec to the ovpn file.

My idea was to use "updatedb" & "locate"; but that's not a great idea
because *multiple* ovpn files with the same name /might/ exist.

So, I like your "/proc" idea better; but when I tested it in a script,
something went wrong because it got the "resolv.conf" part of the way
that I ran openvpn:
#!/bin/bash
# testme.sh
fileName=$(basename $(cat /proc/$(pgrep openvpn)/cmdline))
echo "\$fileName is $fileName"
shortFileName="${fileName%%.*}"
echo "\$shortFileName is $shortFileName"
exit 0

$ testme.sh
$fileName is update-resolv-conf
$shortFileName is update-resolv-conf

I realize this is because you run openvpn differently than I do, since I
run openvpn with "resolv.conf" specified in the command line (so as to
avoid the known DNS leaks problem when running command-line openvpn):

$ pgrep openvpn
5641

$ cat /proc/5641/cmdline
openvpn--configvpngate_US_FL_Florida_FortLauderdale_99.117.129.116-99.117.129.116_udp1195_20160304.ovpn--script-security2--up/etc/openvpn/update-resolv-conf--down/etc/openvpn/update-resolv-conf

Unfortunately, the full filespec to the openvpn config file is *not*
specified when I run openvpn using the "vpnrun.sh" or "vpnstatus.sh"
scripts; but I think you're still correct in using the /proc directory!

Looking at the manpage for /proc, I see that there is a concept of "cwd":
$ man proc
/proc/[pid]/cwd
This is a symbolic link to the current working directory of the process.
To find out the current working directory of process 20, for instance,
you can do this: $ cd /proc/20/cwd; /bin/pwd
Note that the pwd command is often a shell built-in, and might not work
properly. In bash(1), you may use this: $ pwd -P.

So, apparently this "cwd" will tell me the current working directory
of the openvpn script, which (*in my use model*) is also always the same
as the actual directory containing the ovpn file.

But that "cwd" capability is a bit hard to get at because of permissions!

The manpage for proc says this will work; but it fails due to permissions:
$ pgrep openvpn
5641
$ cd /proc/5641/cwd; /bin/pwd -P
bash: cd: /proc/5641/cwd: Permission denied

Giving up on the incorrect manpage for proc, this "ls" seems to tell me the
directory in which openvpn was run (which, *in my use model*, is the same
directory as the openvpn config file):

$ sudo ls -l /proc/5641/cwd
lrwxrwxrwx 1 root root 0 Mar 9 21:20 /proc/5641/cwd -> /home/vpnuser/doc/cert/vpn_winners/US

But I have to clean it up a bit to get "just" the openvpn cwd:
$ sudo ls -l /proc/5641/cwd | awk -F" " '{print $NF}'
/home/vpnuser/doc/cert/vpn_winners/US

Putting that into a generic command-format & assuming awk defaults, I get:
$ sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}'
/home/vpnuser/doc/cert/vpn_winners/US

Putting that into variables for the bash script, I get the following:
openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')

In summary, using your wonderful "proc" idea, I can dispense altogether
with the bad "ps -elfww|grep openwin" idea.

Looking at the proc manpage, I see also that I can test whether openvpn
is even running, because the "pgrep openvpn" will return a null value:
$ set -- $(pgrep openvpn)
$ echo $1
5641

$ set -- $(pgrep notopenvpn)
$ echo $1
<nothing is returned>

So, using your "set --" idea, I think this tests whether openvpn is running:
$ set -- $(pgrep openvpn)
$ if [ -z "$1" ]; then echo "openvpn is not running"
fi
$ if [ -n "$1" ]; then echo "openvpn is not running"
fi

So, since this post is getting long, I will separately add those fixes
into the vpnspeed.sh script and report back when/if successful.

[VPN user]

Marek Novotny wrote in message

U6udnbKMop7TZX3L...@giganews.com

> Okay, how about adding a test to see if openvpn is running and if not,
> then exit.

Thanks for the idea of using /proc (which I had never heard of before)!

Using your suggestions, I tested "proc" out in this bash script snippet.
WIP: I do realize that having two nested "if" commands is redundant though.

# Test whether openvpn is running
set -- $(pgrep openvpn)
if [ -z "$1" ]; then
echo "Oops: Apparently openvpn is not currently running"
exit 1
else
# If openvpn is running, obtain the cwd directory openvpn was run from:
if [ -n "$1" ]; then
echo "Good: Apparently openvpn is running"
openvpnPID=$1
echo "\$openvpnPID is ${openvpnPID}"

openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')

echo "\$openvpnCWD is ${openvpnCWD}"
fi
fi

I will add this snippet to the vpnspeed.sh script momentarily.

Your idea of using "proc" was great!

[VPN user]

/less wrote in message nbr2vf$19h3$1...@gioia.aioe.org

> You guys obviously are having the time, expertness and seem to have a
> lot fun for these kind of 'hobby' projects.
>
> Existing free VPNs are for the proficient ones (there are evidently only
> a few participating here, others seem just steer away from this
> particular thread).

Hi /less/,

Each VPN service has its pros and cons (like everything in life).

There are fewer than ten free public VPN services that I know of.

Of those, only vpngate seems to have *thousands* of servers to choose from.
The rest of the free VPN services only have a small handful (e.g., typically
only a half dozen or so in my humble experience).

So, I *like* that one benefit of the vpngate process which Marek and I
(and about a dozen others such as Jonathan N. Little, Mike Easter,
William Unruh, Aitch, Bit Twister, J.O. Aho, Eef Hartman, JG Miller,
et. al.) have honed over the past year.

Currently, I have about a thousand tested good openvpn config files from
over 50 countries, which, at an average of 4 config files per server is
about 250 different servers.

I'm curious...

Can you get *that* kind of flexibility out of your payware VPN service?

[VPN user]

/less wrote in message nbr2vf$19h3$1...@gioia.aioe.org

> Why is it so difficult developing or working purposefully on a VPN
> application (similar to the paid ones) which can be applied by the
> 'broader' community i.e.technical challenged users?
> Has it something to do with accessibility of world-wide servers and/or
> an inevitably commitment for providing continued services?

To answer that question above, I do NOT think the solution Marek and I
(and the others who pitched in from time to time) will work for the
general non-Linux user.

The reason that it won't work for the general user is that we chose
a specific server that *on purpose* provides bad files in 2 out of 3
cases!

This is to foil censors - but the *reason* they provide bad files on
purpose doesn't matter. What matters is that the bad files exist.

For the general user, this isn't really a big problem because the
general user does things *manually*.

1. They manually download a vpn config text file from vpngate.
2. They manually run openvpn on that file.

If it doesn't work - they manually download another text config file.
On average, 1 out of 3 will be good.

If I summarize what Marek and the others have enabled for me, it is that
they *automated* this manual process above.

The benefit is that I currently have over a thousand good VPN files
(all tested today using Marek's scripts) which is probably about 250
different VPN servers from over 50 different countries.

Can you get *that* kind of flexibility out of your current payware
VPN service?

[VPN user]

VPN user wrote in message nbr3e8$r2k$2...@news.mixmin.net

> So, since this post is getting long, I will separately add those fixes
> into the vpnspeed.sh script and report back when/if successful.

Hi Marek,

Here are snippets which worked great, thanks to your suggestions!

Your suggestions:
a. Removed the need for a "ps" to find whether openvpn was running.
b. Removed the need for "updatedb" & "locate" to guess the ovpn filespec.
c. Simplified the process of obtaining the "short" file name.

This new snippet, thanks to your suggestions, tells us whether openvpn
is currently running, and, if it is running, what directory it was run from.

# If openvpn is currently running, obtain the directory it was run from:
set -- $(pgrep openvpn)
if [ -z "$1" ]; then

echo "Oops: Apparently openvpn is not currently running"
exit 1
else

if [ -n "$1" ]; then

echo "Good: Apparently openvpn is running"
openvpnPID=$1
echo "\$openvpnPID is ${openvpnPID}"

openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')

echo "\$openvpnCWD is ${openvpnCWD}"
fi
fi

This new snippet uses your "proc" suggestion to generate the full path to
the openvpn file, and it uses your suggestion to shorten the file name.

Notice that "my" command below is slightly different than yours simply
because openvpn is run differently for me than for you (because I add
options to eliminate DNS leaks).

$ cat /proc/$(pgrep openvpn)/cmdline
REPORTS:
openvpn--configvpngate_US_FL_Florida_FortLauderdale_99.117.129.116-99.117.129.116_udp1195_20160304.ovpn--script-security2--up/etc/openvpn/update-resolv-conf--down/etc/openvpn/update-resolv-conf

So, the snippet "I" put in my vpnspeed script just now is the following:
# Obtain the full filespec & short name to the openvpn config file:
fileName=$(cat /proc/$(pgrep openvpn)/cmdline | awk -F"config" '{print $2}' | awk -F"--script" '{print $1}')
shortFileName="${fileName%%.*}"

[/less]

Well, since my VPN provider is using openvpn I can only assume that they
are using a sufficient number of config files deemed as adequate by
them. They operate on less servers but it seems that they full-fill
their tasks.

The flexibility of your VPN services is undoubtedly impressive but I
wonder if these multitude of choices are really required or desired and
essential?

Anyway, it's all very educational and I wish your guys all the best for
your projects.

[DecadentLinuxUserNumeroUno]

On Thu, 10 Mar 2016 14:49:58 +0700, /less <ro...@noabode.nil> Gave us:

>Anyway, it's all very educational and I wish your guys all the best for
>your projects.

Except when you chime in like an unsolicited mouthy little bitch.

[VPN user]

/less wrote in message nbr8v7$1g86$1...@gioia.aioe.org

> Well, since my VPN provider is using openvpn I can only assume that they
> are using a sufficient number of config files deemed as adequate by
> them.

Hi /less/,

That's the wrong argument. :)

The fact they use openvpn has nothing to do with how many servers they
allow you to access nor where those servers reside.

> They operate on less servers but it seems that they full-fill
> their tasks.

That was my point.

Each VPN provider has advantages and disadvantages.

Apparently the server farm I'm using contains over 4,000 servers in
about 60 or 75 countries.

That's *different* than what your server farm providers gives you.

It's not better; it's not worse.
It just depends on what your purpose is.

> The flexibility of your VPN services is undoubtedly impressive but I
> wonder if these multitude of choices are really required or desired and
> essential?

I think it's clear that it matters if you are being censored.
Also if spammers are using the servers, then web sites will shut them
down by putting them in black lists (as they do with most Tor Exit Nodes).

Again, I'm just saying that your choice and my choice are *both* valid.
They're just different.

[VPN user]

Marek Novotny wrote:

> shortFileName="${fileName%%.*}"

Hi Marek,

Of all your great ideas, this was the only one that failed. :(
The reason it fails is simply that the IP address also contains "dots".

Snippet...

openvpnFileName=$(cat /proc/$(pgrep openvpn)/cmdline | awk -F"--config" '{print $2}' | awk -F"--script" '{print $1}')
shortFileName="${openvpnFileName%%.*}"
echo "\$openvpnFileName is ${openvpnFileName}"
echo "\$shortFileName is ${shortFileName}"

RESULTS:
$openvpnFileName is vpngate_US_FL_Florida_Hollywood_107.141.185.58-107.141.185.58_udp1195_20160306.ovpn
$shortFileName is vpngate_US_FL_Florida_Hollywood_107

So, I'm going to go back to my sed example (which simply strips off the
".ovpn" extension.

[VPN user]

VPN user wrote in message nbr70g$r2k$6...@news.mixmin.net

> Your suggestions:
> a. Removed the need for a "ps" to find whether openvpn was running.
> b. Removed the need for "updatedb" & "locate" to guess the ovpn filespec.
> c. Simplified the process of obtaining the "short" file name.

Hi Marek (and everyone else who helped out!),

Below is a working vpnspeed script that *appends* speedtest statistics
to the currently running ovpn file name.

Thanks to Marek, this speedtest script now no longer needs "ps" or "locate"
just to figure out what the full filespec is for the current openvpn file.

And, this speedtest script no longer needs the "mkdir -p" (parent) option.

Also thanks to Marek, this speedtest script no longer needs to be run in
any particular directory; but due to the fact that the /proc/[pid]/cmd
symbolic link is protected, this script does (unfortunately) require
administrator (sudo) privileges.

As a failsafe, the script aborts if openvpn isn't running, or if the openvpn
file can't be found, or if any of the speedtest results are non-numeric
(which happens more frequently than we'd like).

Overall, the expected use model is something like the following:
1. Run vpnget & vpncut to download about 300 vpngate ovpn files & shorten names.
2. Run vpnkickorkeep to geotag-rename & move about 100 good files to vpn_winners
(which will also move about 200 bad files, not renamed, to vpn_losers).
3. Optionally, run vpnmove to organize vpn_winners into country directories.
4. Run vpnstatus (or vpnrun + vpnwatch) to start openvpn using a good config file
in the current working directory (keeping watch over sensitive processes)
5. Optionally, run vpnspeed to append speed stats to currently running config files
(which will move the renamed files into a lower-level "renamed" directory).

Note: Every command below has an alias to the shell script similar to this:
alias vpnkok='vpnkickorkeep.sh'

Example:
$ vpnget
$ vpncut
$ vpnkok
$ cd ./vpn_winners
$ vpnmove
$ cd ./US
$ vpnstatus vpngate_US_IN_Indiana_MichiganCity_67.186.113.210-67.186.113.210_udp1595_20160306.ovpn
$ vpnspeed
Good: It's clear that openvpn is running
Good: vpnspeed.sh is now collecting speed test data
Good: All 3 speedtest-cli results are numerical; therefore we can rename the ovpn file...
Good: Moving /home/vpnuser/doc/cert/vpn_winners/US/vpngate_US_IN_Indiana_MichiganCity_67.186.113.210-67.186.113.210_udp1595_20160306.ovpn to /home/vpnuser/doc/cert/vpn_winners/US/renamed/vpngate_US_IN_Indiana_MichiganCity_67.186.113.210-67.186.113.210_udp1595_20160306_8.82dn_0.44up_151ms.ovpn

Here is the vpnspeed.sh script.
(Improvements are always welcome!)

#!/bin/bash
# vpnspeed.sh appends speedtest results to the currently running ovpn file.

# Many thanks to Marek Novotny for patient algorithmic guidance & help.

# Test if openvpn is currently running & if so, find the directory it was run from:

set -- $(pgrep openvpn)
if [ -z "$1" ]; then
echo "Oops: Apparently openvpn is not currently running"
exit 1
else
if [ -n "$1" ]; then

echo "Good: It's clear that openvpn is running"

openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')

fi
fi

# Generate a long & short filename of the currently running openvpn config file:
openvpnFileName=$(cat /proc/$(pgrep openvpn)/cmdline | awk -F"--config" '{print $2}' | awk -F"--script" '{print $1}')
shortFileName=$(echo $openvpnFileName | sed -e 's/.ovpn//')

# Capture speedtest results into 3 variables (courtesy of Marek Novotny):

echo "Good: $(basename $0) is now collecting speed test data"
IFS=$'\n'
set -- $(speedtest-cli --simple --secure --timeout 5)
pingResult="${1/*: /}"
downResult="${2/*: /}"
upResult="${3/*: /}"
IFS=$'\t\n '

pingR=${pingResult/.*/}
downR=${downResult/' Mbit/s'/}
upR=${upResult/' Mbit/s'/}

# Determine if each of the speedtest-cli results are numerical:
isnum() { awk -v a="$1" 'BEGIN {print (a == a + 0)}'; }

isNumPingR=$(isnum "$pingR")
isNumDownR=$(isnum "$downR")
isNumUpR=$(isnum "$upR")

# Rename the current ovpn file only if all 3 speedtest-cli results are numerical:

if [ "$isNumPingR" == "1" ] && [ "$isNumUpR" == "1" ] && [ "$isNumDownR" == "1" ]; then

echo "Good: All 3 speedtest-cli results are numerical; therefore we can rename the ovpn file..."
if [ -f ${openvpnCWD}/${openvpnFileName} ]; then
echo "Good: Moving ${openvpnCWD}/${openvpnFileName} to ${openvpnCWD}/renamed/${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn"
if [ ! -d "${openvpnCWD}/renamed" ]; then
echo "Making ${openvpnCWD}/renamed directory"
mkdir ${openvpnCWD}/renamed
fi
mv ${openvpnCWD}/${openvpnFileName} ${openvpnCWD}/renamed/${shortFileName}_${downR}dn_${upR}up_${pingR}ms.ovpn
else
echo "Oops: ${openvpnCWD}/${openvpnFileName} does not exist; not renaming"
exit 1
fi
else
echo "Oops: Speedtest results are not all numerical; can't rename ${openvpnFileName}"

[VPN user]

Marek Novotny wrote in message

U6udnbKMop7TZX3L...@giganews.com

> Also in my original script I wanted to show two different pieces of info
> and parameter expansion worked for those two stages. Consider a shorter,
> single method to get what you want rather than combining my two stages
> for two purposes to get the end result you want.

Hi Marek,

I have to apologize for not doing this one suggestion.

Basically, the "bash shell operations" in that section below of the script
confound me such that I'm not gonna touch what you wrote, since it's working.

# Capture speedtest into variables (courtesy of Marek Novotny):

echo "Good: $(basename $0) is now collecting speed test data"

IFS=$'\n'
set -- $(speedtest-cli --simple --secure --timeout 5)
pingResult="${1/*: /}"
downResult="${2/*: /}"
upResult="${3/*: /}"
IFS=$'\t\n '
pingR=${pingResult/.*/}
downR=${downResult/' Mbit/s'/}
upR=${upResult/' Mbit/s'/}

# Determine if each of the speedtest-cli results are numerical:

isnum() { awk -v a="$1" 'BEGIN {print (a == a + 0)}'; }

isNumPingR=$(isnum "$pingR")
isNumDownR=$(isnum "$downR")
isNumUpR=$(isnum "$upR")

# Rename the current ovpn file only if all 3 speedtest-cli results are numerical:

if [ "$isNumPingR" == "1" ] && [ "$isNumUpR" == "1" ] && [ "$isNumDownR" == "1" ]; then

... blah blah blah ...

[Jasen Betts]

On 2016-03-10, VPN user <vpn...@example.com> wrote:

use a single percent and it trims off the shortest matching part instead of
the longest

shortFileName="${fileName%.*}"



--
\_(ツ)_

[mechanic]

On Thu, 10 Mar 2016 03:57:58 -0500, DecadentLinuxUserNumeroUno
wrote:

Pot, kettle, black!

[VPN user]

Jasen Betts wrote in message nbrjvc$p3n$1...@gonzo.alcatraz

> use a single percent and it trims off the shortest matching part instead of
> the longest
>
> shortFileName="${fileName%.*}"

That worked!

# Generate a filename & short filename of the currently running openvpn config file:

openvpnFileName=$(cat /proc/$(pgrep openvpn)/cmdline | awk -F"--config" '{print $2}' | awk -F"--script" '{print $1}')

shortFileName="${openvpnFileName%.*}"
# The "xxx" is to see if there's extra white space (which often catches me unaware).
echo "xxx${openvpnFileName}xxx"
echo "xxx${shortFileName}xxx"
exit 0

$ vpnspeed
xxxvpngate_US_AR_Arkansas_Batesville_173.216.94.150-vpn642246005.opengw.net_udp1195_20160310.ovpnxxx
xxxvpngate_US_AR_Arkansas_Batesville_173.216.94.150-vpn642246005.opengw.net_udp1195_20160310xxx

I have modified the vpnspeed script to use this percent syntax which is actually
better than the previous sed because the sed could have caught ".ovpn" in the middle
of the file, where this percent stuff presumably catches it at the end of the file.

Thanks.

[Bit Twister]

On Thu, 10 Mar 2016 07:16:32 -0000 (UTC), VPN user wrote:
>
> This new snippet, thanks to your suggestions, tells us whether openvpn
> is currently running, and, if it is running, what directory it was run from.
>
> # If openvpn is currently running, obtain the directory it was run from:
> set -- $(pgrep openvpn)
> if [ -z "$1" ]; then

Oh by the way.
Off hand I am guessing you are not using "set - u" in your scripts OR
not bothering to test your code. :(

While in the learning stage, I suggest running "man command_here" to
see what command line arguments are provided by the command.

When testing for something it is nice if the actual "test command" does
not put out an error message.

I find it handy to have a little test script (t) for testing snippets of code.

Here save the following as t, somewhere in your $PATH, chmod +x t, and run
t without openvpn running.

-----8<-----8<-----8<--cut below this line---8<-----8<-----8<
#!/bin/bash
set -u

set -- $(pgrep openvpn)
if [ -z "$1" ]; then

echo "no openvpn running"
exit 1
else
echo "ok we are good to go"
fi


exit 0
#********************** end of t ***************************

For your "Is openvpn running" test, "set" is not the best method which
you should see from running t.

I can recommend for those kinds of tests something like

pgrep -c openvpn
if [ $? -ne 0 ] ; then
echo "no openvpn running"
exit 1
else
echo "ok we are good to go"
fi

If you were to think about a little, you should notice that you can
remove the else code. which would look something like:

openvpnPID=$(pgrep openvpn)
if [ $? -ne 0 ] ; then
echo "no openvpn running"
exit 1
fi

echo "Good: Apparently openvpn is running"

echo "\$openvpnPID is ${openvpnPID}"
openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')
echo "\$openvpnCWD is ${openvpnCWD}"

--------------------

Why did you not use the pid variable? Your method can have
intermittent run time problems if second pgrep does not return the
stored value. :(

not good openvpnCWD=$(sudo ls -l /proc/$(pgrep openvpn)/cwd | awk '{print $NF}')
better openvpnCWD=$(sudo ls -l /proc/$openvpnPID/cwd | awk '{print $NF}')


Having some experience in a multi-user system, you might want to
consider what happens when more than one user is running your code. :-D



Today's productivity tip:
I can recommend putting commands and examples with keywords in a file
for reference. That way you can have easy access without having to
research commands you do not use everyday.

If I have to look something up the second time I want use it, I will
put it in my unix.help text file. That way my "uh" script can find it.

$ uh exa pgr
pgrep_example pgrep --count --uid $LOGNAME --full "$_title" > /dev/null

# Completed: uh exa pgr
# hit return to close wb /home/bittwister/tmp/uh.x

I also lurk is several Usenet groups and will paste working examples
into the unix.help file if I like it.

Do you remember needing to know what Marek's bash sub string operators did/do?
I cut/pasted that table I pointed you to and added key words so I
could see which one I might have an interest in. Example:

$ uh sub str
get_last_cmdline_arg_substr printf '%s\n' "${*: -1}"
substr of last char line=${line%?}
bash substr 1'st 2 char printf "%s\n" ${line%??}
bash_substr_for_ip_addy _IPADDR=$(ifconfig enpXXX) ; _IPADDR=${_IPADDR#*"addr:"}
bash substr trim leading 2 char printf "%s\n" ${line#??}
3 char bash shell substring echo ${stringZ:7:3}
create scrip to set permissions rpm -qal --dump | perl -anl -e 'print "chmod $F[0] " . substr($F[4],-4)'
string/line substitution/change perl -pi -e 's|#video.driver:auto|video.driver:openGL|' ~/.config/totem/xine_config
substr_varliable_example IPADDR=${_value:0:${#_value}-1}
code_a echo ${string:$((-3)):2} substr two char from end of line
code_b ${#string} Length of $string or substr
code_b ${string:position} Extract substring from $string at $position (0 relative)
code_b ${string:position:length} Extract $length characters substring from $string at $position (0 relative)
code_b {string-default_string} Set default substring if string is null/empty
code_b ${string#substring} Strip shortest match of $substring from front of $string year=${_date#????}
code_b ${string##substring} Strip longest match of $substring from front of $string ext=${file##*.}
code_b ${string%substring} Strip shortest match of $substring from back of $string base=${file%.*} month=${_date%??}
code_b ${string%%substring} Strip longest match of $substring from back of $string
code_b ${string/substring/replacement} Replace first match of $substring with $replacement
code_b ${string//substring/replacement} Replace all matches of $substring with $replacement
code_b ${string/#substring/replacement} If $substring matches front end of $string, substitute $replacement for $substring
code_b ${string/%substring/replacement} If $substring matches back end of $string, substitute $replacement for $substring
code_c html2text -o "${file%.html}.txt" "$file" substring substitution
code__expr_usage_is _var=$(expr substring_cmd_here)
code__expr_match "$string" '$substring' Length of matching $substring* at beginning of $string
code__expr "$string" : '$substring' Length of matching $substring* at beginning of $string
code__expr expr index "$string" $substring Numerical position in $string of first character in $substring that matches
code__expr_substr $string $position $length Extract $length characters from $string starting at $position
code__expr_match "$string" '\($substring\)' Extract $substring* at beginning of $string
code__expr "$string" : '\($substring\)' Extract $substring* at beginning of $string
code__expr_match "$string" '.*\($substring\)' Extract $substring* at back of $string
code__expr "$string" : '.*\($substring\)' Extract $substring* at back of $string
code_d if [[ "$line" == *UUID=* ]]; then Substring test
code_d uuid1=${line#*UUID=} remove substring from front of line
code_d uuid=${uuid1%%[[:blank:]]*} get first word substring
code_d line=${line//UUID=$uuid/LABEL=$label} substitute substring
code_d newdir=${dir// /_} substring space for underscore

[DecadentLinuxUserNumeroUno]

On Thu, 10 Mar 2016 14:59:08 -0000 (UTC), Bit Twister
<BitTw...@mouse-potato.com> Gave us:

> echo "ok we are good to go"

Hehehe Jody Foster

"Small steps, Ellie... small steps."

[Marek Novotny]

On 2016-03-10, /less <ro...@noabode.nil> wrote:

// snip

> You guys obviously are having the time, expertness and seem to have a
> lot fun for these kind of 'hobby' projects.

That sums it up well enough. I have a little fun doing things like this.
Sometimes anyway. If it's not interesting to me I just ignore it.

> Existing free VPNs are for the proficient ones (there are evidently only
> a few participating here, others seem just steer away from this
> particular thread).

Well, I'm too sure about that. Others have chimed in...

> Why is it so difficult developing or working purposefully on a VPN
> application (similar to the paid ones) which can be applied by the
> 'broader' community i.e.technical challenged users?
> Has it something to do with accessibility of world-wide servers and/or
> an inevitably commitment for providing continued services?

I think you're wrong here. Hope in rhel 6.x and tell me how easy it is
for you to setup OpenVPN for the first time without knowing much of
anything. As someone who just uses a mouse and expects that the VPN
providers out there have something for you. Try it with FreeBSD. Hell,
try and install FreeBSD and let me know how that goes. Are you aware
that FreeBSD doesn't have a GUI by default at all? You have to add one
yourself.

WiTopia is a paid for service. It's the one I use personally. I have a
script that anyone could use for WiTopia. It's one simple script and it
presents a menu of end points and all you do is choose a location you
want to connect to and it connects and tells you the status of that
connection. All the knowledge you need to run it is how to download your
key from WiTopia, how to store the key in a particular location,
$HOME/vpn for example, and how to execute a script. When you're done,
you hit ctrl-c and it auto exits for you.

WiTopia has directions for installing WiTopia on Ubuntu. And if you have
the correct version of NetworkManager it's not that difficult. But my
method is even simpler. You download your keys, save them and run the
script. It's portable. I can be on Red Hat, FreeBSD, Ubuntu and pretty
much anything that runs OpenVPN and it will work. From my point of view
I've done a better job than they did. And they have you setup only one
end point, graphically. If you want more, you set those up graphically
one at a time. With mine, I get a menu of all of them and I just pick
the one I want. It's easier.

Now this project you're referring to which you see here in these posts
is different. Here someone has asked for help because the web site has
over thousand openvpn config files. Each one is a connection point.
Because they give them out some work and some don't. Why would anyone
want to setup something that one at a time, graphically, only to find it
doesn't work and then you have to delete it and create a new one,
graphically.

What is easier is to have a script test them all for you and separate
the good ones from the bad ones. And then you simply execute a good one.

Now, what the OP also wanted to do is some extra stuff. Say he's using a
torrent and the VPN connection dies in the middle of a upload /
download. He doesn't want the torrent to continue on his actual IP
address from ISP. Can your commercial VPN provider monitor and kill your
torrent automatically should your connection drop? Can your commercial
VPN provider kill thunderbird should you try to run it while connected
to your vpn? These are the kinds of functions the op wanted. He doesn't
want to expose himself by logging into his own email account over the
VPN which he is using to obscure his traffic.

So this is more than just a vpn script. It's using the shell to
accomplish a goal. To make something that is otherwise tedious simple to
accomplish and versatile.

[William Unruh]

Or you could also just use basename to strip off the suffix.

>
>
>