SSL/Web migration process

[Patrick Robertson]

Sorry for the radio silence all, just got back from holiday… ;-)

I’ve seen the SSL/App Transport Security issues, and realised it’s probably a good excuse to make the migration to Rob’s servers soon.

We could go ahead and buy a single domain SSL certificate (which is pretty cheap) for qs0.qsapp.com, and fire it up on the current server if we want a quick fix.

(qsapp.com and those domains going through Cloudflare can use the Cloudflare SSL feature, it’s just qs0.qsapp.com which isn’t using Cloudflare where we’d need to be an SSL cert)

Also, it seems timely as the qsapp.com domain is up for renewal (in a month’s time) so we could also move that away from United Hosting to somewhere cheaper (UH is £12.50/yr) and use QS’s PayPal account directly

Here are the steps as I see them:

* Set up new server

* Copy all files & database files etc. over to new server

* Migrate main website and other Cloudflare domains (cdn.qsapp, qs.qsapp) by changing IP addresses in Cloudflare

* Test ‘basic’ functionality is still working

* Migrate plugins system etc. over (change IPs in Cloudflare)

* Set up SSL certificate

* Set up existing email aliases using a new mail server (e.g. donate@, devs@, etc.) - these are all just forwarders so all we really need is a forwarder

* Update website/code to add thanks to the new ‘hosting provider’

* Renew domain by 1st November - transfer to another domain provider and pay with QS’s PayPal

[Rob McBroom]

Whoa! How’d I miss this message? I’ve been planning to send one on
the same subject when I had more to report.

On 6 Oct 2015, at 19:48, Patrick Robertson wrote:

> Sorry for the radio silence all, just got back from holiday… ;-)

Welcome back! C says “hi”.

> I’ve seen the SSL/App Transport Security issues, and realised it’s
> probably a good excuse to make the migration to Rob’s servers soon.

Been working on that very thing the past few days.

> We could go ahead and buy a single domain SSL certificate (which is

> pretty cheap) for qs0.qsapp.com <http://qs0.qsapp.com/>, and fire it

> up on the current server if we want a quick fix.

My company manages certs for all kinds of people, so we have a
“partner” account. They said I could buy our cert under that, which
would be $208 for two years vs. $199 for one. Not sure, but I *think*
that would let us cover qsapp.com, www, and qs0. If that sounds OK, let
me know and I’ll set it up.

> Also, it seems timely as the qsapp.com domain is up for renewal (in a
> month’s time) so we could also move that away from United Hosting to
> somewhere cheaper (UH is £12.50/yr) and use QS’s PayPal account
> directly

I moved my personal domain to Hover. Maybe you’ve heard of them on
various podcasts, but bottom line: I got 5 years for 50 USD and they
don’t do shady things.

> Here are the steps as I see them:
>
> * Set up new server

Done. Currently available at quicksilver.sixfeetup.com until we get some
qsapp DNS pointed at it. All of our public keys are there, so you should
be able to log in as the user `qs`. SSH listens on 2070. The stuff is in
`/data/qsapp.com`. I might also relocate the MySQL files to `/data`, but
haven’t yet.

> * Copy all files & database files etc. over to new server

Done-ish. We’ll want to refresh the databases again before the final
cut-over, but it’s up and running.

Maybe you’ve seen the commits, but I’ve been getting the repo
cleaned up. One of the nice things about the new host is we’ll be able
to update the site with `git pull`, like God intended.

Note that it’s a FreeBSD jail, so there’s no 127.0.0.1. I think
I’ve updated all the right files and changed them to use `localhost`
instead (which points to a working IP).

The current files were a pure pull from the repo (to make sure it’s
all right), followed by some manual copying of things that aren’t
tracked (like the wiki files and qs0). I still need to see if anything
else needs to be tracked, like stuff in /lib. A lot of it seems to be
related to Google Analytics. Are we even using that?

> * Migrate main website and other Cloudflare domains (cdn.qsapp,
> qs.qsapp) by changing IP addresses in Cloudflare

I think we should drop the CDN for now and see how it goes. Also, serve
the app downloads directly instead of from GH Pages (though I couldn’t
figure out where that’s happening).

> * Test ‘basic’ functionality is still working

The main site looks OK, as well as the update system.

I don’t have the wiki working. We’re moving from Apache to Nginx, so
it’s mostly a matter of going though all the many htaccess files and
making sure the functionality is recreated.

> * Set up existing email aliases using a new mail server (e.g. donate@,
> devs@, etc.) - these are all just forwarders so all we really need is
> a forwarder

We’re not doing that for any jails right now, but it should be
possible.

> * Update website/code to add thanks to the new ‘hosting provider’

Yup.

--
Rob McBroom
http://www.skurfer.com/

[Patrick Robertson]

> Welcome back! C says “hi”.

Pass on a 'hi' back, and to the rest of the family ;-)

> My company manages certs for all kinds of people, so we have a “partner” account. They said I could buy our cert under that, which would be $208 for two years vs. $199 for one

Is that wildcard certs? The price sounds like it is.

The reason I'd suggested using Cloudflare SSL was so that we could use it for all domains except qs0.qsapp.com - for that domain we could just buy a single domain cert - really cheap from resellers ($5/yr)

I've used https://cheapsslsecurity.com/ a lot, no problems.

> I moved my personal domain to Hover. Maybe you’ve heard of them on various podcasts, but bottom line: I got 5 years for 50 USD and they don’t do shady things.

Sounds good. Shall we make the change? Can you pay with PayPal to avoid the dance of me having to take money back from PayPal?

>  I still need to see if anything else needs to be tracked, like stuff in /lib. A lot of it seems to be related to Google Analytics. Are we even using that?

/lib isn't included for security reasons. I don't think we could trust ourselves with secure code in that part! Just checked... gAnalytics is still in use. The site is getting -33% hits compared with this time last year :'(

Did you do a push then a pull? I know some things have been changed like the downloads.html page, so that'd need commiting I guess.

> I think we should drop the CDN for now and see how it goes. Also, serve the app downloads directly instead of from GH Pages (though I couldn’t figure out where that’s happening).

As long as you guys have enough bandwidth, that's cool. The GH pages stuff is hard-coded in one of the /lib PHP files. I think in the plugins Class PHP file.

> The main site looks OK, as well as the update system.

I assume you're working on it now? 503 error.

> We’re not doing that for any jails right now, but it should be possible.

The email stuff is important, and something I've actually found difficult to replicate on other servers. It's the only thing that's stopped me for ditching United Hosting entirely to go with a cheaper 'self managed' host. We need to check up we can get email aliasing working before any switch.

Nice work, looks like we're pretty close. git push FTW!

--
You received this message because you are subscribed to the Google Groups "Quicksilver - Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quicksilver---deve...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Patrick Robertson]

> The email stuff is important, and something I've actually found difficult to replicate on other servers. It's the only thing that's stopped me for ditching United Hosting entirely to go with a cheaper 'self managed' host.

...perhaps what we can do is keep United Hosting for email, and just transfer the other domains to Six Feet Up. Hover provides email forwarding for domains... at $5 per forwarder per year!

We already have about 10 forwarders, so sticking with UH would probably be simplest.

[Patrick Robertson]

I’ve just found out United Hosting accept PayPal for payment, so although they are slightly more expensive, I will just renew with them so it’s less hassle. It also gives us more credibility for using their mail servers.

I’m busy trying to sort out the new FreeBSD server. I can’t seem to fix the 503 on quicksilver.sixfeetup.com

It seems to be a firewall issue(?) Because doing a curl from on the server works, but otherwise (from my browser) it returns a 503

[Rob McBroom]

On 9 Oct 2015, at 6:14, Patrick Robertson wrote:

> Is that wildcard certs? The price sounds like it is.
> The reason I'd suggested using Cloudflare SSL was so that we could use
> it
> for all domains except qs0.qsapp.com - for that domain we could just
> buy a
> single domain cert - really cheap from resellers ($5/yr)
> I've used https://cheapsslsecurity.com/ a lot, no problems.

I was giving prices for GeoTrust Quick SSL Premium, which is way cheaper
on that site. What’s the catch?

> /lib isn't included for security reasons.

I’m sure some of that stuff can be checked in though, like the library
for interacting with property lists.

> Did you do a push then a pull? I know some things have been changed
> like
> the downloads.html page, so that'd need commiting I guess.

It should be all up to date. I cloned from the repo on GitHub in one
folder, then pulled down a copy of everything from UH in another folder.
Then I moved `.git*` from the repo to the copy from UH which revealed
everything on the live site that differed from the repo. I then
committed what made sense.

Then like I said, cloned the repo to the new server and manually copied
what was missing (from the copy of the live site).

> As long as you guys have enough bandwidth, that's cool.

Yeah, let’s try it for a while.

> The GH pages stuff is hard-coded in one of the /lib PHP files. I think
> in the plugins Class PHP file.

OK, I’ll see if I Can figure it out.

> I assume you're working on it now? 503 error.

I should have replied sooner, but this is because the load balancer and
web server are only configured to use the qsapp.com names. Add them to
your local `/etc/hosts` to try it out.

Now that I think about it though, we’ll have to change things around
to use HTTPS. I’ll need to put a public IP on that jail because we
don’t want to deal with going through the load balancer.

> The email stuff is important, and something I've actually found
> difficult
> to replicate on other servers.

Sounds like we can keep using UH for this?

[Rob McBroom]

On 9 Oct 2015, at 6:14, Patrick Robertson wrote:

> I've used https://cheapsslsecurity.com/ a lot, no problems.

I’d say go for it. They take PayPal, so I’ll let you do it.

My preference is for GeoTrust QuickSSL Premium, since it’s what we use
and I know it meets the ATS requirements. The default 3 years sounds
good. When it’s done, put the key and cert on the new server and
I’ll get it set up.

[Patrick Robertson]

Just realised this never sent yesterday. I wrote this ~20hrs ago:

OK, domain is renewed for 3 years with United Hosting.

> I was giving prices for GeoTrust Quick SSL Premium, which is way cheaper on that site. What’s the catch?

Good question... I've used them for about 4-5 certs on various different hosting providers across 3 different countries, no problem to report. Not sure exactly what you were quoted, but perhaps the downside is that there's no 'organisation verification'. That is, the cert doesn't show the company name like say GitHub's does. A minor thing I say.

If you wanna have a look at the certs themselves, some places where I'm using them:

https://cloud.patrick.is

https://api.techiaith.org/

> I’m sure some of that stuff can be checked in though, like the library for interacting with property lists.

I've checked more in

> Now that I think about it though, we’ll have to change things around to use HTTPS. I’ll need to put a public IP on that jail because we don’t want to deal with going through the load balancer.

Do we need a load balancer? I fiddled with /etc/hosts but I'm too dumb to figure out what needs doing.

[Patrick Robertson]

> I’d say go for it. They take PayPal, so I’ll let you do it.

OK, so we get the single domain cert, and go for just qs0.qsapp.com?

We'd then use Cloudflare's SSL on other domains (which I actually enabled to test on the current site: https://qsapp.com/ )

It uses SNI, which means it won't work in *really* old browsers, but 'meh' (?)

[Rob McBroom]

On 13 Oct 2015, at 21:10, Patrick Robertson wrote:

>> I’d say go for it. They take PayPal, so I’ll let you do it.
>
> OK, so we get the single domain cert, and go for just qs0.qsapp.com?

Yeah, I guess the update system is the priority.

> We'd then use Cloudflare's SSL on other domains (which I actually
> enabled
> to test on the current site: https://qsapp.com/ )
> It uses SNI, which means it won't work in *really* old browsers, but
> 'meh'

I could not care less about old browsers. And even ATS is happy:

/usr/bin/nscurl --ats-diagnostics https://qsapp.com/

>> I’m sure some of that stuff can be checked in though, like the
>> library for interacting with property lists.
>
> I've checked more in

Did you push, though? :-)

> Do we need a load balancer?

Well, in this case, it’s not balancing any load. It’s just a thing
that has a public interface that can send traffic back to a bunch of
jails that don’t.

It’s out of the picture now. I put us on a public interface. DNS is
updated, so you can use the same name for SSH but it no longer needs to
be on the weird port.

> I fiddled with /etc/hosts but I'm too dumb to figure out what needs
> doing.

Don’t feel bad. There’s nothing you could’ve done from inside the
jail itself (hence the name). I had to add another IP to the host, then
tell the jail to use it and restart.

I think the main thing left to do is go through all the `.htaccess`
files and duplicate whatever they were doing in Nginx. I’ll work on
that hopefully tomorrow, but feel free to take a stab. :-)

[Patrick Robertson]

> I think the main thing left to do is go through all the `.htaccess` files and duplicate whatever they were doing in Nginx. I’ll work on that hopefully tomorrow, but feel free to take a stab. :-)

I've made a start. The only ones left are the wiki ones I think

[Patrick Robertson]

OK, wiki has been fixed.

@Rob did you move the wiki database over, it seems the pages are all missing?

I've also disabled the CDN and pushed the latest changes to the repo.

I think there's still a bit of testing that needs doing. The page http://qsapp.com/plugins.php looks totally wrong to me. Can anybody else confirm?

[Patrick Robertson]

> @Rob did you move the wiki database over, it seems the pages are all missing?

Wiki sql moved over. All is working now.

I think we can go ahead and move the main site over to the new host, leaving qs0 for the time being (until we've run some tests with the latest HEAD of QS on El Cap)

(I have no access to a Mac atm)

[Rob McBroom]

On 15 Oct 2015, at 2:37, Patrick Robertson wrote:

> Wiki sql moved over. All is working now.

Cool. Looks good.

> I think we can go ahead and move the main site over to the new host,
> leaving qs0 for the time being (until we've run some tests with the
> latest
> HEAD of QS on El Cap)

Main site for sure. It all looks OK to me. This morning, I have:

* imported a fresh copy of the plug-ins database
* broken and fixed MySQL a couple of times :-)
* configured MySQL not to listen on the public interface
* built a new QS with the HTTPS URLs
* blew away my prefs and started QS

I was able to get all the plug-in data and upload a crash report. So I
think qs0 is good to go. I say switch them all over and we’ll address
any issues we find. (It’s completely broken for a lot of people now
anyway.)

[Patrick Robertson]

I am going ahead and switching now.

[Patrick Robertson]

OK, I have made the switch.

The only domain I have left untouched is cdn.qsapp.com - this is still going through the old site. We never set it up on the new server, but I don’t think it’ll ever be used now (since the download.php file doesn’t ever point people to cdn.qsapp.com now)

I prefer to use encrypted email. You can find my public key here.
Learn how to encrypt your email with the Email Self Defense Guide

[Rob McBroom]

On 15 Oct 2015, at 20:21, Patrick Robertson wrote:

> OK, I have made the switch.
> The only domain I have left untouched is cdn.qsapp.com - this is still
> going through the old site. We never set it up on the new server, but
> I don’t think it’ll ever be used now (since the download.php file
> doesn’t ever point people to cdn.qsapp.com now)

It looks like qsapp.com is still pointing to the old address.

% host qs0.qsapp.com
qs0.qsapp.com has address 174.47.106.178

% host www.qsapp.com
www.qsapp.com has address 174.47.106.178

% host qsapp.com
qsapp.com has address 104.24.114.201
qsapp.com has address 104.24.115.201
qsapp.com mail is handled by 10 mx1.spamfiltering.com.
qsapp.com mail is handled by 20 mx2.spamfiltering.com.

[Patrick Robertson]

104.24.114.201 is Cloudflare's servers. Because we're still going through Cloudflare (for SSL) it shows up as Cloudflare's IP address.

I saw this and tested that the new server is actually being used behind the scenes

[Rob McBroom]

On 15 Oct 2015, at 22:07, Patrick Robertson wrote:

> 104.24.114.201 is Cloudflare's servers. Because we're still going
> through
> Cloudflare (for SSL) it shows up as Cloudflare's IP address.
> I saw this and tested that the new server is actually being used
> behind the
> scenes

OK, you’re right.

New release is out. I’ll announce on the mailing list and GitHub. Can
you post it from the Twitter account? I don’t have it logged in
anywhere, and don’t have the password. (Could you put it in a file on
the server?)

Should we do a blog post?

Thanks.

[Etienne Samson]

I can't get the update to the new release here, from 10.10 / 1.3.1.

$ host qs0.qsapp.com
qs0.qsapp.com has address 174.47.106.178

Good job on the migration though ;-).

Regards,
Etienne Samson
--
samson....@gmail.com

[Patrick Robertson]

qs0.qsapp.com has address 174.47.106.178

That’s correct, pointing to the new server. What’s your issue?

I prefer to use encrypted email. You can find my public key here.
Learn how to encrypt your email with the Email Self Defense Guide

[Rob McBroom]

On 16 Oct 2015, at 15:00, Etienne Samson wrote:

> I can't get the update to the new release here, from 10.10 / 1.3.1.

I wonder if ATS is stopping you from even attempting the connection
since it’s HTTP. I had a co-worker on 1.3.1 try, then give me her IP
so I could check the user-agent string, and there were no log entries
from her IP at all.

If you want to send me your IP privately, I’ll see if you ever got
through. Or you should have access to check. It’ll be in
`/var/log/nginx/qs0-access.log`.

Another co-worker on 1.3.0 tried to update and it worked, but told him
4013 was the latest. Maybe he was hitting the old server. New one looks
correct:

% http -b 'https://qs0.qsapp.com/plugins/check.php?qsversion=16402'
4014

[Etienne Samson]

> Le 17 oct. 2015 à 05:43, Patrick Robertson <robertso...@gmail.com> a écrit :
>
>> qs0.qsapp.com has address 174.47.106.178
>
> That’s correct, pointing to the new server. What’s your issue?

The issue seems to be that even though there's no ATS shenanigans going on in OS X 10.10, I can't get the update that would save my bacon when/IF I upgrade ;-).

I'm just concerned that we're deliberately forcing everyone to manually download 1.3.2.

> Le 17 oct. 2015 à 06:21, Rob McBroom <mailin...@skurfer.com> a écrit :

>
> On 16 Oct 2015, at 15:00, Etienne Samson wrote:
>
>> I can't get the update to the new release here, from 10.10 / 1.3.1.
>

> I wonder if ATS is stopping you from even attempting the connection since it’s HTTP.

From 10.10 ?? I think I didn't get the memo that it was being back-ported ;-).

[snip]

> Another co-worker on 1.3.0 tried to update and it worked, but told him 4013 was the latest. Maybe he was hitting the old server. New one looks correct:
>
> % http -b 'https://qs0.qsapp.com/plugins/check.php?qsversion=16402'
> 4014

I'm getting the same reply here using curl, but Quicksilver still says 4013 is the latest.

[Etienne Samson]

> Le 17 oct. 2015 à 12:43, Etienne Samson <samson....@gmail.com> a écrit :
>
> I'm getting the same reply here using curl, but Quicksilver still says 4013 is the latest.

Checking the logs, I can see my curl attempts, but not my QS attempts.

Here's a shared secret ;-) : 17/Oct/2015:06:48:35 -0400

Cordialement,
Etienne Samson
--
samson....@gmail.com

[Rob McBroom]

On 17 Oct 2015, at 6:49, Etienne Samson wrote:

>> Le 17 oct. 2015 à 12:43, Etienne Samson <samson....@gmail.com> a
>> écrit :
>>
>> I'm getting the same reply here using curl, but Quicksilver still
>> says 4013 is the latest.
>
> Checking the logs, I can see my curl attempts, but not my QS attempts.

So where is it getting 4013 from?

> The issue seems to be that even though there's no ATS shenanigans
> going on in OS X 10.10, I can't get the update that would save my
> bacon when/IF I upgrade ;-).

I think it depends in the SDK used when building the app, and not the OS
you’re running.

> I'm just concerned that we're deliberately forcing everyone to
> manually download 1.3.2.

Not deliberately, but I don’t see any way around that at this point.
:-(

[Etienne Samson]

> Le 17 oct. 2015 à 20:00, Rob McBroom <mailin...@skurfer.com> a écrit :
>
> On 17 Oct 2015, at 6:49, Etienne Samson wrote:
>
>>> Le 17 oct. 2015 à 12:43, Etienne Samson <samson....@gmail.com> a écrit :
>>>
>>> I'm getting the same reply here using curl, but Quicksilver still says 4013 is the latest.
>>
>> Checking the logs, I can see my curl attempts, but not my QS attempts.
>
> So where is it getting 4013 from?

This is the thing I'm wondering about... I'll try Wiresharking that out, since there's no TLS involved yet.

Okay, so it seems QS speaks to 104.24.114.201 (as cdn.qsapp.com) when it checks here. I can see the HTTP exchange just fine and it gets 4013 as a reply.

>> The issue seems to be that even though there's no ATS shenanigans going on in OS X 10.10, I can't get the update that would save my bacon when/IF I upgrade ;-).
>
> I think it depends in the SDK used when building the app, and not the OS you’re running.

I don't get ATS warnings in Console when clicking "Check Now", and I can see this :
17/10/2015 20:17:46,088 Quicksilver[753]: Fetching plugin data from http://cdn.qsapp.com/plugins/info.php?qsversion=16403
17/10/2015 20:17:47,253 Quicksilver[753]: Downloaded info for 73 plugins
17/10/2015 20:17:47,263 Quicksilver[753]: Quicksilver is up to date.

So I don't think it's SDK-related.

>> I'm just concerned that we're deliberately forcing everyone to manually download 1.3.2.
>
> Not deliberately, but I don’t see any way around that at this point. :-(

What ? I don't see how it couldn't work. TLS is just a nice thing to have at that point. I suspect something strange with the CDN setup. Or it might be worthwhile to add the update to the now old update system so that users in my case get the memo ?

[Patrick Robertson]

> Okay, so it seems QS speaks to 104.24.114.201 (as cdn.qsapp.com) when it checks here. I can see the HTTP exchange just fine and it gets 4013 as a reply.

Oh OK, cdn.qsapp.com is still pointing to the old site. I thought nothing would be hitting it anymore. I’ll update it to point to the new site. Good find Etienne!

[Rob McBroom]

On 17 Oct 2015, at 21:39, Patrick Robertson wrote:

> Oh OK, cdn.qsapp.com <http://cdn.qsapp.com/> is still pointing to the

> old site. I thought nothing would be hitting it anymore. I’ll update
> it to point to the new site. Good find Etienne!

The person that made this change should have realized what was going on.
<facepalm>

https://github.com/quicksilver/Quicksilver/commit/35e9cedf701aba2c8e3311e4806c7f566243fc8a#diff-97d67ded46ac67dd08078f3987c59bca

[Etienne Samson]

Haha, no worries Rob ;-).

My QS successfully updated yesterday, so I think we're good to go now. At least for non-ATS encumbered OS X-es like mine ;-).

Cordialement,
Etienne Samson
--
samson....@gmail.com

[Patrick Robertson]

>The person that made this change should have realized what was going on. <facepalm>

The person that merged that commit and deemed it OK should have spotted it, seeing as that person made the changes to the QS subdomains... <facepalm>

I’d say I’m more to blame than you ;-)