Publishing WeeWX to AWS S3

[Mike Revitt]

It is nearly 2 years since I migrated my Web Site to AWS S3 but that was done in a rather clumsy way. So when I moved my WeeWX deployment to my Rasberry Pi I decided to do it properly.

With time on my hands over Christmas I have finally gotten around to writing up how I did it and for those that are interested  the cost is free for the first 12 months then $1.00 US per month after that.

Instructions are here

[vince]

Very cool.  Thanks for posting the howto.   Pretty amazing you can do this for under a dollar a month.

I did DevOps tooling for a AWS-hosted $job for a few years using ansible/terraform/boto3 and a little CloudFormation (ugh) so I had some questions about what's under the hood a bit more than just the (nice) howto stuff you have in your link.

My current setup is an AWS Lightsail instance ($4/month) with DNS provided by Google DNS ($1/month).

• how do you get your stable .eu.com FQDN onto your website's ip address ?
• is the website ip address stable ?
• are you using Route53 for your DNS ?  What's that cost ?
  • (background - I went Google DNS years ago because Route53 was too expensive.  Google DNS also had some nice email and site aliasing features that Route53 didn't have back then.   I particularly like the ability to add email aliases for the family very easily, pointing to email wherever they want it to be hosted)

My Lightsail instance sees zillions of attack attempts from the usual bots+script kiddies, trying to probe for the usual things (WordPress, PHP, etc.), from the usual suspect countries with distributions about what you'd expect from reading the press. I know geoip lookups are an inexact science, but the data indicate the bots probably aren't trying too hard to falsify their source ip.  They just scan the public address space periodically and run the same probes the next time.

• I use geoip fencing in nginx to only permit in ip addresses from a handful of countries with folks I collaborate with.  That lessens the number of attack attempts by a huge amount.   Does CloudFront have that kind of ability too ?
• I also run fail2ban to block ip from addresses (from permitted countries) who seem to be trying to attack the system.   Can CloudFront also do something like that ?
• do you have anything enabled to alert you if your costs spike up due to somebody successfully getting into your setup ?   I know that a S3 bucket with web data is not any risk from a security standpoint, but I always worry about AWS costs if a 'feature' happens and somebody/something causes your usage to spike up.  AWS sure likes billing on usage numbers (sigh).

Lastly - automation.    Everything AWS is scriptable basically, which is pretty powerful.  When I rebuilt my Lightsail instance that is a simple ubuntu running just nginx so I can rsync up to it from my home LAN weewx box, I automated most of it with Terraform.  Previously I'd done similar things with Ansible.  But regardless it was pretty great getting the AWS unique stuff done fully scripted.  All I had to do afterward was set up LetsEncrypt and do a couple odds+ends.

Have you given any thought on scripting your whole setup with something like Ansible / Terraform / CloudFormation ?    It would be pretty cool to have an accompanying script or equivalent that did that, once you have the manual step of getting an account and saving the AWS credentials of course.

Very cool howto.  Thanks !

[Mike Revitt]

• how do you get your stable .eu.com FQDN onto your website's ip address ?

I moved my service to Rout53 some time ago from DynDNS, this post shows how I did that when I was hosting at home https://www.cougar.eu.com/useful-guides/networking-tips/replacing-dyndns-with-route/index.html

But as the site is now managed by AWS Rout53 and S3 are tightly integrated so I don't have to worry about IP Addresses

• is the website ip address stable ?

Solid as a Rock

• are you using Route53 for your DNS ?  What's that cost ?
  

Yes, and last month it cost $0.51 US, but you only need this if you have your own domain name

• Have you given any thought on scripting your whole setup with something like Ansible / Terraform / CloudFormation ?    It would be pretty cool to have an accompanying script or equivalent that did that, once you have the manual step of getting an account and saving the AWS credentials of course.

I haven't but it is relatively easy to put the S3 and IAM creation into a Cloud Formation template, I could do that if there was interest

[vince]

Looks like the Route53 prices came down a lot over the years, so I might take another look at it.   I'm paying a buck/month for Google hosting my DNS and I like the feature set a lot, but there's something to be said for being all-in on one vendor since AWS's APIs are pretty solid.

I guess I'd say don't bother with the CF thing unless you get lots of interest from other folks.  Your howto helps a lot in connecting the dots for the S3 and IAM parts.

I might take a crack at the automation thing in CF/Ansible/Terraform just for the adventure as time permits, mainly to fiddle around.    I'm very happy with going Lightsail years ago literally the day it was announced, but I just rsync my weewx public_html up to nginx with no other services there, so maybe it's simpler to go S3 and not need to do ubuntu package updates.   Thinking about it...

Thanks for the updates.

[t...@tom.org]

@vince, if you do take on automation, it would be cool to do it in TF if the storage abstractions would help work across AWS and GCP. My stuff lives on GCP (I currently use rsync from GKE Kubernetes to a compute instance to avoid paying for a load balancer). Hopefully, the storage abstractions from TF would allow it to be more generic across clouds. I have not looked into it yet though.