Wiki is down

[Tim Holloway]

Can someone restart the LUG wiki website? It has been down since Tuesday night.

  Tim

[JaxSmart Admin]

Who and where is the WiKi located. I Noticed it was down also. I had referred a prospect to jaxlug.net, but What good is that when nobody is minding the 'store'?

MJB

This message sent via Linux Mint! I'm now Windows PAIN Free!

On Fri, Jul 3, 2015 at 7:44 AM, Tim Holloway <timhol...@gmail.com> wrote:

Can someone restart the LUG wiki website? It has been down since Tuesday night.

  Tim

--
You received this message because you are subscribed to the Google Groups "JaxLUG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jaxlug+un...@googlegroups.com.
To post to this group, send email to jax...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jess Hires]

I'm operating the wiki at the moment, it's hosted at ChicagoVPS. It's not terribly reliable; once every couple of months or so, the VM appears to have been powered off. But, it's cheap and fits what we need.

There's not currently any kind of monitoring on the site, since I took down my Splunk server and we no longer have a Nagios alert from Tim, so I'm relying on the group to let me know if there's an issue. Anybody else with a Nagios server want to add an alert to email me on this?

Jess

[Tim Holloway]

Actually, the Nagios monitor is back up, but since the server went down while internal email routing was still reeling from my recent change in IP addresses, the "down message" got lost. I've been monitoring it manually - which is why I , but I think I'm going to tweak a few options so that it gets a little more insistent when problems occur.

ChicagoVPS isn't the only vendor who's prone to silently take down systems without warning. That's one of the dangers of the Cloud that nobody talks about. While a lot of VM enviromnents support tossing VMs from one physical host to another with minimal downtime, real-world practices tend to be considerably less robust. And that's not even counting stuff like unscheduled OS upgrades.

At least when you own your own cloud (or host), you have more control over that sort of mayhem.

[JaxSmart Admin]

Very interesting. I thought y'all had a server at home. That made sense to me since the Wiki has so little data on it. So what does ChicagoVPS cost the 'LUG'?

I saw this on their page "Using the best networking gear means we can extend a real level of protection to our customers in the form of a 99.9% uptime guarantee."  I guess JaxLUG.net is in the 0.1 percent.

MJB

This message sent via Linux Mint! I'm now Windows PAIN Free!

[wmltho...@gmail.com]

Jaxlug.org was hosted on my severs I had at Peak10. However as of November 2014 I moved out of Peak10. I was unable to find affordable hosting in Jacksonville. I found lots outside but did not want to deal with shipping my servers and having remote hardware. I first when into Rackspace[1] Cloud under their Developer+[2], $50 free a month in hosting and $50 waiver of managed hosting support, $100 a month, $1200 savings over a year, free for 1 yr! But with them being managed hosting, ended up costing me more than others. Rackspace is THE most expensive, beyond even Softlayer who has features other do not. I shopped around. I ended up with Linode[3] in December 2014 and have been there since. The only time there has been reboots is due to xen security updates. However Linode now has KVM which I migrated to a bit ago, my only other reboot beyond my needs They are a great provider. Most my severs are in Atlanta, which Comcast has some IPv6 routing issue with a BGP peer which sucks. (I reported to FCC etc months later problem remains) I have another node out in Fremont, Ca and will be deploying a few more nodes out there and doing East to West coast replication, as well as serving up data locally. West cost servers for west coast clients, east for east, and they have locations in between if I need more, 4 or so in US might be 5. I also plan to replicate over to Digital Ocean[4] as well, both east and west coast. With the end goal being N+3 on 2 cloud providers. I have also heard good things about Vultr[5]. Not sure if I will use Vultr at all or not. Amazon, Softlayer and some others are not an option due to lack of IPv6. I did look at others like Gentoo sponsor Host Virtual[6].

Linode is pretty geeky IMHO, they give you stuff others charge for. They have some better tools than others. Out of the box RRDGraphs on general node stats from the host, no invasive monitoring software you need to install in your nodes like with OpenStack stuff, etc. Really quite pleased with Linode. However they have had some issues with DNS servers and such. Also with my IPv6 issue via Comcast. They are not as quick to admit fault or look within their own network for problems, even when confirmed. But they do have good responsive support, active IRC channel etc. I don't see myself leaving Linode anytime soon if ever. I am also saving ALLOT compared to Peak10, less hardware. But the time I am close to that price again, I will have N+4, and still be saving $100+ a month...

Though I do find all these issues with the wiki being down very ironic. There were never outages like this when I hosted Jaxlug.org wiki, mailing list, dns, etc. The ONLY issues were with ASSP spam proxy not letting some emails to the list, in order to prevent people spamming the ezmlm mailing list. The whole reason to take it from me was over that, and having access/control. But it was not a good thing. They immediately ran into all the issues I had already resolve with the wiki. Then re-created all data. Though I must admit they did a good job with the recreation filling in meeting data beyond what existed in the other wiki. Which I had to re-create data in the other wiki when data was literally lost by the previous admin during outages....

I am proud of my businesses track recording hosting the JaxLug.org. After all my business was hosting the JaxLug, it wasnt a personal thing. Like I could host it on my development server at my house, as others could. Instead I had it on my production servers at Peak10. Though at first did not have nagios monitoring it as I had it monitoring my stuff. Now I would not be as interested as to host it would require me to purchase another node, even at $5-$10 a month, not sure I care enough anymore. Surely after how i was treated, and then previously banned and bashed on list. Professional public shaming, with only behind the scenes apologies. I forgive, attend meetings, but do not have the same interest no motivation as I did before, sad to say...

None the less many great cloud providers out there!

1. http://www.rackspace.com/cloud

2. https://developer.rackspace.com/signup/

3. https://www.linode.com/

4. https://www.digitalocean.com/

5. https://www.vultr.com/

6. https://www.hostvirtual.com/

[Keith Oakes]

Guys,
Thanks for the update this is nice to know.
Keith O

[wmltho...@gmail.com]

On Friday, July 3, 2015 at 11:17:55 AM UTC-4, Jess Hires wrote:

There's not currently any kind of monitoring on the site, since I took down my Splunk server and we no longer have a Nagios alert from Tim, so I'm relying on the group to let me know if there's an issue. Anybody else with a Nagios server want to add an alert to email me on this?

I added a check (check_http) and added you (Jess) as the contact to my a production server running nagios. I have a relaying/send only qmail server running on the same system as nagios, so alerts will be sent no matter what! Even if my main mail server is down, as its not used for sending nagios alerts unless to me. I have additional third party monitoring the server running nagios, so I will know if its down or has any problems, in addition to regular monitoring and alerts from nagios. I think the group should be covered now and alerts will be sent if there is any issue with the wiki.

Why do I have qmail running on the same system as nagios? Even though it just forwards emails to my main server if they are destined for a domain I host. If the email server is down, and I am just using a command line send mail program like ssmtp, mailx, sendmail (the actual command not the MTA), etc. If the mail server cannot be reached the email ends up in a dead.letter file on the server. I did not want that. I wanted undeliverable emails to be queued and delivery attempted more than once. This way if my main mail server is down, any alerts from nagios will remain in queue until the mail server is back up. Now I will be deploying a 2nd mail server out west soon. However in the even ALL mail servers are down. My alerts will remain in queue till one or more is backup. So I still get the alerts. Also alerts can be sent to my cell phone, and else where, without having to rely on my main mail server, which might be down or unreachable. Once again ensuring I get the alerts no matter what.

Now to have something monitoring me, so I can respond to the alerts, no matter what.... :) Likely could have nagios monitor me as well... It can monitor just about anything... I think I saw it in use at a hospital for a patient on life support.... j/k, but maybe someday....

[Jess Hires]

Thanks William, very much appreciated! Great info, too!

--

[Tim Holloway]

Actually, my Nagios server is supposedly sending alerts again. I fixed it after I found out it wasn't handling my new network configuration properly.

I just ran a quick trial, so if it's really all fixed, an email should be arriving.

[Tim Holloway]

OK. It works on my LAN. So the big question is whether or not it's getting forwarded to general email recipients.

Message body should look like this:

***** Nagios *****

Notification Type: CUSTOM

Service: Wiki
Host: jaxlug
Address: www.jaxlug.net
State: OK

Date/Time: Thu Jul 9 14:32:59 EDT 2015

Additional Info:

HTTP OK: HTTP/1.1 301 Moved Permanently - 439 bytes in 5.125 second response time

[Jess Hires]

Nothing wrong with getting alerts from both places. It will be twice as annoying so it gets fixed twice as fast. :)

[Tim Holloway]

Ah, but have you been annoyed yet?

If not, I'll go check my mail routing again.

[Jess Hires]

Negative, but the wiki is currently up. I imagine I'll get all kinds of alerts next time it goes down, though.

[wmltho...@gmail.com]

On Thursday, July 9, 2015 at 2:58:07 PM UTC-4, Tim Holloway wrote:

Ah, but have you been annoyed yet?

If not, I'll go check my mail routing again.

What MTA or are you using command line sendmail to send the emails? If you are using a MTA there should be logs of the email being delivered etc. If using a command line sendmail program, then look for dead.letter files via find or locate. I can always tell if an email has left my network. What others do with it is another story. Though if it does not bounce back, it went some where... 

[Tim Holloway]

Nagios sends the emails via SMTP. On the Red Hat platform, both Sendmail and Postfix log to /var/log/maillog. First Nagios has to post to its localhost MTA (port 25), the localhost MTA forwards to the mousetech master MTA, thence over the Internet to whatever resolves the MX for the recipient's domain. However the maillogs only indicate when the recipient accepted the mail. Various other things can happen such that 2 days ago I got flooded with mail delivery failure reports sent on July 2 notifying me that mails sent out on June 28 hadn't been deliverable - the MTA retries for 5 days and stuff had been backing up until I got my local routing cleaned up.

You might ask why I didn't have Nagios send directly to the mousetech domain mailhost and skip the localhost hop. The answers are A) I didn't think of it B) I'm not sure that Nagios has that option and C) I was working 2 mailhosts while the IP addresses were transitioning. For some reason, the JavaRanch is still attempting to mail to mail2.mousetech.com even though I've had that one shut down for several days now.

So the quickest and easiest way to find out if mail is making it to the destination is simply ask the recipient. Besides, I'm having SOOOO much fun right now. Randomly, the Muninn cache used by Neo4j decides to totally own my CPU and it's been making development of the app I'm working on an exercise in extreme patience today. So I was hoping to avoid digging through the logs along the mail route.

The dead.letter file usually contains stuff that was destined to my domain, not from it, so I use the maillogs.

--