Solid non-Raspbian platform for WeeWx + MariaDB?

[Stephen Rogers]

I have been stuggling for days trying to get all the moving pieces of WeeWx and MariaDB working on a Pi 4 running Raspbian Buster v10.  I had all this working before my Pi3 went south, so this difficulty is surprising.  I used Pi's for a few years and never hit the wall like this.  This morning I decided to reinstall WeeWx and MariaDB from scratch, following the documentation.  Not sure if it was completely successful because there are various error messages in the log and the nominal install location for the web pages doesn't line up with documentation.

But a number of aspects seem to behave so let's assume that WeeWx is really working.  

But the MariaDB deployment is lost in space.  I keep hitting roadblocks, and have searched far and wide for articles on installing it.  I use MariaDB on Ubuntu for my web app and have no trouble installing or managing it.  But so far on Raspbian I have no proof that it can properly.  I have also had no luck getting SSH passwordless login working from Raspbian so I can use rsync to push data to my cloud host.  Several dead ends there.  No problem doing any of this from Windows or Ubuntu.

So my  question is - what is a solid non-Debian micro-cpu platform to replace the troublesome Pi and Raspbian OS for running WeeWx with MariaDB (or MySQL) and pushing the html to cloud (Ubuntu 20.08)? 

[vince]

I think it's well established that the RaspiOS is just fine for all the things you are trying to do, and my 'guess' is you are either running into firewalling issues or you're not running sudo when needed.    Without seeing your logs and/or error messages, nobody can help you at all basically.

There 'is' ubuntu for the pi if you wanted to go that way.  I just installed the 64-bit variant of LTS server on a pi4 yesterday and it worked fine in minimal testing, but personally I see zero reason to run other than the RaspiOS which works in every way for anything I've ever tried to do on it.  It's just another Debian(ish) distro anyway.

I do have to ask, however, why are you trying to use MariaDB rather than the default sqlite3 ?   That ups your complexity factor significantly.   The sqlite3 default just works right away out of the box.  If you can't get something simple like rsync to work, I'd suggest sticking with sqlite3 for the least complicated setup if at all possible.

[Stephen Rogers]

Thanks Vince.  I didn't include error messages because my question was (and is) whether there are other options for a small CPU I can use to replace the Pi/Raspbian.  I appreciate your letting me know that you have confirmed Ubuntu can work on the Pi 4, I may try that.

I see the differences between Ubuntu (which I am getting pretty familiar with) and Raspbian (which has some annoying differences in how users are implemented for example, plus the various issues below) as distracting from my main goals with WeeWx.  

With regard to the SSH passwordless login issue, I have been running into "Putty key format is too new" when using the keys created by Puttygen in Putty to log into a remote server.  So this indicates to me that Puttygen is downrev on the Pi/RaspbiOS.  I'm sure there is a way to make this work but I haven't found it.  There are multiple options for generating keys.  The ones I've tried don't work on Raspbian thus far.  I have updated all the packages and gone through multiple reinstalls to make sure I wasn't tripping on some small detail.

I don't see how firewalls or sudo play into these problems.

I'm hearing you that you haven't had these problems with Pi.  You are clearly much more of an expert and are working from solid install/config procedures that I don't have, since the (multiple) ones I've tried are not getting me through to the result.

As for MariaDB, I have used it for years and the problems I'm running into with RaspiOS are ones I haven't seen on Ubuntu.  Where I am stuck on that now is when running mysql_secure_installation after the basic MariaDB install, the root password is already set somewhere during the (very simple) database install, so it is impossible to set the root password (it's not blank as it is supposed to be, coming out of the install).  This just seems to me like a bug.  My motivation  to use MariaDB is that I'm used to it, I've used sqllite before and it does not support the ability to do remote queries, which I'd like to do from my NodeJS web application to gather specific data from WeeWx data to incorporate into my app.  If this were the only issue I'd probably just fall back to sqllite for now. 

I built an entire NodeJS web application for Pi that included MariaDB a few years ago, so I've seen MariaDB run on Pi successfully.  This issue is new.

Anyway, thanks again.  I was wondering if the quality of the Pi/Raspbian has fallen behind but you are vouching for it so that is good input.   I'll be thinking about my next move.  

What's frustrating is that I've had WeeWx running on a Pi 3 and pushing data via FTP for years to my hosted site, until the hardware gave out.  I didn't think a reinstall on a new Pi 4 would surface all these issues I hadn't seen in my previous experiences with the Pi.

[vince]

Using keys generated on putty for access to a linux server was always painful in my (fortunately ancient) experience.  I always added cygwin to my windows systems to get a real shell and ssh client rather than the putty kludges.  I'm sure what you're trying is doable, you just need to find the magic incantation to generate a key that a modern Linux sshd supports.    It's also possible to generate your key on linux and import it into putty's setup the other way, but it's been so many years since I had to use putty that my memory is hazy on how to do that.

I don't see any difference in users between raspios and ubuntu other than the name of the built-in canned default user.  They're both debian(ish) under the hood.

Resetting the mariadb root password is something I 'have' done occasionally when I mess up the initial setup.  See if (this link) helps any.

If it's me, I'd work the problems one by one rather than punting the base os by reflex out of frustration.

But consider adding cygwin and ssh to your windows setup if you're going to do a lot of ssh'ing into linux hosts.  It'll make your life 'much' easier.

[Doug Jenkins]

Windows 10 has ssh built into its powershell terminal. This is my little checklist I use for all my ubuntu based distributions. 

Create SSH Keys & harden SSH Access

URL : https://kb.iu.edu/d/aews

===================================

1. First ensure you have a RSA key generated on your windows 10 workstation

2. On the remote server, create the ssh directory and authorized_keys file

mkdir -p ~/.ssh

touch ~/.ssh/authorized_keys

3. Run this command on your PC in powershell replacing SERVER

type $env:USERPROFILE\.ssh\id_rsa.pub | ssh SERVER " cat >> .ssh/authorized_keys"

4. login to the console of the new server and harden SSH

a. sudo nano /etc/ssh/sshd_config

i. PasswordAuthentication no

ii. PermitEmptyPasswords no

iii. MaxAuthTries 3

iv. ClientAliveInterval 300   #5 minute idle timeout

v. ClientAliveCountMax 3

vi. PermitRootLogin no

vii. AllowUsers %USERNAME%

b. sudo systemctl reload sshd.service

All you need to do in Windows is start powershell and type ssh USERNAME@server and you are in

DDJ

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9f815966-942d-4a54-b4a1-f9c38e033bfdn%40googlegroups.com.

[Tom Keffer]

I have had this issue with newer versions of MariaDB as well. The solution was to run as root. See this thread on SO.

Or, maybe you already knew that.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1ef03561-aa5a-4a8e-b9b5-9d91548160e5n%40googlegroups.com.

[Stephen Rogers]

Thanks to all of you for the help.  Thanks to Vince idea, I now happily am running Ubuntu Desktop on my Pi 4, which so far is a lot more polished than Raspbian OS (which is toylike by comparison).  I have MariaDB successfully installed as well, which was a seamless process on Ubuntu -- this what I'm used to.

Next step is to get passwordless SSH running at least in test mode from the Pi, as my understanding is that it is required for rsync.  When I can prove that out the path should be clear for rsync approach to pushing weather data to my cloud host.  I doubt that there will be any issues with WeeWx install on Ubuntu + Pi but we'll see!!!  This has been a rocky road.

Doug I haven't had trouble with Win10 passwordless access to my Digital Ocean Ubuntu 20.04 webserver, just got stuck with Raspbian OS as the SSH client.  But your checklist looks useful and I am a novice at RSA key pair authentication so it will likely come in handy.  And I will take a look at Vince's recommendation for cygwin although the only problem I've had with Putty was puttygen key compatibility on Raspbian. 

I will report back when I get through it all...

[vince]

I would be interested in hearing which version of ubuntu you went with, considering the new 22.04.1 LTS just came out today, and whether you have passwordless ssh working from all the client boxes you want to ssh from.

Reason I'm asking is that when I upgraded my AWS Lightsail to 22.04.1 LTS from 20.04 LTS today it broke passwordless ssh from the ancient long out of patchability debian-7 ARM box that I run weewx on.   Symptoms are that ssh with keys that 'used' to work does not work any more from the old debian box, but it works fine from my mac mini.  I don't want to confuse this thread with the gory details and the workaround I found unless others run into similar misadventures.

(my guess is that anything running current os will be fine, but I did want to mention there 'are' changes in newest ubuntu that deprecate old setups from working without workarounds).

[Owen]

I used to run Raspian on a Pi4, but my InfluxDB regularly crashed and sometimes even failed to restart cleanly at all due to corruption.

All sorts of random issues, including out of memory (when it wasn't).  No tuning of config file settings would fix it.  Drove me mental for months.  I believe it to now be a limitation of the 32-bit OS.

Anyhoo....switched to Ubuntu 20.04.4 LTS (64bit) and it's been flawless since.

I've basically ditched Raspian completely on all my Pi's - the Ubuntu release is extremely stable and robust.