Problems setting up 20210214

[Cristian Arezzini]

Hello,

I've had my Pidp running 24/7 with the older version of the software (which I kept somewhat regularly updated) but I wanted to try the newer release, to start a new installation from "clean slate".

My Pidp is connected to my router via wifi. I don't have an easy way to connect it to a display, but I have an USB switch brought out of the case. I've always used it by SSH'ing into it from my PC or from my phone.

Anyway, I prepared a new SD card using the image provided. Using my PC I edited the needed files to have it connect to the wifi. I also set it to get a static IP, and changed the SSH port to 2020 (since I have other raspberrys in my network, and I need to access them individually from outside).

I inserted the SD on the Pidp and powered it up. The front panel did not come up. Shouldn't it come up automatically? The older version did, but I can't remember if I did something to make it autostart.

By checking my router's page, I can see that the pi is up and running and connected to the network, with the correct IP. It also responds to pings.

But if i try to SSH into it, to find out what's going on, I get an immediate error:

cristian@cristian-desktop:~$ ssh 192.168.0.20 -p 2220 -l pidp8i
Connection reset by 192.168.0.20 port 2220

Is there some setting I forgot to make? Do I have to do it locally, or can I fix this by editing files on the SD using my PC?

Are there other options for connecting to it? I should have somewhere two USB-to-serial adapters, maybe I can use them back-to-back to connect to it?

On my router, I set up the port forwarding for outside access, but this should not be relevant anyway from within my local network, right?

Sorry for all the questions, but it's been a long time since I originally set the old version up, and I'm a bit lost...

Cristian

[timr...@gmail.com]

Typo?  Your post says the port was set to 2020.  The SSH command shows 2220.

[Warren Young]

This is linked directly from the top-level README.md file:

    https://tangentsoft.com/pidp8i/doc/trunk/doc/OS-images.md#sshd

[Cristian Arezzini]

Hi, thanks for the reply... I had seen that, but my main problem is that I have no easy way to connect a display to the pi. That's why I was looking for a way to connect a terminal to it. Or a way to install the needed package directly on the SD card without using the pi...

I could try connecting a keyboard to the pi, but I wouldn't know exactly what to type "in the blind" to login and such...

As I said, I have two usb-to-serial cables, I could try to use those connected back-to-back, but I wouldn't know how to configure that.

Tim: yes, that was a typo, the port is 2220 and my command matched what I had setup.

Cristian

[timr...@gmail.com]

You don't have a HDMI cable to attach to a monitor?  And then a USB keyboard and mouse?  That's what I sometimes use to make changes

to a newly set up Rasp Pi.

[Warren Young]

On Tuesday, May 11, 2021 at 11:29:42 AM UTC-6 macav...@gmail.com wrote:

Or a way to install the needed package directly on the SD card without using the pi...

The problem isn't that a package is missing, it is that when we remove /etc/sshd/*keys as part of building the binary OS images, the sshd init script doesn't regenerate them as it does on other OSes. This need for manual reconfiguration is silly, but I don't know a way around that.

There probably *is* a way, else the official Raspberry Pi OS images wouldn't work, but how do you get the packages back into the same state as with a fresh install? Once someone works out that trick, then I can modify my OS image generation script to do that instead of what I'm doing now.

As annoying as it is to answer this same question several times a year, it's not annoying enough for me to go dig into the Debian package scripting stuff surrounding the openssh-server package on Raspbian to work this stuff out, given that I always run my PiDP-8/I boxes either via RS-232 of some kind or HDMI+USB.

 

I could try connecting a keyboard to the pi, but I wouldn't know exactly what to type "in the blind" to login and such...

It's a much better use of your time to work out how to set up the Pi's USB serial console.

Having gotten that working, it's not only useful for getting you past this stumbling block, it's useful in its own right.

[Cristian Arezzini]

OK, since I did some hardware mods to my pidp8 and I have no easy access anymore to the HDMI port, I had to totally change my approach... and I just plopped the SD card into another Raspberry I use as a media center. I should have thought of doing this right away.

Turns out having  access to a keyboard and display is imperative, not only because of the "openssh-server" issue, but also because Rfkill apparently disables the Wifi until you set up your Wifi locale using raspi-config.

Also, you are forced to change the user's password at first boot.

Anyway, the system works now... I simply put the SD card back in my pidp, and everything's good now. I installed, compiled and ran the Fourier "blinkenlichten demo", and it works great.

I also disabled the onboard HDMI port and audio and bluetooth ports, to save power (as I have a backup battery installed in the case).

I have a question on the ILS but I'll post that in a separate thread.

Thanks again

Cristian

[Warren Young]

On Thursday, May 13, 2021 at 2:40:16 AM UTC-6 macav...@gmail.com wrote:

Rfkill apparently disables the Wifi until you set up your Wifi locale using raspi-config

Isn't that only half the problem? How do you select a wifi network and set the PSK without running the Pi via some sort of direct console, whether USB+HDMI or serial, or some other non-network method?

Also, you are forced to change the user's password at first boot.

While true, that's irrelevant to this discussion since you can change the user's password over SSH.

 

I also disabled the onboard HDMI port and audio and bluetooth ports, to save power

I'd be interested to see actual current measurements before and after.

[Cristian Arezzini]

Isn't that only half the problem? How do you select a wifi network and set the PSK without running the Pi via some sort of direct console, whether USB+HDMI or serial, or some other non-network method?

Well, actually that's pretty easy, I do that all the time: I just enter my wifi's SSID and PW in the file /etc/wpa_supplicant/wpa_supplicant.conf

 

Also, you are forced to change the user's password at first boot.

While true, that's irrelevant to this discussion since you can change the user's password over SSH.

Yes and no... Yes, what you say is true; no, since it's not irrelevant because I can't get SSH access to change the password until I've had some other form of access... :)

 

 

I also disabled the onboard HDMI port and audio and bluetooth ports, to save power

I'd be interested to see actual current measurements before and after.

I didn't take direct measurements, but I can say that without disabling the HDMI, audio, and bluetooth interfaces, the power  circuits I'm using (LiPo battery + dedicated charger + regulator) could barely supply enough current to run the Pi + front panel, and keep the LiPo charged. There was no current left to recharge the battery if it got discharged, so after a (partial or total) discharge I had to keep the Pi off for a while, to let the battery recharge.

After disabling those circuits, the available current is enough to recharge the battery while the Pi is up and running, albeit not very quickly.

[Cristian Arezzini]

Il giorno lunedì 17 maggio 2021 alle 00:24:37 UTC+2 Cristian Arezzini ha scritto:

Isn't that only half the problem? How do you select a wifi network and set the PSK without running the Pi via some sort of direct console, whether USB+HDMI or serial, or some other non-network method?

Well, actually that's pretty easy, I do that all the time: I just enter my wifi's SSID and PW in the file /etc/wpa_supplicant/wpa_supplicant.conf

 

Sorry about the double post, but I could not find a way to edit my message.

To clarify, I meant that I edit that file directly on my PC, right after flashing the SD card. No need to have the Pi up and running (and accessible) to make those changes. I use linux so I have easy access to the Pi's file system.

With the same method, I install a basic script to automatically reconnect to the WiFi, I setup the pi's static IP address, setup the SSH ports, disable the aforementioned interfaces...

Cristian

[Warren Young]

On Sunday, May 16, 2021 at 4:24:37 PM UTC-6 macav...@gmail.com wrote:

How do you select a wifi network and set the PSK without running the Pi via some sort of direct console, whether USB+HDMI or serial, or some other non-network method?

Well, actually that's pretty easy, I do that all the time: I just enter my wifi's SSID and PW in the file /etc/wpa_supplicant/wpa_supplicant.conf

Then re-generating /etc/ssh/ssh_host*_key and their corresponding *.pub files using ssh-keygen on the Pi filesystem should be "pretty easy" for you, too.

I'm pretty sure almost everyone having such problems doesn't think these are reasonable solutions.

What we really want is a method that lets me set a flag here at binary OS build time that tells the OS on the first boot after flashing the card with that image that the keys *must* be regenerated.

There is a way to do this on Red Hattish Linuxes. Why not Debian types?

Back when the PiDP-8/I software installed a root-level SysVinit script, we were able to forcibly regenerate the keys, but we're neither running under SysVinit any more, nor starting the software as root, so we can't do this at the pidp8i service level any more.

[Andrew Yeomans]

Can't we do this by simply putting a file called ssh in the boot partition? https://www.raspberrypi.org/documentation/remote-access/ssh/lib/systemd/system/sshswitch.service

That is used by /lib/systemd/system/sshswitch.service to start ssh and then delete the ssh (or ssh.txt) file, looks like it also will invoke regenerate_ssh_host_keys.service

In similar fashion you can put a wpa_supplicant.conf file in the boot partition https://www.raspberrypi.org/documentation/configuration/wireless/headless.md

We can run other headless scripts by putting them in the boot partition e.g. "init=/pidp8i/initscript parameters" at end of cmdline.txt

A virgin Pi image has "init=/usr/lib/raspi-config/init_resize.sh" which resizes the SD card then edits itself out of cmdline.txt.

[I might get a chance to look into this a bit more soon. I was going to do a simple update of my PiDP8 from the new image, but was foiled as it is slightly too large for my SD card of 15196160 512-byte logical blocks: (7.78 GB/7.25 GiB).]

Andrew Yeomans

--
You received this message because you are subscribed to the Google Groups "PiDP-8" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pidp-8+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-8/52ab08c2-441c-40a3-9d98-ccc18ff29a7fn%40googlegroups.com.

[Warren Young]

On Monday, May 17, 2021 at 8:37:10 AM UTC-6 andrew....@gmail.com wrote:

Can't we do this by simply putting a file called ssh in the boot partition? https://www.raspberrypi.org/documentation/remote-access/ssh/lib/systemd/system/sshswitch.service

That gives a 404, and the closest alternative document I found doesn't say anything of the sort:

    https://www.raspberrypi.org/documentation/remote-access/ssh/README.md

 

looks like it also will invoke regenerate_ssh_host_keys.service

Going on blind faith in the Internet, I've made a change to use this mechanism instead of the generic Debian style method we had before. I figure, worst case, we're just back to where we were before, only now we're dependent on Raspberry Pi OS specifically, which was already a given as far as I'm concerned.

Unless someone tests this beforehand — not me, not interested in re-keying my PiDP-8/i — we won't know whether this helps until after the binary OS media is re-generated, most likely timed to be after a Debian 11 based Raspberry Pi OS is released. September-ish is my guess.

 

it is slightly too large for my SD card of 15196160 512-byte logical blocks: (7.78 GB/7.25 GiB).]

That doesn't make a lot of sense. You should have something like 5 GB free on the root partition with a fresh image as downloaded from my site, on an 8 GB card. Even a really big update should take less than a gig, and that only temporary, for the downloaded packages.

The boot partition will be quite constrained, with only a few hundred megs free, but you shouldn't be putting big files there.

[Andrew Yeomans]

Hi Warren,

Sorry about the bad paste! Yes, section 3 of https://www.raspberrypi.org/documentation/remote-access/ssh/README.md was correct.

Re the image size:

pidp8i-2021.02.14-nls-buster-lite.img.zip contains pidp8i-2021.02.14-nls-buster-lite.img which is 7948206080 bytes long so won't fit on my card. Even though the image has 5GB of "free space",

The root partition really should be shrunk, so that "init=/usr/lib/raspi-config/init_resize.sh" will regrow it.

https://github.com/Drewsif/PiShrink is one tool to do that; I've not tried it, but have in the past done similar shrinking manually. Somewhat tedious!

I've also previously shrunk the boot partition so everything just fits on a 2GB SD card. Older Raspbian images used a smaller /boot partition, the new ones allocate 256MB even though they only use 50MB or so.

Note it's probably a good idea to make this a nice multiple of the eMMC erase block size - my current card has a 4MByte erase block. On my system this value is found at:-

$ cat /sys/devices/platform/soc/3f202000.mmc/mmc_host/mmc0/mmc0:aaaa/preferred_erase_size
4194304

There may be nicer ways to read the value! Some systems have "mtdinfo" command, but that's not a default raspios command.

Andrew

--

You received this message because you are subscribed to the Google Groups "PiDP-8" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pidp-8+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-8/442e391e-7d38-4380-aea1-191f61ade6c3n%40googlegroups.com.

[Warren Young]

On Monday, May 17, 2021 at 10:58:44 AM UTC-6 andrew....@gmail.com wrote:

pidp8i-2021.02.14-nls-buster-lite.img.zip contains pidp8i-2021.02.14-nls-buster-lite.img which is 7948206080 bytes long so won't fit on my card.

Then you don't have an "8 GB" SD card, since whether you're talking SI-units gigabytes (7.948...GB) or gibibytes (7.40 GB) it's under the limit.

I made that image using an actual 8 GB card.

 

The root partition really should be shrunk

The prior OS images were shrunk, but the process we used failed on the new cards, resulting in unbootable media when tested.  Since the process of making new images took something like 8 hours this last time (SD card reading and rewriting is sllllloooooowwww, and we have to do it several times) I gave up and nuked the resizing step to get to the next step in the process.

You could image the card using the prior version and upgrade from there...

 

https://github.com/Drewsif/PiShrink

423 lines of shell script just to do that??

Maybe I'll look into what that script does to justify itself when it comes time to do the next release, but it's rather disheartening since the prior process was done in about 25 lines of shell script. See the commit taking the "bosi shrink" feature out.

If you'd be willing to distill the other script to a restoration of the "bosi shrink" process, it'd help me to evaluate the pros and cons. I'm hoping that the other script does a bunch of stuff that my BOSI process doesn't actually need, so we can strip that out to get it down to similar size levels.

have in the past done similar shrinking manually. Somewhat tedious!

More than tedious: tricky to get right, because it's a potentially lossy process.  You have to do it *just so* else you make the OS unbootable.

I'm sure what went wrong in my prior process is that upstream changed something between Debian Stretch and Buster, and my old "bosi shrink" process didn't take that into account.

 

I've also previously shrunk the boot partition so everything just fits on a 2GB SD card.

The Buster based Raspberry Pi OS just barely fits into that space so that once all of the PiDP-8/I stuff is added, you're over the limit.

The Foundation actually says "8 GB" as the minimum, probably because that's the smallest cards you can reasonably find these days. I actually went looking for 4 GB cards before deciding to make the Foundation's advice my own recommendation and couldn't find anything other than old remaindered stock and such.

I repeated that search, and we're pretty much at that point now with 8 and 16 GB cards, with 32+ being the sweet spot for cheap cards.

 

Note it's probably a good idea to make this a nice multiple of the eMMC erase block size - my current card has a 4MByte erase block.

Why? It's only at the suboptimal size during the first boot, and it'll only matter if you happen to access the very last block. Once it expands, it's at the full SD card size, which is presumably an even multiple of the native medium block size.

[Cristian Arezzini]

I had the same size problem as Andrew. I used a brand new, virgin Toshiba Exceria 8GB card. But in my experience none of these cards are *exactly* 8GB, the "usable" space is always somewhat less. I had to extract the image, mount it, shrink the partition, then truncate the modified image file at the right place, so that the cloning software wouldn't complain and fail. Here's the page describing the process I followed:

https://askubuntu.com/questions/1174487/re-size-the-img-for-smaller-sd-card-how-to-shrink-a-bootable-sd-card-image

it's the longest answer in the thread, and it worked perfectly for me, though being a bit tedious...

Cristian

[Andrew Yeomans]

I did a bit of checking and testing.

Rather than shrink the rootfs partition, it is easier to not grow it in the first place! Note this should speed up any later image copying, as only 2-2.5 GiB will need to be read off the card rather than 8 GiB.

1. Copy the buster lite image to SD card. The Raspberry Pi mager is a good way to do this as it verifies the image

2. Edit boot/cmdline.txt to remove "quiet init=/usr/lib/raspi-config/init_resize.sh" before using the card in a pi.

3. Resize partition 2 to grow it to the minimum size needed, around 2 GiB. There are several ways to do this:-

3a. Use a partition tool, such as gparted on Linux or gparted live or other partition tool, on the system used to copy the image

3b. Boot on pi then use parted or fdisk or cfdisk or sfdisk to increase the size of partition 2 (e.g. "sudo parted /dev/mmcblk0p2"). You might need to reboot. Then use "sudo resize2fs /dev/mmcblk0p2" to grow the filesystem into the enlarged partition.

You now have a larger root partition but which does not fill the sd card.

After updating the OS, if you replace the original /boot/cmdline.txt, then the partition will get resized to fill the SD card, but unfortunately the filesystem won't be enlarged on reboot. You can use "sudo raspi-config" to fix, or directly use "sudo resize2fs /dev/mmcblk0p2".  [I think another file might need restoring to get the file system resize working, I've not followed all the logic yet.]

In case you want to shrink an SD card further, you can shrink the boot partition too. I wouldn't recommend this unless you really need to squeeze out the last few MB.

The Buster Lite 2021-03-04 image looks like:

Start sector	Size	Contents
0	512 bytes	MBR = Master Boot Record – map of partitions (no actual boot loader needed)
1	4 MiB-512 B	Free space, unused
8192	256 MiB	boot - FAT32 filesystem (type 0x0c) with boot code (about 52 MiB used)
532480	1520 MiB	root – EXT4 filesystem (type 0x83) with operating system (about 1185 MiB used, 279 MiB free)
3645440		Free space, root will be extended into this on first boot

Earlier versions of Raspbian used only 60 MB for the boot partition, this has been enlarged to allow for additional kernels and device tree overlays.

The latest versions of Gparted cannot resize FAT32 partitions smaller than 512MB, so easiest to create a new card layout. Boot could start at 1 MB (sector 1024) of size 70-100 MB, but best to make it start at 4 MB (sector 4096) to align with a common eMMC erase block size – so as not to wear out the first erase block with writes to the first partition. Then copy the files from a mounted disk image.

By the way, a quick way to erase and reset an SD card to factory condition is to run the blkdiscard command. The SD/eMMC technology only allows large blocks (e.g. 4MiB) to be erased, so the memory controller on the card will re-map logical sectors to free areas as they get written. This may result in significant slow-down of writes to the card as time goes on. You can do this on a pi, e.g. "sudo blkdiscard /dev/mmcblk0" will wipe and reset the card. After executing. take out the card immediately (!) before odd blocks get written back to it, and power down pi. It would be nicer to the OS if you were not running off the card, but it works! Unfortunately you can't use blkdiscard on USB memory sticks, the USB controller won't pass on the low-level commands.

Andrew Yeomans

--

You received this message because you are subscribed to the Google Groups "PiDP-8" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pidp-8+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-8/ae1c9788-337a-4b52-a703-20654270642fn%40googlegroups.com.