debian sid no boot after this morning's update

[Frank]

Just went back to my Debian Sid installation this evening to discover
it won't boot.
There were 88 updates this morning so I suspect my problem is
related to that.

Has anyone else run into this? How would I go about diagnosing
what's wrong? I tried booting in emergency mode but don't know
what to look for.

The emergency prompt suggested journalctl -xb so that's what
I did but I didn't see anything obviously wrong.

Right now I am at my wits end. Good thing I have a working
Fedora partition.

Help!

[Frank]

On 2023-01-04 7:30 p.m., Frank wrote:
> Just went back to my Debian Sid installation this evening to discover
> it won't boot.
> There were 88 updates this morning so I suspect my problem is
> related to that.
>
> Has anyone else run into this? How would I go about diagnosing
> what's wrong? I tried booting in emergency mode but don't know
> what to look for.
>

  Adding more info.


  All i get is a flashing cursor on a black screen.

  Digging around in the emergency shell I found two logs related to
lightdm. It seems it is having problems running the greeter.

This is part of the lightdm log

[+0.33s] DEBUG: Seat seat0: Display server ready, starting session
authentication
[+0.33s] DEBUG: Session pid=734: Started with service 'lightdm-greeter',
username 'lightdm'
[+0.35s] DEBUG: Session pid=734: Authentication complete with return
value 0: Success
[+0.35s] DEBUG: Seat seat0: Session authenticated, running command
[+0.35s] DEBUG: Session pid=734: Running command
/usr/sbin/lightdm-gtk-greeter
[+0.35s] DEBUG: Creating shared data directory /var/lib/lightdm/data/lightdm
[+0.35s] DEBUG: Session pid=734: Logging to
/var/log/lightdm/seat0-greeter.log
[+0.37s] DEBUG: Activating VT 7
[+0.37s] DEBUG: Activating login1 session c5
[+0.37s] DEBUG: Seat seat0 changes active session to c5
[+0.37s] DEBUG: Session c5 is already active
[+0.37s] DEBUG: Greeter closed communication channel
[+0.37s] DEBUG: Session pid=734: Exited with return value 1
[+0.37s] DEBUG: Seat seat0: Session stopped
[+0.37s] DEBUG: Seat seat0: Stopping; failed to start a greeter
[+0.37s] DEBUG: Seat seat0: Stopping
[+0.37s] DEBUG: Seat seat0: Stopping display server
[+0.37s] DEBUG: Sending signal 15 to process 720
[+0.38s] DEBUG: Seat seat0 changes active session to
[+0.52s] DEBUG: Process 720 exited with return value 0
[+0.52s] DEBUG: XServer 0: X server stopped

This is the greeter log

** (process:734): WARNING **: 22:32:38.355: Error reading existing
Xauthority: Failed to open file ?/var/lib/lightdm/.Xauthority?:
Permission denied
Error writing X authority: Failed to open X authority
/var/lib/lightdm/.Xauthority: Permission denied

[Greg Wooledge]

On Wed, Jan 04, 2023 at 11:03:15PM -0500, Frank wrote:
> ** (process:734): WARNING **: 22:32:38.355: Error reading existing
> Xauthority: Failed to open file ?/var/lib/lightdm/.Xauthority?: Permission
> denied
> Error writing X authority: Failed to open X authority
> /var/lib/lightdm/.Xauthority: Permission denied

Does that file exist? If so, ls -ld /var/lib/lightdm/.Xauthority

If not, ls -ld /var/lib/lightdm

Hell, just do both regardless of whether the file currently exists.
It's one command with two lines of output. Should be the first thing
you do.

See if you can figure out which user lightdm is trying to run as.
Try to make it so that user can write that file, either by removing
the existing file, or chowning it, or fixing the permissions on the
directory. Whatever is indicated by the lightdm changelog. Hopefully
there's a note in the lightdm changelog about this. Or in NEWS.

[Frank]

On 2023-01-04 11:23 p.m., Greg Wooledge wrote:
> On Wed, Jan 04, 2023 at 11:03:15PM -0500, Frank wrote:
>> ** (process:734): WARNING **: 22:32:38.355: Error reading existing
>> Xauthority: Failed to open file ?/var/lib/lightdm/.Xauthority?: Permission
>> denied
>> Error writing X authority: Failed to open X authority
>> /var/lib/lightdm/.Xauthority: Permission denied
> Does that file exist? If so, ls -ld /var/lib/lightdm/.Xauthority
>
> If not, ls -ld /var/lib/lightdm
>
> Hell, just do both regardless of whether the file currently exists.
> It's one command with two lines of output. Should be the first thing
> you do.

  Yes, the file exists and is owned by lightdm. I don't know whether
this is
correct as I can't find any info on who should own it.
I tried moving it out of the way hoping lightdm or the greeter would
recreate but no luck.

>
> See if you can figure out which user lightdm is trying to run as.
> Try to make it so that user can write that file, either by removing
> the existing file, or chowning it, or fixing the permissions on the
> directory. Whatever is indicated by the lightdm changelog. Hopefully
> there's a note in the lightdm changelog about this. Or in NEWS.
>

 There are no notes or news file for lightdm so I am in the dark.

It is strange as there were no updates yesterday of lightdm or the
greeter. The GTK
greeter was updated the day before but Debian started and ran fine after
that.

There are other strange things going on which may or not be related. I
tried to
reinstall lightdm and the greeter using apt. (The Debian partition is
mounted
on my Fedora partition) but apt failed saying it could not contact
debian.deb to
gain access to the needed files. I downloaded the files from Fedora and
moved them
to Debian but dpkg said it could not access them !!??

Is there a way to reinstall lightdm and the greeter from Fedora ??
Perhaps using chroot
which I have zero knowledge of.

I am getting close to reinstalling Debian ( shades of Windows) !

Thanks for any help.

[Eduardo M KALINOWSKI]

On 05/01/2023 11:12, Frank wrote:
> Is there a way to reinstall lightdm and the greeter from Fedora ??
> Perhaps using chroot
> which I have zero knowledge of.

Since the problem seems to be the display manager/greeter, there's a
good chance that you can press Ctrl+F1 (or F2, F3...) to reach a
console, so no need to a console to reinstall debian packages. Try also
Ctrl+Alt+Fn.


--
To give of yourself, you must first know yourself.

Eduardo M KALINOWSKI
edu...@kalinowski.com.br

[Greg Wooledge]

On Thu, Jan 05, 2023 at 09:12:58AM -0500, Frank wrote:
> On 2023-01-04 11:23 p.m., Greg Wooledge wrote:
> > On Wed, Jan 04, 2023 at 11:03:15PM -0500, Frank wrote:
> > > ** (process:734): WARNING **: 22:32:38.355: Error reading existing
> > > Xauthority: Failed to open file ?/var/lib/lightdm/.Xauthority?: Permission
> > > denied
> > > Error writing X authority: Failed to open X authority
> > > /var/lib/lightdm/.Xauthority: Permission denied
> > Does that file exist? If so, ls -ld /var/lib/lightdm/.Xauthority
> >
> > If not, ls -ld /var/lib/lightdm
> >
> > Hell, just do both regardless of whether the file currently exists.
> > It's one command with two lines of output. Should be the first thing
> > you do.
>
>   Yes, the file exists and is owned by lightdm. I don't know whether this is
> correct as I can't find any info on who should own it.
> I tried moving it out of the way hoping lightdm or the greeter would
> recreate but no luck.

Hmm, that's weird indeed. If the file is owned by lightdm (the user),
that implies that lightdm (the program) was able to write it in the past.
If it can't do so now, then we have to try to guess what changed which
could lead to this result.

Possibility 1: lightdm (the program) no longer runs as lightdm (the user).

Pissibility 2: a new restriction has been added to the service which
launches lightdm (the program). Something at the AppArmor level, or at
the systemd unit level, perhaps.

> It is strange as there were no updates yesterday of lightdm or the greeter.

How about in recent days? Maybe a change occurred a few days ago, but
you hadn't rebooted, so the changes didn't actually have any impact yet.

> The GTK
> greeter was updated the day before but Debian started and ran fine after
> that.

I don't know what this is. A separate package?

> There are other strange things going on which may or not be related. I tried
> to
> reinstall lightdm and the greeter using apt. (The Debian partition is
> mounted
> on my Fedora partition)

This is confusing. Are you saying that you have a multi-boot system,
with both Debian and Fedora installed on the same hard drive? And that
you booted into Fedora, then mounted the Debian root file system somewhere,
and then chrooted into it?

> but apt failed saying it could not contact
> debian.deb to
> gain access to the needed files. I downloaded the files from Fedora and
> moved them
> to Debian but dpkg said it could not access them !!??

Details would be helpful here. Actual commands, actual error messages.

> Is there a way to reinstall lightdm and the greeter from Fedora ?? Perhaps
> using chroot
> which I have zero knowledge of.

Wait... you DID NOT chroot into the Debian file system after mounting it?!

How in the hell did you expect ANYTHING to work...?

OK, basic primer:

mkdir /debian
mount /dev/whatever /debian
chroot /debian

There's fancy crap you can do involving bind mounts of /proc and so on,
but you probably don't need all that.

That said, I think you've jumped ahead too far. Your Debian system
apparently boots just fine -- it just doesn't run X. So boot Debian,
either normally, or with the "systemd.unit=multi-user.target" kernel
parameter, login on a regular text console, and fix it from there.

Another thing you could try would be to boot your previous kernel, in
case it was a kernel update that broke lightdm. It might not make any
difference, but hey, it's worth trying.

Another other thing you could try would be to run 'startx' from a
regular text console after booting Debian and logging in on said console.
If X starts correctly that way, then you know the problem is truly in
the DM, not in X, video drivers, video firmware, etc. We suspect this
already, but confirmation is always good.

[Frank McCormick]

On 1/5/23 09:24, Eduardo M KALINOWSKI wrote:
> On 05/01/2023 11:12, Frank wrote:
>> Is there a way to reinstall lightdm and the greeter from Fedora ??
>> Perhaps using chroot
>> which I have zero knowledge of.
>
> Since the problem seems to be the display manager/greeter, there's a
> good chance that you can press Ctrl+F1 (or F2, F3...) to reach a
> console, so no need to a console to reinstall debian packages. Try also
> Ctrl+Alt+Fn.
>
>

I never tried that unfortunately. It might have saved me a lot of
trouble. I finally got lightdm and it's greeter reinstalled after
fixing some other problems and the system then booted fine. The strange
thing in my mind is how the whole thing happened, as only the lightdm
greeter had been updated and that was a few days ago.
Thanks for the suggestion - Ill try to keep it in mind.

--
Frank McCormick

[Frank McCormick]

On 1/5/23 09:53, Greg Wooledge wrote:
/snip/

Lightdm uses various greeters which provide various login appearances to
the user.

>
>
> This is confusing. Are you saying that you have a multi-boot system,
> with both Debian and Fedora installed on the same hard drive? And that
> you booted into Fedora, then mounted the Debian root file system somewhere,
> and then chrooted into it?
>

Yes I have a multi boot system. I only mounted Debian in Fedora to read
log files, and move the two deb files to debian. I downloaded them in
fedora.

>> but apt failed saying it could not contact
>> debian.deb to
>> gain access to the needed files.

> Wait... you DID NOT chroot into the Debian file system after mounting it?!
>
> How in the hell did you expect ANYTHING to work...?

No, I was not trying to use Debian commands from Fedora. I was in the
Debian emergency shell. I am no system admin, but I do know that won't work.
As it turns out dpkg failed because of other problems. A bunch of files
and directories related to sudo had been changed to being owned by me
instead of root. This I absolutely do not understand!! I had to fix them
one by one in the Debian emergency shell.

>
> OK, basic primer:
>
> mkdir /debian
> mount /dev/whatever /debian
> chroot /debian
>
> There's fancy crap you can do involving bind mounts of /proc and so on,
> but you probably don't need all that.
>
> That said, I think you've jumped ahead too far. Your Debian system
> apparently boots just fine -- it just doesn't run X.

That wasn't the problem. Seemingly it ran X but lightdm didn't run so X
was killed. That's what it looked like.

So boot Debian,
> either normally, or with the "systemd.unit=multi-user.target" kernel
> parameter, login on a regular text console, and fix it from there.
>

Once I fixed the sudo problem, dpkg worked and I reinstalled lightdm
and the greeter it uses. The system then booted fine but I have no idea
how it got into that state.

> Another thing you could try would be to boot your previous kernel, in
> case it was a kernel update that broke lightdm. It might not make any
> difference, but hey, it's worth trying.
>
> Another other thing you could try would be to run 'startx' from a
> regular text console after booting Debian and logging in on said console.
> If X starts correctly that way, then you know the problem is truly in
> the DM, not in X, video drivers, video firmware, etc. We suspect this
> already, but confirmation is always good.
>

These are good ideas I will try then next time I have a similar
problem, hopefully never.

Thanks for all your help and advice.
--
Frank McCormick

[Andrew M.A. Cater]

On Thu, Jan 05, 2023 at 09:12:58AM -0500, Frank wrote:
>

> There are other strange things going on which may or not be related. I tried
> to
> reinstall lightdm and the greeter using apt. (The Debian partition is
> mounted
> on my Fedora partition) but apt failed saying it could not contact
> debian.deb to
> gain access to the needed files. I downloaded the files from Fedora and
> moved them
> to Debian but dpkg said it could not access them !!??
>

If you're using raw dpkg tp install them - you are root? If using su, it
needs to be su - to get roots login environment.

> Is there a way to reinstall lightdm and the greeter from Fedora ?? Perhaps
> using chroot
> which I have zero knowledge of.
>
> I am getting close to reinstalling Debian ( shades of Windows) !
>

Debian sid is unstable by name mostly because of package churn and large
transitions like this. Unfortunately, it's not intended for inexeperienced
users - you are expected to be able to resolve most problems yourself - "if
it breaks, you get to keep both pieces".

Lots of folk go to use sid because they want to be very up to date when
the boringness of Debian stable with security support and defined upgrade
paths is more appropriate for them.

If something is going to update 80 packages at once, try manually using
apt or apt-get to install five at a time - take it more steadily and take
notes of any package breakage, maybe.

Look in the Debian bugs database / the Debian package tracker to see
what bugs have been recorded against lightdm recently.

With every good wish, as ever,

Andy Cater

> Thanks for any help.
>
>

[Greg Wooledge]

On Thu, Jan 05, 2023 at 06:12:07PM +0000, Andrew M.A. Cater wrote:
> On Thu, Jan 05, 2023 at 09:12:58AM -0500, Frank wrote:
> >
> > There are other strange things going on which may or not be related. I tried
> > to
> > reinstall lightdm and the greeter using apt. (The Debian partition is
> > mounted
> > on my Fedora partition) but apt failed saying it could not contact
> > debian.deb to
> > gain access to the needed files. I downloaded the files from Fedora and
> > moved them
> > to Debian but dpkg said it could not access them !!??
> >
>
> If you're using raw dpkg tp install them - you are root? If using su, it
> needs to be su - to get roots login environment.

Unless you create an /etc/default/su file to fix what Debian broke.

unicorn:~$ cat /etc/default/su
ALWAYS_SET_PATH yes

That restores the PATH behavior to how it was before Debian switched
upstream sources for su in buster.

Anyway, it sounded like the OP had some permission/ownership changes
which were breaking things, but they managed to find them and revert
them. Or some of them. We'll have to await further details.

[Charles Curley]

On Thu, 5 Jan 2023 09:12:58 -0500
Frank <debia...@videotron.ca> wrote:

>   Yes, the file exists and is owned by lightdm. I don't know whether
> this is
> correct as I can't find any info on who should own it.

It would be more useful to those trying to help you if you showed the
actual results rather than telling us what you find significant. Like
so:

root@hawk:~# ls -ld /var/lib/lightdm
drwxr-x--- 6 lightdm lightdm 4096 Oct 29 2021 /var/lib/lightdm
root@hawk:~# ls -la /var/lib/lightdm
total 28
drwxr-x--- 6 lightdm lightdm 4096 Oct 29 2021 .
drwxr-xr-x 70 root root 4096 Sep 30 16:10 ..
drwxr-xr-x 7 lightdm lightdm 4096 Oct 29 2021 .cache
drwx------ 5 lightdm lightdm 4096 Mar 5 2022 .config
drwxr-xr-x 4 root root 4096 Dec 19 11:54 data
drwxr-xr-x 3 lightdm lightdm 4096 Oct 29 2021 .local
-rw------- 1 lightdm lightdm 686 Jan 5 11:35 .Xauthority
root@hawk:~#

Note that I include the trailing prompt to show that the command ended
rather than that the result was truncated.

This is from a working system which I have rebooted since most of those
dates. You may find it useful to check against your directory.

--
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[Frank McCormick]

On 1/5/23 14:45, Charles Curley wrote:
> On Thu, 5 Jan 2023 09:12:58 -0500
> Frank <debia...@videotron.ca> wrote:
>
>>   Yes, the file exists and is owned by lightdm. I don't know whether
>> this is
>> correct as I can't find any info on who should own it.
>
> It would be more useful to those trying to help you if you showed the
> actual results rather than telling us what you find significant.

Agreed in retrospect

>Like
> so:
>
> root@hawk:~# ls -ld /var/lib/lightdm
> drwxr-x--- 6 lightdm lightdm 4096 Oct 29 2021 /var/lib/lightdm
> root@hawk:~# ls -la /var/lib/lightdm
> total 28
> drwxr-x--- 6 lightdm lightdm 4096 Oct 29 2021 .
> drwxr-xr-x 70 root root 4096 Sep 30 16:10 ..
> drwxr-xr-x 7 lightdm lightdm 4096 Oct 29 2021 .cache
> drwx------ 5 lightdm lightdm 4096 Mar 5 2022 .config
> drwxr-xr-x 4 root root 4096 Dec 19 11:54 data
> drwxr-xr-x 3 lightdm lightdm 4096 Oct 29 2021 .local
> -rw------- 1 lightdm lightdm 686 Jan 5 11:35 .Xauthority
> root@hawk:~#
>
> Note that I include the trailing prompt to show that the command ended
> rather than that the result was truncated.
>
> This is from a working system which I have rebooted since most of those
> dates. You may find it useful to check against your directory.
>

I did indeed. In checking what I had there were a couple of
differences which hadn't seemed to make much difference. But I have
fixed those now.

I appreciate your help.


--
Frank McCormick

[Frank McCormick]

I managed to fix most of the problem permissions, and just now fixed
the rest with the help of Charles Curley's posting of the permission on
his system.

I think we are now back to normal, whatever that means.

Thanks all

--
Frank McCormick

[Frank McCormick]

On 1/5/23 13:12, Andrew M.A. Cater wrote:

>>
>> I am getting close to reinstalling Debian ( shades of Windows) !
>>
>
> Debian sid is unstable by name mostly because of package churn and large
> transitions like this. Unfortunately, it's not intended for inexeperienced
> users - you are expected to be able to resolve most problems yourself - "if
> it breaks, you get to keep both pieces".
>
> Lots of folk go to use sid because they want to be very up to date when
> the boringness of Debian stable with security support and defined upgrade
> paths is more appropriate for them.
>
> If something is going to update 80 packages at once, try manually using
> apt or apt-get to install five at a time - take it more steadily and take
> notes of any package breakage, maybe.
>
> Look in the Debian bugs database / the Debian package tracker to see
> what bugs have been recorded against lightdm recently.

I've been running Sid for about 10 years and yes it has broken from
time to time, but, with some knowledge and a lot of help from the people
on this list I have always managed to put the two pieces back together.
At the same time I also run Fedora on another partition which I
believe is a little more leading edge than Sid. It too breaks time to
time. And to add to the mix I have just
started running OpenSuse Tumbleweed which keeps me on
my toes. I guess I am a bear for punishment. One day soon I will drop
two of the three. Until then I appreciate
everyone's help.

>
> With every good wish, as ever,
>
> Andy Cater
>
>> Thanks for any help.
>>
>>
>

--
Frank McCormick

[songbird]

Frank McCormick wrote:
...

> my toes. I guess I am a bear for punishment. One day soon I will drop
> two of the three. Until then I appreciate
> everyone's help.

always a good idea to keep a stable booting partition handy,
especially when running sid (my opinion of course :) ).

normally i stick with testing and it seems ok for what i'm
doing but i also keep the stable partition bootable just in
case.

nothing i'm doing is production grade or involves possible
harm to anyone else so that is the other consideration.


songbird

[Charles Curley]

On Thu, 5 Jan 2023 19:01:11 -0500
Frank McCormick <debia...@videotron.ca> wrote:

> I did indeed. In checking what I had there were a couple of
> differences which hadn't seemed to make much difference. But I have
> fixed those now.
>
> I appreciate your help.

Glad to help. Now you might wonder what might have caused the
discrepancies between your system and mine. It might help prevent a
recurrence. Since you didn't tell us *exactly* what you did to repair,
you are the person best situated to do this.