can you help me think through a system reclamation problem?

[Paul Johnson]

On a RedHat Enterprise system, the RPM system was running and
updating a lot of packages in the change from 5.3 to 5.4. Because
his system got slow, the user did the worst thing possible. He
turned it off. Then when it restarted, the upgrade started again,
and he turned it off again. After that, it would not boot, so I
booted it from a Redhat 5.3 CD.

If I had RedHat 5.4 on disk I would use that to just reinstall.
However, for some reason I cannot fathom, the people who are in
charge of KU's satellite server for RedHat iso files don't stay up
to date, so I don't got no 5.4 disk.

Anyway, I engineered an almost super human feat. First, I booted the
system, and ran the yum-complete-transaction thing, and I watched in
horror as the system methodically removed about 200 rpms. Something
had gone wrong so that the new rpm were not installed and the old
were removed, including rpm, basically everything. Then the super
human part came into play. In the system's
/var/cache/yum/rhel-i386-client folder, the new 5.4 upgrade rpms
were available, and I had the disk with the 5.3 rpms. I tell you it
was a mighty flurry of commands. Mighty!

I worked it back to the place where the system will start. The
kernel is found, all the services start, and it comes to a login
prompt.

But no users can log in. It lets you type in a user name, but it
does not ask for a password. It just pops back saying username:
again. Some rpm package that controls the login process must not
have gotten installed. Or else a post-install script did not complete.

If you could help me figure out which rpms might need to be
re-installed, then I could go back at this tomorrow morning and succeed.

pj

--
Paul E. Johnson email: paul...@ku.edu
Professor, Political Science http://pj.freefaculty.org
1541 Lilac Lane, Rm 504
University of Kansas Office: (785) 864-9086
Lawrence, Kansas 66044-3177 FAX: (785) 864-5700

[Dario Landazuri]

> If you could help me figure out which rpms might need to be
> re-installed, then I could go back at this tomorrow morning and succeed.

It would probably take you less time to back up the user's data and
reinstall the machine. You *do* do installs via kickstart, I hope...

-Dario

--
**************************************************************
Dario Landazuri Triangle Fraternity Minn97Ok
da...@landazuri.net
http://www.landazuri.net
**************************************************************
Put 30 kids in a room with an easel, a computer, and a guitar.
You'll get one Van Gogh, one Von Neumann, and one Van Halen
(or Van Morrison). And 27 burger flippers.
-Dr. Damian Conway

[Jeffrey Watts]

Yeah, yum-complete-transaction can be dangerous if you've aborted an install at an inappropriate time.  Basically it loses state and does the cleanup (remove) but doesn't finish the install.

To be honest you're wasting your time.  I'd boot off of a live USB stick, back up the user data, and do a fresh install.  You've lost state on that machine and you won't know what's missing.  Most likely little weird problems are going to pop up again and again as you discover more missing packages.

A backup and reinstall will only take an hour or two.

Jeffrey.

--

"He that would make his own liberty secure must guard even his enemy from oppression; for if he violates this duty he establishes a precedent that will reach to himself." -- Thomas Paine

[Steve Nordquist]

> A backup and reinstall will only take an hour or two.
> Jeffrey.

Nice flurry; if /etc/passwd looks willing and the chain of scripts for
gdm (or kdm, etc.) viable, you surely already have core and deps
installed...either it logs in or swaps drives and compare later.

[David Hageman]

I do agree with Dario and Jeffrey on this. A re-install would probably
be the optimal way to get the machine back up and running.

However, there is another way.

In theory, you should be able to find a file in /root called
anaconda-ks.cfg This is a file that you could pass to a kickstart to
rebuild the machine in the future. Towards the bottom it lists all the
RPMs that were put on the machine at install time. You can feed those
scripts to yum and rebuild the machine. If you see a name that starts
with a @, that is a group. You need to feed that to "yum groupinstall /
yum groupupdate". I recommend you use update as much as you can to
prevent duplicates.

Of course, all packages installed after the fact - won't be listed.

Jeffrey Watts wrote:
> Yeah, yum-complete-transaction can be dangerous if you've aborted an
> install at an inappropriate time. Basically it loses state and does the
> cleanup (remove) but doesn't finish the install.
>
> To be honest you're wasting your time. I'd boot off of a live USB
> stick, back up the user data, and do a fresh install. You've lost state
> on that machine and you won't know what's missing. Most likely little
> weird problems are going to pop up again and again as you discover more
> missing packages.
>
> A backup and reinstall will only take an hour or two.
>
> Jeffrey.
>
> On Thu, Sep 24, 2009 at 9:45 PM, Paul Johnson <paul...@ku.edu

--
========================================================
D. Hageman <dhag...@dracken.com>
Dracken Technology, Inc. http://www.dracken.com/
========================================================

[Stephen Figgins]

The start up process goes init->getty->login->shell. The init process uses getty (or some getty replacement like mingetty) to configure the tty and prompt for the user name.  Upon receiving the username it is supposed to kick off a login process that collects the password.

So I would check to make sure you have a login command and the rpm that it came with is all installed properly.  You might also check to see if you have an /etc/nologin file, which I believe will cause login to bail.  Delete it if it exists.

--
Stephen

[Jeffrey Watts]

The only concern with that is if he's upgraded the box in the interim (obsoletes) it won't represent the current system, though it's certainly a place to start from.

For normal users by the way, "upgrade" is what they ought to be doing as it will obsolete deprecated packages.  Not using upgrade can cause issues when something significant is deprecated, such as when they switch between subsystems (like the sound system).

Of course, if there are specific packages that are needed, even if they're deprecated, update is a good course but then again they could just manually exclude those.

Jeffrey.

On Fri, Sep 25, 2009 at 8:59 AM, David Hageman <dhag...@dracken.com> wrote:

I do agree with Dario and Jeffrey on this.  A re-install would probably
be the  optimal way to get the machine back up and running.

However, there is another way.

In theory, you should be able to find a file in /root called
anaconda-ks.cfg  This is a file that you could pass to a kickstart to
rebuild the machine in the future.  Towards the bottom it lists all the
RPMs that were put on the machine at install time.  You can feed those
scripts to yum and rebuild the machine.  If you see a name that starts
with a @, that is a group.  You need to feed that to "yum groupinstall /
yum groupupdate".  I recommend you use update as much as you can to
prevent duplicates.

Of course, all packages installed after the fact - won't be listed.

[David Hageman]

I don't see how obsoletes would be a concern. The yum process would
acquire the most recent information and install all the appropriate
packages based on the current dependency tree in the repository at that
time. Packages that are obsolete should be skipped and the replacement
should be installed.

The process is no different from doing a "yum update" with an already
installed machine.

Yum wouldn't be a very good tool if it didn't work that way.

I might have mis-interpreted your e-mail though.

--

[Jeffrey Watts]

The problem is that "update" will not process obsoletes.

In other words, if he has upgraded the box in the time between when he did his initial install (anaconda-ks.cfg) to the present, the list of RPMs in anaconda-ks.cfg might not be the defaults today.

He's using RHEL, so if for example, the sound system packages were obsoleted and a new subsystem was put in, if he used the original anaconda-ks.cfg from (let's say) RHEL5.1 and used it to do a yum update against the current RHEL5.4 repository, it might install obsoleted packages instead of the current ones.  Using "upgrade" solves that, and for regular users "upgrade" is what they ought to be doing.

"update" is more appropriate in a setting where you want to make sure that nothing changes other than individual package updates.  This is often in a server setting.

That all said, if his anaconda-ks.cfg is vanilla it'll most likely all be groups, and the packages will get handled.  I'm just saying that "upgrade" is the more appropriate choice for what he's doing (and the more appropriate choice for 90% of upgrades, especially in desktop settings).

Jeffrey.

On Fri, Sep 25, 2009 at 11:17 AM, David Hageman <dhag...@dracken.com> wrote:

I don't see how obsoletes would be a concern.  The yum process would
acquire the most recent information and install all the appropriate
packages based on the current dependency tree in the repository at that
time.  Packages that are obsolete should be skipped and the replacement
should be installed.

The process is no different from doing a "yum update" with an already
installed machine.

Yum wouldn't be a very good tool if it didn't work that way.

I might have mis-interpreted your e-mail though.

[Andrew Beals]

My thought (mailed privately) was that something in the authentication mechanism (PAM) was broken.  Suggested he check /etc/pam.d and look for the modules it references being there. 

But all y'all are right - back-up user data, install anew.  That's the right answer for just about any "It's broken" problem these days.  (eSATA drive frames recommended)

Andy

[Dario Landazuri]

> But all y'all are right - back-up user data, install anew. That's the
> right answer for just about any "It's broken" problem these days.
> (eSATA drive frames recommended)

I think you're overgeneralizing that statement - it's just an ongoing
evaluation of the trade-off between how much time spent trying to
debug/fix a problem vs. nuking it. Nuking it is not always the right
answer.

-Dario

--

************************************************************
Dario Landazuri Triangle Fraternity Minn97Ok
da...@landazuri.net
http://www.landazuri.net
************************************************************

"It don't mean a thing if it ain't got that certain
je ne sais quoi."
-Peter Schickele

[David Hageman]

I think I understand what you are trying to convey, but I think you
might be over analyzing the issue.

a) obsolete processing is the default. So upgrade and update work the same.

b) If during an update, the person kills off the process and the update
wasn't completed - the old packages were removed and the new packages
weren't installed - you essentially have a bare bones system.

Using the original anaconda-ks.cfg shouldn't be an issue as *most* of
the packages listed are in groups. The group names really don't change,
but what is included in them might. So, you are pointed at the new
repository because this all started because you were doing an
update/upgrade. When you do an update/upgrade on a bare bones systems
... you shouldn't really have to worry about obsoletes.

I guess if you are overly concerned and want to be pedantic, use upgrade
or "updates --obsoletes" to ensure you get the results you want.

At any rate, as I said before - I would just do a clean install.

[Andrew Beals]

On Fri, Sep 25, 2009 at 11:57 AM, Dario Landazuri <da...@landazuri.net> wrote:

But all y'all are right - back-up user data, install anew.  That's the right answer for just about any "It's broken" problem these days.  (eSATA drive frames recommended)

I think you're overgeneralizing that statement - it's just an ongoing evaluation of the trade-off between how much time spent trying to debug/fix a problem vs. nuking it.  Nuking it is not always the right answer.

Two geeks, three opinions.




By this point, he's either gotten it back up, or he blew his deadline.
 

[Jeffrey Watts]

Dave, read the man page.  'yum update' does not process obsoletes.  The only difference between 'yum update' and 'yum upgrade' is that the latter processes obsoletes.  It is equivalent to 'yum --obsoletes update'.  I personally think that "update" ought to be deprecated as it causes tons of confusion.  They would just need to add a --no-obsoletes option.

As far as the groups go, yes I pointed that out earlier.  I'm not being pedantic, I just pointed out that 'yum upgrade' is the better way to go and it's possible if he selected individual packages for install that 'yum update' may not get him the results he wanted.  You suggested he go by the anaconda-ks.cfg, and thus obsoletes can be a concern.

To be honest I find the suggestion that I'm being pedantic to be a bit insulting, as using "upgrade" is the proper method and the differences are a very important distinction when working with RHEL.  On Fedora it's usually not an issue as people rarely use yum to move between Fedora releases (they use the installer), and I don't believe Fedora has ever obsoleted a package mid-release.  However, on RHEL obsoletes processing is something that average users expect to occur in a minor release upgrade.

Regardless, we both agree that a reinstall is the best approach.

Jeffrey.

On Fri, Sep 25, 2009 at 12:42 PM, David Hageman <dhag...@dracken.com> wrote:

I think I understand what you are trying to convey, but I think you
might be over analyzing the issue.

a) obsolete processing is the default.  So upgrade and update work the same.

b) If during an update, the person kills off the process and the update
wasn't completed - the old packages were removed and the new packages
weren't installed - you essentially have a bare bones system.

Using the original anaconda-ks.cfg shouldn't be an issue as *most* of
the packages listed are in groups.  The group names really don't change,
but what is included in them might.  So, you are pointed at the new
repository because this all started because you were doing an
update/upgrade.  When you do an update/upgrade on a bare bones systems
... you shouldn't really have to worry about obsoletes.

I guess if you are overly concerned and want to be pedantic, use upgrade
or "updates --obsoletes" to ensure you get the results you want.

At any rate, as I said before - I would just do a clean install.

[David Hageman]

Quoth the man page in the *update* section taken from a RHEL box:

"If the main obsoletes configure option is true (default) or the
--obsoletes flag is present yum will include package obsoletes
in its calculations - this makes it better for distro-version
changes, for example: upgrading from somelinux 8.0 to somelinux
9."

As I said in my last message, the *default* is that obsoletes option is
true. Thus, upgrade and update is essentially the same out of the box.
Yes, you can change the behavior to match what you describe, but that
is not the *default*.

You can verify this by looking at your /etc/yum.conf and looking for:

obsoletes=1

Sorry man, I do know how run a Linux box. ;-)

[Jeffrey Watts]

Hey Dave, what OS did you do that on?  Here's what I see on RHEL5.4:

[watts@aotus ~]$ rpm -q yum
yum-3.2.22-20.el5
[watts@aotus ~]$ man yum
[... snip ...]
              If the --obsoletes flag is  present  yum  will  include  package
              obsoletes  in  its  calculations - this makes it better for dis-

              tro-version changes, for example: upgrading from  somelinux  8.0
              to somelinux 9.

[... snip ...]

According to the documentation for yum on RHEL5.4, obsoletes is not the default.  Perhaps the man page is incorrect on RHEL, or perhaps it's a change introduced in Fedora.

Jeffrey.

On Fri, Sep 25, 2009 at 1:56 PM, David Hageman <dhag...@dracken.com> wrote:

Quoth the man page in the *update* section taken from a RHEL box:

"If the main obsoletes configure option is true (default) or the
--obsoletes flag is present yum will include  package  obsoletes
in  its  calculations  - this makes it better for distro-version
changes, for example: upgrading from somelinux 8.0 to  somelinux
9."

As I said in my last message, the *default* is that obsoletes option is
true.  Thus, upgrade and update is essentially the same out of the box.
  Yes, you can change the behavior to match what you describe, but that
is not the *default*.

You can verify this by looking at your /etc/yum.conf and looking for:

obsoletes=1

Sorry man, I do know how run a Linux box. ;-)

[Justin Dugger]

On Fri, Sep 25, 2009 at 2:26 PM, Jeffrey Watts
<jeffrey...@gmail.com> wrote:
> Hey Dave, what OS did you do that on?  Here's what I see on RHEL5.4:
>
> [watts@aotus ~]$ rpm -q yum
> yum-3.2.22-20.el5
> [watts@aotus ~]$ man yum
> [... snip ...]
>               If the --obsoletes flag is  present  yum  will  include
> package
>               obsoletes  in  its  calculations - this makes it better for
> dis-
>               tro-version changes, for example: upgrading from  somelinux
> 8.0
>               to somelinux 9.
> [... snip ...]
>
> According to the documentation for yum on RHEL5.4, obsoletes is not the
> default.  Perhaps the man page is incorrect on RHEL, or perhaps it's a
> change introduced in Fedora.

Never trust the documentation:

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.4 (Tikanga)
# grep obsoletes /etc/yum.conf
obsoletes=1

Justin

[David Hageman]

[root@mail ~]# cat /etc/redhat-release

Red Hat Enterprise Linux Server release 5.4 (Tikanga)

[root@mail ~]# rpm -q yum
yum-3.2.22-20.el5

It doesn't paste well, but here is other relevant information about yum:

Build Date: Tue 07 Jul 2009 02:15:42 PM CDT
Build Host: ls20-bc2-14.build.redhat.com
Signature : DSA/SHA1, Wed 29 Jul 2009 06:37:48 AM CDT, Key ID
5326810137017186

And I did a verify on the RPM to ensure it hadn't been modified and it
came back clean.

Sorry man, I can't tell you why your system is different from mine.

FYI - the Fedora page reads the same as the RedHat.

Just out of curiosity, even though your man page doesn't say it - what
does your yum.conf say?

[Jeffrey Watts]

Yeah it looks like they changed the default on me sometime in the past.  Looks like obsoletes is now enabled by default, not sure when that snuck in.

What's weird is that my yum man page shows as unmodified in rpm -Vvv.  My yum version, build time, built host, etc are the same.

Jeffrey.

On Fri, Sep 25, 2009 at 2:38 PM, David Hageman <dhag...@dracken.com> wrote:

[root@mail ~]# cat /etc/redhat-release

Red Hat Enterprise Linux Server release 5.4 (Tikanga)

[root@mail ~]# rpm -q yum

yum-3.2.22-20.el5

It doesn't paste well, but here is other relevant information about yum:

Build Date: Tue 07 Jul 2009 02:15:42 PM CDT
Build Host: ls20-bc2-14.build.redhat.com
Signature   : DSA/SHA1, Wed 29 Jul 2009 06:37:48 AM CDT, Key ID
5326810137017186

And I did a verify on the RPM to ensure it hadn't been modified and it
came back clean.

Sorry man, I can't tell you why your system is different from mine.

FYI - the Fedora page reads the same as the RedHat.

Just out of curiosity, even though your man page doesn't say it - what
does your yum.conf say?

[David Nicol]

> Regardless, we both agree that a reinstall is the best approach.

"Rebuild, don't repair."

[Steve Nordquist]

Can't the reality distortion fields just get along?
You're trying to communicate the notion of a staged cockfight between
Swisher Sweets and Clove aficionados, and doing a ...rendering
accurately, let's say... so let us suppose that for someone who
sometimes takes the care to delete all the default configuration files
and type them in to suit, and --edit manpages to match-- the cases
might --both-- be as described.

Bloody courtesy.

[Paul Johnson]

Jeffrey Watts wrote:

> Regardless, we both agree that a reinstall is the best approach.
>

I agree. If I had RedHat 5.4, I would install a clean system.

But the KU guy that is supposed to download the updates is slow, and
I don't want to install 5.3 again only to have to wrestle with
updates to 5.4.

In the meanwhile, I've used my super cow powers to make the system
work again.

In case you are ever stubborn like me, here is a neat trick you can
do if you boot from the rescue disk.

The Rescue mounts the existing installed system on

/mnt/sysimage

and it invites you to chroot to access it. But if you have no vitals
like "ls" or "mount" or "MAKEDEV", that's a bad idea.

However, while running from the CD, you can do commands like

rpm -Uvh fileutils.XXX,XXX.rpm --relocate=/=/mnt/sysimage

and the rpm system will install all of the files onto /mnt/sysimage
as if it were /.

This works even for packages that are not relocatable!

I expect I will make a fresh install of 5.4 once I get the disk, and
I'll make a system image for all our systems from it. Until then,
that guy's system is running in boy scout mode.

[jamesmi...@googlemail.com]

On Fri, Sep 25, 2009 at 4:45 AM, Paul Johnson <paul...@ku.edu> wrote:
>
> I worked it back to the place where the system will start.  The
> kernel is found, all the services start, and it comes to a login
> prompt.
>
> But no users can log in.  It lets you type in a user name, but it
> does not ask for a password.  It just pops back saying username:
> again.  Some rpm package that controls the login process must not
> have gotten installed.  Or else a post-install script did not complete.
>

I have had a problem like that, you need to boot from a live cd and
restore or recreate the /etc/shadow file.

mike