[gentoo-user] Package management, depclean and new installs

[coa...@tuta.io]

Hi y'all new confused user regarding package management

How do you guys manage and protect your packages?

Do you just put everything on world and end up with a huge world file?

Do you have basic system files on world and the rest you protect or omit?

Do you create new(personalised) files depending on category and somehow link them in any of the above 3?

It's just emerging everything you are not sure you will keep with -1 seems cumbersome to me especially if at some point I want to transfer to a new device and want to copy my settings over ,select what to keep and discard the rest,depcleaning with pretend all the time seems annoying on the long term as well so there should be lots of different solutions from different people(at least thats what I think)

I just think this is one of the things its better I learn now rather than later and forum or wiki info is too "on-point" on a specific situation so I thought I'd ask the userbase

[cal]

I rarely use -1; pretty much exclusively for the case where I just need
to run something once right now and will not need it again. Otherwise,
I just install it normally, which will add it to my world file, and it
will be preserved during depcleans. If I later decide I no longer need
this package, I can always remove it later.

My world file has 161 packages; I'm not sure what you consider "huge",
but glancing through the list I don't see a lot of bloat -- I know what
pretty much each of them does/provides to my system.

As far as "system" packages, I just rely on my profile's @system to keep
those. Things in @world are applications/resources that I installed for
my particular usage.

I'm not sure from your question exactly why you're using -1 frequently
enough to find it cumbersome; if you clarified what you're doing it
might be easier to clarify whether that is "normal" usage or there is a
better way of doing things.

cal

[Dale]

This is the way I've done it for a while, a couple decades.  Once I get
my new install done, everything installed that I want, I change the
default for the emerge command and add --oneshot or -1.  That way if I'm
trying to work through a upgrade problem, I don't have to remember to
use -1 to keep world clean.  My default emerge options look like this:

EMERGE_DEFAULT_OPTS="--with-bdeps y --backtrack=100 --keep-going -v -j5
--quiet-build=n -1 --unordered-display"

Obviously, depending on CPU, available memory etc, you may want to
adjust those to your case.  The key points here, the -1, --with-bdeps y,
--backtrack=100 and --keep-going options.  99% of the time, when I do my
updates with the command emerge -auDN world, that gives me a easy
upgrade.  There may be exceptions at times but generally that works and
gives me a stable system. 

Keep in mind, you don't have to worry about @system much if any.  It's
handled by the devs.  As for world, you only keep in there the packages
you use.  When you update or install something new, emerge takes care of
whatever depends on the package you want.  As a example, if you want a
full KDE install, you just emerge the kde meta package and it gets
recorded in the world file.  The emerge command will take care of all
the other packages that depend on the meta package.  That is a LOT of
packages too.  My biggest advice, if you find yourself fighting emerge
to get things done, you're doing something wrong.  These days, emerge
-av <package> should work virtually 100% of the time.  You may have to
adjust USE flags or something but it should just work.   

Over the years, I've adjusted options until I got a easy update path. 
This works really, really well.  I update once a week, usually Sunday
night.  Lately, I start late Saturday night or Sunday morning.  That way
I'm done and can update my backups Sunday night. 

Hope that helps.  Welcome to Gentoo and the source of good heat, lots of
compiling.  lol

Dale

:-)  :-) 

[Arve Barsnes]

On Mon, 4 Oct 2021 at 05:30, <coa...@tuta.io> wrote:
>
> Hi y'all new confused user regarding package management
>
> How do you guys manage and protect your packages?
> Do you just put everything on world and end up with a huge world file?
> Do you have basic system files on world and the rest you protect or omit?
> Do you create new(personalised) files depending on category and somehow link them in any of the above 3?

I'm not sure I understand what you are asking about when you say
'protect' here? As cal said, putting the actual programs you use in
world is the way to go, and portage will take care of dependencies for
you. Putting more than needed in world can sometimes confuse portage
to the point where you could have problems that can be annoying to
troubleshoot.

> It's just emerging everything you are not sure you will keep with -1 seems cumbersome to me especially if at some point I want to transfer to a new device and want to copy my settings over ,select what to keep and discard the rest,depcleaning with pretend all the time seems annoying on the long term as well so there should be lots of different solutions from different people(at least thats what I think)

As you can see from the two earlier replies, from cal and Dale, this
depends on how you want to manage your system. If you do a lot of
installing, putting -1 in your defaults might be the way to go, but
personally I'm in cal's camp, I would much rather remember the -1 when
I'm doing something ad hoc, and have any programs I want added to
world without adding another flag to my emerge command.

As for depclean, I have added an alias for depclean with --pretend
that I use after world updates. Unless you have installed something
with -1, most of the time that list is empty. And the output gives
nice clean lists of what packages it wants to remove, with exact
version strings and slots, so if I agree, I just do 'emerge -C <copied
list>' and I'm good to go.

Cheers,
Arve

[William Kenworthy]

Apps I am trying out or wont keep for long, install with -1

Apps I want, install without -1

Updates - always use -1 and after a successful update is complete run
emerge --depclean -p and make sure whats being removed is really correct
then remove the -p.

Occasionally go through "/var/lib/portage/world" and turf things you no
longer need or don't know the use of, then run emerge --depclean -p and
make sure whats being removed is really correct then remove the -p.

BillK

[coa...@tuta.io]

I thank the four of you for the insight I learn more in 5 mins then I did in an afternoon,I have two last question tho

As a example, if you want a

full KDE install, you just emerge the kde meta package and it gets

recorded in the world file.  The emerge command will take care of all

the other packages that depend on the meta package.  That is a LOT of

packages too

Theoretically I can make my own meta package and place in the localrepo I have and set it to pull packages from the official repos

Firstly is there any dependency hell that I can fall into when placing lots of different packages with (unexpectedly) conflicting deps on my own meta package?Has anyone (reading this) that has done it before and worked out a niche way to avoid falling into that trap?

Secondly(I know I will surely find this one in the wiki but)can I set a priority to pull from the local repo first if package exists and then have the official repo as a backup?

Lastly thank you for your previous replies forgot to add it on my last mail and I didnt want to bloat the mailing list like im editing a forum post with asterisks :D 

.

.

.

Thank you (ah I'm learning)

[Miles Malone]

I would strongly, STRONGLY discourage you from creating your own meta
package. There are very few meta packages in the tree (in the scheme
of things) for very good reasons, they take one hell of a lot of
maintenance. They're really only there for things like kde, where you
might just want a bare bones kde environment, or you might be
expecting the full-fat desktop environment with all the side packages
you'd get if you were using a distro that gave you no option out of
the box.

If you really want to group a bunch of packages into a set that gets
emerged with one command, I would do exactly that: create a custom
set. Similar to @world, @system, @security, etc. You can do that
quite easily, see https://wiki.gentoo.org/wiki/Package_sets .

But really there's not a lot of use cases for it, mostly if you use a
package and it's not just a dep of something (or several things) you
should just have it in your world file, *for most people's use cases*.
Going through your world file and cleaning out cruft is a part of
regular gentoo maintenance, should be done at a minimum annually imo.
Much like cleaning out distfiles and whatnot (see eclean, from
app-portage/gentoolkit. And, indeed, pretty much every other useful
utility in gentoolkit. Also flaggie for use-flag management.)

[Arve Barsnes]

On Mon, 4 Oct 2021 at 08:05, <coa...@tuta.io> wrote:
> Firstly is there any dependency hell that I can fall into when placing lots of different packages with (unexpectedly) conflicting deps on my own meta package?Has anyone (reading this) that has done it before and worked out a niche way to avoid falling into that trap?

Probably depends on what you intend this 'meta' package to do.
Something like the KDE meta package is rarely useful outside of DE's
in my estimate, and exist purely to create a KDE 'package' that users
can easily install without much consideration.

If you want to create your own groups of packages that you want to
install with a single command, I would look into sets. @system and
@world are sets that everyone uses, but it's easy to create your own
for whatever purpose.

Portage is usually pretty good at helping you figure out any
dependency conflicts, so I wouldn't worry about it. Might be worth
looking deeper into the way portage prints dependency errors if you
encounter problems though. As evidenced by many a thread on this list,
it can sometimes be very hard to understand, simply because there can
be a lot of it when there are conflicts, and it's easy to get
side-tracked by information that isn't directly related to your
problem.

> Secondly(I know I will surely find this one in the wiki but)can I set a priority to pull from the local repo first if package exists and then have the official repo as a backup?

You configure your repos in /etc/portage/repos.conf. For each repo you
have the option of setting a priority. I think "official third-party"
repos installed through layman gets a priority = 50, and if I'm not
mistaken, the official repo have a default of 100. If you want your
own repo to be the first choice, give your repo a higher priority.

Cheers,
Arve

[Miles Malone]

There's one thing that springs instantly to mind that uses a complex
meta package that isnt a desktop environment is texlive. And jesus do
the texlive team (all... two of them?) work hard. Special shout out.

[coa...@tuta.io]

Lol i just installed that earlier, didnt know gentoo is THAT understaffed,looking at the history i know of i still dont understand if the wiki dying was a good or a bad thing for the community in one hand hardcore fans stayed and rewrote(of what i can see) some epic documentation in comparison to other distors,on the other hand. . . lonelyness?Eh to its their own i guess :D 

Cheers,

good night

4 Oct 2021, 07:51 by m.ma...@homicidalteddybear.net:

[coa...@tuta.io]

That is what i waa looking day-1 when i started gentoo

If it works as I expect it to id say problems solved case closed

And if i can add other sets inside world even better,with that being said my questions from my last email still stand,even tho now unneeded someone reading this mailing list in the future might need it so if you know go ahead

For me that is it, not only im satisfied but happy with the gentoo community as a new user, I'll stop mailing now so i dont annoy other users and goodnight by me{wow its day now :(  }

(Also while typing this Avre replied as well, so ill say everything here, I remember seeing priority somewhere sometime in the past few weeks but Id forgotten it, should relook it up by evening?¿ appreciated cheers )

4 Oct 2021, 07:36 by arve.b...@gmail.com:

On Mon, 4 Oct 2021 at 08:05, <coa...@tuta.io> wrote

Secondly(I know I will surely find this one in the wiki but)can I set a priority to pull from the local repo first if package exists and then have the official repo as a backup?

You configure your repos in /etc/portage/repos.conf. For each repo you

have the option of setting a priority. I think "official third-party"

repos installed through layman gets a priority = 50, and if I'm not

mistaken, the official repo have a default of 100. If you want your

own repo to be the first choice, give your repo a higher priority.

Cheers,

Arve

4 Oct 2021, 07:13 by m.ma...@homicidalteddybear.net:

[Neil Bothwick]

On Mon, 4 Oct 2021 06:31:53 +0200, Arve Barsnes wrote:

> As you can see from the two earlier replies, from cal and Dale, this
> depends on how you want to manage your system. If you do a lot of
> installing, putting -1 in your defaults might be the way to go, but
> personally I'm in cal's camp, I would much rather remember the -1 when
> I'm doing something ad hoc, and have any programs I want added to
> world without adding another flag to my emerge command.

I pretty much do the same. I sometimes need to install a package for a
short while, rather than a quick test, so I have a @temp set I can add
these to and clean it out regularly.

>
> As for depclean, I have added an alias for depclean with --pretend
> that I use after world updates. Unless you have installed something
> with -1, most of the time that list is empty. And the output gives
> nice clean lists of what packages it wants to remove, with exact
> version strings and slots, so if I agree, I just do 'emerge -C <copied
> list>' and I'm good to go.

Why not use --ask instead of --pretend? Then you don't have to copy
anything or re-run commands. "emerge -ca" is short enough to not need an
alias.


--
Neil Bothwick

Beware of the opinion of someone without any facts.

[Arve Barsnes]

Mostly because, while I am not a prolific installer of temporary
packages, I fairly often (compared to how often depclean wants to
remove anything at all) have one or two packages in the list that I
want to delay removing for a little while. Most common after a kernel
upgrade where I still have not gotten around to reboot and confirm
that the new version actually works. Copying the package list now and
then seems less of a hassle to me than having to say no to --ask most
of the time.

Regards,
Arve

[Peter Humphrey]

On Monday, 4 October 2021 07:57:10 BST coa...@tuta.io wrote:
> Lol i just installed that earlier, didnt know gentoo is THAT
> understaffed,looking at the history i know of i still dont understand if
> the wiki dying was a good or a bad thing for the community in one hand
> hardcore fans stayed and rewrote(of what i can see) some epic documentation
> in comparison to other distors,on the other hand. . .

Eh? What's this about the wiki dying?

--
Regards,
Peter.

[Wols Lists]

On 04/10/2021 07:13, Miles Malone wrote:
> I would strongly, STRONGLY discourage you from creating your own meta
> package. There are very few meta packages in the tree (in the scheme
> of things) for very good reasons, they take one hell of a lot of
> maintenance. They're really only there for things like kde, where you
> might just want a bare bones kde environment, or you might be
> expecting the full-fat desktop environment with all the side packages
> you'd get if you were using a distro that gave you no option out of
> the box.
>
> If you really want to group a bunch of packages into a set that gets
> emerged with one command, I would do exactly that: create a custom
> set. Similar to @world, @system, @security, etc. You can do that
> quite easily, see https://wiki.gentoo.org/wiki/Package_sets .
>
> But really there's not a lot of use cases for it, mostly if you use a
> package and it's not just a dep of something (or several things) you
> should just have it in your world file, *for most people's use cases*.
> Going through your world file and cleaning out cruft is a part of
> regular gentoo maintenance, should be done at a minimum annually imo.
> Much like cleaning out distfiles and whatnot (see eclean, from
> app-portage/gentoolkit. And, indeed, pretty much every other useful
> utility in gentoolkit. Also flaggie for use-flag management.)
>

Yup. Emerge the packages you want, let portage take care of the rest.
And I've recently joined the group who let emerge default to --oneshot,
and use --select if I want to keep it ... :-)

And do learn how to use package.use. Don't clutter your make.conf with
loads of flags you don't understand (or want), just to get packages to
emerge. I use autounmask-write if I need to, then I DON'T let etc-update
update some huge package.use file. I just rename the new bunch of flags
into package.use/what-ive-just-emerged.date. That way, if I know I still
want that program, I let the file accumulate cruft :-), and if I'm
wondering what the hell it is or it looks outdated as heck, I just
carefully delete them one by one, and see what happens.

One last tip for a newbie - have you already got X, harfbuzz, and
freetype installed? If not, squirrel this info away until you need it -
harfbuzz often won't install because freetype isn't there. But freetype
won't install because harfbuzz isn't there! Find out what's pulling them
both in, do an emerge -C on that, then install either harfbuzz or
freetype with USE="-theother". Then an emerge --update should clean up
the mess :-)

Cheers,
Wol

[Neil Bothwick]

On Mon, 4 Oct 2021 10:33:58 +0200, Arve Barsnes wrote:

> > Why not use --ask instead of --pretend? Then you don't have to copy
> > anything or re-run commands. "emerge -ca" is short enough to not need
> > an alias.
>
> Mostly because, while I am not a prolific installer of temporary
> packages, I fairly often (compared to how often depclean wants to
> remove anything at all) have one or two packages in the list that I
> want to delay removing for a little while. Most common after a kernel
> upgrade where I still have not gotten around to reboot and confirm
> that the new version actually works. Copying the package list now and
> then seems less of a hassle to me than having to say no to --ask most
> of the time.

I picked up this tip some years ago to avoid depcleaning kernel sources.

% cat /etc/portage/sets.conf
[kernels]
class = portage.sets.dbapi.OwnerSet
world-candidate = False
files = /usr/src

and emerge -n @kernels

I also have this in the file to allow multiple GCC versions.

[gcc]
class = portage.sets.dbapi.OwnerSet
world-candidate = False
files = /usr/x86_64-pc-linux-gnu/gcc-bin


--
Neil Bothwick

With free advice you often get what you pay for.

[Arve Barsnes]

On Mon, 4 Oct 2021 at 20:09, Neil Bothwick <ne...@digimed.co.uk> wrote:
> On Mon, 4 Oct 2021 10:33:58 +0200, Arve Barsnes wrote:
> I picked up this tip some years ago to avoid depcleaning kernel sources.
>
> % cat /etc/portage/sets.conf
> [kernels]
> class = portage.sets.dbapi.OwnerSet
> world-candidate = False
> files = /usr/src
>
> and emerge -n @kernels

Interesting way to do it. I want them depcleaned though, as soon as I
confirm that the new kernel works as expected. I use a custom
patch-set, so I never have any updates unless I add them myself, so
I'm not sure if using a set would give me anything.

Cheers,
Arve

[Neil Bothwick]

On Mon, 4 Oct 2021 21:31:35 +0200, Arve Barsnes wrote:

> > On Mon, 4 Oct 2021 10:33:58 +0200, Arve Barsnes wrote:
> > I picked up this tip some years ago to avoid depcleaning kernel
> > sources.
> >
> > % cat /etc/portage/sets.conf
> > [kernels]
> > class = portage.sets.dbapi.OwnerSet
> > world-candidate = False
> > files = /usr/src
> >
> > and emerge -n @kernels
>
> Interesting way to do it. I want them depcleaned though, as soon as I
> confirm that the new kernel works as expected. I use a custom
> patch-set, so I never have any updates unless I add them myself, so
> I'm not sure if using a set would give me anything.

Fair enough. I use standard gentoo-sources but always like to keep at
least one previous version available "just in case". I unmerge old
versions manually, I have to delete kernel sources manually anyway as
emerge only removes the files it installed, leaving all the files created
when compiling the kernel.


--
Neil Bothwick

Power corrupts - absolute power is even more fun.

[Steve Evans]

I use an alias for depclean to achieve the same result:

alias depclean='emerge -va --depclean --exclude gentoo-sources --exclude gcc --exclude mythweb --exclude php --exclude owncloud'

Steve
--
____________________________________________________________________
Steve Evans E-mail: mailto:ste...@gorbag.com
Registered Linux user #217906: http://counter.li.org
Public Encryption Key: http://www.gorbag.com/public-key.html
____________________________________________________________________

5.10.61-gentoo Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz GNU/Linux

22:46:01 up 14 days, 12:25, 5 users, load average: 1.25, 0.68, 0.65

"I remember when I was a kid I used to come home from Sunday School and
my mother would get drunk and try to make pancakes."
-- George Carlin

[Dale]

I "borrowed" your method a good while back and do it the same way.  Even
if I use emerge to remove it, it always leaves cruft behind so I just do
it manually then tell emerge to remove it from its info. 

I sometimes end up with 3 to 4 kernels.  Before the init thingys came
along, I didn't worry about removing kernels.  I just left them since
space wasn't a issue.  With the init thingys tho, it doubles or so the
space needed for each kernel.  So, when I get a couple new stable ones,
I delete old ones to keep a little breathing room. 

Your way is nifty.  No more editing package files every time I want to
upgrade or something. 

Dale

:-)  :-) 

[Neil Bothwick]

On Mon, 4 Oct 2021 18:02:55 -0500, Dale wrote:

> > Fair enough. I use standard gentoo-sources but always like to keep at
> > least one previous version available "just in case". I unmerge old
> > versions manually, I have to delete kernel sources manually anyway as
> > emerge only removes the files it installed, leaving all the files
> > created when compiling the kernel.
> >
> >
>
> I "borrowed" your method a good while back and do it the same way.  Even
> if I use emerge to remove it, it always leaves cruft behind so I just do
> it manually then tell emerge to remove it from its info. 

When I checked it in the past

rm -fr /usr/src/linux-version
emerge -C gentoo-sources-version

was significantly faster than

emerge -C gentoo-sources-version
rm -fr /usr/src/linux-version


--
Neil Bothwick

If at first you don't succeed, redefine success.

[Dale]

That's what I found out too.  That's why I delete manually first then
tell emerge to.  Plus, you have to delete manually anyway so may as well
save time.  ;-)

Dale

:-)  :-) 

[Peter Humphrey]

On Tuesday, 5 October 2021 00:11:42 BST Neil Bothwick wrote:

> When I checked it in the past
>
> rm -fr /usr/src/linux-version
> emerge -C gentoo-sources-version
>
> was significantly faster than
>
> emerge -C gentoo-sources-version
> rm -fr /usr/src/linux-version

Yes, it would be. The first way, portage just scans the list of files and tries
to delete what isn't there; the second way it actually has to delete real
files.

--
Regards,
Peter.

[coa...@tuta.io]

So what about other packages? Do you guys find it faster to

rm -rf $(equary f PACKAGE) && emerge -C PACKAGE? Is that slower cuz equery is also called or could that cause unwanted files from other packages to be wiped?maybe filters?

[Wols Lists]

I doubt that's the problem. After all, rm has to delete the files too.

I guess portage scans the files and looks up whether it can delete them
or not. If you've deleted them already, that step no longer happens ...

Cheers,
Wol

[Neil Bothwick]

On Tue, 5 Oct 2021 04:17:32 +0200 (CEST), coa...@tuta.io wrote:

> > was significantly faster than
> >
> > emerge -C gentoo-sources-version
> > rm -fr /usr/src/linux-version
> >
> So what about other packages? Do you guys find it faster to
> rm -rf $(equary f PACKAGE) && emerge -C PACKAGE? Is that slower cuz
> equery is also called or could that cause unwanted files from other
> packages to be wiped?maybe filters?

Very few, if any, other packages install all their files in one
directory. Nor do they install the sheer number of files

% qlist \=sys-kernel/gentoo-sources-5.10.61 | wc -l
70655

But the real benefit is that you have to run rm -fr anyway to remove the
object files etc. you are running the same commands either way, it is
just the order that makes a difference.

For any other package, I just let portage do its job.


--
Neil Bothwick

Electrocution, n.:
Burning at the stake with all the modern improvements.

[Neil Bothwick]

On Tue, 5 Oct 2021 05:43:27 +0100, Wols Lists wrote:

> > Yes, it would be. The first way, portage just scans the list of files
> > and tries to delete what isn't there; the second way it actually has
> > to delete real files.
> >
> I doubt that's the problem. After all, rm has to delete the files too.
>
> I guess portage scans the files and looks up whether it can delete them
> or not. If you've deleted them already, that step no longer happens ...

That's what I thought, it has to check whether each file has been
modified before deleting it, then it deletes that one file. rm just
chucks the whole lot in the bin.


--
Neil Bothwick

Half of being smart is knowing what you're dumb at.