EASY way to install packages from trixie/sid to stable?

[Hans]

Hi folks,

is there a very easy way, if I want to install packages from trixie oder sid
into my bookworm installation?

I read about apt pinning, but as far as I understood, I have to name
explicitily each package I want to install from sid. This can be much work,
when installing a high number of packages.

I suppose, I then have also to install all dependencies of the packaes from
sid, even if they are related to the system.

In my case I wanted to install virtualbox from sid, as it has all packages
ready. However, virtualbox requires and depends also the newer gcc compiler
and some compiler libs, thus I took distance from installing for now.

At the moment I am not using pinning. My actual way of doing is

1. adding the sid repo into /etc/apt/sources.list

2. then aptitude -u

3. then searching for the required package and mark it as install (or
upgradeble

4. Then install, if wanted.

Yes, I know, pinning would be the better way, but it is very, very seldom, I
need to install something from a higher repo.

And yes, I know, mixing repos is no good idea, so I am using this only for
applications, which are using theire own libraries (or libs, they are only for
this special application).

Do you know another way, except pinning or my (weired) way?

Oh, last but not least, I know, Oracle has its own debian-repo for virtualbox,
but it looks somehow not well set up IMHO.

Thanks for reading this.

Best regards

Hans

[Alexander V. Makartsev]

On 26.10.2023 18:39, Hans wrote:

Hi folks, 

is there a very easy way, if I want to install packages from trixie oder sid 
into my bookworm installation?

I read about apt pinning, but as far as I understood, I have to name 
explicitily each package I want to install from sid. This can be much work, 
when installing a high number of packages.

I suppose, I then have also to install all dependencies of the packaes from 
sid, even if they are related to the system.

In my case I wanted to install virtualbox from sid, as it has all packages 
ready. However, virtualbox requires and depends also the newer gcc compiler 
and some compiler libs, thus I took distance from installing for now.

At the moment I am not using pinning. My actual way of doing is 

1. adding the sid repo into /etc/apt/sources.list

2. then aptitude -u

3. then searching for the required package and mark it as install (or 
upgradeble

4. Then install, if wanted.

Yes, I know, pinning would be the better way, but it is very, very seldom, I 
need to install something from a higher repo.

And yes, I know, mixing repos is no good idea, so I am using this only for 
applications, which are using theire own libraries (or libs, they are only for 
this special application).

Do you know another way, except pinning or my (weired) way?

I've always did it "the right way", by making simple backports [1] of required package and its selected dependencies if newer versions are required.
For some exotic packages this approach is not feasible, because you might end up with dozens of packages to backport as dependencies and dependencies of their dependencies.
"php", "nodejs" and "golang" to name a few tend to snowball a lot.

In your situation, it could be better to use officially distributed package [2] from Oracle for now.
Foreign packages like these could be installed into separate location like into "/opt/". This way they won't interfere with the rest of your system.
I don't use virtualbox (KVM does everything and more for me) so I can't vouch for the quality of packages from Oracle.

[1] https://wiki.debian.org/SimpleBackportCreation
[2] https://www.virtualbox.org/wiki/Linux_Downloads

--
With kindest regards, Alexander.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀⠀⠀⠀

[to...@tuxteam.de]

On Thu, Oct 26, 2023 at 03:39:23PM +0200, Hans wrote:
> Hi folks,
>
> is there a very easy way, if I want to install packages from trixie oder sid
> into my bookworm installation?

That will depend very much on the package.

1. If you try a naive install, the package will pull in its
dependencies. If those are very fundamental (think libc,
for example), that will wreak havoc in your system.

Most people will tell you not to do that (you will create
a FrankenDebian [1], the horrors!). I have actually been
doing that for a while. If you try:

- be prepared to learn a lot
- always have a plan B for the case an upgrade renders
your system useless
- ask here, but please, don't complain :)

2. A significantly less traumatic option would be to see whether
the package version you are after is in backports [2]. You say
you are on bookworm, so this [2a] would be relevant to you.

3. In case (2) fails, you can still try to build your package
from source. Debian makes that as easy as possible for you:
in a nutshell: download your source package, install the
building machinery (build-essential, possibly dpkg-dev, the
package's build dependencies, change into the package's
"debian" subdir, and issue there the magic incantation
"dpkg-buildpackage -uc -us" or thereabouts [3].

Of course, that might fail, because the build dependencies
can't be satisfied by your bookworm or something.

Of course, I left out a lot :)

Cheers

[1] https://wiki.debian.org/DontBreakDebian
[2] https://backports.debian.org/
[2a] https://backports.debian.org/bookworm-backports/overview/
[3] https://www.debian.org/doc/manuals/maint-guide/build.en.html

--
t

[Michael Kjörling]

On 26 Oct 2023 21:37 +0500, from avb...@gmail.com (Alexander V. Makartsev):

> I don't use virtualbox (KVM does everything and more for me) so I can't
> vouch for the quality of packages from Oracle.

I switched from VirtualBox to KVM at one point; as I recall a Debian
kernel upgrade broke VirtualBox and still after two weeks or so Oracle
hadn't updated their for-Debian repository with a version that
incorporated a fix. (This was the respective "stable" versions at the
time, and while I don't recall the details, the breakage made
VirtualBox useless for my use case.) I had planned to do such a
migration anyway; the breakage just somewhat forced the issue.

KVM/QEMU/virt-manager and friends perhaps aren't as streamlined for
the typical end user who just wants to quickly spin up a VM with
minimal hassle, but they are also very much more powerful if you're
willing to do a little reading. (For example, I had to do a fair bit
of digging to figure out how to get guest networking to work reliably
without turning off the host firewall.[1]) This is in line with their
respective intended usage: VirtualBox is at best a power user tool,
whereas KVM is intended for large server deployments but _can_ be used
on workstation virtualization hosts as well.

[1] https://michael.kjorling.se/blog/2022/linux-kvm-host-nftables-guest-networking/

--
Michael Kjörling 🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[Hans]

Am Donnerstag, 26. Oktober 2023, 19:03:15 CEST schrieb Michael Kjörling:
This is interesting information! Looks like KVM and Virt-Manager are better
and faster than Virtualbox.

Obviously it seems (regarding to other people), these solutions are more
stable, too.

That looks great, as I am not so happy of beeing dependent on Oracle.

However, often I get some Images as OVA files, but I still could not get, how
to import OVA files to Virt-Manager or KVM.

A graphical way is preferred, but if not possible, also the CLI way will be
acceptable

I installed aqemu (a GUI for qemu), virt-manger (a little bit complex GUI) and
virtualbox (from Oracle, but without guest-additions annd ext-pack).

Virtualbox is easy to use, but I like AQEMU, too (it is using KVM).

Are the other solutions capable, to import and export OVA or VDI?

Best

Hans

[exceptbees]

> Do you know another way, except pinning or my (weired) way?
>
> Oh, last but not least, I know, Oracle has its own debian-repo for virtualbox,
> but it looks somehow not well set up IMHO.

The latest version of Virtualbox for bookworm is available from the Fast
Track repository [1] [2]. I've been using it for a couple of months,
haven't encountered any problems.

The instructions at [1] are for bullseye, the codename should be changed
to 'bookworm' for it to work.

[1] https://fasttrack.debian.net/
[2] https://wiki.debian.org/FastTrack

Cheers,
exceptbees

[Michael Kjörling]

On 26 Oct 2023 17:23 +0200, from hans.u...@loop.de (Hans):

> I installed aqemu (a GUI for qemu), virt-manger (a little bit complex GUI) and
> virtualbox (from Oracle, but without guest-additions annd ext-pack).

You generally shouldn't install multiple hypervisors on the same
system. I have seen fairly sternly worded warnings against having
VirtualBox and KVM/QEMU installed simultaneously on the same host.
(They probably can _coexist_, but trying to _run_ both at the same
time may well cause issues.) There should be no problems with using
different front-ends for the same hypervisor, however; I regularly use
both virt-manager and the command-line utilities (including
virt-install and virsh) depending on what I need.

> Virtualbox is easy to use, but I like AQEMU, too (it is using KVM).
>
> Are the other solutions capable, to import and export OVA or VDI?

I think you can use `qemu-img convert -f vdi -O qcow2` to convert from
a VDI disk image to a QCOW2 or raw disk image, which QEMU/KVM in turn
can use. (qemu-img convert can also convert to and from many other
formats; see the output of qemu-img --help.) Compared to raw disk
images, QCOW2 adds a number of nice features, not least of which disk
snapshots.

In Debian, qemu-img is packaged in qemu-utils (in Bookworm, at least).

Apparently, a OVA is just a tarball of a hard disk image and a XML
file describing the VM. It shouldn't be too difficult to convert the
disk image and then create a similar KVM VM using the information in
the XML file. It looks like there's a tool named virt-v2v which can do
the conversion, although I have never had a need to try it.

[Hans]

Yes, a little bit "googleing" showed me that way. However, it looks, that none
of the other solutions can import OVA via the graphical interface.

However, for me the CLI will be ok, but I am lazy and would like to do it
graphical, if possible.

But looks like none of the other solutions are capable of this. It is a pity,
but ok.

Thanks for the advice anyway

Best regards

Hans

[Jeffrey Walton]

On Thu, Oct 26, 2023 at 1:24 PM Hans <hans.u...@loop.de> wrote:
>
> Am Donnerstag, 26. Oktober 2023, 19:03:15 CEST schrieb Michael Kjörling:
> This is interesting information! Looks like KVM and Virt-Manager are better
> and faster than Virtualbox.

libvirt is also available on Fedora, while Virtual Box is not. On
Fedora, you have to build Virtual Box from sources, and configure DKMS
to rebuild it for each kernel upgrade.

So if you want a virtualization platform that just works just about
everywhere, then choose libvirt/QEMU/KVM.

Jeff

[Andrew M.A. Cater]

Apt-get install virt-manager will pull in all the associated qemu/KVM
packages you might need. It should be at least as straightforward to
use as Virtualbox.

I use this for testing when we do the testing for every Debian point release
- it's straightforward.

Andy

[Peter Hillier-Brook]

It works well for me, running Bookworm with several Trixie guests. VBox
7.0.12 from VirtualBox.org (Oracle, obviously), obtained via a
"sources.list.d"

Peter HB

[The Wanderer]

On 2023-10-26 at 15:28, Andrew M.A. Cater wrote:

> Apt-get install virt-manager will pull in all the associated
> qemu/KVM packages you might need. It should be at least as
> straightforward to use as Virtualbox.

I've seen people state or suggest multiple times that virt-manager
should be, as you say, as straightforward to use as VirtualBox, and that
was what I expected to find when I tried to use it myself - but even
once I got around the issues that arose from my not using systemd et
cetera and could actually use the program, it is not what I actually found.

(I very much hope that what I am about to describe is wrong, and that
people will explain how/why it's wrong, such that I can get out of the
situation described and into a state where I can actually use
virt-manager in a way that I could find useful. It is not my intention
to spread FUD or falsehoods, even if some of the below may look like it
would fall within those categories.)


From what I recall from having used VirtualBox in the past, its workflow
is fairly similar to what I see in using VMWare Workstation in a
(work-related) Windows environment. When you create a new VM, it prompts
you for where the VM's files should be stored, and then for details
about the VM's configuration (disk sizes, hardware devices, et cetera),
and optionally lets you specify what will be done to install the OS that
will run in the VM - and then with that done, the VM is ready, and you
can boot it up or create a snapshot (using a graphical
snapshot-management interface) or make further configuration edits or do
whatever else you will with it.

With virt-manager, from what I recall (it has been a while since I last
tried), the workflow was quite different. IIRC, I didn't even try using
qemu as a backend, because AFAIK it doesn't support hierarchical VM
snapshots and that's a feature I very much expect to rely on; instead I
think I went with KVM. With that backend, AFAIR I didn't even get
prompted for where the VM's file should be stored; instead, the location
where the system stores files appears to be defined in a system-wide
config file, and to not be modifiable on a per-VM basis (except relative
to that system-wide root). That's a problem, because when I partitioned
this system I expected to be able to store VM files in the same massive
data partition as I allocated for other large data; the default
system-wide location doesn't have the space to do much with. It also
doesn't work when the system may have multiple users who may want to
manage VMs separately from one another (though, fortunately, this is
more an abstract concern rather than one that affects me in practice).

With VMWare Workstation and what I think I I recall from VirtualBox,
once a VM is created, the resulting files are owned by the user who ran
the program. With what I recall from when I tried virt-manager, even if
I redirected the file storage location to be under the larger data
partition, the files were owned by another user, related to libvirt.
That's undesirable when trying to store VM files per-user in a per-user
location, since the user won't be able to work with them (moving them
around, editing details, etc.) except through programs running with that
other user's access.

When I accepted that and tried to proceed anyway, for the sake of
experimentation, IIRC, I ran into obstacles trying to set up the
necessary virtual hardware for the VM - in particular, IIRC, a virtual
CD drive pointed at the ISO that would be used to install the OS. (This
part I am less certain about than even the above; it's been rather a
while, and I was stressed enough by the time I hit this point that I may
have blanked out more of the details in self-defense.) At that point, I
gave up, at least in part for the sake of not piling more and more
stress on myself trying to get the ability to do things that would
hopefully enable me to reduce stress in other areas.

(Writing this mail is already bringing back up all that stress, and I
hope it will not just wind up making things worse.)


So... either I somehow have managed to do things *100% completely
wrong*, or the workflow with virt-manager is not even remotely as
straightforward(ly usable) as the one I see with VMWare Workstation and
think I remember seeing with VirtualBox.

I would *love* to be wrong about that, because there is a *lot* of stuff
that I'd like to do that would be *far* easier if I had discardable VM
snapshots to do it in. However, I also do not have the personal stress
to spare for experimenting with this blindly and bashing my head against
walls getting nowhere in those experiments.

If there *is* a way to get virt-manager to support a VMWare( and, I
think, VirtualBox)-like workflow - with support for hierarchical nested
snapshots, and graphical management thereof, among other things - and
have things more-or-less Just Work, I would *love* to learn about it.

--
The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw

[Andrew M.A. Cater]

On Fri, Oct 27, 2023 at 05:17:46AM -0400, The Wanderer wrote:
> On 2023-10-26 at 15:28, Andrew M.A. Cater wrote:
>
> > Apt-get install virt-manager will pull in all the associated
> > qemu/KVM packages you might need. It should be at least as
> > straightforward to use as Virtualbox.
>
> I've seen people state or suggest multiple times that virt-manager
> should be, as you say, as straightforward to use as VirtualBox, and that
> was what I expected to find when I tried to use it myself - but even
> once I got around the issues that arose from my not using systemd et
> cetera and could actually use the program, it is not what I actually found.
>
> (I very much hope that what I am about to describe is wrong, and that
> people will explain how/why it's wrong, such that I can get out of the
> situation described and into a state where I can actually use
> virt-manager in a way that I could find useful. It is not my intention
> to spread FUD or falsehoods, even if some of the below may look like it
> would fall within those categories.)
>

People's experience varies: I've used Virtualbox in the past, Hyper-V under
Windows and VMWare Workstation - Virt-manager provides as easy a front end to
KVM for me.

It does help if you select the "Customise configuration" option before
beginning install at step 5 of 5

<snip>

See the "configure before install" which opens this up more - you can
also see this by viewing/modifying settings - but you normally have to make
sure that the VM is shut down.

> (Writing this mail is already bringing back up all that stress, and I
> hope it will not just wind up making things worse.)
>
>
> So... either I somehow have managed to do things *100% completely
> wrong*, or the workflow with virt-manager is not even remotely as
> straightforward(ly usable) as the one I see with VMWare Workstation and
> think I remember seeing with VirtualBox.
>
> I would *love* to be wrong about that, because there is a *lot* of stuff
> that I'd like to do that would be *far* easier if I had discardable VM
> snapshots to do it in. However, I also do not have the personal stress
> to spare for experimenting with this blindly and bashing my head against
> walls getting nowhere in those experiments.
>
> If there *is* a way to get virt-manager to support a VMWare( and, I
> think, VirtualBox)-like workflow - with support for hierarchical nested
> snapshots, and graphical management thereof, among other things - and
> have things more-or-less Just Work, I would *love* to learn about it.
>

All the VM solutions essentially have a text backend and something
configurable: virt-manager is just a nice front end to things like virsh
and means that you can get stuff done quickly.

That said, I tend to be simplistic: one or two VMs which have access via
the host network, no bonding, nothing too fancy.
I think there's probably more documentation for KVM out there than anything
else, one way and another.

> --
> The Wanderer
>
> The reasonable man adapts himself to the world; the unreasonable one
> persists in trying to adapt the world to himself. Therefore all
> progress depends on the unreasonable man. -- George Bernard Shaw
>

All the very best, as ever,

Andy

[The Wanderer]

On 2023-10-27 at 10:46, Minecraftchest1 wrote:

> With Virt-Manager, you should have the option to choose an existing
> disk image.

That only helps if you've already created a disk image, which will not
be the case when creating a new VM from scratch. Having to resort to the
command line (or to other tools) to create the initial disk image - even
if, potentially, just creating an empty file (or a file of specified
size, filled with zeroes) would work - is not as friendly or as
straightforward a workflow as being able to do it from the GUI.

> In that dialog, you can create an image in any of the pools (you can
> also add pools in that dialog), and that will let you change the file
> name and disk size.

Hold up. Where do "pools" (which I'm guessing is short for "storage
pools") come into things? The other virtualization solutions I've seen
and worked with (short of full-system-level things, such as I understand
VMWare ESXi to be) don't require being aware of or handling storage
pools; they work with disk image files (or, for the case of hierarchical
snapshots, cascading stacks thereof) directly, and do not require those
files to be part of any "storage pool" in any way that the user needs to
be aware of.

If you're starting with the assumption that "storage pools" will - much
less need to - be involved somewhere, you're already not matching the
convenience etc. of the workflow I know from those other tools.

Just offhand, I would expect that a "storage pools" paradigm would block
off some of the convenient things that can be done in that other
workflow, such as being able to move or copy a virtual machine by moving
or copying the directory that its files (disk images, configuration
files, et cetera) are stored in - because you'd also have to fiddle with
whatever it is that defines the "storage pool" that those files are part
of, and that definition would presumably be outside of the directory
that defines the virtual machine.

> I am not ay my laptop currently, but I can take and share some
> screenshots later today.

Regardless of the above, that might be useful.

[Charles Curley]

On Fri, 27 Oct 2023 14:46:52 +0000
Minecraftchest1 <wyattcja...@outlook.com> wrote:

> With Virt-Manager, you should have the option to choose an existing
> disk image.

It can also create a disk image for you. On which you will have to make
partitions and file systems.

--
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

[Max Nikulin]

On 28/10/2023 02:02, The Wanderer wrote:
> for the case of hierarchical snapshots

qemu-img(1) allows to create snapshots of disk images that are stored in
the same file. In addition the "create" command has the "-b
BACKING_FILE" option

> If the option BACKING_FILE is specified, then the image will record only
> the differences from BACKING_FILE.

Is it something close to "hierarchical snapshots"?

[The Wanderer]

On 2023-10-28 at 00:25, Max Nikulin wrote:

> On 28/10/2023 02:02, The Wanderer wrote:
>
>> for the case of hierarchical snapshots
>
> qemu-img(1) allows to create snapshots of disk images that are stored
> in the same file. In addition the "create" command has the "-b
> BACKING_FILE" option

Does virt-manager expose this feature via a convenient
create-a-new-snapshot GUI, showing the tree of which snapshots descend
from what? I believe I was under the impression that, for the case of
qemu, no such thing was available.

(I note, again, that it's been a long time since I tried this; I have
spent the past couple of years, at least, attempting with various
degrees of desperation to avoid more stress than I can handle, and after
my last try with virt-manager hit that wall this conversation is the
first time I've been able to handle any kind of try-it-again.)

>> If the option BACKING_FILE is specified, then the image will record
>> only the differences from BACKING_FILE.
>
> Is it something close to "hierarchical snapshots"?

If one snapshot can descend from another, and you can delete any
snapshot (again, from that GUI) and have any references to it in other
snapshots automagically cleaned up to point to any relevant new parent
(such that the data seen through those other snapshots is still the same
as before the deletion), then probably.

It sounds like it might be worth my giving this another try, with qemu
as a backend, and seeing if I get any better results. Not sure exactly
when I may do that - I'm dealing with major stress from another source
right now, and would probably have difficulty handling it if the attempt
failed - but the idea is at least on my radar.

[Michael Kjörling]

On 28 Oct 2023 06:33 -0400, from wand...@fastmail.fm (The Wanderer):
> virt-manager

Do keep in mind that virt-manager is just _one_ possible front-end for
KVM (although perhaps the most common GUI one). AQEMU has already been
mentioned in this thread. Technically virsh and friends is another
front-end, although a command-line one and quite technical. I'm sure
there are other freely available alternatives as well.