SVG in Copy to image

[Lukas-Fabian Moser]

Folks,

after trying Frescobaldi 4.0 a bit and being impressed overall, I hit a roadblock: As explained in the commit message to 8db25c906, the lack of Poppler support in PyQt6 makes it impossible to export vector formats in "Copy to image".

Unfortunately, this is dealbreaker for my workflow - auto-cropped svg export of selected portions of a LilyPond document is an ideal bridge to LibreOffice Writer and other such programs. (Using PNGs as a replacement is at best acceptable to me in an emergency.) So this means I probably have to go back to Frescobaldi 3.3 as long as it works. (Does the installation via pipx allow for co-existence of multiple Frescobaldi versions on one system?)

What's the picture here: Is the issue of missing Poppler support something that we have to (and can) wait out, or is this unlikely to happen at all, so auto-cropped vector export is gone for good?

Lukas

[Jean Abou Samra]

Le vendredi 24 janvier 2025 à 09:58 -0800, 'Lukas-Fabian Moser' via
Frescobaldi a écrit :

> Folks,
>
> after trying Frescobaldi 4.0 a bit and being impressed overall, I hit
> a roadblock: As explained in the commit message to 8db25c906, the
> lack of Poppler support in PyQt6 makes it impossible to export vector
> formats in "Copy to image".
>
> Unfortunately, this is dealbreaker for my workflow - auto-cropped svg
> export of selected portions of a LilyPond document is an ideal bridge
> to LibreOffice Writer and other such programs. (Using PNGs as a
> replacement is at best acceptable to me in an emergency.) So this
> means I probably have to go back to Frescobaldi 3.3 as long as it
> works. (Does the installation via pipx allow for co-existence of
> multiple Frescobaldi versions on one system?)

pipx doesn't allow this, but pipx is merely a convenient wrapper
around creating the virtual environments yourself.

I actually think it was a mistake when I wrote these instructions
to use pipx. The main other problem we're getting is that there's
no way around first installing frescobaldi with wrong
python-ly and qpageview versions, then injecting the correct
versions, which is an issue if the wrong versions cannot even
build.

Unfortunately, Python packaging is a moving target with
a myriad of different tools, none of which does its job
so well that you can forget about the others.

> What's the picture here: Is the issue of missing Poppler support
> something that we have to (and can) wait out, or is this unlikely to
> happen at all, so auto-cropped vector export is gone for good?

It is unlikely to happen, at least in the way it was implemented
previously.

For years, the single biggest problem of Frescobaldi has always
been packaging at large, and especially the creation of installers,
which was in large part due to the python-poppler-qt5 module,
a Python binding to poppler-qt5 developed by Wilbert for Frescobaldi.
There was a commutative diagram of dependencies (😉)

Qt5 → poppler-qt5
↓ ↓
PyQt5 → python-poppler-qt5

which was an absolute PITA because it makes it impossible,
for example, to use the PyQt PyPI packages, as those bundle
Qt but not in a way that we can reuse to build Poppler
against them so the result is ABI-compatible. So we had to
find ways to get compatible builds of all of these, which
for example on MacOS was achieved by (horror!) taking
packages built by MacPorts, relying on lots of their
implementation details, and doing fragile dark magic
that nobody really understood to rebundle them in a way
accepted by Apple's obscure undocumented "framework" format
in app bundles.

Being able to use a native Qt6 module is frankly a life saver.
It is still incredibly complex to build (QtPdf relies on Chromium's
Web engine), but those build complexities are not on us anymore.
I really do not see us going back now that we finally have
a hope of being able to make Frescobaldi releases semi-regularly.

But it could be acceptable to implement the feature differently,
if that doesn't need Poppler. For example, instead of exporting
fragments of PDF to SVG, we could implement this for the SVG
view where we already have SVG available, produced by LilyPond.
It should be trivial to crop an SVG by just resizing its bounding
box; it's probably much less trivial to make that process
actually trim its size, but if the fonts are embedded, they
might be the main contributor to size anyway?

[Lukas-Fabian Moser]

Hi Jean,

thanks for all the explanations!

>> So this means I probably have to go back to Frescobaldi 3.3 as long
>> as it works. (Does the installation via pipx allow for co-existence
>> of multiple Frescobaldi versions on one system?)
> pipx doesn't allow this, but pipx is merely a convenient wrapper
> around creating the virtual environments yourself.

I see - looking at ~/.local/pipx/venvs, this seems to be transparent
enough. So I'll simply create the venvs myself (or even try letting pipx
create one for the old Frescobaldi version and moving this to a safe
place; if there are no hardcoded absolute paths, this might actually work).

> So we had to
> find ways to get compatible builds of all of these, which
> for example on MacOS was achieved by (horror!) taking
> packages built by MacPorts, relying on lots of their
> implementation details, and doing fragile dark magic
> that nobody really understood to rebundle them in a way
> accepted by Apple's obscure undocumented "framework" format
> in app bundles.

It harrows me with fear and wonder.

> But it could be acceptable to implement the feature differently,
> if that doesn't need Poppler. For example, instead of exporting
> fragments of PDF to SVG, we could implement this for the SVG
> view where we already have SVG available, produced by LilyPond.
> It should be trivial to crop an SVG by just resizing its bounding
> box; it's probably much less trivial to make that process
> actually trim its size, but if the fonts are embedded, they
> might be the main contributor to size anyway?

Well, I wasn't a huge fan of the SVG backend since I often had the
impression that its results different subtly from the PDF (I don't
remember the details right now). But of course, this has probably
changed with the Cairo backend.

Trimming an SVG with a bounding box would probably acceptable to me; my
standard use case is "taking one or two systems from a 1-2 page PDF file
with musical examples" and embedding them in 1-10 page LibreOffice
Writer documents, so file sizes won't matter all that much.

But there's one point I don't yet understand: Namely, the relationship
between Poppler (which, as I understand it, deals with PDF files) and
Qt. I understand that they must be intertwined if we use Poppler for
displaying PDFs, but aren't there "abstract" Poppler bindings,
independent of Qt, that one could use on any platform? So basically:
Frescobaldi uses native Qt tools for displaying PDFs, and when I want to
export a cropped box, Frescobaldi takes the coordinates of the rectangle
I selected and uses that information to process the PDF with Poppler?

I'm perfectly willing to try my hand at this (since the cropped SVG
export feature is really essential to me), but if this route is doomed
to fail anyway, I'll be grateful for a heads-up. :-)

Lukas

[Jean Abou Samra]

Le mardi 28 janvier 2025 à 14:57 +0100, 'Lukas-Fabian Moser' via Frescobaldi a écrit :
> I see - looking at ~/.local/pipx/venvs, this seems to be transparent
> enough. So I'll simply create the venvs myself (or even try letting pipx
> create one for the old Frescobaldi version and moving this to a safe
> place; if there are no hardcoded absolute paths, this might actually work).

Beware that you actually cannot move venvs (this is a
known shortcoming).

>
> Well, I wasn't a huge fan of the SVG backend since I often had the
> impression that its results different subtly from the PDF (I don't
> remember the details right now).

IIRC, that is because it does not embed the fonts in the SVG file.

> But of course, this has probably changed with the Cairo backend.

Indeed.

> Trimming an SVG with a bounding box would probably acceptable to me; my
> standard use case is "taking one or two systems from a 1-2 page PDF file
> with musical examples" and embedding them in 1-10 page LibreOffice
> Writer documents, so file sizes won't matter all that much.
>
> But there's one point I don't yet understand: Namely, the relationship
> between Poppler (which, as I understand it, deals with PDF files) and
> Qt. I understand that they must be intertwined if we use Poppler for
> displaying PDFs, but aren't there "abstract" Poppler bindings,
> independent of Qt, that one could use on any platform? So basically:
> Frescobaldi uses native Qt tools for displaying PDFs, and when I want to
> export a cropped box, Frescobaldi takes the coordinates of the rectangle
> I selected and uses that information to process the PDF with Poppler?

Poppler has four APIs: one for Qt6, one for Qt5, one for GLib
(used e.g. with GTK), and an "internal" API. The Qt6 one basically
provides you with a Qt widget which renders a given PDF file.
It also has a CLI providing among other things a `pdftocairo`
command that you can use to convert a PDF to any format for
which Cairo has an output backend, like SVG. That command
appears to have arguments for cropping so it would be conceivable
to call it in a subprocess. I'm not sure how easy it would
be to ship Poppler built against Cairo in the installers, but
if it's not for a feature most users need, we might just declare
that you need to have poppler-utils separately installed just
like you need a separate tool if you want to listen to MIDI.

Cheers,
Jean

[Lukas-Fabian Moser]

Hi Jean,

Beware that you actually cannot move venvs (this is a
known shortcoming).

Thanks for the warning! Then I'll think of something else.

But there's one point I don't yet understand: Namely, the relationship
between Poppler (which, as I understand it, deals with PDF files) and
Qt. I understand that they must be intertwined if we use Poppler for
displaying PDFs, but aren't there "abstract" Poppler bindings,
independent of Qt, that one could use on any platform? So basically:
Frescobaldi uses native Qt tools for displaying PDFs, and when I want to
export a cropped box, Frescobaldi takes the coordinates of the rectangle
I selected and uses that information to process the PDF with Poppler?

Poppler has four APIs: one for Qt6, one for Qt5, one for GLib
(used e.g. with GTK), and an "internal" API. The Qt6 one basically
provides you with a Qt widget which renders a given PDF file.
It also has a CLI providing among other things a `pdftocairo`
command that you can use to convert a PDF to any format for
which Cairo has an output backend, like SVG. That command
appears to have arguments for cropping so it would be conceivable
to call it in a subprocess. I'm not sure how easy it would
be to ship Poppler built against Cairo in the installers, but
if it's not for a feature most users need, we might just declare
that you need to have poppler-utils separately installed just
like you need a separate tool if you want to listen to MIDI.

That sounds great! I'll look into this as soon as I have time; since this feature is really important to me, I'm bound to hit that problem sooner or later.

Best
Lukas