fonts, bitmap and the (old TeX) PK file format
223 views
Skip to first unread message
Sebastien Binet
Aug 27, 2021, 1:28:54 PM8/27/21
to golang-nuts
hi there,
I am working on providing a pure-Go TeX engine.
to achieve such a thing, I'd need to be able to decode and use the
so-called "PacKed font file" format (PK fonts).
(I know it's a very old & deprecated file format, but I figured it would
be easier on me to start "old tech" first)
I think I managed to properly decode it, and managed to extract the
bitmap data of each glyph:
- https://play.golang.org/p/dA8cev_x4TQ
the program above displays the bitmap data extracted for glyph 'a':
$> go run ./main.go
...........********...................
........**************................
......*****.......******..............
.....***............*****.............
....*****............******...........
...*******............******..........
...********...........******..........
...********............******.........
...********............******.........
...********.............******........
....******..............******........
.....****...............******........
........................******........
........................******........
........................******........
........................******........
.................*************........
.............*****************........
..........*********.....******........
........*******.........******........
......*******...........******........
....********............******........
...*******..............******........
..********..............******........
.********...............******........
.*******................******........
.*******................******......**
*******.................******......**
*******.................******......**
*******.................******......**
*******................*******......**
*******................*******......**
********..............********......**
.*******.............***.*****......**
.********............**...*****....**.
..********.........****...*****....**.
....*******......****......*********..
......**************........*******...
.........********............*****....
now, the question is: how should I packaged that data so that I can
interoperate best with, say, golang.org/x/image/font ?
would I just use a golang.org/x/image/font/basicfont.Face ?
cheers,
-s
I am working on providing a pure-Go TeX engine.
to achieve such a thing, I'd need to be able to decode and use the
so-called "PacKed font file" format (PK fonts).
(I know it's a very old & deprecated file format, but I figured it would
be easier on me to start "old tech" first)
I think I managed to properly decode it, and managed to extract the
bitmap data of each glyph:
- https://play.golang.org/p/dA8cev_x4TQ
the program above displays the bitmap data extracted for glyph 'a':
$> go run ./main.go
...........********...................
........**************................
......*****.......******..............
.....***............*****.............
....*****............******...........
...*******............******..........
...********...........******..........
...********............******.........
...********............******.........
...********.............******........
....******..............******........
.....****...............******........
........................******........
........................******........
........................******........
........................******........
.................*************........
.............*****************........
..........*********.....******........
........*******.........******........
......*******...........******........
....********............******........
...*******..............******........
..********..............******........
.********...............******........
.*******................******........
.*******................******......**
*******.................******......**
*******.................******......**
*******.................******......**
*******................*******......**
*******................*******......**
********..............********......**
.*******.............***.*****......**
.********............**...*****....**.
..********.........****...*****....**.
....*******......****......*********..
......**************........*******...
.........********............*****....
now, the question is: how should I packaged that data so that I can
interoperate best with, say, golang.org/x/image/font ?
would I just use a golang.org/x/image/font/basicfont.Face ?
cheers,
-s
Howard C. Shaw III
Sep 1, 2021, 4:11:44 PM9/1/21
to golang-nuts
You would implement the Face interface to have your fonts available to Go's text drawing routines.
You can try using basicfont directly, however it is fixedwidth only. But just implementing the interface does not seem that onerous.
Have a look at https://github.com/hajimehoshi/bitmapfont for an example of implementing Face for a mostly fixed-size bitmap font (by the author of the Ebiten game engine). hajimehoshi's file gives an example of dealing with a *mostly* fixed width file, where certain characters (i.e. East Asian glyphs) can be double-width.
However, while that would gain interoperability of your PK fonts with Go's text flow engine.... isn't TeX emulation pretty much going to require you roll your own text flow engine anyway? I mean, that is kind of the heart of TeX, using font metrics to flow text. And so in that case, looking at https://github.com/usedbytes/fonts, which ignores the Face and golang Font and rolls its own text flow for rendering bitmap fonts onto Images might actually be a useful example.... but only so far, because I believe it also assumes a fixed width font.
And no form of fixed width font handling is going to suffice to mimic TeX; but again, really, with TeX what you need to be parsing from the .pk file is the font metrics, really. TeX never cared about the glyphs. That is why TeX's output was a DVI (DeVice Independent) file that had to be combined with font files to render an actual printable or viewable output. Except that I don't know that the PK file even has the font metrics - I think back then they were in a separate .tfm file?
This is relevant because one of the important elements that TeX handled was the concept that glyphs, properly kerned, can *overlap*. That you cannot use the width of the glyph to know how far to move forward to draw the next one - that was a separate metric. (Not saying it was the first, just that it is one of the important features.)
Again, not to take anything away from having success parsing the .pk file format - kudos to you. I'm just not sure not only whether it will actually be helpful in emulating TeX, but whether it is even needed. Theoretically, you should be able to fully emulate TeX in processing a TeX-format file into a DVI without ever touching any font glyphs at all, just the metrics.
Sebastien Binet
Sep 1, 2021, 8:02:20 PM9/1/21
to Howard C. Shaw III, golang-nuts
Howard,
On Wed Sep 1, 2021 at 18:11 CET, Howard C. Shaw III wrote:
> You would implement the Face interface to have your fonts available to
> Go's
> text drawing routines. You can try using basicfont directly, however it
> is
> fixedwidth only. But just implementing the interface does not seem that
> onerous.
>
> Have a look at https://github.com/hajimehoshi/bitmapfont for an example
> of
> implementing Face for a mostly fixed-size bitmap font (by the author of
> the
> Ebiten game engine). hajimehoshi's file gives an example of dealing with
> a *mostly* fixed width file, where certain characters (i.e. East Asian
> glyphs) can be double-width.
>
> Also https://github.com/textmodes/font
>
> Not https://github.com/usedbytes/fonts - this one does not implement the
> Face interface.
thanks for these pointers.
I'll have a look.
(funny my 'bitmap' queries on pkg.go.dev didn't turn them out)
>
> However, while that would gain interoperability of your PK fonts with
> Go's
> text flow engine.... isn't TeX emulation pretty much going to require
> you
> roll your own text flow engine anyway? I mean, that is kind of the heart
> of
> TeX, using font metrics to flow text. And so in that case, looking at
> https://github.com/usedbytes/fonts, which ignores the Face and golang
> Font
> and rolls its own text flow for rendering bitmap fonts onto Images might
> actually be a useful example.... but only so far, because I believe it
> also
> assumes a fixed width font.
yes, TeX has its own set of text shaping algorithms.
in view of being able to display LaTeX equations in Gonum plots, I had
first tried to implement the "matheq" subset of LaTeX:
- https://github.com/go-latex/latex
but, in the end, I went with implementing the full kaboodle:
- https://star-tex.org
- https://star-tex.org/x/tex
>
> And no form of fixed width font handling is going to suffice to mimic
> TeX;
> but again, really, with TeX what you need to be parsing from the .pk
> file
> is the font metrics, really. TeX never cared about the glyphs. That is
> why
> TeX's output was a DVI (DeVice Independent) file that had to be combined
> with font files to render an actual printable or viewable output. Except
> that I don't know that the PK file even has the font metrics - I think
> back
> then they were in a separate .tfm file?
correct.
> This is relevant because one of the important elements that TeX handled
> was
> the concept that glyphs, properly kerned, can *overlap*. That you cannot
> use the width of the glyph to know how far to move forward to draw the
> next
> one - that was a separate metric. (Not saying it was the first, just
> that
> it is one of the important features.)
>
> Again, not to take anything away from having success parsing the .pk
> file
> format - kudos to you. I'm just not sure not only whether it will
> actually
> be helpful in emulating TeX, but whether it is even needed.
> Theoretically,
> you should be able to fully emulate TeX in processing a TeX-format file
> into a DVI without ever touching any font glyphs at all, just the
> metrics.
yes, DVI is the main output of "old" TeX (nowadays, most TeX engines
directly produce PDFs), with only the "boxes" of the glyphs (using the
TFM data.)
it's only when one issues, say, "dvipdf" that the "bounding boxes" of
each glyph is filled with the actual glyph, drawn from PK fonts (in the
very old days), from Type-1 or TTF fonts.
I have the pure-Go TeX engine that produces a .dvi from a .tex file.
now, with either PK or Type-1 fonts, I'd like to implement a "dvipng"
or "dvipdf" command; hence my query about PK fonts.
(probably the PK fonts will only be workable w/ the "dvipng" command)
Implementing support for Type-1 fonts is quite a bigger toll, as this
means implementing a PostScript interpreter.
Eventually, I'd like to implement a Gio-based viewer for DVI files.
Lots of fun ahead :)
Thanks again for the pointers.
-s
On Wed Sep 1, 2021 at 18:11 CET, Howard C. Shaw III wrote:
> You would implement the Face interface to have your fonts available to
> Go's
> text drawing routines. You can try using basicfont directly, however it
> is
> fixedwidth only. But just implementing the interface does not seem that
> onerous.
>
> Have a look at https://github.com/hajimehoshi/bitmapfont for an example
> of
> implementing Face for a mostly fixed-size bitmap font (by the author of
> the
> Ebiten game engine). hajimehoshi's file gives an example of dealing with
> a *mostly* fixed width file, where certain characters (i.e. East Asian
> glyphs) can be double-width.
>
> Also https://github.com/textmodes/font
>
> Not https://github.com/usedbytes/fonts - this one does not implement the
> Face interface.
I'll have a look.
(funny my 'bitmap' queries on pkg.go.dev didn't turn them out)
>
> However, while that would gain interoperability of your PK fonts with
> Go's
> text flow engine.... isn't TeX emulation pretty much going to require
> you
> roll your own text flow engine anyway? I mean, that is kind of the heart
> of
> TeX, using font metrics to flow text. And so in that case, looking at
> https://github.com/usedbytes/fonts, which ignores the Face and golang
> Font
> and rolls its own text flow for rendering bitmap fonts onto Images might
> actually be a useful example.... but only so far, because I believe it
> also
> assumes a fixed width font.
in view of being able to display LaTeX equations in Gonum plots, I had
first tried to implement the "matheq" subset of LaTeX:
- https://github.com/go-latex/latex
but, in the end, I went with implementing the full kaboodle:
- https://star-tex.org
- https://star-tex.org/x/tex
>
> And no form of fixed width font handling is going to suffice to mimic
> TeX;
> but again, really, with TeX what you need to be parsing from the .pk
> file
> is the font metrics, really. TeX never cared about the glyphs. That is
> why
> TeX's output was a DVI (DeVice Independent) file that had to be combined
> with font files to render an actual printable or viewable output. Except
> that I don't know that the PK file even has the font metrics - I think
> back
> then they were in a separate .tfm file?
> This is relevant because one of the important elements that TeX handled
> was
> the concept that glyphs, properly kerned, can *overlap*. That you cannot
> use the width of the glyph to know how far to move forward to draw the
> next
> one - that was a separate metric. (Not saying it was the first, just
> that
> it is one of the important features.)
>
> Again, not to take anything away from having success parsing the .pk
> file
> format - kudos to you. I'm just not sure not only whether it will
> actually
> be helpful in emulating TeX, but whether it is even needed.
> Theoretically,
> you should be able to fully emulate TeX in processing a TeX-format file
> into a DVI without ever touching any font glyphs at all, just the
> metrics.
directly produce PDFs), with only the "boxes" of the glyphs (using the
TFM data.)
it's only when one issues, say, "dvipdf" that the "bounding boxes" of
each glyph is filled with the actual glyph, drawn from PK fonts (in the
very old days), from Type-1 or TTF fonts.
I have the pure-Go TeX engine that produces a .dvi from a .tex file.
now, with either PK or Type-1 fonts, I'd like to implement a "dvipng"
or "dvipdf" command; hence my query about PK fonts.
(probably the PK fonts will only be workable w/ the "dvipng" command)
Implementing support for Type-1 fonts is quite a bigger toll, as this
means implementing a PostScript interpreter.
Eventually, I'd like to implement a Gio-based viewer for DVI files.
Lots of fun ahead :)
Thanks again for the pointers.
-s
Howard C. Shaw III
Sep 1, 2021, 9:24:03 PM9/1/21
to Sebastien Binet, golang-nuts
Cool.
As to the Postscript interpreter, look at
Also, font specific
Not sure how much either helps towards Type 1 support. Even Adobe is end of lifing Type 1 support, though, so it might be worth seeing if you can find a table of maps to help you find the closest OpenType match instead.
Best of luck!
Andy Balholm
Sep 2, 2021, 8:22:26 PM9/2/21
to golan...@googlegroups.com
You don't need a full PostScript interpreter to use a Type 1 font. They
use a very limited subset of PS.
To embed one in a PDF, I don't think you need to parse it at all, if you
know the metrics and the encoding already. You can just embed it as a
binary blob, if I'm not mistaken.
Andy
use a very limited subset of PS.
To embed one in a PDF, I don't think you need to parse it at all, if you
know the metrics and the encoding already. You can just embed it as a
binary blob, if I'm not mistaken.
Andy
Sebastien Binet
Sep 3, 2021, 6:54:20 AM9/3/21
to Andy Balholm, golan...@googlegroups.com
On Thu Sep 2, 2021 at 22:21 CET, Andy Balholm wrote:
> You don't need a full PostScript interpreter to use a Type 1 font. They
> use a very limited subset of PS.
I surmised as much reading through:
> You don't need a full PostScript interpreter to use a Type 1 font. They
> use a very limited subset of PS.
- https://adobe-type-tools.github.io/font-tech-notes/pdfs/T1_SPEC.pdf
(which is my next target as PK support is about to be committed.)
>
> To embed one in a PDF, I don't think you need to parse it at all, if you
> know the metrics and the encoding already. You can just embed it as a
> binary blob, if I'm not mistaken.
thanks all for the input and pointers,
-s
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/2de7f944-3a8f-2e17-0804-cd9b0406d40c%40gmail.com.
Sebastien Binet
Sep 14, 2021, 4:21:38 PM9/14/21
to Sebastien Binet, golang-nuts
On Fri Aug 27, 2021 at 15:28 CET, 'Sebastien Binet' via golang-nuts wrote:
> hi there,
>
> I am working on providing a pure-Go TeX engine.
> to achieve such a thing, I'd need to be able to decode and use the
> so-called "PacKed font file" format (PK fonts).
> (I know it's a very old & deprecated file format, but I figured it would
> be easier on me to start "old tech" first)
>
> I think I managed to properly decode it, and managed to extract the
> bitmap data of each glyph:
>
> - https://play.golang.org/p/dA8cev_x4TQ
FYI, star-tex can now convert some DVI files into PNG files, using PK
> hi there,
>
> I am working on providing a pure-Go TeX engine.
> to achieve such a thing, I'd need to be able to decode and use the
> so-called "PacKed font file" format (PK fonts).
> (I know it's a very old & deprecated file format, but I figured it would
> be easier on me to start "old tech" first)
>
> I think I managed to properly decode it, and managed to extract the
> bitmap data of each glyph:
>
> - https://play.golang.org/p/dA8cev_x4TQ
file fonts.
as star-tex doesn't know how to rescale PK fonts (this involves support
for MetaFont: not sure it's worth the complexity), it must be instructed
to generate a PNG file with the "correct" DPI (one that matches that of
the PK font).
but apart from that, it works.
- https://git.sr.ht/~sbinet/star-tex/tree/main/item/cmd/dvi-cnv/testdata/hello_golden.png
- https://git.sr.ht/~sbinet/star-tex/tree/main/item/cmd/dvi-cnv/testdata/xcolor_golden.png
now working on handling Type-1 fonts so star-tex can generate PNGs at
"any" resolution.
-s
Reply all
Reply to author
Forward
0 new messages