bazel and latex
jrgle...@gmail.com
First off, I am very new to bazel. I am one of those people who always want to transition, but every time I try I seem to hit a fairly steep learning curve and sadly go back to make because in the end of the day there is a limit to how much I can spend my workdays mucking around with a build system.
So I was going to type up some documentation today and thought, "hey, I was about to write a makefile for this, the project is stupidly simple, why not use this as a starting point for using bazel". As it is latex, I will also get a feel for how bazel work outside of the predefined rules. Because that has to me always been a huge issue. If you want to compile a simple C++ program you look at the tutorial, and as long as your problem can be accomplished by the steps described there, it is roses all the way. However, the moment I want to customize anything, the whole thing falls apart for me, and digging through the documentation for a couple of hours doesn't really seem to help. E.g. I needed to turn off -std=c++11 for a program I was working on, and getting bazel to do that took me multiple hours.
Sorry about the long setup, it is not a rant, I really want to learn, it can just feel a bit daunting at times. Here is my problem:
I have a folder with a .tex file in it. I want to compile this with either pdflatex, xelatex or use latexmk, anything to create a .pdf file. My first though was to use a genrule to do it, but while I was at it, I thought I might as well define a macro. So in my root folder I define a file "buildutils.bzl" which contains the following:
def _replace_extension(input_str, ext_from, ext_to):
return input_str.split(ext_from)[0] + ext_to
def latex_pdf(name, src):
native.genrule(
name = name,
srcs = [src],
outs = [_replace_extension(src, ".tex", ".pdf")],
cmd = "latexmk -pdf -xelatex $(location " + src + ")"
)
Simple enough for now I though. No need for anything fancy like checking that src really is a .tex file etcetc. I'm sure you see the mistake immediately, but this is my attempt. Then in the folder my .tex file is I have this BUILD file
load("//:buildutils.bzl", "latex_pdf")
latex_pdf(
name = "documentation",
src = "documentation.tex"
)
Trying to compile this does not work, it complains that it never produced the pdf file. The reason for this I assume is that the files are produced in the wrong folders. This is because latexmk unfortunately produces its output in the folder it is called from. Not that much I can do about that. latexmk does provide a "-cd" option which it claims make it output relative to the source file, but that didn't work for reasons unknown to me.
So my question is this. How do I proceed? I would somehow need to either cd to the folder before I call the command, something along:
cmd = "cd $(dir " + src + ") && latexmk -pdf -xelatex $(location " + src + ")"
Or find a way for the macro to compute the directory, and pass that as a parameter to latexmk via the "-outdir" option.
Besides this I do have some questions regarding where bazel stores things. I understand that they are placed in cashe directories and symlinked in place, which is fine. However, latex produces quite a bit of auxillary files, and these are used in latex compilations to speed up the build process. Will these be available to pdflatex/xelatex/latexmk on later builds? These e.g. help with cross-referencing, as well as making it so that you only have to build separate sections if they are changed etcetc.
Do you have any guidelines for going about adapting bazel to work with tex? Also, how e.g. would one get it to work so that if one .tex file produces a figure that is needed by another tex file? Maybe it "just works" whenever you get the basics working.
Sorry for the long question.
Cheers,
Jonas
Damien Martin-Guillerez
--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/aaf008dd-3b79-4269-828c-01603bb3e235%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Klaus Aehlig
Hi Jonas,
> I have a folder with a .tex file in it. I want to compile this with either
> pdflatex, xelatex or use latexmk, anything to create a .pdf file.
- The location of the outputfile is not necessarily, what is suitable
for bazel, as you already noticed.
- pdflatex generates a bunch of temporary files in various places
(all those *.aux, *.log, *.toc etc).
- The output of pdflatex contains quite some timestamps[1], whereas
for bazel we prefer reproducible builds.
- pdflatex is verbose, even if no error occurs.
The second point motivates to do the actual latex invocation in a
temporary directory, as you want you rules to work independently of
sandboxing (i.e., they should also work with standalone execution
strategy without putting garbage in bazel's internal directories. If
you're copying in and out of the work directory, copying to the right
location is no harder, hence solving the first problem. The two remaining
points are more of an esthetic thing, but if we write a script anyway,
it's not hard to fix this, so let's do it. My script (not yet handling
bibtex) is attached.
Such a script together with a genrule works well enough for simple
cases. Using latex more seriously, file groups will turn out to be a
good idea, as usually you split up your latex document into individual
files. If you split up things extensively (e.g., step-by-step diagrams
for overlay slides), you might want to replace the genrule by a proper
rule and provide the files to be used in a parameter file (yes, there
is a limit to the size of a command line; it is not terribly small, but
you can hit it).
A next thing, you might notice is that you need some other files implictly
implied with a generated tex file (e.g., you have a file group of
definitions and want a file that \input's them all). This can be done
using providers[2].
I attach the relevant fragment of my personal bazel rules. They're not
ready for prime time yet (in particular bibtex is missing and I currently
model document fragments simply by a filegroup of the used files with
an implicit understanding of the first element being the entry point),
but they might serve as a starting point to understand the latex-specific
considerations. Together with the skylark rules documentation[3], you
should be able to build something for your needs.
Regards,
Klaus
[1] http://www.linta.de/~aehlig/techblog/2017-02-19.html
[2] https://docs.bazel.build/versions/master/skylark/rules.html#providers
[3] https://docs.bazel.build/versions/master/skylark/rules.html
--
Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschaeftsfuehrer: Matthew Scott Sucherman, Paul Terence Manicle
e...@nuxi.nl
I hope you're doing great. Say hi to some of the others in Munich!
Op maandag 24 juli 2017 13:19:07 UTC+2 schreef Klaus Aehlig:
> I attach the relevant fragment of my personal bazel rules.
Thanks for publishing this. Very useful!
At my current employer we also want to have some basic library/inheritance functionality for LaTeX, so that we can easily reuse templates for requirements specification/design/testing results documents. I've just published our fork of your rules on our company's GitHub page:
https://github.com/ProdriveTechnologies/bazel-latex
Ed
Klaus Aehlig
Hi Ed,
good to hear from you again.
And thanks for properly documenting and publishing the latex rules. I
see, if I can merge some your ideas back to my personal rules (I can't
use them as they are, as I too much depend on includefile and the
Provider it returns being honored); in particular the idea of guessing
the entry point if not provided seems useful.
> At my current employer we also want to have some basic library/inheritance
> functionality for LaTeX, so that we can easily reuse templates for requirements
> specification/design/testing results documents.
filegroup(
name="company_style",
srcs = ...,
)
filegroup(
name = "my_report_lib",
srcs = glob([
"my_report.tex",
"chapters/*.tex",
"figures/*",
]) + [":company_style"],
)
latex(
name = "my_report",
srcs = [":my_report_lib"],
main = "my_report.tex",
)
Filegroups also work nicely when generating various versions of a graphic
to be included in slides (see the slightly hackish but very useful
ps_family in the aehlig_rules_latex.shar attached in my original post)
which I used, e.g., for my FrOSCon 2017 talk[1]. These filegroups are
also the reason why I had to change the genrule to an action, as I had
to write the parameter file as well (with filegroups all the way down,
the command line quickly grew longer than the maximal allowed length).
In the long run, my plan is to add a proper latex_fragment rule that
basically is just a bag of sources, organised in a provider. The reason
is that in the long run, a latex fragment is more that just the tex
files: there are .bib files that all need to be merged and included, a
collection of files to be \input before the \begin{document}, e.g.,
notations, etc. Unfortunately, I didn't have the time to do so yet.
I'll report back if and when I found the time to do so. In the mean
time thanks for picking up my draft of latex rules for bazel!
Klaus
[1] http://www.linta.de/~aehlig/university/slides/conf/froscon2017.pdf
That talk also, in the "Extending Bazel" section, shows how and why
my rules evolved from the simple macro expanding to a genrule to
a full action working with providers.
--
Klaus Aehlig
Google Germany GmbH, Erika-Mann-Str. 33, 80636 Muenchen
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Ed Schouten
2018-05-23 14:49 GMT+02:00 Klaus Aehlig <aeh...@google.com>:
> And thanks for properly documenting and publishing the latex rules. I
> see, if I can merge some your ideas back to my personal rules (I can't
> use them as they are, as I too much depend on includefile and the
> Provider it returns being honored); in particular the idea of guessing
> the entry point if not provided seems useful.
for me; no need to scrub any PDFs.
>> At my current employer we also want to have some basic library/inheritance
>> functionality for LaTeX, so that we can easily reuse templates for requirements
>> specification/design/testing results documents.
>
> I've been always using filegroups for this. E.g.,
>
> filegroup(
> name="company_style",
> srcs = ...,
> )
>
> filegroup(
> name = "my_report_lib",
> srcs = glob([
> "my_report.tex",
> "chapters/*.tex",
> "figures/*",
> ]) + [":company_style"],
> )
>
> latex(
> name = "my_report",
> srcs = [":my_report_lib"],
> main = "my_report.tex",
> )
the code in my bazel-latex repository to get rid of latex_library()
and use filegroup() instead, but it doesn't seem to be a drop-in
replacement. Right now my latex_library() function adds support for
include paths in a relatively crude way (by merging tarballs). Having
such include paths is going to be necessary for making classes in
external repositories work; LaTeX's \ProvideClass{} gets fairly
annoyed if use "external/.../some_document_class"; the name has to
match exactly with what's specified in the class.
> Filegroups also work nicely when generating various versions of a graphic
> to be included in slides (see the slightly hackish but very useful
> ps_family in the aehlig_rules_latex.shar attached in my original post)
> which I used, e.g., for my FrOSCon 2017 talk[1].
blog post you wrote. Will take a look!
> These filegroups are
> also the reason why I had to change the genrule to an action, as I had
> to write the parameter file as well (with filegroups all the way down,
> the command line quickly grew longer than the maximal allowed length).
> In the long run, my plan is to add a proper latex_fragment rule that
> basically is just a bag of sources, organised in a provider.
upwards. I searched/tested around a bit and the TEXINPUTS=...
environment variable can be used for that purpose, it seems.
But maybe I'm overthinking things. Maybe we shouldn't even allow
setting arbitrary include paths. Maybe it makes sense to set TEXINPUTS
to just external/*, so that you can only include paths relative to the
root of all workspaces? Will experiment with this.
> The reason
> is that in the long run, a latex fragment is more that just the tex
> files: there are .bib files that all need to be merged and included, a
> collection of files to be \input before the \begin{document}, e.g.,
> notations, etc. Unfortunately, I didn't have the time to do so yet.
> I'll report back if and when I found the time to do so. In the mean
> time thanks for picking up my draft of latex rules for bazel!
test anything, etc.
--
Ed Schouten <e...@nuxi.nl>
Nuxi, 's-Hertogenbosch, the Netherlands