LuaTeX
Randy Yates
forthcoming remake of the old TeX/PDFTeX.
While I realize that my opinion doesn't really mean much, I was hoping
to open a discussion on this.
While I am happy to see a new form of (La)TeX (I've been a user since
1988) developed that weds it to a more conventional programming
language, from what I've seen I am very disappointed to see Lua chosen
as that language.
In my opinion, the ideal marriage would be to Scheme, or possibly some
other LISP variation. The combination of Scheme's text processing power
and formal language capabilities with (La)TeX's typesetting abilities
would yield a typesetting of almost unimaginable power.
Am I the only one who has such thoughts?
--
% Randy Yates % "With time with what you've learned,
%% Fuquay-Varina, NC % they'll kiss the ground you walk
%%% 919-577-9882 % upon."
%%%% <ya...@ieee.org> % '21st Century Man', *Time*, ELO
http://www.digitalsignallabs.com
Joost Kremers
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
> and formal language capabilities with (La)TeX's typesetting abilities
> would yield a typesetting of almost unimaginable power.
>
> Am I the only one who has such thoughts?
no:
<http://www.fractalconcept.com/asp/y-Bz/sdataQGxjQerZQy9vDM==/sdataQucgleWmCuSG9eWI$Nmw>
of course, a lisp-like language is probably not going to be accepted as
easily by many people, because of its alien looks (all those
parentheses...)
personally, i tend to agree with you, though. i've often considered
learning a bit of TeX programming, but TeX as a programming language seems
more complex than necessary. now if i could use lisp instead... ;-)
--
Joost Kremers joostk...@yahoo.com
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)
Timothy Murphy
> The people who write the FAQ feel that LuaTeX is now the major
> forthcoming remake of the old TeX/PDFTeX.
>
> While I realize that my opinion doesn't really mean much, I was hoping
> to open a discussion on this.
>
> While I am happy to see a new form of (La)TeX (I've been a user since
> 1988) developed that weds it to a more conventional programming
> language, from what I've seen I am very disappointed to see Lua chosen
> as that language.
>
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
> and formal language capabilities with (La)TeX's typesetting abilities
> would yield a typesetting of almost unimaginable power.
>
> Am I the only one who has such thoughts?
I'm afraid my thoughts are much more radical,
or should I say conservative?
The only "improvement" to LaTeX that has actually increased
the number of users has been the introduction of PDFTeX and beamer.
People don't use LaTeX because it is too complicated,
not because it lacks features they want or need.
If only They would stop saying, "How can we improve LaTeX?",
and start saying, "How can we get more people to use LaTeX?".
--
Timothy Murphy
e-mail: gayleard /at/ eircom.net
tel: +353-86-2336090, +353-1-2842366
s-mail: School of Mathematics, Trinity College Dublin
Joseph Wright
> The people who write the FAQ feel that LuaTeX is now the major
> forthcoming remake of the old TeX/PDFTeX.
>
> While I realize that my opinion doesn't really mean much, I was hoping
> to open a discussion on this.
>
> While I am happy to see a new form of (La)TeX (I've been a user since
> 1988) developed that weds it to a more conventional programming
> language, from what I've seen I am very disappointed to see Lua chosen
> as that language.
>
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
> and formal language capabilities with (La)TeX's typesetting abilities
> would yield a typesetting of almost unimaginable power.
>
> Am I the only one who has such thoughts?
I doubt it :-) Not everyone is convinced that LuaTeX is "the best way
forward", with various different arguments put forward on this. There
are people who don't think this type of integration is a good thing
whatever language is used. At the same time, there are always
arguments for and against almost any programming language (which is
partly why there are some many in the first place). From what I've
looked at, Lua doesn't look like as good a choice as many other
possibilities.
The reality is that the LuaTeX people have actually sat down to write
something. It may not be perfect, but it is being done. They had to
make a decision, and they did so some time ago. Unless someone else is
prepared to do a similar amount of work to come up with an
alternative, then Lua + TeX is what we will have. Somehow, I don't see
this being likely.
--
Joseph Wright
Ulrike Fischer
> The people who write the FAQ feel that LuaTeX is now the major
> forthcoming remake of the old TeX/PDFTeX.
>
> While I realize that my opinion doesn't really mean much, I was hoping
> to open a discussion on this.
>
> While I am happy to see a new form of (La)TeX (I've been a user since
> 1988) developed that weds it to a more conventional programming
> language, from what I've seen I am very disappointed to see Lua chosen
> as that language.
>
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
> and formal language capabilities with (La)TeX's typesetting abilities
> would yield a typesetting of almost unimaginable power.
>
> Am I the only one who has such thoughts?
The luatex team has made its choice. They rejected scheme
explicitly:
"Why not use language X instead of Lua?
We needed a language that matched all of the following criteria:
Freely available
Embeddable integrate within pdfTeX
Very small footprint
Portable
Easy to extend with pdfTeX-specific functionality
Fun to work with
Lua was the first language to match all the criteria. The known
scripting languages tend to be much too large for our use.
Specifically, we have rejected Java, Perl, Python, Ruby, Scheme on
one or more of those criteria."
So I don't see much sense in a discussion about alternatives --
unless you are actually willing to recruit people to develop a
schemetex.
--
Ulrike Fischer
Joseph Wright
> Randy Yates wrote:
> > The people who write the FAQ feel that LuaTeX is now the major
> > forthcoming remake of the old TeX/PDFTeX.
>
> > While I realize that my opinion doesn't really mean much, I was hoping
> > to open a discussion on this.
>
> > While I am happy to see a new form of (La)TeX (I've been a user since
> > 1988) developed that weds it to a more conventional programming
> > language, from what I've seen I am very disappointed to see Lua chosen
> > as that language.
>
> > In my opinion, the ideal marriage would be to Scheme, or possibly some
> > other LISP variation. The combination of Scheme's text processing power
> > and formal language capabilities with (La)TeX's typesetting abilities
> > would yield a typesetting of almost unimaginable power.
>
> > Am I the only one who has such thoughts?
>
> I'm afraid my thoughts are much more radical,
> or should I say conservative?
>
> The only "improvement" to LaTeX that has actually increased
> the number of users has been the introduction of PDFTeX and beamer.
>
> People don't use LaTeX because it is too complicated,
> not because it lacks features they want or need.
>
> If only They would stop saying, "How can we improve LaTeX?",
> and start saying, "How can we get more people to use LaTeX?".
The two are not one and the same. LuaTeX is an engine, and by
providing better programming tools hopefully makes it easier to
produce good formats, packages, modules or whatever.
On the other hand, LaTeX is built on TeX as a format. Currently, there
are lots of design decisions in LaTeX that I don't think most people
would make if starting today. ConTeXt demonstrates this: it makes some
much better calls than LaTeX2e does (at least at the kernel level).
Now, having better programming tools should make it easier to produce
better TeX formats. The ConTeXt people are trying to do this in
ConTeXt Mark IV. Depending on your point of view, the LaTeX situation
is either (1) better as it has been stable (at the base level) for
many years or (2) not great as it has not evolved. An easier-to-use
LaTeX is a good idea, and if LuaTeX helps that to happen I'm all for
it.
Making LaTeX easier to use *is* improving it, in my opinion. But that
means moving on from "take LaTeX2e and load ...", which will be a big
step if and when it happens. LuaTeX might help this happen directly,
or might have a more indirect influence.
--
Joseph Wright
David Kastrup
> On May 20, 11:09�am, Randy Yates <ya...@ieee.org> wrote:
>
>> In my opinion, the ideal marriage would be to Scheme, or possibly
>> some other LISP variation. The combination of Scheme's text
>> processing power and formal language capabilities with (La)TeX's
>> typesetting abilities would yield a typesetting of almost
>> unimaginable power.
>>
>> Am I the only one who has such thoughts?
>
> I doubt it :-) Not everyone is convinced that LuaTeX is "the best way
> forward", with various different arguments put forward on this. There
> are people who don't think this type of integration is a good thing
> whatever language is used.
>
> At the same time, there are always arguments for and against almost
> any programming language (which is partly why there are some many in
> the first place). From what I've looked at, Lua doesn't look like as
> good a choice as many other possibilities.
I'm quite versed in wagonloads of programming languages. And Lua is, in
my opinion, quite about the best choice available.
a) small. A good programmer can learn the basics of the language in two
days. And there is not much more than the basics in the language.
That means that it won't scare people away for which language X is
_not_ something they already love.
b) coroutines. Really, really essential if you want to program with
passing stuff from and back a TeX engine or other encapsulated
objects
c) can wrap foreign data structures into its own naturally
d) feature complete
e) efficient
Here are some drawbacks:
a) only byte strings
b) no native list type, and TeX is about processing lists
> The reality is that the LuaTeX people have actually sat down to write
> something. It may not be perfect, but it is being done.
And one of the reasons is that the language does not throw stumbling
blocks in your way like others.
> They had to make a decision, and they did so some time ago.
_After_ considering a lot of choices.
> Unless someone else is prepared to do a similar amount of work to come
> up with an alternative, then Lua + TeX is what we will have.
I disagree with that assessment. Until someone else is prepared to do
quite a _dissimilar_ amount, namely _wagonloads_ more of work, this is
what we'll get. A major point of picking Lua was that it makes going
forward quite easy.
I don't necessarily agree with the sanity of putting callbacks and
surgical bypasses into TeX the way the LuaTeX team does: I think it
would be more straightforward to recode the control flow of TeX in Lua
rather than tap into the resulting and consumed data streams.
Maybe we'll see that at some point of time.
--
David Kastrup
UKTUG FAQ: <URL:http://www.tex.ac.uk/cgi-bin/texfaq2html>
Jim
> So I don't see much sense in a discussion about alternatives --
> unless you are actually willing to recruit people to develop a
> schemetex.
I have the idea that much of the LuaTeX development involves putting
in hooks, whereby the person programming in Lua can step into the
engine's process. If I have that right, once the hooks are in, and in
the right places, wouldn't it then be much easier to bring in another
language?
Jim
Timothy Murphy
>> I'm afraid my thoughts are much more radical,
>> or should I say conservative?
>>
>> The only "improvement" to LaTeX that has actually increased
>> the number of users has been the introduction of PDFTeX and beamer.
>>
>> People don't use LaTeX because it is too complicated,
>> not because it lacks features they want or need.
>>
>> If only They would stop saying, "How can we improve LaTeX?",
>> and start saying, "How can we get more people to use LaTeX?".
>
> The two are not one and the same. LuaTeX is an engine, and by
> providing better programming tools hopefully makes it easier to
> produce good formats, packages, modules or whatever.
>
> On the other hand, LaTeX is built on TeX as a format. Currently, there
> are lots of design decisions in LaTeX that I don't think most people
> would make if starting today. ConTeXt demonstrates this: it makes some
> much better calls than LaTeX2e does (at least at the kernel level).
I'm sure you are right.
But for 99% of users, LaTeX is perfectly acceptable as is.
It does what virtually all users want.
LaTeX2e was an enormous improvement,
because LaTeX209 did not work properly.
But LaTeX2e works perfectly,
and does what nearly everybody wants.
To me, current developments are like people re-designing the bicycle,
trying to work out the most efficient ratio
between the radii of the front and back wheels.
All very interesting, but it won't increase the number of people
riding bicycles.
The reasons why people don't ride bicycles are much more basic,
and so are the reasons why people don't use LaTeX.
Ulrike Fischer
> The only "improvement" to LaTeX that has actually increased
> the number of users has been the introduction of PDFTeX and beamer.
You forgot xetex.
> People don't use LaTeX because it is too complicated,
> not because it lacks features they want or need.
One thing that is complicated in LaTeX is its font handling and the
handling of non-western scripts. And this is because TeX lacks the
feature to use fonts directly and always needs a tfm, and because it
lacks the feature to handle utf8-files natively.
> If only They would stop saying, "How can we improve LaTeX?",
> and start saying, "How can we get more people to use LaTeX?".
E.g. by simplifying the use of fonts (system fonts, open types fonts
...) and by adding native utf8 support to the engine.
--
Ulrike Fischer
Joseph Wright
> looked at, Lua doesn't look like as good a choice as many other
looks
Sorry about that!
--
Joseph Wright
Joseph Wright
> I'm quite versed in wagonloads of programming languages. And Lua is, in
> my opinion, quite about the best choice available.
David, thanks for the careful analysis. I meant to say that "Lua *is*
as good a choice as others", not the other way around (asks self: hop
did that happen?).
> > They had to make a decision, and they did so some time ago.
>
> _After_ considering a lot of choices.
Yes, I should have said exactly that.
> > Unless someone else is prepared to do a similar amount of work to come
> > up with an alternative, then Lua + TeX is what we will have.
>
> I disagree with that assessment. Until someone else is prepared to do
> quite a _dissimilar_ amount, namely _wagonloads_ more of work, this is
> what we'll get. A major point of picking Lua was that it makes going
> forward quite easy.
I was under the impression that LuaTeX was already quite a bit of
work! Fair point that other approaches could be harder. You obviously
know more about this than me.
> I don't necessarily agree with the sanity of putting callbacks and
> surgical bypasses into TeX the way the LuaTeX team does: I think it
> would be more straightforward to recode the control flow of TeX in Lua
> rather than tap into the resulting and consumed data streams.
I'm still trying to see how all of this will work: I'm not really sure
I quite see how it all fits together, just yet.
--
Joseph Wright
David Kastrup
> Joseph Wright wrote:
>
>>> I'm afraid my thoughts are much more radical,
>>> or should I say conservative?
>>>
>>> The only "improvement" to LaTeX that has actually increased
>>> the number of users has been the introduction of PDFTeX and beamer.
>>>
>>> People don't use LaTeX because it is too complicated,
>>> not because it lacks features they want or need.
>>>
>>> If only They would stop saying, "How can we improve LaTeX?",
>>> and start saying, "How can we get more people to use LaTeX?".
>>
>> The two are not one and the same. LuaTeX is an engine, and by
>> providing better programming tools hopefully makes it easier to
>> produce good formats, packages, modules or whatever.
>>
>> On the other hand, LaTeX is built on TeX as a format. Currently, there
>> are lots of design decisions in LaTeX that I don't think most people
>> would make if starting today. ConTeXt demonstrates this: it makes some
>> much better calls than LaTeX2e does (at least at the kernel level).
>
> I'm sure you are right.
> But for 99% of users, LaTeX is perfectly acceptable as is.
> It does what virtually all users want.
No, it doesn't. It does not work with international fonts reasonably.
It does not produce log output in proper utf-8, even if the input is
proper utf-8. It does not deal at all well with using external fonts.
It can't do hyphenation worth squat if you need more than 256 characters
at once. It does not produce PDF/A. It does not produce tagged PDF,
and "accessibility" issues actually rule out untagged PDF on more or
less all government sites increasingly. It does not work for magazines
with more than 2 columns (and not well with 2 columns either) since it
is unprepared to place figures. It does not provide for two-column
display math. If you put together packages like line numbering and
verbatim modes, things explode in so many ways that it is not funny.
It is not straightforward to program and things fall apart at a number
of weird and totally incomprehensible places if you stress it too much.
> LaTeX2e was an enormous improvement,
> because LaTeX209 did not work properly.
> But LaTeX2e works perfectly,
You are living in fairy dream land.
> The reasons why people don't ride bicycles are much more basic,
> and so are the reasons why people don't use LaTeX.
The reason people don't use LaTeX are that its use is complex, and you
get lots of incomprehensible problems without obvious workarounds.
There is only a limited number of solution for various typesetting
problems, and quite a few incompatible with one another.
Joseph Wright
>
> I'm sure you are right.
> But for 99% of users, LaTeX is perfectly acceptable as is.
> It does what virtually all users want.
>
> LaTeX2e was an enormous improvement,
> because LaTeX209 did not work properly.
> But LaTeX2e works perfectly,
> and does what nearly everybody wants.
This depends, of course, on how you view "works perfectly". My problem
with LaTeX2e is that there is too much that requires either (1) add
ons or (2) altering internals. For example, to use the standard
classes to produce anything at all reasonable, I always end up loading
a lot of packages (fontenc, lmodern, whatever fonts I actually want,
float, caption, ...). I don't think that is really a good thing: the
kernel should deal with all of this. That said, if you know what you
are doing then most things can be done (there are some complex output
routine problems, but I'm not sure they are TeX-solvable at all).
> To me, current developments are like people re-designing the bicycle,
> trying to work out the most efficient ratio
> between the radii of the front and back wheels.
>
> All very interesting, but it won't increase the number of people
> riding bicycles.
> The reasons why people don't ride bicycles are much more basic,
> and so are the reasons why people don't use LaTeX.
You are quite right that most people will not switch to LaTeX due to a
few minor alterations. The fundamental idea is never going to change.
However, I don't think that means LaTeX cannot be improved for people
who do choose to use it. That means looking at user ideas, but also
trying to make the programming more accessible.
--
Joseph Wright
Leo
> This depends, of course, on how you view "works perfectly". My problem
> with LaTeX2e is that there is too much that requires either (1) add
> ons or (2) altering internals. For example, to use the standard
> classes to produce anything at all reasonable, I always end up loading
> a lot of packages (fontenc, lmodern, whatever fonts I actually want,
> float, caption, ...). I don't think that is really a good thing: the
> kernel should deal with all of this. That said, if you know what you
> are doing then most things can be done (there are some complex output
> routine problems, but I'm not sure they are TeX-solvable at all).
Yes, I'd like to see this improved. So the new TeX engine LuaTeX is
welcome though I'm also disappointed by the choise of lua.
BTW, Joseph, I saw your blog regarding adding new primitives to XeTeX.
Will that be easier with LuaTeX?
Best wishes,
--
.: Leo :. [ sdl.web AT gmail.com ] .: I use Emacs :.
Turgut Durduran
> The reason people don't use LaTeX are that its use is complex, and you
> get lots of incomprehensible problems without obvious workarounds.
> There is only a limited number of solution for various typesetting
> problems, and quite a few incompatible with one another.
You are forgetting how so many people are willing to deal with the
horrible mess called MS-Word but complain about LaTeX at the slightest
bit of problem.
While I appreciate the more technical (and certainly orders of
magnitude more knowledgable) approach you bring to these questions, don't
under stress the other elements of this story.
ugdc
Timothy Murphy
>> LaTeX2e was an enormous improvement,
>> because LaTeX209 did not work properly.
>> But LaTeX2e works perfectly,
>
> You are living in fairy dream land.
On the contrary, it is you who are completely out-of-touch
with ordinary would-be LaTeX users,
eg the physics PhD student who is just starting to write his thesis.
He is simply not interested in the issues you list.
He knows what he wants to say, and he just wants to get out
a decent looking document acceptable to his local thesis office.
Sadly, he is going to write his thesis in MS Word,
because that is recommended to him by the graduate students office,
and it is perfectly straightforward.
So the document may not be perfect, but that is not his top priority.
Simon Spiegel
> David Kastrup wrote:
>
>>> LaTeX2e was an enormous improvement,
>>> because LaTeX209 did not work properly.
>>> But LaTeX2e works perfectly,
>>
>> You are living in fairy dream land.
>
> On the contrary, it is you who are completely out-of-touch
> with ordinary would-be LaTeX users,
> eg the physics PhD student who is just starting to write his thesis.
In a previous post you talked about "for 99% of LaTeX users", now you
talk about "physics PhD students". Please do accept, that theses are
not 99% of LaTeX users. There are people who use LaTeX for all kind of
purposes; I have never used LaTeX to print a single formula (and I
probably never will). I use LaTeX because of its (or better biblatex's)
bibliograpic features and because it creates beautiful documents. And I
would love if I could use these abilities with any font I want (and
still make use of the microtypographic features).
> He is simply not interested in the issues you list.
> He knows what he wants to say, and he just wants to get out
> a decent looking document acceptable to his local thesis office.
>
> Sadly, he is going to write his thesis in MS Word,
> because that is recommended to him by the graduate students office,
> and it is perfectly straightforward.
> So the document may not be perfect, but that is not his top priority.
If he's happy with MS Word, why even considering him in this
discussion? There's no point in forcing a software on someone who's
happy with what he alrady has.
simon
Simon Spiegel
> The people who write the FAQ feel that LuaTeX is now the major
> forthcoming remake of the old TeX/PDFTeX.
>
> While I realize that my opinion doesn't really mean much, I was hoping
> to open a discussion on this.
>
> While I am happy to see a new form of (La)TeX (I've been a user since
> 1988) developed that weds it to a more conventional programming
> language, from what I've seen I am very disappointed to see Lua chosen
> as that language.
>
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
> and formal language capabilities with (La)TeX's typesetting abilities
> would yield a typesetting of almost unimaginable power.
>
> Am I the only one who has such thoughts?
I don't have any technical expertise, but the discussion seems quite
moot to me. If you don't like Lua and want another language instead, be
free to implement it your own. The people behind LuaTeX have made their
decision and are actually delievering something. I don't see how this
kind of ex post criticism should be constructive.
simon
Randy Yates
> Joseph Wright <joseph...@morningstar2.co.uk> writes:
>
>> On May 20, 11:09�am, Randy Yates <ya...@ieee.org> wrote:
>>
>>> In my opinion, the ideal marriage would be to Scheme, or possibly
>>> some other LISP variation. The combination of Scheme's text
>>> processing power and formal language capabilities with (La)TeX's
>>> typesetting abilities would yield a typesetting of almost
>>> unimaginable power.
>>>
>>> Am I the only one who has such thoughts?
>>
>> I doubt it :-) Not everyone is convinced that LuaTeX is "the best way
>> forward", with various different arguments put forward on this. There
>> are people who don't think this type of integration is a good thing
>> whatever language is used.
>>
>> At the same time, there are always arguments for and against almost
>> any programming language (which is partly why there are some many in
>> the first place). From what I've looked at, Lua doesn't look like as
>> good a choice as many other possibilities.
>
> I'm quite versed in wagonloads of programming languages. And Lua is, in
> my opinion, quite about the best choice available.
>
> a) small. A good programmer can learn the basics of the language in two
> days. And there is not much more than the basics in the language.
> That means that it won't scare people away for which language X is
> _not_ something they already love.
Hi David,
A good programmer should be able to learn Scheme in not too much longer.
It's really not that bad. See, e.g.,
http://groups.csail.mit.edu/mac/projects/scheme/documentation/scheme_toc.html
> b) coroutines. Really, really essential if you want to program with
> passing stuff from and back a TeX engine or other encapsulated
> objects
It seems that coroutines are essentially subroutines with native
muli-threading. I'm not sure how the applications you mention require
this. And, while perhaps not native, you could certainly build such
things into a scheme-based implementation.
> c) can wrap foreign data structures into its own naturally
I don't see anything being more suitable than Scheme to this task.
> d) feature complete
Yep, I think Scheme's feature complete. Should be after 30-some-odd
years.
> e) efficient
Storage-wise, or execution-wise? I'm not sure why either one is
really required in this day and age - at least not for the task
of typesetting.
> Here are some drawbacks:
>
> a) only byte strings
You mean no unicode handling?
> b) no native list type, and TeX is about processing lists
Well, yeah...
>> The reality is that the LuaTeX people have actually sat down to write
>> something. It may not be perfect, but it is being done.
And they have my respect. I meant to respectfully open a discussion
on alternate directions.
>> Unless someone else is prepared to do a similar amount of work to come
>> up with an alternative, then Lua + TeX is what we will have.
>
> I disagree with that assessment. Until someone else is prepared to do
> quite a _dissimilar_ amount, namely _wagonloads_ more of work, this is
> what we'll get. A major point of picking Lua was that it makes going
> forward quite easy.
Most people have found Scheme, and LISP in general, to be an excellent
tool for solving very complex problems in a reasonable amount of code,
due largely to Scheme's ability to design into itself important pieces
necessary for a specific problem.
> I don't necessarily agree with the sanity of putting callbacks and
> surgical bypasses into TeX the way the LuaTeX team does: I think it
> would be more straightforward to recode the control flow of TeX in Lua
> rather than tap into the resulting and consumed data streams.
Sounds like it to me too, but I'm ignorant of such things within
(La)TeX.
--
% Randy Yates % "How's life on earth?
%% Fuquay-Varina, NC % ... What is it worth?"
%%% 919-577-9882 % 'Mission (A World Record)',
%%%% <ya...@ieee.org> % *A New World Record*, ELO
http://www.digitalsignallabs.com
Randy Yates
So that's it then? People should just move from their own idea of
what's good into the implementation phase without discussing it
with the community?
I don't see how discouraging discussion can be constructive.
--
% Randy Yates % "Bird, on the wing,
%% Fuquay-Varina, NC % goes floating by
%%% 919-577-9882 % but there's a teardrop in his eye..."
%%%% <ya...@ieee.org> % 'One Summer Dream', *Face The Music*, ELO
http://www.digitalsignallabs.com
Simon Spiegel
So what do you expect? That, thanks to your input, Hans and Taco
suddenly realize that they have backed the wrong horse and start new
from scratch, but this time with Scheme? In other words: What is the
point of this discussion? Coming up with a theoretical perfect
successor to pdftex? This has been done many times before. Personally,
I prefer the not so perfect but actually existing LuaTeX.
simon
Randy Yates
> So what do you expect?
To explore.
> In other words: What is the point of this discussion?
To explore.
--
Turgut Durduran
>> Sadly, he is going to write his thesis in MS Word,
>> because that is recommended to him by the graduate students office,
>> and it is perfectly straightforward.
>> So the document may not be perfect, but that is not his top priority.
>
> If he's happy with MS Word, why even considering him in this
> discussion? There's no point in forcing a software on someone who's
> happy with what he alrady has.
This is a discussion amongst the converts and it repeats itself here quite
often. I am not sure what the point is but it amuses me so I participate
with my two cents knowledge (no offense to OP here, that was a different
question/issue).
FWIW, I know 10s of people in natural sciences and medicine who are *not*
happy with MS-Word but are either forced (one way or another) to a degree
to use it or do not know better so they stick to it.
Do we have to care to evangelize to convert them? There are self-fish
advantages for me if more people used it due increased
development of packages, increased availability of templates, easier
document sharing and collaboration etc but that is about it. I also have
an educational/academic interest. I believe I benefited a lot from LaTeX
use and I owe that mostly to few researchers who introduced LaTeX to me
when I first started doing research so I want to "educate" the people I
supervise. I see LaTeX as an important tool in scientific research. As a
group leader, I see that LaTeX could increase the group's efficiency and
help us to get more done.
Turgut
Simon Spiegel
Look, I 'converted' several people to LaTeX, so I definitely don't have
a problem with that per se. But Timothy was referring to some
theoretical graduate student whose "document may not be perfect, but
that is not his top priority". Well, if his priorities are that clear,
why should he bother to use a different tool? I'm more than willing to
help people who are not happy with Word (and I know there are many of
them), but why should someone whose top priorities are taken care of by
Word use something different? I will certainly not force LaTeX on
someone who actually thinks that he already uses the right tool. It's
just a waste of time.
simon
Timothy Murphy
>>>> LaTeX2e was an enormous improvement,
>>>> because LaTeX209 did not work properly.
>>>> But LaTeX2e works perfectly,
>>>
>>> You are living in fairy dream land.
>>
>> On the contrary, it is you who are completely out-of-touch
>> with ordinary would-be LaTeX users,
>> eg the physics PhD student who is just starting to write his thesis.
>
> In a previous post you talked about "for 99% of LaTeX users", now you
> talk about "physics PhD students".
Eg means "for example".
(Also I said "student" not students".)
> Please do accept, that theses are
> not 99% of LaTeX users. There are people who use LaTeX for all kind of
> purposes; I have never used LaTeX to print a single formula (and I
> probably never will). I use LaTeX because of its (or better biblatex's)
> bibliograpic features and because it creates beautiful documents. And I
> would love if I could use these abilities with any font I want (and
> still make use of the microtypographic features).
I would say that 99% or thereabouts of LaTeX users
use it because of its ability to print mathematics.
>> Sadly, he is going to write his thesis in MS Word,
>> because that is recommended to him by the graduate students office,
>> and it is perfectly straightforward.
>> So the document may not be perfect, but that is not his top priority.
>
> If he's happy with MS Word, why even considering him in this
> discussion? There's no point in forcing a software on someone who's
> happy with what he alrady has.
I didn't say he was _happy_ to use MS Word.
He would probably prefer to use LaTeX,
because he sees mathematical documents printed with LaTeX,
and it is evident to him that this produces nicer documents.
But that is not his priority.
He spends 95% of his working life carrying out experiments,
and 5% of the time writing up his work.
He is not going to spend hours trying to work out
how to write his thesis in LaTeX.
Joseph Wright
> BTW, Joseph, I saw your blog regarding adding new primitives to XeTeX.
> Will that be easier with LuaTeX?
XeTeX is still TeX, in the sense that ultimately everything has to
expand into a primitive or a literal (numbers, letters, etc.). So to
add new things to XeTeX you have to add new primitives.
On the other hand, LuaTeX has the break-out to Lua, which can do
things that cannot be expressed as TeX primitives. So adding ideas to
LuaTeX should be possible "after the fact": not in the binary but at
run-time piece. That said, LuaTeX will include many of the primitives
that have been added to pdfTeX beyond e-TeX (although not necessarily
with identical names). So one way or another LuaTeX should provide a
superset of the primitives functionality available in pdfTeX.
--
Joseph Wright
Turgut Durduran
It is bit late if it has gotten to the dissertation writing point but
earlier on, please see my point about (1) education, (2) efficiency.
ugdc
Joseph Wright
> So that's it then? People should just move from their own idea of
> what's good into the implementation phase without discussing it
> with the community?
>
> I don't see how discouraging discussion can be constructive.
Sometimes there is a need to just do rather than discuss, particularly
if there is a danger that discussion is open-ended and never ends. On
the other hand, I'm all in favour of asking people what they need
before doing it: makes it much more likely that things will be useful.
I'm not sure it is necessarily an easy balance.
The thing with the LuaTeX situation is that it has already been going
for some time (the roadmap mentions a first demo in 2005). There has
been discussion, which is why Lua was decided on, as I understand it.
I'd point out that I wondered myself why they'd picked Lua, when I
first looked at what was going on. There are lots of other "obvious"
choices, but as Ulrike has already mentioned these were considered and
decided against. If we look at other projects, for example XeTeX,
sometimes success needs a small team (maybe just one person) with not
everything decided by committee.
--
Joseph Wright
Simon Spiegel
>> Please do accept, that theses are
>> not 99% of LaTeX users. There are people who use LaTeX for all kind of
>> purposes; I have never used LaTeX to print a single formula (and I
>> probably never will). I use LaTeX because of its (or better biblatex's)
>> bibliograpic features and because it creates beautiful documents. And I
>> would love if I could use these abilities with any font I want (and
>> still make use of the microtypographic features).
>
> I would say that 99% or thereabouts of LaTeX users
> use it because of its ability to print mathematics.
That's a statement I would challenge; but since we both don't have any
statistics ...
>
>>> Sadly, he is going to write his thesis in MS Word,
>>> because that is recommended to him by the graduate students office,
>>> and it is perfectly straightforward.
>>> So the document may not be perfect, but that is not his top priority.
>>
>> If he's happy with MS Word, why even considering him in this
>> discussion? There's no point in forcing a software on someone who's
>> happy with what he alrady has.
>
> I didn't say he was _happy_ to use MS Word.
> He would probably prefer to use LaTeX,
> because he sees mathematical documents printed with LaTeX,
> and it is evident to him that this produces nicer documents.
>
> But that is not his priority.
> He spends 95% of his working life carrying out experiments,
> and 5% of the time writing up his work.
> He is not going to spend hours trying to work out
> how to write his thesis in LaTeX.
I'm not really sure what you're trying to get at. Would it be nice if
it was easier to use LaTeX? Well, it certainly would. But then we
arrive at a fundamental problem: IMO one of the great strenghts of
LaTeX is the huge number of packages, but at the same time, that's also
its big weakness. Learning to write a simple document in LaTeX can be
done in two hours (certainly in less time than figuring out Word's
style system). It's the finetuning where you will spend a lot of time.
But that's something that will always be true. Sure you could provide
better standard documenclasses, sure some things could be simplified,
but people will always want to finetune things and that will always
need extra knowledge.
There certainly are areas where things could be dramatically
simplified, but most of them are actually in the list David posted in
this thread – like font handling.
simon
Dominik Waßenhoven
> Simon Spiegel wrote:
>> There are people who use LaTeX for all kind of purposes; I have never
>> used LaTeX to print a single formula (and I probably never will). I
>> use LaTeX because of its (or better biblatex's) bibliograpic
>> features and because it creates beautiful documents. And I would
>> love if I could use these abilities with any font I want (and still
>> make use of the microtypographic features).
>
> I would say that 99% or thereabouts of LaTeX users
> use it because of its ability to print mathematics.
I don't think so. I also never used math up to now and probably will
never do so. I use LaTeX for similar reasons as Simon, and I now of
quite a bunch of people in the humanities who use LaTeX and don't use it
to print mathematics. I think this is one of the most prominent problems
for the acceptance of LaTeX in the humanities: A lot of people think of
LaTeX as a fancy tool for natural scientists or only for mathematicians.
(A student said exactly that to me just yesterday.)
Regards,
Dominik.-
--
UK-TeX-FAQ: http://www.tex.ac.uk/cgi-bin/texfaq2html
minimal example: http://www.minimalbeispiel.de/mini-en.html
biblatex styles: http://biblatex.dominik-wassenhoven.de/?en
Robin Fairbairns
>The people who write the FAQ feel that LuaTeX is now the major
>forthcoming remake of the old TeX/PDFTeX.
>
>While I realize that my opinion doesn't really mean much, I was hoping
>to open a discussion on this.
remember that, with my extended illness, it's a *very* long time since
the last update of the faq. at the time i wrote that, i'd had
essentially no contact with the luatex people. one thing is now clear
to me: that luatex _isn't_ a replacement for pdftex, it's another
mechanism for doing typesetting work, and providing most things that
pdftex does.
as for scheme (or whatever -- i've not programmed lisp since the
1960s) i don't believe anyone is working on a tex replacement using
such a language. the nearest, istm, is ant, which is programmed in
another functional language -- ocaml[*]. ant may or may not be active
-- the last release was december 2007 ... but then it's almost a year
since the faq was last released, and that's definitely being worked
on.
[*] i really know nothing useful about ocaml, but i understand it to
be an fl.
--
Robin Fairbairns, Cambridge
Joseph Wright
wrote:
> But then we
> arrive at a fundamental problem: IMO one of the great strenghts of
> LaTeX is the huge number of packages, but at the same time, that's also
> its big weakness. Learning to write a simple document in LaTeX can be
> done in two hours (certainly in less time than figuring out Word's
> style system). It's the finetuning where you will spend a lot of time.
> But that's something that will always be true. Sure you could provide
> better standard documenclasses, sure some things could be simplified,
> but people will always want to finetune things and that will always
> need extra knowledge.
>
> There certainly are areas where things could be dramatically
> simplified, but most of them are actually in the list David posted in
> this thread – like font handling.
The problems don't divide up neatly, but there are some general
classes:
- Things related to the engine, such as system font use. That is down
to the engine in use, but I can imagine having a higher-level font
interface (similar to fontspec) in the kernel and "degrading
gracefully" if pdfTeX is begin used and it's not possible (directly).
- Things that really should be in the kernel: page geometry, basic
float creation, etc.
UTF8 input goes somewhere here: the kernel should provide support if
the engine does not.
- Things that should be supported but not perhaps as part of the
default kernel. I'd suggest reading external data files (CSV, tab
sep.), other "data manipulation" stuff. Quite a lot of things are
potentially in this category.
- Layout material. The current kernel mixes layout and function too
much, making it hard to separate out.
- Genuine add-on functions for specialists: where the line falls for
this depends very much on point of view.
In all cases, improving the way the kernel handles things at the user
interface layer would be beneficial. There are also a lot of overlaps
in the current LaTeX packages, and at least to some extent an improved
kernel would be helpful. David has already mentioned the issues that
can arise when there are package conflicts.
--
Joseph Wright
Timothy Murphy
Dominik Waßenhoven wrote:
>> I would say that 99% or thereabouts of LaTeX users
>> use it because of its ability to print mathematics.
>
> I don't think so. I also never used math up to now and probably will
> never do so. I use LaTeX for similar reasons as Simon, and I now of
> quite a bunch of people in the humanities who use LaTeX and don't use it
> to print mathematics. I think this is one of the most prominent problems
> for the acceptance of LaTeX in the humanities: A lot of people think of
> LaTeX as a fancy tool for natural scientists or only for mathematicians.
> (A student said exactly that to me just yesterday.)
I wish what you think were true.
Sadly, I fear your student has a better grasp of reality.
Unfortunately, TeX has never really found a "market" outside mathematics,
as Knuth certainly hoped it would.
There are of course many honourable exceptions,
of which you seem to be one.
In my view, one of the reasons for this
is precisely that too little time and too few brain-cells
have been expended on "selling" LaTeX,
and too much of both on improving it.
Zeljko Vrba
>
> In my opinion, the ideal marriage would be to Scheme, or possibly some
> other LISP variation. The combination of Scheme's text processing power
>
you have also the power of macros.
Zeljko Vrba
>
> a) only byte strings
>
c) Only a single numeric type (int or double, configurable at compile-time
of the interpreter)
T3X
wrote:
> In all cases, improving the way the kernel handles things at the user
> interface layer would be beneficial. There are also a lot of overlaps
> in the current LaTeX packages, and at least to some extent an improved
> kernel would be helpful.
Sorting out the "package mess" is useful but by itself doesn't solve
the fundamental problem of LaTeX -- there is no definition of what
LaTeX really is. Or to be more precise the concepts embodied by LaTeX
are accessible only to human beings -- TeX engine is oblivious to the
document structure and nothing else is able to grok LaTeX sources in
their full glory. This turns LaTeX niche into LaTeX ghetto.
Cheers,
Toemk
Joseph Wright
> In my view, one of the reasons for this
> is precisely that too little time and too few brain-cells
> have been expended on "selling" LaTeX,
> and too much of both on improving it.
What would you see as good ways of "selling" LaTeX? As a member of the
UK-TUG Committee and the LaTeX team, this type of thing is of some interest!
--
Joseph Wright
David Kastrup
> David Kastrup wrote:
>
>>> LaTeX2e was an enormous improvement,
>>> because LaTeX209 did not work properly.
>>> But LaTeX2e works perfectly,
>>
>> You are living in fairy dream land.
>
> On the contrary, it is you who are completely out-of-touch
> with ordinary would-be LaTeX users,
> eg the physics PhD student who is just starting to write his thesis.
> He is simply not interested in the issues you list.
That must be the reason why this group and other lists are full of
problems and solution involving \makeatletter.
> He knows what he wants to say, and he just wants to get out
> a decent looking document acceptable to his local thesis office.
>
> Sadly, he is going to write his thesis in MS Word,
> because that is recommended to him by the graduate students office,
> and it is perfectly straightforward.
So that's your defense of "LaTeX2e works perfectly"? You _really_ are
living in fairy dream land.
--
David Kastrup
UKTUG FAQ: <URL:http://www.tex.ac.uk/cgi-bin/texfaq2html>
Joseph Wright
> Sorting out the "package mess" is useful but by itself doesn't solve
> the fundamental problem of LaTeX -- there is no definition of what
> LaTeX really is. Or to be more precise the concepts embodied by LaTeX
> are accessible only to human beings -- TeX engine is oblivious to the
> document structure and nothing else is able to grok LaTeX sources in
> their full glory. This turns LaTeX niche into LaTeX ghetto.
This issue has not escaped the notice of the LaTeX team: of course, good
ideas on how to solve this type of issue are always welcome.
--
Joseph Wright
David Kastrup
> Simon Spiegel wrote:
>
>> Please do accept, that theses are not 99% of LaTeX users. There are
>> people who use LaTeX for all kind of purposes; I have never used
>> LaTeX to print a single formula (and I probably never will). I use
>> LaTeX because of its (or better biblatex's) bibliograpic features and
>> because it creates beautiful documents. And I would love if I could
>> use these abilities with any font I want (and still make use of the
>> microtypographic features).
>
> I would say that 99% or thereabouts of LaTeX users
> use it because of its ability to print mathematics.
You would.
> But that is not his priority.
> He spends 95% of his working life carrying out experiments,
> and 5% of the time writing up his work.
> He is not going to spend hours trying to work out
> how to write his thesis in LaTeX.
Why would he blow the 95% he invested by blundering on the 5%?
David Kastrup
> Unfortunately, TeX has never really found a "market" outside mathematics,
> as Knuth certainly hoped it would.
> There are of course many honourable exceptions,
> of which you seem to be one.
Since TeX is the honorable exception within math, there is nothing
special about it being so outside of math.
> In my view, one of the reasons for this
> is precisely that too little time and too few brain-cells
> have been expended on "selling" LaTeX,
> and too much of both on improving it.
It is expensive to sell what does not work.
David Kastrup
Turgut Durduran
FWIW, thinking exclusively of natural sciences and high level research
work;
1- A concentrated effort to "kindly" ask journals to start, continue and
expand offering LaTeX style files. Even provide cheap expertise for
development/maintainance of them. Provide forums for users to support
each other.
2- A similar effort to influence major funding agencies. Start with those
that are funded by governments, i.e. tax-payers. Even provide cheap
expertise for devepment/maintainance of them. Provide forums for users
to support each other.
3- A similar effort to create "official" support for dissertation styles
internationally. Start at major research institutions and continously
expand. Even provide cheap expertise for devepment/maintainance of
them. Provide forums for users to support each other.
4- Offer to deliver presentations/workshops at institutions. This could be
accomplished free or for really cheap if local TUGs pick it up.
A lot of this is now based on goodwill of a few users, the style files in
many journals are antiquated and no longer supported, etc.
There is a chick-flick on TV and this is few cents worth of brainstorming.
Turgut
Turgut Durduran
>> is precisely that too little time and too few brain-cells
>> have been expended on "selling" LaTeX,
>> and too much of both on improving it.
>
> It is expensive to sell what does not work.
yeah but one could success, see M$.
Turgut
Turgut Durduran
PS: I am fully aware that some of this is already done and these are
things discussed a multitude of times.
Timothy Murphy
> Timothy Murphy <gayl...@eircom.net> writes:
>
>> Unfortunately, TeX has never really found a "market" outside mathematics,
>> as Knuth certainly hoped it would.
>> There are of course many honourable exceptions,
>> of which you seem to be one.
>
> Since TeX is the honorable exception within math, there is nothing
> special about it being so outside of math.
That just isn't so, in my experience.
Virtually all research mathematics, at least that I see,
is written in LaTeX.
It would be rare indeed for a group theorist to write a paper in MS Word.
But for some reason TeX does not seem to have made any great inroads
even into physics and computer science, again in my experience.
If anything there are fewer physicists using LaTeX
than there were 10 years ago.
>> In my view, one of the reasons for this
>> is precisely that too little time and too few brain-cells
>> have been expended on "selling" LaTeX,
>> and too much of both on improving it.
>
> It is expensive to sell what does not work.
As I said, LaTeX works perfectly well for straightforward documents
(at least in English).
There are some problems including graphics, but not serious ones.
That is not the issue.
It simply isn't true that physicists, say, don't use LaTeX
because of the esoteric problems you have raised.
These just don't enter the equation.
And the advent of LuaTeX will not have the slightest effect,
good or bad, in this context.
John Haltiwanger
or may not have anything to say about this that folks haven't heard
before. However I felt like piping up because a) I don't plan on
invoking math at all and am still quite excited about TeX, and b) I
don't think the "it's too hard" argument holds water.
For instance, I think the quickest and most visceral demonstration of
LaTeX's power is the standard letter template. Here is something that
quite literally takes way too much work in Word that is only coloring
by numbers in LaTeX. Not only is it easy, it looks _brilliant_. Very
much a 'tractor app'. Now, since people are not used to thinking in
terms of a markup language, it could be argued that some sort of GUI
interface with text boxes that ask for input to fill in what the
template expects would be a helpful tool for evangelism. However, many
many people are not only capable of comprehending mark up but may in
the end decide they quite prefer it.
Also, who on earth thinks having Lisp be the language of the scripting
engine will make things _easier_ or "more likely to be adopted" ? Lisp
has never moved beyond its own academic birthing grounds and likely
never will. Lua on the other hand is definitely a "language to know"
right now, all the more so because it is used by so many applications.
But either way, here is an "embedded Lisp" interpreter for Lua
http://blog.davber.com/2006/09/07/embedded-lisp-via-lua/ . He says
that he didn't know Lua before writing the interpreter and then claims
that that represents the beauty of the Lisp language. Maybe so, but I
think it would be nonsensical to argue that it doesn't also represent
the beauty of Lua. No way someone is going to embed Lua in Lisp (or
write a bridge between the two for that matter) the first night they
encounter the ()'s. Or I'll eat my Ruby interpreter.
T3X
wrote:
I'm glad to hear that. I follow your blog closely and other sources on
LaTeX3 I can find but I couldn't find any information on that subject
beyond some general ideas from a few years back. In particular, I
think that interoperability with XML-based formats and reuse of
content in LaTeX2e format are the two aspects that will determine the
fate of LaTeX3. Having a discussion on such topics would warrant a
separate post/thread.
Cheers,
Tomek
Timothy Murphy
>> He spends 95% of his working life carrying out experiments,
>> and 5% of the time writing up his work.
>> He is not going to spend hours trying to work out
>> how to write his thesis in LaTeX.
>
> Why would he blow the 95% he invested by blundering on the 5%?
It obviously isn't a _blunder_ to produce a thesis in MS Word,
otherwise these intelligent youngsters would have discovered that.
But the final document certainly does not look as good as it could.
Also, I'm pretty sure that it would in fact have been easier
in most cases to have used LaTeX, not only for formulae
but also for tables.
I don't really understand your attitude.
Do you regard it as an act of God that most physicists
and computer scientists don't write their theses in LaTeX?
Timothy Murphy
interest!You seem to think the same is true of mathematicians.
One suggestion which I have made every year for the last 10 years
is that there should be an official thesis class,
with help to institutions to produce an additional package
to cover local variants.
If in fact, as I am told every year, the report class
is almost sufficient then the work involved would be miniscule.
Joris
This must be the most successful thread on comp.text.tex ever. I'm
surprised to see that physicists use word. In economics there are
some word users, but most people use LaTeX, or rather Scientific
Workplace :-( . J.
John Haltiwanger
relationship does LaTeX have to luaTeX? From an outsider perspective
it looks like ConTeXt is the only thing using luaTeX anyway.
Harald Hanche-Olsen
> On May 20, 10:19�pm, Joseph Wright <joseph.wri...@morningstar2.co.uk>
> wrote:
>> T3X wrote:
>> This issue has not escaped the notice of the LaTeX team: of course, good
>> ideas on how to solve this type of issue are always welcome.
>
> I'm glad to hear that. I follow your blog closely
The LaTeX team has a blog?
--
* Harald Hanche-Olsen <URL:http://www.math.ntnu.no/~hanche/>
- It is undesirable to believe a proposition
when there is no ground whatsoever for supposing it is true.
-- Bertrand Russell
Joseph Wright
LuaTeX is an "engine", like pdfTeX or XeTeX. So it provides certain
functions, which might or might not get used. The end-user can use them
directly, but this is really not helping most people as they don't want
to do this type of low-level work. So in the main it is down to either
the TeX format (ConTeXt, LaTeX, ...) or an add on (LaTeX package,
ConTeXt module, ...) to take advantage of them.
In the ConTeXt case, the entire format is being re-written as "Mark IV",
which will use a lot of Lua. This of course means things fundamentally
change compared to early versions of ConTeXt, and ties it to a single
engine. In the LaTeX case, the format (latex.ltx) is written to work on
plain TeX, no extensions at all. That is not about to change:
essentially, LaTeX2e will *never* be "updated" in that sense. The
current LaTeX3 plans don't envisage requiring LuaTeX, although I'd hope
that some low-level support will be included if LaTeX3 ever becomes a
reality. There will, though, be LaTeX packages that use Lua: I'm sure
that at some point there will be a fontspec-like package for LuaTeX, for
example.
--
Joseph Wright
Zeljko Vrba
> Zeljko Vrba <mordor...@fly.srk.fer.hr> writes:
>
>> On 2009-05-20, David Kastrup <d...@gnu.org> wrote:
>>>
>>> Here are some drawbacks:
>>>
>>> a) only byte strings
>>> b) no native list type, and TeX is about processing lists
>>>
>>
>> c) Only a single numeric type (int or double, configurable at compile-time
>> of the interpreter)
>
> Where is the drawback with that?
>
Because feature sets of floating-point and integer arithmetic are only
overlapping, i.e., neither is a subset of the other. Maybe it does not
affect LuaTeX (TeX already does use fixed-point arithmetic via integers,
right?), but I always have that nagging feeling when I use floating-point
where integers would suffice.
Actually, in comparison with other languages, Scheme (as a language, not as
engine for TeX) has one strong point going for it, and that's its numeric
tower.
Jonathan Fine
> The people who write the FAQ feel that LuaTeX is now the major
> forthcoming remake of the old TeX/PDFTeX.
>
> While I realize that my opinion doesn't really mean much, I was hoping
> to open a discussion on this.
Thank you, Randy, for starting this thread. There's been much
discussion of the merits of various languages, the needs and nature of
the TeX community, and some of the problems and opportunities we have.
Here I'm starting a subthread on an important technical issue, which on
how to combine a typesetting engine (such as TeX) with a dynamic
language (such as Lua).
There are two basic ways to do this, embed and extend. For people new
to this question, the difference between the two is sometimes hard to
understand.
Here are some URLs that explain the difference:
http://twistedmatrix.com/users/glyph/rant/extendit.html
http://docs.python.org/extending/
http://docs.python.org/extending/extending.html
http://docs.python.org/extending/embedding.html
These happen to be all Python URLs. I'd appreciate seeing other, for
different languages.
This is not simply a Python issue. Here is a quote from "Beginning Lua
Programming" (pub: Wrox)
===
If you're developing a library that will perform services for an
application, you'll be extending Lua. If, however, you're developing an
application, you've got a choice between embedding Lua and extending it.
===
If you understand this statement, you have got the gist of the
difference between embed and extend, and also one of the most important
practical implications.
The LuaTeX project has decided to embed (the Lua interpreter into an
program derived from TeX) and so will not provide a library that a
standard Lua interpreter (and hence a Lua script) can use.
Once the interface to a typesetting engine (or a database, for that
matter) has wrapped to provide it as an extension for one programming
language, it is fairly easy to do the same for another.
I think it very important that we move towards a typesetting engine that
functions as a service rather than as an application. This service can,
of course, be presented as an application.
Creating a service is taking the extend branch, which allows for both
service and application. The LuaTeX project has taken the embed branch,
which can only produce an application.
In short, for me the most important question is not the choice of a
language but whether or not the software will provide typesetting as a
service for other applications to use.
Thank you for reading to the end of this long and somewhat technical post.
--
Jonathan
Joseph Wright
> > I'm glad to hear that. I follow your blog closely
>
> The LaTeX team has a blog?
No, *I* do: www.texdev.net. The focus is "whatever catches my
interest", so there is some LaTeX3 stuff, things about my own packages
(siunitx and achemso mainly, but others come up), plus anything else I
have time to write about.
--
Joseph Wright
Joseph Wright
> One suggestion which I have made every year for the last 10 years
> is that there should be an official thesis class,
> with help to institutions to produce an additional package
> to cover local variants.
> If in fact, as I am told every year, the report class
> is almost sufficient then the work involved would be miniscule.
At the LaTeX project level, I doubt this will happen. I've already
said that the LaTeX kernel is not flexible enough, in my opinion.
Trying to create a usable thesis class just using the kernel is not a
realistic goal: "report" is simply not up to the job. To get good
results you need to use an extended class (KOMA, memoir, etc.) or lots
of packages. Not only that, but there is the whole "no major changes
to LaTeX2e" point: I don't think there is any chance of adding new
classes to the base LaTeX distribution.
There are of course some very good pieces of general thesis advice,
for example:
- http://theoval.sys.uea.ac.uk/~nlct/latex/thesis/
- http://www.sunilpatel.co.uk/thesistemplate.php
I agree that this is not a great situation, but I just don't see a
change happening unless the LaTeX team can actually deliver on LaTeX3.
--
Joseph Wright
Turgut Durduran
> This must be the most successful thread on comp.text.tex ever. I'm
> surprised to see that physicists use word. In economics there are
> some word users, but most people use LaTeX, or rather Scientific
> Workplace :-( . J.
There are a *lot* of physicists who use word.
turgut
David Kastrup
> Creating a service is taking the extend branch, which allows for both
> service and application. The LuaTeX project has taken the embed
> branch, which can only produce an application.
I think this is a somewhat limited view: Lua has the concept of
coroutines, and LuaTeX includes the Coco patches which make those work
across "C function calls". As a result, you can present the programmer
with a programming model where the programmer-visible control flow
happens in Lua and TeX just gets fed scraps of material to execute,
returning expressions evaluated within TeX (like box dimensions) to the
Lua engine.
So the programming paradigm you use in the end is actually not really
dictated by the _underlying_ architecture: the language offers all the
low-level features to conceptually turn the system inside out and treat
it as if it were working from the other side.
Basically, you can have separate objects with separate control flow,
talking by message passing. While this made Lua quite attractive for
programming games (like Worlds of Warcraft), getting both a service and
an application view out of the same code is an obvious advantage for TeX
encapsulation as well.
> In short, for me the most important question is not the choice of a
> language but whether or not the software will provide typesetting as a
> service for other applications to use.
With Lua, you get your pick either way.
T3X
wrote:
> current LaTeX3 plans don't envisage requiring LuaTeX, although I'd hope
> that some low-level support will be included if LaTeX3 ever becomes a
> reality.
That's a pity. This decision has a hidden cost, namely it forces
package writers to stick with TeX macros for programming. Of course
some of them might decide to make LuaTeX mandatory (and I hope that
many will do). Still, making LuaTeX mandatory for LaTeX3 would
simplify the picture a lot for package writers and end users alike. I
wonder how much benefit is in supporting all of those other engines.
How many users would reject LaTeX3 specifically because of LuaTeX
requirement?
Cheers,
Tomek
T3X
> Jonathan Fine <jf...@pytex.org> writes:
> > Creating a service is taking the extend branch, which allows for both
> > service and application. The LuaTeX project has taken the embed
> > branch, which can only produce an application.
>
> I think this is a somewhat limited view: Lua has the concept of
> coroutines, and LuaTeX includes the Coco patches which make those work
> across "C function calls". As a result, you can present the programmer
> with a programming model where the programmer-visible control flow
> happens in Lua and TeX just gets fed scraps of material to execute,
> returning expressions evaluated within TeX (like box dimensions) to the
> Lua engine.
>
> So the programming paradigm you use in the end is actually not really
> dictated by the _underlying_ architecture: the language offers all the
> low-level features to conceptually turn the system inside out and treat
> it as if it were working from the other side.
Still, having TeX engine cleanly separated as a library could make it
interesting for a lot of projects. Are there any practical reasons
behind the desicion of not going that route?
Cheers,
Tomek
Robin Fairbairns
>On May 20, 6:44 am, Ulrike Fischer <ne...@nililand.de> wrote:
>> So I don't see much sense in a discussion about alternatives --
>> unless you are actually willing to recruit people to develop a
>> schemetex.
>
>I have the idea that much of the LuaTeX development involves putting
>in hooks, whereby the person programming in Lua can step into the
>engine's process. If I have that right, once the hooks are in, and in
>the right places, wouldn't it then be much easier to bring in another
>language?
sure, but remember you've still to write a whole bunch of things:
a) the integration of tex data structures
b) the translation of the lua that's already been written by taco
c) the redevlopment of the tex code that's already been written to
work with luatex
since (context aside) item c is rather small, now would be the time to
start.
oh ... and remember we already have a schemetex (which prepares scheme
for typesetting with latex, or some such thing ;-)
--
Robin Fairbairns, Cambridge
Jonathan Fine
> Jonathan Fine <jf...@pytex.org> writes:
>
>> Creating a service is taking the extend branch, which allows for both
>> service and application. The LuaTeX project has taken the embed
>> branch, which can only produce an application.
>
> I think this is a somewhat limited view: Lua has the concept of
> coroutines, and LuaTeX includes the Coco patches which make those work
> across "C function calls". As a result, you can present the programmer
> with a programming model where the programmer-visible control flow
> happens in Lua and TeX just gets fed scraps of material to execute,
> returning expressions evaluated within TeX (like box dimensions) to the
> Lua engine.
Thank you for your comments, David. However, I do not see the relevance
of what you are saying.
> So the programming paradigm you use in the end is actually not really
> dictated by the _underlying_ architecture: the language offers all the
> low-level features to conceptually turn the system inside out and treat
> it as if it were working from the other side.
I am now beginning to understand, perhaps, what you are saying. I think
you are saying that, at least in some circumstance, you can write Lua
code that uses the typesetting engine which will work with both the
embed and extend implementations.
I agree with this statement, and take it further. There are aspects of
LuaTeX that rely on Lua being embedded in the typesetting engine. I
don't think I'd miss these features if they weren't there. But this is
going off-topic.
> Basically, you can have separate objects with separate control flow,
> talking by message passing. While this made Lua quite attractive for
> programming games (like Worlds of Warcraft), getting both a service and
> an application view out of the same code is an obvious advantage for TeX
> encapsulation as well.
I don't need a typsettting library that that has "object with separate
control flow, talking by message passing" and consider it to be an
unneccesary complication.
--
Jonathan
Jonathan Fine
[discussion of coroutines snipped]
>> In short, for me the most important question is not the choice of a
>> language but whether or not the software will provide typesetting as a
>> service for other applications to use.
>
> With Lua, you get your pick either way.
This statement, David, seems to contradict something I wrote in my
original post, namely: This is not simply a Python issue. Here is a
quote from "Beginning Lua Programming" (pub: Wrox)
===
If you're developing a library that will perform services for an
application, you'll be extending Lua. If, however, you're developing an
application, you've got a choice between embedding Lua and extending it.
===
Are you saying that because Lua has coroutines the authors of the book
are mistaken about the choice between embed and extend?
--
Jonathan
Robin Fairbairns
>Am Wed, 20 May 2009 11:37:45 +0100 schrieb Timothy Murphy:
>
>> The only "improvement" to LaTeX that has actually increased
>> the number of users has been the introduction of PDFTeX and beamer.
>
>You forgot xetex.
>
>> People don't use LaTeX because it is too complicated,
>> not because it lacks features they want or need.
>
>One thing that is complicated in LaTeX is its font handling and the
>handling of non-western scripts. And this is because TeX lacks the
>feature to use fonts directly and always needs a tfm, and because it
>lacks the feature to handle utf8-files natively.
>
>> If only They would stop saying, "How can we improve LaTeX?",
>> and start saying, "How can we get more people to use LaTeX?".
>
>E.g. by simplifying the use of fonts (system fonts, open types fonts
>...) and by adding native utf8 support to the engine.
total agreement.
however, this doesn't satisfy TM: he (afaict) is only interested in
anglophone physicists or mathematicians.
what's more, i don't understand how we (his capitalised "they") should
be trying to interest more people in latex. i, for one, am definitely
not on the latex team because of my teaching, or of my public speaking
credentials. should we be recruiting trainers to the project? --
seems a worthy cause, but i really don't think we're the people to do
it.
--
Robin Fairbairns, Cambridge
David Kastrup
No, I am saying that because of the coroutine model, one can present the
programmer with a different view rather painlessly. The choice still
has to be made in some manner or other, but for the person _using_ the
resulting system, the difference can be wrapped to a degree where he
does not need to bother about the difference.
As an illustration, I put here a file called "ls-u.lua", use it like
lua ls-u.lua .
It implements an iterator called dirtree returning an iterator function
that returns for each call another directory entry from a recursive tree
traversal. Now the point is that the function used for getting one
directory entry has no concept of recursion. So one just programs a
normal, recursive directory traversal that produces the elements for
processing one by one. But the processing one-by-one has its control
flow and loops. So there are actually _two_ systems running in
parallel, and each system is programmed as if it is the one in control,
because its dynamic state (local variables and call stack) are
maintained separately and independently.
This conceptual inversion of control flow can also be used in LuaTeX to
put a Lua program conceptually in control of TeX as its typesetting
engine.
That the underlying implementation has been coded the other way round is
an implementation detail that only affects a small, fixed amount of
code.
require "lfs"
function dirtree(dirs)
local function yieldtree(dir)
for entry in lfs.dir(dir) do
if entry ~= "." and entry ~= ".." then
entry=dir.."/"..entry
local attr=lfs.attributes(entry)
coroutine.yield(entry,attr)
if attr.mode == "directory" then
yieldtree(entry)
end
end
end
end
return coroutine.wrap(function()
for i,dir in ipairs(dirs) do
assert(dir and dir ~= "", "directory parameter is missing or empty")
if string.sub(dir, -1) == "/" then
dir=string.sub(dir, 1, -2)
end
yieldtree(dir)
end end)
end
for name,attr in dirtree(arg) do
print(attr.mode,name)
end
Joseph Wright
> That's a pity. This decision has a hidden cost, namely it forces
> package writers to stick with TeX macros for programming. Of course
> some of them might decide to make LuaTeX mandatory (and I hope that
> many will do). Still, making LuaTeX mandatory for LaTeX3 would
> simplify the picture a lot for package writers and end users alike. I
> wonder how much benefit is in supporting all of those other engines.
> How many users would reject LaTeX3 specifically because of LuaTeX
> requirement?
This all depends on how you see LaTeX3 being used. For end users, with
their own TeX system on their own PC, it's very likely that the
majority will have LuaTeX available well before any kind of LaTeX3 can
possibly be written. So in this context the reply to your question is
"not many people", I think. However, if you look at multi-user systems
or publishers set ups, the picture is very different. These tend to be
upgraded only very slowly, and so there would be the danger of
preventing LaTeX3 use in these environments. I'm not sure that is such
a great idea.
There are some other issues. First, the number of Lua experts in the
TeX community is rather less than the number of TeX experts, I
suspect. Obviously the ConTeXt team now have a lot of Lua experience,
but that is not true more widely. This is bound to impact on what is
seen as "the best way forward". Of course, the only way to learn is to
do, and I'm not saying that expertise won't arise over time.
Second, if the LaTeX team decided to require Lua in the way ConTeXt
Mark IV does (a large amount of basic work in Lua not TeX), then most
of the code that has been written to date for LaTeX3 would be
obsolete. You can argue that this is not a bad thing (I'm sure there
will be someone who does), but unless there is then real progress on a
Lua-based LaTeX it doesn't really achieve anything. At present, I
think the real challenges for LaTeX3 are probably not at the
programming level per se and more things like user interface and back-
compatibility. Lua doesn't make those issues disappear instantly.
I've argued that a sensible way forward is to work with pdfTeX now,
and get some of the basic issues addressed. I'd then look to moving to
requiring XeTeX or LuaTeX at some point so that system fonts and UTF-8
input can be used directly, and perhaps later to LuaTeX only. In that
sense, I'd suggest a progression of LaTeX3 versions, rather than
imagining that you can deliver LaTeX3 as a fixed product and not
change the basic requirements of years. In this model, the TeX low-
level coding could be replaced in later versions by Lua, with the
interface staying the same. The problem with LaTeX2e is that there is
no interface as such: you have to know the low-level code detail to
alter anything. (This is why making any significant changes to LaTeX2e
basically kills back-compatibility entirely.) My thinking here is that
if you look at things like biblatex or LuaTeX itself, you can get a
working system that people can test and work with well before you fix
all of the design and finalise the requirements.
Timothy Murphy
>>> If only They would stop saying, "How can we improve LaTeX?",
>>> and start saying, "How can we get more people to use LaTeX?".
>>
>>E.g. by simplifying the use of fonts (system fonts, open types fonts
>>...) and by adding native utf8 support to the engine.
>
> total agreement.
>
> however, this doesn't satisfy TM: he (afaict) is only interested in
> anglophone physicists or mathematicians.
First of all may I say that you (or You), Robin,
do more than anyone else to hold high the flag of TeX.
You are indeed the human face of They.
As to myself, I have in fact always been interested in the use of LaTeX
for various foreign languages.
I recall being very impressed at an early TeX meeting
when Knuth introduced a Chinese speaker who sat at
what looked like an organ keyboard, writing Chinese in TeX.
I look back on many wasted hours trying to print Cyrillic and Hebrew.
(Does everybody in Israel still have their own private TeX system?)
I even once updated a LaTeX209 package to print ancient Greek
(which I do not understand) so that I could put on my door,
"Let no man ignorant of geometry enter here."
I'd love to see LaTeX used more by chess-players and lawyers.
I took physics and computer science as examples
because these are the countries nearest to mathematics,
and therefore ripest for colonisation.
The failure of LaTeX to spread there is the most obvious symptom
of the general failure I was lamenting.
Timothy Murphy
>> One suggestion which I have made every year for the last 10 years
>> is that there should be an official thesis class,
>> with help to institutions to produce an additional package
>> to cover local variants.
>> If in fact, as I am told every year, the report class
>> is almost sufficient then the work involved would be miniscule.
>
> At the LaTeX project level, I doubt this will happen. I've already
> said that the LaTeX kernel is not flexible enough, in my opinion.
> Trying to create a usable thesis class just using the kernel is not a
> realistic goal: "report" is simply not up to the job. To get good
> results you need to use an extended class (KOMA, memoir, etc.) or lots
> of packages. Not only that, but there is the whole "no major changes
> to LaTeX2e" point: I don't think there is any chance of adding new
> classes to the base LaTeX distribution.
Sigh.
Some say it is too easy, and therefore not worth doing.
Others say it is too difficult, and therefore not worth trying.
I believe it would be a great selling point,
if you could say to a student starting his thesis,
"Just start with \documentclass[uccd]{thesis}.
You may find you have to add some packages later
for special features you want to include.
But this will do to start with.
Just list your chapters with their titles,
and you are on the way."
Jonathan Fine
> Jonathan Fine <J.F...@open.ac.uk> writes:
>> This statement, David, seems to contradict something I wrote in my
>> original post, namely: This is not simply a Python issue. Here is a
>> quote from "Beginning Lua Programming" (pub: Wrox)
>> ===
>> If you're developing a library that will perform services for an
>> application, you'll be extending Lua. If, however, you're developing
>> an application, you've got a choice between embedding Lua and
>> extending it.
>> ===
>>
>> Are you saying that because Lua has coroutines the authors of the book
>> are mistaken about the choice between embed and extend?
>
> No, I am saying that because of the coroutine model, one can present the
> programmer with a different view rather painlessly. The choice still
> has to be made in some manner or other, but for the person _using_ the
> resulting system, the difference can be wrapped to a degree where he
> does not need to bother about the difference.
[discussion of iterators snipped]
Thank you for the example of how to use "coroutine.yield" in Lua. In
Python there's a keyword "yield" that does something similar.
> This conceptual inversion of control flow can also be used in LuaTeX to
> put a Lua program conceptually in control of TeX as its typesetting
> engine.
When we extend Lua with a typesetting engine, the Lua script is in
charge of the typesetting engine, at least at the beginning.
When we embed Lua in a typesetting engine, the typesetting engine is in
charge, at least at the beginning.
This is stated more clearly than I am able to in one of the URLs I quoted:
http://docs.python.org/extending/embedding.html
which reads, in part
===
So if you are embedding Python, you are providing your own main program.
One of the things this main program has to do is initialize the Python
interpreter. At the very least, you have to call the function
Py_Initialize. There are optional calls to pass command line arguments
to Python. Then later you can call the interpreter from any part of the
application.
===
(By the way, your use of the term "Lua program" is potentially
misleading, but I know you don't mean the running instance of the Lua
interpreter.)
But all this is a side issue. If we extend Lua with a typesetting
engine, we can use the engine to provide services to the standard Lua
interpreter, and thus to any Lua script. (And that script can function
as an application.)
If we embed Lua in a typesetting program then we have an application,
but we cannot provide services to the standard Lua interpreter.
Instead, we have to invoke our special program Lua+engine to access
typesetting.
To clarify the issue, suppose we have a typesetting engine and a
database that we wish to combine with Lua (where combine means embed or
extend).
If we extend Lua, than the standard Lua interpreter has access to (and
is in control of) both the typesetting and database engines.
If we go down the embed route then we are stuck. Do we embed Lua in the
typesetting engine, the database engine, or both. Or do we create a
combined typesetting-database program and embed Lua in that?
I wish to provide typesetting as a service for other programs, such as
Lua, Python, Perl, Ruby and and Scheme to use. I'm fairly certain that
LuaTeX does not provide typesetting as a service.
--
Jonathan
Robin Fairbairns
>Simon Spiegel <si...@removethsi.simifilm.ch> writes:
>>> Am I the only one who has such thoughts?
>>
>> I don't have any technical expertise, but the discussion seems quite
>> moot to me. If you don't like Lua and want another language instead,
>> be free to implement it your own. The people behind LuaTeX have made
>> their decision and are actually delievering something. I don't see how
>> this kind of ex post criticism should be constructive.
>
>So that's it then? People should just move from their own idea of
>what's good into the implementation phase without discussing it
>with the community?
that would have been bad, had it actually happened. however, there's
a thread of papers (in several places) about how the luatex idea
arose, and there have been presentations at several conferences (some
of which you can see, to some extent, on the net.
i've stopped going to conferences now, but there was a session on
extensions at the last meeting i went to (eurotex at pont a mousson)
with a lively exchange of views.
>I don't see how discouraging discussion can be constructive.
of course not. and all simon claimed was that discussing what the
luatex people _ought_ to have done isn't going to change anything
_now_. he didn't (and the people now working on luatex didn't)
discourage discussion of their model of the world before they started
implementing.
ex post facto discussion is an interesting academic exercise, and some
academics (e.g., counterfactual historians) make an amusing display
with it.
real progress, istm, comes from those who acknowledge what's there,
and discuss how to proceed, or what first principles to make a clean
start from.
--
Robin Fairbairns, Cambridge
David Kastrup
> I even once updated a LaTeX209 package to print ancient Greek
> (which I do not understand) so that I could put on my door,
> "Let no man ignorant of geometry enter here."
I find it ironical to put up a warning against ignorance that the writer
is not able to understand.
> The failure of LaTeX to spread there is the most obvious symptom
> of the general failure I was lamenting.
So basically everything else is sour grapes?
Joseph Wright
> Sigh.
> Some say it is too easy, and therefore not worth doing.
> Others say it is too difficult, and therefore not worth trying.
>
> I believe it would be a great selling point,
> if you could say to a student starting his thesis,
> "Just start with \documentclass[uccd]{thesis}.
I didn't say it was too easy (it's not if you want to get half-decent
output), nor that it was too difficult (although writing a class
without loading extra packages and maintaining inter-operability might
be a little work!). I was simply saying that it is not going to happen
with LaTeX2e: there aren't going to be any changes to the base part of
LaTeX from a user point of view (leaving open the possibility of
adding things to allow interaction with, for example, LuaTeX).
--
Joseph Wright
John Collins
>
> But for some reason TeX does not seem to have made any great inroads
> even into physics and computer science, again in my experience.
> If anything there are fewer physicists using LaTeX
> than there were 10 years ago.
In the areas of physics I am familiar with, the opposite is true. LaTeX
has close to 100% penetration -- http://arxiv.org/form/ (In some prominent
fields of physics, almost all papers written appear in this archive, so it
provides a representative sample of papers.)
All the main journals accept LaTeX submissions, and provide appropriate
class files.
John
Turgut Durduran
> In the areas of physics I am familiar with, the opposite is true.
This on itself also tells something. I was at a physics department for ~15
years not too far away from you (PSU vs UPENN). I see the numbers
declining in general.
Turgut
T3X
points.
On May 21, 1:24 pm, Joseph Wright <joseph.wri...@morningstar2.co.uk>
wrote:
> On May 21, 11:03 am, T3X <t34...@googlemail.com> wrote:
> > How many users would reject LaTeX3 specifically because of LuaTeX
> > requirement?
[...]
> majority will have LuaTeX available well before any kind of LaTeX3 can
> possibly be written. So in this context the reply to your question is
> "not many people", I think.
Yes, that is my expectation as well.
> However, if you look at multi-user systems
> or publishers set ups, the picture is very different. These tend to be
> upgraded only very slowly, and so there would be the danger of
> preventing LaTeX3 use in these environments. I'm not sure that is such
> a great idea.
This argument doesn't compute for me at all. If anything, I would
expect just the opposite trend: publishers adopting LuaTeX rather than
LaTeX3.
LuaTeX is almost a drop in replacement for pdf(e)tex with the added
value of more flexible and controled typsetting. This might be very
attractive for some publishers.
OTOH, LaTeX3 means supporting additional source format in parallel
with those currently in use (nobody is going to drop support for
LaTeX2e in favour of LaTeX3 any time soon). There will have to be some
very good incentive to do that. The only one that comes to my mind is
better integration into XML based workflow leading to increased level
of automation of the whole submission-to-publication process. It's all
about the economics.
For example, let's say that you can validate LaTeX3 sources against
certain set of requirements. A puplisher house can set up a web
service for submission of papers and do an automated validation of
documents according to its house style and reject any non-compliant
ones. This way the responsibility of getting documents into shape is
shifted from the receiving end to the sending end, which has an
obvious economic incentive.
> Second, if the LaTeX team decided to require Lua in the way ConTeXt
> Mark IV does (a large amount of basic work in Lua not TeX), then most
> of the code that has been written to date for LaTeX3 would be
> obsolete.
That wasn't my point. The requirement of LuaTeX is not about rewriting
LaTeX3 or any other package in Lua at all. It's about laying out
foundations for the future and sending a clear message about how we
want to shape that future. I really don't think that having a similar
split as for eTeX extensions is the best way forward. Many problems
might become much easier if people are not afraid to take adventage of
Lua extensions. Of course, as long as there will be a legitimate
demand to keep the system working with older engines, this will be
supported in one way or another. I just think we shouldn't officially
encourage people to hinge forever onto old ways (and many will unless
the carpet is pulled away from under them).
> Lua-based LaTeX it doesn't really achieve anything. At present, I
> think the real challenges for LaTeX3 are probably not at the
> programming level per se and more things like user interface and back-
> compatibility.
Yes, you're absolutely right. LuaTeX aside, finding the right
solutions to those challenges will make or break LaTeX3, IMO. I would
be very interested (and I'm sure many others as well) to hear more
about the ideas that LaTeX team has in this regard.
> I've argued that a sensible way forward is to work with pdfTeX now,
> and get some of the basic issues addressed. I'd then look to moving to
> requiring XeTeX or LuaTeX at some point so that system fonts and UTF-8
> input can be used directly, and perhaps later to LuaTeX only. In that
> sense, I'd suggest a progression of LaTeX3 versions, rather than
This is a reasonable compromise, not as good as painting a clear
picture from the begining, but certainly acceptable. The only drawback
to this model is that once you accumulate some legacy material, things
are usually not that simple any more.
Cheers,
Tomek
Robin Fairbairns
>I agree with this statement, and take it further. There are aspects of
>LuaTeX that rely on Lua being embedded in the typesetting engine. I
>don't think I'd miss these features if they weren't there. But this is
>going off-topic.
just to pursue this a bit further ... one of the things now
implemented in lua is reading hyphenation patterns; they're not
storeable in a format file any more. standard tex-live setup
therefore doesn't work satisfactorily, and will need to be changed for
a "production" luatex (we're not there yet, though we may be
approaching feature specification stability).
so, on the whole, i doubt you'll be happy with a luatex that omits all
lua-implemented stuff.
[note: i understand taco's reasons for doing this, but i think it's a
bit sad.]
--
Robin Fairbairns, Cambridge
John Haltiwanger
wrote:
So then I'd ask Tim how LuaTeX is actually a diversion of resources
for LaTeX, if LaTeX isn't even troubling itself with LuaTeX atm?
And I'd also ask why LaTeX document classes are a better fit for a
standardized thesis format over a ConTeXt based solution? Is the
problem that LaTeX is more well-documented and thus better for
students (i.e. more familiar/lower entry barrier)? Maybe it's the fact
that Mk IV is not finished yet? Or are there some limitations to
ConTeXt that I am not seeing? Because from my angle it looks like a
more configurable and ultimately sensible TeX to be using/learning at
this point.
Jonathan Fine
> Jonathan Fine <J.F...@open.ac.uk> writes:
>> I agree with this statement, and take it further. There are aspects of
>> LuaTeX that rely on Lua being embedded in the typesetting engine. I
>> don't think I'd miss these features if they weren't there. But this is
>> going off-topic.
>
> just to pursue this a bit further ... one of the things now
> implemented in lua is reading hyphenation patterns; they're not
Thank you for your contribution, Robin. I think you mean LuaTeX here.
Or maybe luatex.
> storeable in a format file any more. standard tex-live setup
> therefore doesn't work satisfactorily, and will need to be changed for
> a "production" luatex (we're not there yet, though we may be
> approaching feature specification stability).
I don't see how this is relevant. The TeX model is that if one sends to
the typesetting engine a string of the form '\patterns{ ..... }' when
the engine is in an appropriate state then hyphenation patterns will be
stored. I'd be surprised if LuaTeX were any different in this respect.
You do seem to be saying that, unlike TeX, LuaTeX does not use format
files (for reloading at high speed material that had been previously
digested).
So far as I can tell, this is a matter of the capabilities of the
typesetting engine, and quite separate from whether the engine is
wrapped to provide an extension of Lua or whether Lua is embedded in the
typesetting engine to provide scripting.
--
Jonathan
Joseph Wright
> This argument doesn't compute for me at all. If anything, I would
> expect just the opposite trend: publishers adopting LuaTeX rather than
> LaTeX3.
I was probably thinking rather the other way around to you. If I send
something to a publisher that needs a TeX file they don't have, I can
include in in my submission and hopefully be lucky. On the other hand,
if they don't have the engine I've relied on, I'm stuck. (I've had
this experience when creating the achemso class. It turned out that
the American Chemical Society servers don't have e-TeX. A pain, as I'd
rather relied on it. There was quite a bit of re-writing.)
> > Second, if the LaTeX team decided to require Lua in the way ConTeXt
> > Mark IV does (a large amount of basic work in Lua not TeX), then most
> > of the code that has been written to date for LaTeX3 would be
> > obsolete.
>
> That wasn't my point. The requirement of LuaTeX is not about rewriting
> LaTeX3 or any other package in Lua at all. It's about laying out
> foundations for the future and sending a clear message about how we
> want to shape that future.
Okay, two things here. First, when I originally suggested planning a
move "beyond pdfTeX" for LaTeX3 on my blog I said something like:
While extensions beyond pdfTeX are not required, then we should aim
to have a system
which works with pdfTeX. However, it should be made clear *from the
start* that at
some point XeTeX/LuaTeX will be required: this will happen when
adding system font
support if not before.
So I was being explicit that I would consider planning requiring a
more flexible engine than pdfTeX.
Second, my point about "making the existing code obsolete" was that if
you are going to require LuaTeX, it really makes sense to do all of
the programming in Lua and reserve TeX purely for typesetting. This is
more-or-less the model the ConTeXt team are using, I think.
> I just think we shouldn't officially
> encourage people to hinge forever onto old ways (and many will unless
> the carpet is pulled away from under them).
This is always a difficult one. One of the things many people like
about LaTeX is that it has been a constant over many years.
> > Lua-based LaTeX it doesn't really achieve anything. At present, I
> > think the real challenges for LaTeX3 are probably not at the
> > programming level per se and more things like user interface and back-
> > compatibility.
>
> Yes, you're absolutely right. LuaTeX aside, finding the right
> solutions to those challenges will make or break LaTeX3, IMO. I would
> be very interested (and I'm sure many others as well) to hear more
> about the ideas that LaTeX team has in this regard.
On back-compatibility it is pretty clear that you can't address the
problems of LaTeX2e and still run documents without changes (in the
sense that almost all real documents make low level changes either
directly or by loading packages which do). The current (very vague)
plan is to have some kind of bail-out system, so that if a file
doesn't start with a LaTeX3-specific macro then everything is handed
over to the 2e kernel.
User syntax is more moot. To quote from the LaTeX3 news letter "A high-
level user syntax needs to be designed ... " (http://www.latex-
project.org/l3news/l3news01.pdf). My feeling is that the basics of a
LaTeX document will stay (\begin{...}, \textbf{...}, \emph{...}, etc.)
but that a lot of control structures will alter.
> > I've argued that a sensible way forward is to work with pdfTeX now,
> > and get some of the basic issues addressed. I'd then look to moving to
> > requiring XeTeX or LuaTeX at some point so that system fonts and UTF-8
> > input can be used directly, and perhaps later to LuaTeX only. In that
> > sense, I'd suggest a progression of LaTeX3 versions, rather than
>
> This is a reasonable compromise, not as good as painting a clear
> picture from the begining, but certainly acceptable. The only drawback
> to this model is that once you accumulate some legacy material, things
> are usually not that simple any more.
As I said earlier, I think there is an argument for this being flagged
well in advance. I'll certainly be arguing for decisions which will
make such a transition easier, so at the user level there should be
nothing which depends on an older engine. For example, I'm very keen
that LaTeX3 assumes UTF-8, and so effectively does
\usepackage[utf8]{inputenc}
if the engine is pdfTeX, and leaves well alone otherwise. Fonts are
the other obvious candidates for issues, but I'd hope a little
imagination can give a single interface which is engine-aware.
Probably something like fontspec, or perhaps:
\SetDocumentFonts{
body = <font name>,
monospaced = <other font name>,
}
with the exact implementation "behind the scenes".
--
Joseph Wright
Zeljko Vrba
>
> Thank you for the example of how to use "coroutine.yield" in Lua. In
> Python there's a keyword "yield" that does something similar.
>
Python's generators are asymmetric.
>
> To clarify the issue, suppose we have a typesetting engine and a
> database that we wish to combine with Lua (where combine means embed or
> extend).
>
> If we extend Lua, than the standard Lua interpreter has access to (and
> is in control of) both the typesetting and database engines.
>
> If we go down the embed route then we are stuck. Do we embed Lua in the
> typesetting engine, the database engine, or both. Or do we create a
> combined typesetting-database program and embed Lua in that?
>
Modern operating systems provide programs with the capability to dynamically
load modules at run-time, so the issue you raise is completely moot. You
embed Lua in TeX, and as soon as Lua code gets control, you can do whatever
you want, including loading an extension module to connect to some database.
Given this, the only difference between "extend" and "embed" is where the main
function is: in some script or in some file. Why does such irrelevant detail
bother you? (Yes, Lua can be embedded, and you could still e.g., start some
TCP server to listen to remote clients and provide a "service" to other
programs.)
Furthermore, Lua has much simpler, smaller and better-documented C API than
Python (Yes, I've tried to work with both some time ago.) That in itself is a
huge plus.
Jonathan Fine
> Modern operating systems provide programs with the capability to dynamically
> load modules at run-time, so the issue you raise is completely moot. You
> embed Lua in TeX, and as soon as Lua code gets control, you can do whatever
> you want, including loading an extension module to connect to some database.
Hello Zeljko. Thank you for this. You almost get the point, but then
at the last minute you swerve away and miss it.
What you suggest works because Lua can load "an extension module to
connect to some database". If Lua scripting for the database was
provided by embedding Lua in a database then you could not load the
extension module.
You find it perfectly reasonable that the database connection is
provided as an extension module for Lua. And so do I.
But when it comes to providing typesetting services you say that embed
is fine, whereas I continue to ask for extend.
What you are in effect is saying is this: if you want to use a
typesetting engine with Lua, you must use a specially prepared Lua
interpreter that has the typesetting engine built in to it.
What I am saying is that I'd like the typesetting engine to by
dynamically loaded, just like the database connection.
> Given this, the only difference between "extend" and "embed" is where the main
> function is: in some script or in some file. Why does such irrelevant detail
> bother you? (Yes, Lua can be embedded, and you could still e.g., start some
> TCP server to listen to remote clients and provide a "service" to other
> programs.)
The one thing I want, and I think you should want it to, is that
typsetting is available as a service to other programs. You can of
course add that capability to a typesetting engine. LuaTeX /does not/
provide that capability (although as you say, it could be added).
Instead, LuaTeX provides a capability which I don't want, which is very
close interaction between Lua scripts and typesetting internals.
[Lua is better than Python at ... snipped]
--
Jonathan
Joseph Wright
wrote:
> So then I'd ask Tim how LuaTeX is actually a diversion of resources
> for LaTeX, if LaTeX isn't even troubling itself with LuaTeX atm?
LaTeX2e will not be using LuaTeX in the kernel, but that doesn't mean
that the LaTeX team are not taking an interest in LuaTeX. As well as
potential LaTeX3 things, there is a need to think about how LuaTeX
will work with LaTeX: there are things to consider. The whole callback
business means that there will need to be LaTeX packages to correctly
manage the LuaTeX interaction.
> And I'd also ask why LaTeX document classes are a better fit for a
> standardized thesis format over a ConTeXt based solution? Is the
> problem that LaTeX is more well-documented and thus better for
> students (i.e. more familiar/lower entry barrier)? Maybe it's the fact
> that Mk IV is not finished yet? Or are there some limitations to
> ConTeXt that I am not seeing? Because from my angle it looks like a
> more configurable and ultimately sensible TeX to be using/learning at
> this point.
ConTeXt is, I've always thought, slightly closer than LaTeX in some
ways to plain TeX. LaTeX aims to give the end user a system that works
without needing to worry too much about layout. That doesn't always
work out, but that is the aim. On the other hand ConTeXt tends to
leave more up to the user. In that sense, ConTeXt is better for
creating individual documents, whereas LaTeX is more about "mass
production". A lot of people using LaTeX for their thesis do so
without wanting to learn more about TeX, typesetting or anything
related. In that sense, LaTeX is a sensible choice *if* someone
provides them with a reasonable document class to start from. That is,
of course, where an "official" thesis class would be a good idea. That
doesn't mean that ConTeXt is not also a good way to go, but probably
requires that someone really does think about things first. Given the
difficulty of finding University-sanctioned LaTeX templates or
classes, this all seems a bit of an unlikely outcome.
You are quite right, though, that more people know LaTeX than ConTeXt.
There is ultimately a "critical mass" issue.
--
Joseph Wright
Turgut Durduran
begining. It is now amongst the top ranked journals in Optics.
They have just placed this warning on their site. Look how ridiculous it
is. They are saying that in order to produce "archival quality XML" , we
should stop using LaTeX or incur delays and lose quality because they need
MS-Word documents to generate XML.
My last paper to Optics Express, before they came up with this statement,
was already problematic, because their LaTeX style files were antique and
no-longer maintained to match with latest guidelines. I had to manually
change/tweak them which made the editors unhappy. And OSA is the leading
optics association and its journals are used by thousands of physicists.
Unfortunately, the community is more or less silent to this. I think both
LaTeX community and physics/optics community should be more vocal and
active about these. I already have received complaints from students whom
I "taught" to use LaTeX for all their work, they regret the switch.
--
http://www.opticsinfobase.org/oe/submit/style/attention.cfm
Optics Express is making significant changes to its production process by
creating archival-quality XML along with the PDF output. XML is the
industry standard for producing and archiving scientific journal articles
and is used in producing all other OSA journals. Having full-text XML will
allow Optics Express to be indexed more accurately and completely in
MEDLINE, PubMed Central, and other databases; it will also allow the
journal to meet its archival obligations and to prepare for new services
such as full-text semantic search and repurposing of content.
In order to prevent delays in production, we ask that authors carefully
adhere to the following new guidelines:
* Word instead of LaTeX. OSA strongly encourages authors to submit
papers in MS Word rather than in LaTeX. All LaTeX submissions will be
converted into Word to enable XML tagging. LaTeX submissions will incur
delays in production and will be subject to the layout limitations of MS
Word.
---
There is more to this story. They can't even deal with different versions
of MS-Word properly:
http://www.opticsinfobase.org/submit/templates/word2007.cfm
Zeljko Vrba
> dynamically loaded, just like the database connection.
>
Python as a procedural language, because Python needs access to DB internals
in order to be able to efficiently execute stored procedures.
When a Python program uses a database *as a service*, it does not load the
database as a module into its address space. Instead, it loads an *interface*
module that uses some protocol to talk to the databae executable.
>
> The one thing I want, and I think you should want it to, is that
> typsetting is available as a service to other programs. You can of
>
Service capability can be independently layered upon luatex (and, indeed,
current tex). From what David told about coroutines, they will allow such
interface to be made, if somebody sees the need for it.
Anyway, IIRC, there have been some threads talking about making TeX available
as a DLL (shared library). Maybe you want to search c.t.t archives for that.
T3X
wrote:
> On May 21, 4:07 pm, T3X <t34...@googlemail.com> wrote:
> if they don't have the engine I've relied on, I'm stuck. (I've had
> this experience when creating the achemso class. It turned out that
> the American Chemical Society servers don't have e-TeX. A pain, as I'd
> rather relied on it. There was quite a bit of re-writing.)
Now change in the above TeX to pdfTeX and eTeX to LuaTeX and that
exactly makes my point. Making some requirements early on can prevent
this kind of problems from happening. I don't think that this is a
critical issue, but nonetheless such re-writes are wasteful.
> Second, my point about "making the existing code obsolete" was that if
> you are going to require LuaTeX, it really makes sense to do all of
> the programming in Lua and reserve TeX purely for typesetting.
Not if you already have loads of stuff coded in TeX. You can always
think of re-writes later, but you are much better off if you prepare
for it early on.
> This is always a difficult one. One of the things many people like
> about LaTeX is that it has been a constant over many years.
Yes, but let's not forget that there are no such expectations of
LaTeX3, i.e changes are expected. This is a one time opportunity that
should not be wasted.
> The current (very vague)
> plan is to have some kind of bail-out system, so that if a file
> doesn't start with a LaTeX3-specific macro then everything is handed
> over to the 2e kernel.
Do you mean that that if you feed LaTeX3 with LaTeX2e document it will
automatically switch to legacy mode? If so, this is not exactly what I
was asking for. Some people have a considerable investment in material
stored in LaTeX2e format. If porting such material to the new format
can not be reasonably automated (to get at least most of it right),
people won't make a switch.
Also going the other way LaTeX3 --> LaTeX2e will be important, so that
you can fallback on the legacy, where the new stuff is not an option
(like submitting a paper to your favourite behind-the-time publisher).
> User syntax is more moot. To quote from the LaTeX3 news letter "A high-
> level user syntax needs to be designed ... " (http://www.latex-
> project.org/l3news/l3news01.pdf). My feeling is that the basics of a
> LaTeX document will stay (\begin{...}, \textbf{...}, \emph{...}, etc.)
> but that a lot of control structures will alter.
The actual syntax is of less interest to me. What really matters is
the semantics. I think that it would be highly beneficial if the new
format is (or ties to be) isomorphic with some well established XML
format (TEI, DocBook) with extensions only where necessary.
There are probably no silver bullets and a lot of tough choices and
compromises will be involved in designing the new format, but the
silence surrounding those issues worries me somewhat.
Cheers,
Tomek
T3X
>
> They have just placed this warning on their site. Look how ridiculous it
> is. They are saying that in order to produce "archival quality XML" , we
> should stop using LaTeX or incur delays and lose quality because they need
> MS-Word documents to generate XML.
This is not as ridiculous as it seems. Despite the claims in this
thread and other palces that LaTeX is oh so wonderful and covers 99%
of users' needs, it has this one unfortunate property -- you can flush
it only one way -- down the TeX engine guts. For a long time this was
sufficient but those days are over. It's too late to fix LaTeX2e but
if we won't manage to come up with a successor to solve that problem,
we better all start looking for a good LaTeX to Word converter.
Cheers,
Tomek
Joseph Wright
> This is not as ridiculous as it seems. Despite the claims in this
> thread and other palces that LaTeX is oh so wonderful and covers 99%
> of users' needs, it has this one unfortunate property -- you can flush
> it only one way -- down the TeX engine guts. For a long time this was
> sufficient but those days are over. It's too late to fix LaTeX2e but
> if we won't manage to come up with a successor to solve that problem,
> we better all start looking for a good LaTeX to Word converter.
At least in my area, I get the impression that the publishers use a set
of scripts to get the paper into their database for typesetting, web
output, etc. So of course they only want one input format. (I also note
that the printed output is typeset by TeX: the irony.)
--
Joseph Wright
Jonathan Fine
> On 2009-05-21, Jonathan Fine <J.F...@open.ac.uk> wrote:
>> What I am saying is that I'd like the typesetting engine to by
>> dynamically loaded, just like the database connection.
>>
> I'm not convinced that this is reasonable. PostgreSQL, for example, *embeds*
> Python as a procedural language, because Python needs access to DB internals
> in order to be able to efficiently execute stored procedures.
I'm sorry, Zeljko. I've searched on postgresql.org, and haven't found
anything that fits what you say. Please would you supply a URL.
> When a Python program uses a database *as a service*, it does not load the
> database as a module into its address space. Instead, it loads an *interface*
> module that uses some protocol to talk to the databae executable.
Yes. The database, for example, provides a socket interface. And then
a Python module (possibly written in C for efficiency) wraps that
interface and makes it available to Python.
[BTW, I think in "load the database" you mean 'load the database
software'. To mean, a database is the data. PostgresSQl, for example,
describes itself as an "object-relational database system". But you and
I know what the other means here.]
>> The one thing I want, and I think you should want it to, is that
>> typsetting is available as a service to other programs. You can of
>>
> Service capability can be independently layered upon luatex (and, indeed,
> current tex).
Neither LuaTeX or TeX provides a socket interface. Under Unix/Linux
named pipes can be used to provide an emulation of this, which is what
the TeX daemon (developed by myself) does.
http://sourceforge.net/projects/texd
The TeX daemon is used to power the MathTran web service:
http://www.mathtran.org
> Anyway, IIRC, there have been some threads talking about making TeX available
> as a DLL (shared library). Maybe you want to search c.t.t archives for that.
I think you'll find that I'm a participant in these threads.
We can take a typesetting engine and make it available as an extension
to a scripting language (such as Lua) or embed in it a scripting
language (such as Lua). Or even, if we wish, do both. Or even do
something else, such as embed Java and give make it available as a
Python extension.
We have choices to make between embed and extend (or choose both).
Embed is not the same as extend, and will have differences. This I
think you will agree with.
Having understood the difference between embed and extend, which I think
we do, the next step is to understand the implications for software
such as LuaTeX.
Thank you, Zeljko, for discussing this with me.
--
Jonathan
Joseph Wright
>> This is always a difficult one. One of the things many people like
>> about LaTeX is that it has been a constant over many years.
>
> Yes, but let's not forget that there are no such expectations of
> LaTeX3, i.e changes are expected. This is a one time opportunity that
> should not be wasted.
I'd like to agree, but I'm not sure everyone sees things in the same
way. Some members of the LaTeX community *really* value stability.
>> The current (very vague)
>> plan is to have some kind of bail-out system, so that if a file
>> doesn't start with a LaTeX3-specific macro then everything is handed
>> over to the 2e kernel.
>
> Do you mean that that if you feed LaTeX3 with LaTeX2e document it will
> automatically switch to legacy mode? If so, this is not exactly what I
> was asking for. Some people have a considerable investment in material
> stored in LaTeX2e format. If porting such material to the new format
> can not be reasonably automated (to get at least most of it right),
> people won't make a switch.
This is indeed a problem. The issue is that a lot of "serious" LaTeX2e
documents include sections of the type:
\makeatletter
% Alter some internal function
\makeatother
Given the point that the entire kernel needs rewriting, these will
basically all break. On the other hand, things like:
\usepackage[options]{some-package}
may well be okay, as long as there is some way to "translate" what the
package does into LaTeX3.
> Also going the other way LaTeX3 --> LaTeX2e will be important, so that
> you can fallback on the legacy, where the new stuff is not an option
> (like submitting a paper to your favourite behind-the-time publisher).
Assuming that the default syntax stays similar (which looks likely),
then we can perhaps imagine some LaTeX2e package which "covers the gaps"
and something like:
\begin{legacy-support}
Code here
\end{legacy-support}
which is ignored by the LaTeX3 kernel and defined by my hypothetical
"covers the gaps" package to simply run the code in LaTeX2e.
> What really matters is
> the semantics. I think that it would be highly beneficial if the new
> format is (or ties to be) isomorphic with some well established XML
> format (TEI, DocBook) with extensions only where necessary.
The idea, as I understand it, is that at a low level LaTeX3 should be
independent of the input set up. For example, a lot of the current
internals rely on a certain number of optional arguments, which then
ties you to LaTeX syntax. The idea of xparse (part of the current
experimental LaTeX3) is that all internal macros have a fixed number of
arguments with defined meaning. You then use xparse as "glue" to bind
these internal functions to user input. So the idea is that the
internals can then cope with different input. If you've got ideas, I'm
sure the team will be happy to discuss them (perhaps a separate thread
here or on the LaTeX-L mailing list).
>
> There are probably no silver bullets and a lot of tough choices and
> compromises will be involved in designing the new format, but the
> silence surrounding those issues worries me somewhat.
As it says in the LaTeX3 Newsletter "Momentum is again starting to build
behind the LaTeX3 project". Part of that is trying to get some publicity
and discussion, which has been rather limited for a while. As I said,
input on these things is definitely welcome.
--
Joseph Wright
John Haltiwanger
> We can take a typesetting engine and make it available as an extension
> to a scripting language (such as Lua) or embed in it a scripting
> language (such as Lua). Or even, if we wish, do both. Or even do
> something else, such as embed Java and give make it available as a
> Python extension.
I think it would be sensible to leverage the power of the Parrot VM if
a bridge were to be built. The ability of Parrot languages to (be made
able to) share not only loadable modules but runtime objects would be
quite useful.
For instance I just read "The Nature of Lisp" and have a completely
different appreciation for the language than I did before this thread
started. I definitely see how it could be useful in relation to TeX. I
still think Lua is a better embedded language, but an embedded Lua-to-
Parrot bridge should allow anyone who wants to interface with LuaTeX
from another language to do so. Obviously Parrot is not there quite
yet, but momentum is building.
Zeljko Vrba
>> I'm not convinced that this is reasonable. PostgreSQL, for example, *embeds*
>> Python as a procedural language, because Python needs access to DB internals
>> in order to be able to efficiently execute stored procedures.
>
> I'm sorry, Zeljko. I've searched on postgresql.org, and haven't found
> anything that fits what you say. Please would you supply a URL.
>
Stored procedures are stored in the database itself, and the database does
not spawn an external interpreter to run them. Thus, the only other
possibility to execute stored procedures written in python is to embed the
interpreter in the database.
Also: http://www.zope.org/Members/pupq/zope_in_pg
Quote from the page: "PostgreSQL can use PL/Python, an embedded Python
interpreter that lets you execute Python functions in the database."
>
> Having understood the difference between embed and extend, which I think
> we do, the next step is to understand the implications for software
> such as LuaTeX.
>
Well, I personally have no interest to push towards the "service" direction.
I want a high-quality, stable application for typesetting.
Turgut Durduran
exactly. I have seen that in a clinical journal when at the last step I
got back a TeX marked-up version from them.
If they can go Word -> XML LaTeX -> Word -> XML, I think with reasonable
amount of effort they can probably go LaTeX --> XML too. Look how much
they restricted the word formatting, they could have restricted LaTeX too.
Furthermore, this particular example , Optics Express , requires
print-ready submissions. I am yet to see a nice looking one done with
MS-Word in 10s of reviews I receive from them.
turgut
Jonathan Fine
>> Please would you supply a URL.
>>
> http://www.postgresql.org/docs/8.3/interactive/plpython.html
>
> Stored procedures are stored in the database itself, and the database does
> not spawn an external interpreter to run them.
Thank you, Zeljko, for the URL. So we have here an example where both
embed and extend are available. This will be very helpful for
explaining the difference.
What Postgres calls PL/Python (PL = procedural language) works because
it embeds Python in Postgres.
Meanwhile, there are interface wrappers for Postgres (and other db
engines) that extend Python with a database. For example, see:
http://docs.djangoproject.com/en/dev/topics/install/#database-installation
> Thus, the only other
> possibility to execute stored procedures written in python is to embed the
> interpreter in the database.
I don't agree with that conclusion. For example, the stored procedures
could be available on a separate process that communicates with the
database using sockets. But I think this is off-topic.
> Also: http://www.zope.org/Members/pupq/zope_in_pg
> Quote from the page: "PostgreSQL can use PL/Python, an embedded Python
> interpreter that lets you execute Python functions in the database."
Thank you also for this URL.
>> Having understood the difference between embed and extend, which I think
>> we do, the next step is to understand the implications for software
>> such as LuaTeX.
>>
> Well, I personally have no interest to push towards the "service" direction.
> I want a high-quality, stable application for typesetting.
I'm pleased we're now discussing the relative merits of embed and
extend, and in the context of expressed user needs also. Good.
Well, there's TeX. That's certainly a high-quality and stable
application for typesetting. I think you also want the ability to use
code written in a high-level language (such as Lua) rather than having
to use the TeX macro language.
Recall that if you want to provide a service, you have to use extend,
while for an application you can choose between embed and extend. Even
though you have no personal interest in providing a service, it may be
that the service route will work best for providing the application you
want.
So let's now discuss how embed and extend work for providing a
typesetting application. I'm keen for us to know that LuaTeX has made
a choice, and for us to understand the consequences of that decision.
--
Jonathan
Jonathan Fine
>> We can take a typesetting engine and make it available as an extension
>> to a scripting language (such as Lua) or embed in it a scripting
>> language (such as Lua). Or even, if we wish, do both. Or even do
>> something else, such as embed Java and give make it available as a
>> Python extension.
>
> I think it would be sensible to leverage the power of the Parrot VM if
> a bridge were to be built. The ability of Parrot languages to (be made
> able to) share not only loadable modules but runtime objects would be
> quite useful.
There's not only Parrot but also Microsoft's CLI and the open source
Mono (both guided by the same ECMA standard).
http://en.wikipedia.org/wiki/Mono_(software)
> For instance I just read "The Nature of Lisp" and have a completely
> different appreciation for the language than I did before this thread
> started.
Thank you. I'm glad this thread has help you understand Lisp better.
> I definitely see how it could be useful in relation to TeX. I
> still think Lua is a better embedded language, but an embedded Lua-to-
> Parrot bridge should allow anyone who wants to interface with LuaTeX
> from another language to do so.
It can be very helpful for applications to provide a service API,
because that allows them to be used as an extension to a dynamic
language (such Lua, Lisp, etc).
For example, Emacs provides
===
$ emacsclient --help
Usage: emacsclient [OPTIONS] FILE...
Tell the Emacs server to visit the specified files.
===
which I find very useful when a command line program wants to ask the
user to edit a file.
Similarly, inverse search from dvi to source works by providing a
service API. So yes, providing a service API can be very useful.
--
Jonathan
T3X
wrote:
> > Yes, but let's not forget that there are no such expectations of
> > LaTeX3, i.e changes are expected. This is a one time opportunity that
> > should not be wasted.
>
> I'd like to agree, but I'm not sure everyone sees things in the same
> way. Some members of the LaTeX community *really* value stability.
Yes, TeX community is well known for its conservatism. But this
shouldn't discourage those, who are able to see beyond the current
status quo, from pushing for the right decisions.
> So the idea is that the
> internals can then cope with different input. If you've got ideas, I'm
> sure the team will be happy to discuss them (perhaps a separate thread
> here or on the LaTeX-L mailing list).
Well, nothing particularly well thought out atm. And I'm no expert on
the subject. I'm merely trying to look at the bigger picture and
pinpoint the fundamental issues involved. If that's of any use, I am
be more than happy to participate in any discussions on the subject.
Cheers,
Tomek
Robin Fairbairns
>Joseph Wright wrote:
>>> One suggestion which I have made every year for the last 10 years
>>> is that there should be an official thesis class,
>>> with help to institutions to produce an additional package
>>> to cover local variants.
>>> If in fact, as I am told every year, the report class
>>> is almost sufficient then the work involved would be miniscule.
>>
>> At the LaTeX project level, I doubt this will happen. I've already
>> said that the LaTeX kernel is not flexible enough, in my opinion.
>> Trying to create a usable thesis class just using the kernel is not a
>> realistic goal: "report" is simply not up to the job. To get good
>> results you need to use an extended class (KOMA, memoir, etc.) or lots
>> of packages. Not only that, but there is the whole "no major changes
>> to LaTeX2e" point: I don't think there is any chance of adding new
>> classes to the base LaTeX distribution.
>
>Sigh.
>Some say it is too easy, and therefore not worth doing.
>Others say it is too difficult, and therefore not worth trying.
this is an encapsulation of the problem. for a skilled macro hacker,
it's often quite easy; for that same skilled macro hacker to do the
same thing with variations for every college in the usa, is just too
much to expect.
there was a case recently on texhax[*]: a woman in the usa wanted to
write her thesis in latex, but couldn't get latex to do the layout.
she was working to a tight deadline, but eventually a distinguished
(retired) latex hacker found time to do the job for her. a major part
of the exercise was reading the college's 90-page(!) thesis layout
manual.
despite the distinction of the hacker, it took several days' work and
more than one pass at "seeking layout approval" before the job was
deemed done.
multiply those several days by (say) half the number of phd-granting
colleges in the usa, and you've some measure of the problem.
>I believe it would be a great selling point,
>if you could say to a student starting his thesis,
>"Just start with \documentclass[uccd]{thesis}.
>You may find you have to add some packages later
>for special features you want to include.
>But this will do to start with.
>Just list your chapters with their titles,
>and you are on the way."
sure. it would be a great start. but unless there's a confident
latex hacker in the institution, it's not going to happen. so,
realistically, it's only going to happen in a few select places.
but why would anyone want to do this? most places write thesis
specifications closely linked to what was thought good practice in the
days of typewriters -- i.e., ugly and difficult to read fluently.
m$ word was designed precisely with the aim of replacing typewriters,
so (apart from mathematics, which tended to be tricky on typewriters)
it's good enough for most thesis specs. the only real difference from
the days of the typewriter is that no-one now expects you to submit
corrections in the form of inscriptions on the original... (one of
this place's most distinguished students was required to write in
pencil on his thesis. someone ought to type it up, as otherwise the
graphite is going to disappear.)
istm that journals are a better target, actually.
[*] i'm persona non grata there, for posting, but there's nothing to
stop me reading it.
--
Robin Fairbairns, Cambridge
Robin Fairbairns
>Robin Fairbairns wrote:
>> just to pursue this a bit further ... one of the things now
>> implemented in lua is reading hyphenation patterns; they're not
>
>Thank you for your contribution, Robin. I think you mean LuaTeX here.
>Or maybe luatex.
i mean lua. lua, as in "the scripting language embedded in luatex".
surely that's clear enough?
>> storeable in a format file any more. standard tex-live setup
>> therefore doesn't work satisfactorily, and will need to be changed for
>> a "production" luatex (we're not there yet, though we may be
>> approaching feature specification stability).
>
>I don't see how this is relevant. The TeX model is that if one sends to
>the typesetting engine a string of the form '\patterns{ ..... }' when
>the engine is in an appropriate state then hyphenation patterns will be
>stored. I'd be surprised if LuaTeX were any different in this respect.
luatex doesn't bind itself to the tex model. if you have \patterns in
a luatex initialisation run, it stores the \patterns command in the format.
>You do seem to be saying that, unlike TeX, LuaTeX does not use format
>files (for reloading at high speed material that had been previously
no. i'm saying that luatex treats \patterns differently.
>So far as I can tell, this is a matter of the capabilities of the
>typesetting engine, and quite separate from whether the engine is
>wrapped to provide an extension of Lua or whether Lua is embedded in the
>typesetting engine to provide scripting.
i was merely trying to suggest that your flip assumption ("i don't care
what the lua engine will do for me") was probably incorrect.
to first order, i don't care which way round the embedding/extension
goes: i find it easiest to view the setup as a sort of symbiosis.
ymmv.
--
Robin Fairbairns, Cambridge