\bibliographystyle error with UTF-8 character
Seb
With file foo.bib containing the single entry:
---<---------------cut here---------------start-------------->---
@ARTICLE{1,
author = {Surname1, Ø. and Surname2, A. and Surname3, B.},
title = {Title X},
journal = {Journal X},
year = {2008},
volume = {1},
pages = {1-2},
number = {1}
}
---<---------------cut here---------------end---------------->---
and a minimalistic foo_utf8.tex, UTF-8 encoded:
---<---------------cut here---------------start-------------->---
\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
Letter Ø refers here \cite{1}.
\bibliographystyle{ieeetr}
\bibliography{foo}
\end{document}
---<---------------cut here---------------end---------------->---
and then:
$ pdflatex foo_utf8.tex
$ bibtex foo_utf8
$ pdflatex foo_utf8.tex
gives the error:
---<---------------cut here---------------start-------------->---
LaTeX Warning: Citation `1' on page 1 undefined on input line 7.
(./foo_utf8.bbl
! Package inputenc Error: Unicode char \u8:Ã. not set up for use with LaTeX.
---<---------------cut here---------------end---------------->---
The problem seems to be with certain bibliography styles, such as
ieeetr. Is there some way to get around this issue? Thanks.
Cheers,
--
Seb
Joseph Wright
BibTeX *cannot* safely handle UTF8 characters. This is a known and
long-standing issue. The problem is the BibTeX(8) binaries, so there is
nothing that can be done at the LaTeX end. Stick to latin1 in your
database and keys, and you should be fine.
--
Joseph Wright
Robin Fairbairns
>--=-=-=
>[re-posting, characters got mangled previously]
>
>
> author = {Surname1, Ø. and Surname2, A.},
> title = {Title X},
> journal = {Journal X},
> year = {2008},
> volume = {1},
> pages = {1-2},
> number = {1}
>}
>
>
>LaTeX Warning: Citation `1' on page 1 undefined on input line 7.
>
>(./foo_utf8.bbl
>
>! Package inputenc Error: Unicode char \u8:Ã. not set up for use with LaTeX.
given that bibtex is a 7bit application, this isn't terribly
surprising (utf-8 is a solidly 8-bit encoding of the rather wider
unicode space).
recently, someone advised someone, here, to use bibtex8 for this
issue. have you tried that?
if it doesn't work, there's no real way forward other than recoding
the .bib file. (there are newer alternatives[*] to bibtex, but i've
never been satisfied that any is ready for immediate use ... and not
all of them, even, work with utf-8.)
[*] note that current bibtex was released in the late 1980s. i doubt
unicode was more than somebody's working paper, at that time (i worked
quite a lot with the character coding people in iso, and they never
mentioned unicode at all).
--
Robin Fairbairns, Cambridge
Jeffrey Goldberg
> (there are newer alternatives[*] to bibtex, but i've
> never been satisfied that any is ready for immediate use ... and not
> all of them, even, work with utf-8.)
My apologies if this is a FAQ, but what is the current state of
alternatives to BibTeX?
In the 80s I used something called TiB which seemed a promising design,
but it was coded somewhat sloppliy and very difficult to port. I made a
failed attempt to move it to ANSI C and squash as many of buffer overflows
as I could, but I seemed to do more harm than good, and I'm not sure that
if I could even find the back-up tapes that is on, that I could find
something to read it.
Cheers,
-j
--
Jeffrey Goldberg http://www.goldmark.org/jeff/
I rarely read top-posted, over-quoting or HTML postings.
http://improve-usenet.org/
Simon Spiegel
> In <ggk8b4$8fb$1...@gemini.csx.cam.ac.uk>, Robin Fairbairns wrote:
>
>> (there are newer alternatives[*] to bibtex, but i've
>> never been satisfied that any is ready for immediate use ... and not
>> all of them, even, work with utf-8.)
>
> My apologies if this is a FAQ, but what is the current state of
> alternatives to BibTeX?
>
> In the 80s I used something called TiB which seemed a promising design,
> but it was coded somewhat sloppliy and very difficult to port. I made
> a failed attempt to move it to ANSI C and squash as many of buffer
> overflows as I could, but I seemed to do more harm than good, and I'm
> not sure that if I could even find the back-up tapes that is on, that I
> could find something to read it.
There was a thread on this recently, but the bottom line is that there
currently isn't a BibTeX replacement. Jean-Michel Hufflen has been
presenting his MlBibTeX on different TeX meetings for several years
now, but it seems to be vapoware so far, is it isn't available
everywhere. Charles Neugebauer has mentioned ExBib on this list
recently. I actually tried to compile it something like a week ago and
failed (which may be completely my mistake). There is CrossTeX which is
available and works but which doesn't really solve BibTeX's fundamental
problems. And then there's CSL which might be Next Big Thing, but
hasn't been implemented for LaTeX so far.
To make this short: the current situation is quite terrible. We have
biblatex which solves most formatting and document type problems, but
when it comes to the backend, we're still stuck with BibTeX(8). From
the different anouncements it looks like ExBib might be the most likely
candidate for the not so distant future, but so far I've been unable to
make it work.
simon
Simon Spiegel
<si...@removethis.simifilm.ch> said:
> On 2008-11-27 19:38:31 +0100, Jeffrey Goldberg <nob...@goldmark.org> said:
>
>> In <ggk8b4$8fb$1...@gemini.csx.cam.ac.uk>, Robin Fairbairns wrote:
>>
>>> (there are newer alternatives[*] to bibtex, but i've
>>> never been satisfied that any is ready for immediate use ... and not
>>> all of them, even, work with utf-8.)
>>
>> My apologies if this is a FAQ, but what is the current state of
>> alternatives to BibTeX?
>>
>> In the 80s I used something called TiB which seemed a promising design,
>> but it was coded somewhat sloppliy and very difficult to port. I made
>> a failed attempt to move it to ANSI C and squash as many of buffer
>> overflows as I could, but I seemed to do more harm than good, and I'm
>> not sure that if I could even find the back-up tapes that is on, that I
>> could find something to read it.
>
> There was a thread on this recently, but the bottom line is that there
> currently isn't a BibTeX replacement. Jean-Michel Hufflen has been
> presenting his MlBibTeX on different TeX meetings for several years
> now, but it seems to be vapoware so far, is it isn't available
> everywhere. Charles Neugebauer has mentioned ExBib on this list
That is, of course, Gerd Neugebauer ...
simon
Lars Madsen
given biblatex, isn't the current bibtex(8) problem now sorting? See if
we could use the sorting from xindy ... (xindy is now available for
windows via TeXLive).
/daleif
Simon Spiegel
There are several severe limitations IMO:
- No proper Unicode support.
- Sorting.
- Limited flat data model.
I'm not a programmer and it's always easy for an outsider like me to
make these kind of statements, but I believe that at least the first
two points aren't really that complicated. As you say, xindy has
already solved them, and when it comes to sorting there actually are,
for most languages defined standards, how names and such should be
sorted. Point 3, the data model, is more complicated. So I think, it
shouldn't be too difficult to come up with a BibTeX successor which can
take care of Unicode support and sorting, but then again: That's
probably easier to say than to do.
simon
Robin Fairbairns
>On 2008-11-28 00:32:26 +0100, Lars Madsen <dal...@imf.au.dk> said:
>> given biblatex, isn't the current bibtex(8) problem now sorting? See if
>> we could use the sorting from xindy ... (xindy is now available for
>> windows via TeXLive).
biblatex deals with one part of the bibtex problem -- flezibility of
output. (it does this brilliantly, mind you; it's just that it
doesn't actually solve anything.
>There are several severe limitations IMO:
>
>- No proper Unicode support.
unicode sometimes creeps through cracks; it's not supported at all.
>- Sorting.
>- Limited flat data model.
>
>I'm not a programmer and it's always easy for an outsider like me to
>make these kind of statements, but I believe that at least the first
>two points aren't really that complicated. As you say, xindy has
>already solved them, and when it comes to sorting there actually are,
>for most languages defined standards, how names and such should be
>sorted. Point 3, the data model, is more complicated. So I think, it
>shouldn't be too difficult to come up with a BibTeX successor which can
>take care of Unicode support and sorting, but then again: That's
>probably easier to say than to do.
i think (as a programmer ... of sorts[*]) that your summary is pretty
reasonable. it's part of the reason i pursued the csl/citeproc thread
so over-enegertically (to the point that i was rebuked for
extrapolating from the lack of answers...).
i'm not sure why i'm so keen on this issue, but i am. i've never been
a big publisher of papers, so my experience of bibtex is largely based
on support of others' efforts.
[*] my programming reflexes are pretty sclerotic now
--
Robin Fairbairns, Cambridge
Simon Spiegel
I'm also really keen on this issue. To me, the whole situation is a bit
frustrating. After years of Endnote's domination all kind of free and
better solutions appear, but so far, they don't fit together. There's
biblatex, there's Zotero/CSL, there are web based solutions lime
citeulike, Wikindx, Aigaion and Refbase, but so far, it's really not
possible to move your data hasslefree between these different setups.
Not strictly related to this thread: In some ways I'm also a bit
astonished that there isn't more coordinated energy behind this whole
thing. To some degree, most of what is happening now seems to be
initiated by individuals like Philipp Lehmann and Bruce D'Arcus. One
would think that the subject of proper exchange of bibliographic data
is something of great interest for all branches of reasearch ...
simon
Simon Spiegel
<si...@removethis.simifilm.ch> said:
> On 2008-11-27 19:38:31 +0100, Jeffrey Goldberg <nob...@goldmark.org> said:
>
>> In <ggk8b4$8fb$1...@gemini.csx.cam.ac.uk>, Robin Fairbairns wrote:
>>
>>> (there are newer alternatives[*] to bibtex, but i've
>>> never been satisfied that any is ready for immediate use ... and not
>>> all of them, even, work with utf-8.)
>>
>> My apologies if this is a FAQ, but what is the current state of
>> alternatives to BibTeX?
>>
>> In the 80s I used something called TiB which seemed a promising design,
>> but it was coded somewhat sloppliy and very difficult to port. I made
>> a failed attempt to move it to ANSI C and squash as many of buffer
>> overflows as I could, but I seemed to do more harm than good, and I'm
>> not sure that if I could even find the back-up tapes that is on, that I
>> could find something to read it.
>
> There was a thread on this recently, but the bottom line is that there
> currently isn't a BibTeX replacement. Jean-Michel Hufflen has been
> presenting his MlBibTeX on different TeX meetings for several years
> now, but it seems to be vapoware so far, is it isn't available
> everywhere. Charles Neugebauer has mentioned ExBib on this list
> recently. I actually tried to compile it something like a week ago and
> failed (which may be completely my mistake). There is CrossTeX which is
> available and works but which doesn't really solve BibTeX's fundamental
> problems.
Something I forgot: While CrossTeX does read .bib files and generates
.bbl files, it can't handle just any .bst style. At the moment,
CrossTeX only handles abbrv, alpha, full and plain which makes it
uninteresting for me ...
simon
Lars Madsen
isn't the biggest problem that most journals are in English, so they
don't see any problem in the lack of sorting in bibtex for other languages.
Doesn't bibtex8 handle UTF8? and as far as I know xindy runs UTF8 internally
--
/daleif (remove RTFSIGNATURE from email address)
LaTeX FAQ: http://www.tex.ac.uk/faq
LaTeX book: http://www.imf.au.dk/system/latex/bog/ (in Danish)
Remember to post minimal examples, see URL below
http://www.tex.ac.uk/cgi-bin/texfaq2html?label=minxampl
http://www.minimalbeispiel.de/mini-en.html
Robin Fairbairns
>isn't the biggest problem that most journals are in English, so they
>don't see any problem in the lack of sorting in bibtex for other languages.
this situation isn't going to last (assuming journal publishing
continues at all in the decades to come: i shall spare you my
political analysis of the future ;-).
>Doesn't bibtex8 handle UTF8? and as far as I know xindy runs UTF8 internally
bibtex8 will deal with certain utf-8 characters (in the sense that it
won't mangle them). it is presumably, though, likely to split
multi-byte sequences, at end of line, with %. of course, it will make
a pig's ear of sorting anything with utf-8 in it.
xindy is a utf-8 capable app. apart from it being a model of the way
forward (intelligent users giving up on i18n of makeindex), i don't
see that it's relevant to the present case.
--
Robin Fairbairns, Cambridge
Lars Madsen
i18n? I just mention that it can handle much of the multilanguage
sorting, if one could take out that part of xindy, it could be useful in
other applications
Simon Spiegel
>>>>
>>>
>>> i think (as a programmer ... of sorts[*]) that your summary is pretty
>>> reasonable. it's part of the reason i pursued the csl/citeproc thread
>>> so over-enegertically (to the point that i was rebuked for
>>> extrapolating from the lack of answers...).
>>>
>>> i'm not sure why i'm so keen on this issue, but i am. i've never been
>>> a big publisher of papers, so my experience of bibtex is largely based
>>> on support of others' efforts.
>>
>> I'm also really keen on this issue. To me, the whole situation is a bit
>> frustrating. After years of Endnote's domination all kind of free and
>> better solutions appear, but so far, they don't fit together. There's
>> biblatex, there's Zotero/CSL, there are web based solutions lime
>> citeulike, Wikindx, Aigaion and Refbase, but so far, it's really not
>> possible to move your data hasslefree between these different setups.
>>
>> Not strictly related to this thread: In some ways I'm also a bit
>> astonished that there isn't more coordinated energy behind this whole
>> thing. To some degree, most of what is happening now seems to be
>> initiated by individuals like Philipp Lehmann and Bruce D'Arcus. One
>> would think that the subject of proper exchange of bibliographic data
>> is something of great interest for all branches of reasearch ...
>>
>> simon
>>
>
> isn't the biggest problem that most journals are in English, so they
> don't see any problem in the lack of sorting in bibtex for other
> languages.
I guess that's true for most journals in the hard sciences, but there's
a whole world of academic research going on beside that. And exactly
this has been BibTeX's problem right from the beginning –> if you look
at the standard document types and fields, it's obvious that it's
really geared towards English speaking hard sciences and not for much
else . For example, it has been mainly because of one field that I
moved to biblatex as early as possible, and that's 'bookauthor'. The
situation that you have an essay in a book which itself has an author
but not an editor is rather common in my field. Until biblatex, there
was no clean solution for this in BibTeX. And it's interesting to see
how this specific issue always comes up again and again. For example,
in the Zotero forums some people ask for this and other don't even
understand why this would be useful. That's what I also find
frustrating about many new efforts in the area of bibliographic
software: Someone comes up with some interesting idea, releases a first
version, and again, no 'bookauthor' (this only being an example for
some popular missing feature). That's also one of the great things
about biblatex: Here, you really get the impression that someone has
actually looked beyond the basic formats and tried to cover as much as
possible (CSL would do that too, *in principle*, unfortunately, its
current implementation in Zotero is still lacking).
simon
Sebastian P. Luque
rf...@cl.cam.ac.uk (Robin Fairbairns) wrote:
[...]
> given that bibtex is a 7bit application, this isn't terribly
> surprising (utf-8 is a solidly 8-bit encoding of the rather wider
> unicode space).
> recently, someone advised someone, here, to use bibtex8 for this
> issue. have you tried that?
I did, but got the same failure with the Ø character, so I guess I'm
stuck with recoding the *.bib file... Thanks anyway for the feedback!
--
Seb
Jeffrey Goldberg
> On 2008-11-27 19:38:31 +0100, Jeffrey Goldberg <nob...@goldmark.org> said:
>> My apologies if this is a FAQ, but what is the current state of
>> alternatives to BibTeX?
> There was a thread on this recently, but the bottom line is that there
> currently isn't a BibTeX replacement. Jean-Michel Hufflen has been
> presenting his MlBibTeX on different TeX meetings for several years now,
> but it seems to be vapoware so far, is it isn't available everywhere.
> [Gerd] Neugebauer has mentioned ExBib on this list recently. I actually
> tried to compile it something like a week ago and failed (which may be
> completely my mistake). There is CrossTeX which is available and works
> but which doesn't really solve BibTeX's fundamental problems. And then
> there's CSL which might be Next Big Thing, but hasn't been implemented
> for LaTeX so far.
Thank you for that excellent summary. I'll try to find ExBib (I couldn't
find it on CTAN) and see what it does. And I suppose I should learn what
CSL is.
Simon Spiegel
> In <6p89abF...@mid.individual.net>, Simon Spiegel wrote:
>
>> On 2008-11-27 19:38:31 +0100, Jeffrey Goldberg <nob...@goldmark.org> said:
>
>>> My apologies if this is a FAQ, but what is the current state of
>>> alternatives to BibTeX?
>
>> There was a thread on this recently, but the bottom line is that there
>> currently isn't a BibTeX replacement. Jean-Michel Hufflen has been
>> presenting his MlBibTeX on different TeX meetings for several years
>> now, but it seems to be vapoware so far, is it isn't available
>> everywhere. [Gerd] Neugebauer has mentioned ExBib on this list
>> recently. I actually tried to compile it something like a week ago and
>> failed (which may be completely my mistake). There is CrossTeX which is
>> available and works but which doesn't really solve BibTeX's fundamental
>> problems. And then there's CSL which might be Next Big Thing, but
>> hasn't been implemented for LaTeX so far.
>
> Thank you for that excellent summary. I'll try to find ExBib (I
> couldn't find it on CTAN) and see what it does. And I suppose I should
> learn what CSL is.
Here's what Gerd posted on ExBib about a month ago:
> If you are willing to act as early adopter you can have a look at ExBib.
> It is a Program written in Java which uses dynamic allocation of memory
> without predefined limit (except those of the JVM)
>
> The BibTeX compatibility has been tested. The BibTeX8 features might not
> all be in place yet.
>
> The source can be retrieved from the Subversion repository. See
>
> http://www.extex.org/sources/repository.html
>
> In the subfolder ExBib you can find the sources of ExBib. (in addition
> ExTeX-resource and CLI are neede to build). Details on the building can
> be found in the ReadMe files. When the first public release is finished
> a installer will be provided for download. This can already be build
> from the sources.
As I said, I didn't manage to compile it.
As for CSL: This is not only a replacement for BibTeX (the program),
but a replacement of the whole chain. It provides a data model and a
complete formatting language for bibliographies. At the moment, the
most useful implementation is part of Zotero (http://www.zotero.org),
some details can be found here:
http://www.zotero.org/support/dev/csl_syntax_summary . But for LaTeX
users, this currently only interesting in theory as it hasn't been
ported to LaTeX (yet).
simon