patch to make saddpkg locale-independent

[Gianvacca]

saddpkg was not working properly for me and quite some time ago I realised it was because Smart in my system is localised to Italian.

The bug occurs when in the spec file there's something like

BuildRequires:   package > version

In this case the script queries (through smart) for the available package version and asks the user if it's correct.

However, the lines of code aimed at extracting the package name and version from smart output rely on grep searching for the words 'Loading', 'Saving', 'Updating' at the beginning of a line.

Grep will never find these words is Smart is localised to any other language than english, though.

The patch is meant to subsitute those words (actually the full sentences 'Loading cache...' and so on) with their translation if relevant.

I'll leave to Steve the choice of implementing it 'as is' or doing some changes (change variable names, position of code, ...).

By the way, do you think that a similar issue might affect other scripts? I could provide a patch also for those.

PS. Tell me if you didn't get the attachment.

[Stumpy842]

Thanks I'll take a look at it ASAP. I'm sure there are other scripts
which will have the same problem, I've just never had the knowledge of
what to do with them to make them more language-friendly...

[Stumpy842]

After applying the patch successfully and looking over the code again,
it occurs to me that there are other strings which would need
translated also to make the script work in all cases.

I will refer to line numbers in the new patched version...

line 142: "failed build dependencies"
line 154: "failed dependencies"
line 157: "no such file"
line 164: "error" AND "file not found"

These lines would prevent any rpm errors from being detected since
they are all strings returned by rpm. So if the package has errors or
missing files then the script will not detect this problem unless
those strings are translated also. Sorry I do not know more about how
to fix this else I could do it myself but I think I will need your
help for this one, OK?

Steve

[Gianvacca]

Steve, my new patch is almost ready but I still need some information.

In which case do you get the 'file not found' message? I need to know to do a simple test.

I hope you can also help me with this...

'No such file or directory' is not an rpm message, it comes from somewhere else. Have you got any idea where it could be? I've checked the coreutils but it doesn't come from there.

[Stumpy842]

First my apologies for not getting back to you sooner on this, I've
been very busy this weekend...

I've had to make several changes in saddpkg because it seems I never
really had it updated fully to handle the differences in output since
it was originally written for rpm 4.x and apt-get, and of course now
needs to work with rpm 5.x and smart...

I've committed the changes to svn in the /projects/pkgutils folder and
we still need to have translation capability for the following:

line 141: "failed build dependencies" (from rpmbuild)
line 144: "failed dependencies" (from rpm)
line 159: "error" AND "file not found" (from rpmbuild)

The string "file not found" is displayed by rpmbuild if the
dependencies are already met and it is looking for the files from the
package, because I'm using rpmbuild -bl as the command to check for
missing deps. Try rpmbuild -bl with a package for which you already
have all the BuildRequires installed and you will see something like
this:

$ rpmbuild -bl bc.spec
Processing files: bc-1.06-26.x86_64
error: File not found:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/bin/bc
error: File not found:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/bin/dc
error: File not found by glob:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/share/man/man1/bc.1*
error: File not found by glob:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/share/man/man1/dc.1*
error: File not found by glob:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/share/info/bc.info*
error: File not found by glob:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/share/info/dc.info*
error: File not found:
/home/stumpy/src/rpm/BUILDROOT/bc-1.06-26-unity2009.x86_64/usr/share/doc/bc

So saddpkg needs to filter out those lines by searching for "file not
found" (line 159).

If you need more info please let me know...

P.S. - I don't think I've mentioned it before, but saddpkg can be used
in two ways, to install missing deps for either a spec file (assumes
you have the package in your rpm build tree in the SPECS & SOURCES
folders), or just for installing the deps for an rpm file... so "sudo
saddpkg bc-1.06-26-unity2009.x86_64.rpm" is legal also...

[David Smid]

Stumpy842 napsal(a):

It is not difficult, Steve.
TEXTDOMAIN environment variable defines which translation file(s) should the
script use (in this case the same files as Smart - e.g.
/usr/share/locale/it/LC_MESSAGES/smart.mo). TEXDOMAIN refers to a MO file name
without the .mo extension.
Setting TEXTDOMAINDIR is optional because /usr/share/locale is the default.

$(gettext -e "message")
gives you "message" translated to current locale.

$"message"
is the simpler equivalent. This is what I prefer because I find it more readable
than the former. Works only in bash 3.0 or higher AFAIK.

Of course "message" is supposed to be present in the translation file(s). That's
why Gianvacca sets msg1 to "Loading cache...", not just "Loading" - "Loading"
alone is not present in the translated messages list. See smart.pot file or
locale specific .po files in Smart source tarball.
FYI, human readable .po file becomes .mo file after compilation. Only .mo files
are actually installed and used by gettext for L10n.
.pot file is a template for .po files: usually list of all translatable phrases
generated from sources by "xgettext" or "bash --dump-po-strings"

To sum up, now you're ready to make internationalized bash scripts :) You just
set the TEXTDOMAIN to anything unique and then use $"..." construct for phrases
you wish to have translated. I also advise using printf instead of echo - it
makes things much simpler when part of internationalized message is a variable.

E.g. instead of
echo $"File " $FILE " not found."
use
printf $"File %s not found.""\n" $FILE

The reason is that the word order can be different in every language.

David

[Stumpy842]

OK thanks for the info and I'll probably need some help until I get
used to using this, not to mention I don't have any other
localizations installed to test out, I wouldn't be able to operate
with them if I did since I'm mono-lingual, lol...

[Gianvacca]

Don't worry, Steve, I quite enjoy localising bash scripts. I'll take care of this matter and I'm going to write something up as well, on our wiki I mean. I'm sure you'll learn one day.

I'm sorry but Sunday night and all yesterday I wasn't available for hacking and I couldn't even tell you. But take for granted a patch later.

[Gianvacca]

Steve, I've got a new patch and two questions:

1) why a '\' at this line?

if echo "$i" | grep -i "^error" | grep -iqv "${es}\|file not found"; then

2) in the following example I get the translated string and substitute %s with an empty string

gprintf "File not found by glob: %s" ""

I can use the output with grep because the variable is at one string edge and it is for sure in all languages.

But if it had been

gprintf "File %s not found by glob" ""

that wouldn't work with grep because searcing for "File  not found by glob" is not like searching for "File /x/y/z not found by glob". Do you know what I mean?

Well, the easiest solution is to substitute the variable %s with ".*". I tried but I havn't been successful, do you know the trick?

Thanks, Steve

[Stumpy842]

To answer the first point, the \| means OR to grep, so it does a
case-insensitive search for either $es OR file not found. The
backslash char escapes the | symbol to make it work as OR, I think you
can do the same but omit the \, if you use grep -E (in our case grep
-Eiqv), so you may want to try that.

For the second case I don't really have an answer other than to say
that the filename is not relevant at all, so all we need is to find
the string equivalent to "File not found" if that is possible.

[Gianvacca]

2009/12/16 Stumpy842 <stum...@gmail.com>

Yes, Steve, this is already taken care in the patch I've sent you. What I pointed out is that this was a  fortunate event, since the variable (%s) is at the end of the line in the original strings:

File not found: %s\n

                                                              and

File not found by glob: %s\n

What I've done was to consider just the first line substitute %s with 'empty'.

You obtain:

'File not found: '

                                                              and

'File not found by glob: '

grep will search for these and will find them. Problem solved in this particular case, job done.

But what if the string was something like

File %s not found.

If I used the same trick, the string grep would search for will be:

'File  not found.'

and grep will never find it because in real life there will be something in place of %s. As David pointed out, you can only search for the whole exact string through gettext no bits and pieces. 

One workaround could be to take just the left part (preceding the variable), or the right part (following the variable), and search for it with grep.

The drawback is that you can't predict in other languages were the variable will be. In English it might be in the middle, but in other languages it could be at the beginning or at the end, who knows. Well, if in Italian the variable is at the beginning and grep is instructed to search for the left part of the string, then the searching key will be empty or almost empty, thus giving meaningless results.

The real solution, valid for any possible case is to substitute '.*' to the variable. In that case grep will search for:

'File .* not found'

Therefore, whatever is between File and not will be picked up. My problem is that I'm not able to do this substitution. It seems that the * in .* gets intepreted and if I escape it, I get .\* instead of just .*

You know bash better than me, that's why I asked you.

[Stumpy842]

OK first thing is, the patch is very good but there is one error
contained in it. The line:

if echo "$i" | grep -i "^$(gprintf "error:")" | grep -iqv
"${es}\|${fnf1}|${fnf2}"; then

should be:

if echo "$i" | grep -i "^$(gprintf "error:")" | grep -iqv
"${es}\|${fnf1}\|${fnf2}"; then

Notice that there is a backslash before each pipe symbol, and the
patch is missing the one between ${fnf1}|${fnf2}

I have made a test setup but of course I don't have any other
localizations installed so I can only test this for the English
locale. I am able to use the .* in the definition of the search
strings for gprintf, so if you have time please download these files
and play with the test script, which is named "t", and the data files
(foo, bar & baz).

The script defines some strings to search for, using the same examples
found in saddpkg, and also adding the example of the following:

fnf1=$(gprintf "File %s not found\n" ".*" | head -n 1)
fnf2=$(gprintf "File %s not found by glob\n" ".*" | head -n 1)

It seems to work fine here, let me know if you do not get good results with it.