Attn: Marc! New extension lib for fopen and friends in GAWK.

[Kenny McCormack]

1) To get it, do:
Use "wget" or "curl" (or similar) to fetch:
http://shell.xmission.com:PORT/fopen.zip

(where "PORT" is 65401)

2) To build it, do (as mentioned in the .c file):
a) unzip fopen -d fopen
b) cd fopen
c) gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c

3) To test it, do:
gawk -l ./fopen 'BEGIN { print fopen_version() }'

4) Assuming this all goes off without any problems, you can test further
with the included "Marc.awk" file. When I run this on my /etc/group file,
the output is as shown below:

$ gawk4 -f Marc.awk /etc/group
Library version: fopen extension: version 1.0 (Apr 26 2017 at 06:51:03)
fd = 3
fd = 3
String: sys:*:3:root
String: netusers:*:52:
$

Notes:
1) This was all developed on OSX, so reflects a POSIX-centered
world-view, but it should work on Windows using Cygwin. My
understanding is that you (Marc) are primarily a Windows user, but
it should be OK for you if you can get Cygwin installed and
working on your system. I do not have a Cygwin-configured Windows
system at the moment to test with, so I am relying on you for that
part of the effort.
2) The bugaboo in all of this is going to be what exactly to use for
the offset type, in the ftell and fseek calls. The current,
barebones, implementation just uses 'long' (i.e., the simple 'long'
versions of the calls), but that probably only works for files up
to 2G size. It is my understanding that you want to be able to use
this on files larger than that. There are ways to do that, but I
wanted to keep this first implementation as simple as possible.

Anyway, let me know if/when you are able to get this working.

P.S. Just to re-iterate: It should on Cygwin (Cygwin bash shell) exactly
as it does in OSX. It should just compile and should "just work".

--
The randomly chosen signature file that would have appeared here is more than 4
lines long. As such, it violates one or more Usenet RFCs. In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
http://user.xmission.com/~gazelle/Sigs/Aspergers

[Marc de Bourget]

Thank you very much indeed, Kenny.
I'll test it the next few days after I have installed Cygwin etc.
One first question concerning:

gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c

This requires the MiNGW GCC compiler, doesn't it?
I have already installed it (hopefully with all required options).

[Kenny McCormack]

In article <8d5e1419-acaf-4c14...@googlegroups.com>,
Marc de Bourget <marcde...@gmail.com> wrote:
...

>Thank you very much indeed, Kenny.
>I'll test it the next few days after I have installed Cygwin etc.
>One first question concerning:
>gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c
>This requires the MiNGW GCC compiler, doesn't it?
>I have already installed it (hopefully with all required options).

I'm not sure about MiNGW. I've used it in the past, but I think I agree
with Kaz's basic point that it is a stripped down version of Cygwin and
there's really no reason to use it. Or, to put it another way:

1) The good thing about MinGW is that it requires no additional support
libraries and it only depends on MSVCRT.DLL.

2) The bad thing about MinGW is that it doesn't use the very nice and
complete cygwin1.dll support library and it depends on MSVCRT.DLL.

Given that MSVCRT is becoming deprecated by MS, it seems kind of a lonely
road to be on.

That all said, all you can do is try it and see what happens.

Be aware that my code does use POSIX features (e.g., getline()), so it
(probably) won't work on a non-POSIX-y toolchain.

If you are using Windows 10, you might try the new "Linux subsystem for
Windows" - or whatever MS is calling it. From what I hear that includes a
compiler, but I know next to nothing about it. From what I can tell, this
subsystem is kinda the same thing as Cygwin, but with MS's official backing
and support.

--
In politics and in life, ignorance is not a virtue.
-- Barack Obama --

[Marc de Bourget]

I have now finally installed Cygwin with default settings.
It seems to be a bit more complicated as expected:

$ gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c
fopen.c:1:0: error: -fPIC ignored for target (all code is position independent) [-Werror]
fopen.c: in function ¦do_fgetline¦:
fopen.c:124:20: Error: Implicit declaration of function ¦getline¦ [-Werror=implicit-function-declaration]
if ((linelen = getline(&buff,&bufflen,fp)) < 1)
cc1.exe: all warnings being treated as errors

BTW, strange: Why "cc1.EXE"? I would have supposed this is kind of Linux :-)
I have MingW installed on the same PC but this shouldn't interfere, does it?

[Kenny McCormack]

In article <aea92324-ddde-4885...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:
...

>I have now finally installed Cygwin with default settings.
>It seems to be a bit more complicated as expected:
>
>$ gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c
>fopen.c:1:0: error: -fPIC ignored for target (all code is position independent) [-Werror]

OK - you should drop -fPIC from the command line.

>fopen.c: in function ¦do_fgetline¦:
>fopen.c:124:20: Error: Implicit declaration of function ¦getline¦
>[-Werror=implicit-function-declaration]
> if ((linelen = getline(&buff,&bufflen,fp)) < 1)
>cc1.exe: all warnings being treated as errors

Well, that is the crux of it. If something was going to go wrong, that
would be it. If you Google "getline in cygwin", you'll get smatterings of
text that say a) Cygwin doesn't have getline and b) Cygwin does have
getline and c) Cygwin doesn't have getline, but you can get it via
"gnulib".

I'm not sure which is correct, but I note that a lot of what I read is old
(from c. 2006), so you'd think they have fixed it all by now. Note that I
make no bones about my not caring about the C standards (alone); I program
for POSIX and make no bones about that. So, I don't care that getline() is
not in the C standards, but the point is that it should be in POSIX and
thus it should be in Cygwin. FWIW, on a current Linux system, "man
getline" says this:

* Both getline() and getdelim() were originally GNU extensions. They were
* standardized in POSIX.1-2008.

And on OSX (where I do most of my primary development and where the fopen
library was developed), it says:

* The getdelim() and getline() functions conform to IEEE Std 1003.1-2008
* (``POSIX.1'').

So, it really should work in Cygwin. I think we'll need someone else, more
familiar with Cygwin to confirm or deny this point.

>BTW, strange: Why "cc1.EXE"? I would have supposed this is kind of Linux :-)
>I have MingW installed on the same PC but this shouldn't interfere, does it?

It's OK. The executable files created by Cygwin, as well as the
executables that actually implement Cygwin, are all .EXE files. They have
to be, in order to be a) findable and b) executable by the Windows OS.

--
Pensacola - the thinking man's drink.

[Kenny McCormack]

In article <oeasuo$3ad$1...@news.xmission.com>,

Kenny McCormack <gaz...@shell.xmission.com> wrote:
>In article <aea92324-ddde-4885...@googlegroups.com>,
>Marc de Bourget <marcde...@gmail.com> wrote:
>...
>>I have now finally installed Cygwin with default settings.
>>It seems to be a bit more complicated as expected:
>>
>>$ gcc -shared -W -Wall -Werror -fPIC -o fopen.so fopen.c
>>fopen.c:1:0: error: -fPIC ignored for target (all code is position independent)
>[-Werror]
>
>OK - you should drop -fPIC from the command line.
>
>>fopen.c: in function ¦do_fgetline¦:
>>fopen.c:124:20: Error: Implicit declaration of function ¦getline¦
>>[-Werror=implicit-function-declaration]
>> if ((linelen = getline(&buff,&bufflen,fp)) < 1)
>>cc1.exe: all warnings being treated as errors

Update: I've put out a new version, called fopen_with_getline.zip, that
includes a free (GPL'd) version of getline(). Please try it out.

Note: Once you've unpacked this latest zipfile in the same directory as the
previous stuff, the compile command should now be:

gcc -shared -W -Wall -Werror -o fopen.so fopen_with_getline.c

--
Just for a change of pace, this sig is *not* an obscure reference to
comp.lang.c...

[Marc de Bourget]

Thank you Kenny. I was now able to create "fopen.so". So, what's next? :-)

[Marc de Bourget]

Sorry, I have forgotten about the steps 3 and 4 at the beginning of your post.
I had to change @load "./fopen" to @load "./fopen.so" in marc.awk, then it
worked. It works properly for Windows line endings, not for Linux files.
With Linux file endings (e.g. original getline.c) wrong lines are printed.
But this is no big problem since I use CR/LF file endings.

[Kenny McCormack]

In article <0c4a09a8-3d10-4efb...@googlegroups.com>,

Marc de Bourget <marcde...@gmail.com> wrote:
...

>Sorry, I have forgotten about the steps 3 and 4 at the beginning of your post.
>I had to change @load "./fopen" to @load "./fopen.so" in marc.awk, then it
>worked. It works properly for Windows line endings, not for Linux files.
>With Linux file endings (e.g. original getline.c) wrong lines are printed.
>But this is no big problem since I use CR/LF file endings.

OK, so if you have got the Marc.awk script to work, then you should have
what it is you were seeking originally. The next step is to go ahead and
use it in your real application space, and let me know how it works out.

Regarding your questions:

1) I'm not totally surprised that you had to add ".so". On both Linux
and Mac OSX, it works without it, but it looks like your version of
the main GAWK executable doesn't have that as a default extension
to look for. Do you know which compiler/development tools was used
to compile your version of GAWK.EXE?

2) Could you show me an example of how it fails on Linux formatted files?

--
I shot a man on Fifth Aveneue, just to see him die.

[Marc de Bourget]

Thank you very much indeed, Kenny.

To answer your questions:
1. $ gcc --version
gcc.exe (GCC) 5.3.0

2. $ gawk -f marc.awk getline.c
Library version: fopen extension: version 1.0 (May 6 2017 at 10:21:43)

fd = 3
fd = 3

String: FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
String: *p;