[anic] Building anic with mingw

[3Jane]

Hello,

my platform is WindowsXP with gcc under MingW for C-development, main
language python3.

I hate make/autoconf. Any modern compiler or byte-compiler hides more
or less completely what
make is doing (ghc for Haskell still needs so-called "cabal"-files,
but they are much shorter than
Makefiles and one for each project (not one in each directory) is
enough, when i had got it right) -
this includes all highly portable languages.

Trying to build anic in the msys shell i was amazed, that "make all"
only resulted in this
short error list:

$ make all
anic ANI Compiler Makefile

Cleaning build output...
Building version controller...
Building lexer structure generator...
Generating lexer structures...
Compiling lexer structure object...
Building parser structure generator...
Building parser table generator...
Constructing parser table...
Generating parser structures...
Stamping version...
c:\MinGW\bin\sha256sum.exe: -: Bad file descriptor
c:\MinGW\bin\sha256sum.exe: standard input: Bad file descriptor
Version stamp is 0.66.4.
Building main executable...
chmod: changing permissions of `anic': Bad file number
make: *** [anic] Error 1

I got a working anic.exe nevertheless and have no idea know how
serious these
errors are - but i miss the effect of these lines in the makefile:
@echo Generating installater script...
echo Installation complete.
(besides: "installater" - a typo ?)

Firing up "make test" in the msys shell then rsulted in a lot
of lines protocolling term reductions ending with a syntax tree
followed by:

anic: ERROR: SEMMER: tst/test.ani:114:5: -- (input type is null).
anic: ERROR: SEMMER: tst/test.ani:111:1: cannot resolve term's output
type.

anic: ERROR: SEMMER: tst/test.ani:111:1: -- (input type is null).
anic: Semantic mapping generated inconsistencies.

anic: ERROR: fatal error code 1 -- stop.
--------------------------------
Failed default test cases!
NOTE: This is the expected behaviour for now; there's still work to be
done
.
prepended with about 50 other lines ".* -- (input type is null)"
Now i ask: Is it true, that this is the expected behavior" ? Is my
anic.exe working correctly thus in spite of the errors of make ?

I feel i should beg your pardon for asking this - after all i could
try to find
out, how to modify the Makefile to get it work correctly. But that
could
mean whole days of work for me - perhaps the issue is quite trivial
for
someone knowing make better.

Thanks for reading anyway

[Adrian]

On May 8, 1:17 pm, 3Jane <1...@depikt.net> wrote:
> Hello,
>
> my platform is WindowsXP with gcc under MingW for C-development, main
> language python3.
>
> I hate make/autoconf. Any modern compiler or byte-compiler hides more
> or less completely what
> make is doing (ghc for Haskell still needs so-called "cabal"-files,
> but they are much shorter than
> Makefiles and one for each project (not one in each directory) is
> enough, when i had got it right) -
> this includes all highly portable languages.

Yeah, modern compiler systems do take over a lot of the job, except
gcc doesn't do this (and probably won't for a long time to come, since
gcc => unix, and the unix philosophy is to have a whole bunch of small
tools that are each good at one and only thing (and therefore, gcc
"shouldn't" be doing change management, only compilation). I
personally don't agree with this philosophy, but gcc really *is* the
most portable compiler suite out there, so relying on Make, I think,
is a small price to pay for free portability to virtually any
architecture out there.

Yes, this is the expected behavior (the scripts aren't lying to
you :P!). The main branch (which is quite old) has quite severely
broken typing and binding, particularly for "recall identifiers",
which is what's throwing all of those errors in the test cases.

There's aggressive work being done on a new development branch, "new-
typing", that addresses these issues (which mainly stemmed from an
initially rushed type system implementation, which sadly is expected
of a compiler for a language that doesn't have much precedent to go
on).

>
> I feel i should beg your pardon for asking this - after all i could
> try to find
> out, how to modify the Makefile to get it work correctly. But that
> could
> mean whole days of work for me - perhaps the issue is quite trivial
> for
> someone knowing make better.

It's not actually a Makefile issue; make is doing its job correctly
and everything is being built fine. The problem is in the compiler
core (written in C++), which isn't fully working (but is being worked
on, quite literally, every day).

>
> Thanks for reading anyway

No problem, I love helping people out when it comes to anic.

Cheers!
Adrian (project lead)

[3Jane]

Okay - thanks !

"aggressive work" is sounding well and gives me hope
that we can see a nice version this summer. All in
all the project looks great. There is no really living
dataflow language today - stating this i found ani.

And i like the grammar (including the backslash).
Code and the tutorial are very short in regard of
the expressive power of the language - apparently
not only easy matter has left being easy, some
difficult matter has been made loooking easy too.

For me it is obvious, that a lot of thought and
decision finding have gone into this - so i hesitate
to offer help. What i'd definitely like to do were
to write some parts of the planned standard lib.
There are enough simple tasks for learning
ani in the same move (obviously Python's
standard lib has been developed by newbie
coders with the same idea. Python's quality
is still suffering from that and ani should avoid
this failure. A real open culture of debate -
open also to so-called "non-constructive"
critics - would be useful).

Not so simple are utf-8 and utf-16. These
are crucial - a language without unicode
support has no chance finding users at all today.
The nontrivial problem is index subscription.

Also crucial seems to me a powerful engine of
string methods including find, index, split,
rsplit, lsplit, trim, rtrim, ltrim, join, startswith,
endswith, replace, regex.replace and more.
Having this index subscription is needed
surprisingly seldom - but it stays needed.

There is no alternative to counting the
occurrences of bytes >= 0xC0 (for utf-8),
thus the complexity of .[n] is unavoidably
>= O(n). But one could reason about a way
of cashing the results of such counting ...

But work on a (more or less canonical)
standard lib would be much easier with
a working compiler. Moreover, i do not
find any import- or include-mechanism
in the tutorial (my preference were include
with header files, a personal note beside:
Using gcc, not Visual Studio, on Windows
is no more any issue of costs or moral -
i really do like gcc and mingw).

Having <include> or <import> another
question arises: How to interface with
C-code. Providing this must come
before any standard lib. Is there really
anybody out there, who wants to translate
the 100.000 lines of sqlite source-code ?
And maintain this translation in the
sequel ?

And a decision is needed here before:

Do you want to have a C-interface or
an interface to machine code in shared libs
first (a living language will come to support
for both sooner or later - but that is future) ?

Good luck, Joost

[Adrian]

On May 8, 8:57 pm, 3Jane <1...@depikt.net> wrote:
> Okay - thanks !
>
> "aggressive work" is sounding well and gives me hope
> that we can see a nice version this summer. All in
> all the project looks great. There is no really living
> dataflow language today - stating this i found ani.
>
> And i like the grammar (including the backslash).
> Code and the tutorial are very short in regard of
> the expressive power of the language - apparently
> not only easy matter has left being easy, some
> difficult matter has been made loooking easy too.

It's good to know the simplicity of the language is appreciated.
Personally, I think that too many languages try to have their code
"read like english" and fail because the semantics of the language are
so complicated that the analogy to reading natural language actually
ends up interfering with understanding: an excellent example here is C+
+, which (ironically) the compiler is written in. Python does a much
better job, but when you get into the more advanced Python
programming, you still have to refer to the manual from time to time
(at least I do).

That's why ANI's syntax has no built-in keywords (although there is a
standard namespace for the primitive object types and such).
Everything is defined using consistent nested punctuation, which
allows the user to define arbitrary data structures and algorithms
without the language syntax getting in the way; this is unlike C++,
which is pretty strict about crazy and confusing things like ambiguous
overloads, class hierarchies, virtual-ness of methods, etc. and forces
you to fight with the language and compiler to get your point across
far more than necessary.

>
> For me it is obvious, that a lot of thought and
> decision finding have gone into this - so i hesitate
> to offer help. What i'd definitely like to do were
> to write some parts of the planned standard lib.
> There are enough simple tasks for learning
> ani in the same move (obviously Python's
> standard lib has been developed by newbie
> coders with the same idea. Python's quality
> is still suffering from that and ani should avoid
> this failure. A real open culture of debate -
> open also to so-called "non-constructive"
> critics - would be useful).

Yeah, that's something to look out for. But for the time being, I'm
willing to personally review all of the ANI standard lib code that
people write to check for bugs and poor quality code. And I completely
agree that we should be fully open to criticism of code; compilers and
libraries are almost by definition used by a *lot* of different people
and therefore their design and implementation should seek as many
diverse opinions as possible.

>
> Not so simple are utf-8 and utf-16. These
> are crucial - a language without unicode
> support has no chance finding users at all today.
> The nontrivial problem is index subscription.
>
> Also crucial seems to me a powerful engine of
> string methods including find, index, split,
> rsplit, lsplit, trim, rtrim, ltrim, join, startswith,
> endswith, replace, regex.replace and more.
> Having this index subscription is needed
> surprisingly seldom - but it stays needed.
>
> There is no alternative to counting the
> occurrences of bytes >= 0xC0 (for utf-8),
> thus the complexity of .[n] is unavoidably>= O(n). But one could reason about a way
>
> of  cashing the results of such counting ...

Yeah, UTF is going to be non-trivial. However, keep in mind that with
well-written ANI code, the compiler could probably get it down to O(n/
c) time, where c is the number of processors you have available.
Counting byte matches is something that anic can easily make parallel,
even if you implement it in the naive way of filtering a character
stream, byte-by-byte.

>
> But work on a (more or less canonical)
> standard lib would be much easier with
> a working compiler. Moreover, i do not
> find any import- or include-mechanism
> in the tutorial (my preference were include
> with header files, a personal note beside:
> Using gcc, not Visual Studio, on Windows
> is no more any issue of costs or moral -
> i really do like gcc and mingw).

There is an import-type mechanism in ANI already. I'm not sure if it's
mentioned in the Tutorial, but basically it look like this:

@someLib.someObject; // @ is the import operator
/* blah blah */ someObject /* blah blah */;

Essentially, the @ operator finds an identifier in some other
namespace and puts it into the current one, which allows you to import
basically anything into where you want it. This means that someLib
needs to be defined somewhere in the source. If it's defined in the
same file as the import declaration, everything will work perfectly
fine. If someLib is defined in another file, that's fine too, but in
that case, the file that contains the definition of someLib needs to
be in the list of files to compile given to anic on the command line,
or anic will complain that it doesn't know how to bind the reference
to someLib.someObject (which makes sense - anic can't go looking
around the filesystem for extra source files to use). Of course, the
standard libraries do not need to be listed on the command line: the
compiler automatically knows about them.

There is currently no way to currently "include" files into ANI
source, but I don't see much need for this. #include just invites
confusion, complexity, and bugs... and if you have #include, then you
need #ifdef include guards to protect against recursive include, and
then you have exactly the same mess that you have with C.

>
> Having <include> or <import> another
> question arises: How to interface with
> C-code. Providing this must come
> before any standard lib. Is there really
> anybody out there, who wants to translate
> the 100.000 lines of sqlite source-code ?
> And maintain this translation in the
> sequel ?
>
> And a decision is needed here before:
>
> Do you want to have a C-interface or
> an interface to machine code in shared libs
> first (a living language will come to support
> for both sooner or later - but that is future) ?
>
> Good luck, Joost

C code interfaces are planned, and will probably be implemented in the
language as a way to declare ANI filters without defining them. This
will cause anic to give these filters gcc-linkable labels in the
assembler output, allowing you to link the ANI code with external C
functions that have the same signature.

However, I don't want to encourage this too much because this presents
a whole bunch of problems for ANI code: pretty much everything in ANI
runs in parallel, and the external C functions would need to be
written with that in mind - i.e. it's the programmer's job to make
sure that no concurrent data access problems like deadlock or race
conditions arise in the interfaced code.

If it's just ANI code, anic can make sure that everything is safe,
efficient, and conflict-free. But if you call arbitrary C code, making
sure that you're not stomping on internal ANI data structures while
they are being accessed by another thread is completely up to you, and
anic can't really help you. And I can't even imagine the problems that
would arise if the C code does something crazy like fork() or spawn a
new thread that the ANI runtime doesn't know about, or if the function
call is really long and holds up ANI's real-time context switching.

I realize that we *do* want to reuse the C code that's out there, but
the nature of ANI programs means that the C code would need to be
specially rewritten with ANI in mind if things are going to work
properly.

It's good that you're bringing these issues up, though. It's best to
think about them sooner rather than later.
Adrian

[3Jane]

Okay. I do not need <include>.

Concerning C: For the beginning we
could simply forbid fork(), threading and
so on in called code - so far as i know
sqlite, that would not pose any problems
there. More difficult seems the
maintainance of updatable C-objects
like sqlite3_stmt with that different
meaning of = in C for me - obviously
we can represent them neither as
pipes/streams nor constants. But
i might misunderstand ANI here.

Thus i should better get some
experience with ANI coding and look
forward to the next working compiler.
Interfacing to C isn't trivial with
CPython also (not at all i can tell
from experience) - some coding
work can always be expected from
those, who want to access C-code.

Good luck, Joost

[Adrian]

On May 9, 3:09 am, 3Jane <1...@depikt.net> wrote:
> Okay. I do not need <include>.
>
> Concerning C: For the beginning we
> could simply forbid fork(), threading and
> so on in called code - so far as i know
> sqlite, that would not pose any problems
> there. More difficult seems the
> maintainance of updatable C-objects
> like sqlite3_stmt with that different
> meaning of = in C for me - obviously
> we can represent them neither as
> pipes/streams nor constants. But
> i might misunderstand ANI here.

We can't technically forbid fork() and pthread_create(), in the sense
that there's not really any good way to make sure people don't link
with C code that contains these things. We can, however, say that "if
you use fork(), the results are undefined", and it might work
sometimes, or it might break randomly. I think that's the best
solution.

But actually, the interfacing would be pretty clean if we made sure
that the only things that are interfaced are ANI filters that take in
latches. This way, the contents of the parameter latches *are* mutable
(ANI allows this), but if you want to get the latched contents back,
you'll have to return them back from the filter as part of the return
value. This will all work pretty cleanly; the only problem is that C
allows arbitrary data accesses (that can completely screw up the ANI
runtime) and C, just like ANI, likes to "own" the memory space in that
it has a heap (for malloc), it needs its own stacks, it needs a place
to put globals, etc. I think it's doable; it's just I'd rather focus
on compiling pure ANI for the time being.

[3Jane]

I never doubted or wanted to raise any doubts,
that a C-interface will be principally doable. My
thoughts were by some kind of roadmap and
the thread here, where creation of a standard lib was
more or less inaugurated. Sqlite support was
among the eventual content of that standard lib -
i only wanted to state, that a general interface
to foreign code (C enough) will be needed, and
that unforeseen problems could linger around
there.

I should not write this, i should not read in
the ANI tutorial - these weeks i should do
all to finish a commercial project. And after
it complete my pet project (a python
interface to gtk - that is fairly advanced C)

sourceforge.net/projects/depikt

However - development of depikt will not
be continued beyond what is announced
there.

Thus i leave and look forward to the
working version.

Good luck and success ! Joost