[Niteration] June 2015

9 views
Skip to first unread message

Jean Privat

unread,
Jul 8, 2015, 10:01:45 AM7/8/15
to nitla...@googlegroups.com

Welcome to this year's seventh issue of Niteration, the newsletter for the Nit project.

Combined Changes for June 2015

  • changes: v0.7.5 .. v0.7.6
  • shortstat: 1031 files changed, 45201 insertions(+), 29497 deletions(-)
  • pull requests: 73
  • non-merge commits: 401

Have contributed (or co-authored patches): Alexis Laferrière, Jean Privat, Alexandre Terrasa, Lucas Bajolet, Romain Chanoir, Julien Pagès, Renata Carvalho, Anis Boubaker, Arthur Delamare, Mehdi Ait Younes, Herve Matysiak, Simon Zeni, Istvan Szalai and Jean-Philippe Caissy.

Highlight of The Month

High Score

This month, some psychological figures where exploded

  • 8000+ commits in the master branch of the repository.
  • 1500+ issues & pull requests on github.
  • 25+ contributors.
  • 3000+ constructions for the continuous integration job of PR in github

Nit Coding Party

The first Nit Coding Party was held at UQAM, on June, 10. 13 participants, including first-time contributors, met up together to write 36 pull requests for tasks of RosettaCode

  • Merge: example: add 24 game task of Rosetta code e923941
  • Merge: rosetta_code: sha1 b7430c1
  • Merge: example: add boolean usage example from rosetta code 39e2aff
  • Merge: Rosetta, batch 1 d0a4ffa
  • Merge: Second batch of RosettaCode 4124b22
  • Merge: example: add arithmetic mean example from rosetta code bbc9a52
  • Merge: Third batch of rosettacode 359fd1f
  • Merge: examples: add json example from rosetta code 76faebc

FFI

We introduced an FFI for the interpreter. This is a first draft/proof of concept to support the light FFI and a few more features, more work is needed to implement the full FFI and support for other languages.

When the interpreter has to execute an extern method, it compiles all the extern code of the modules into an external native shared library (.so). Then the interpreter dynamically loads the module, and invokes the C implementation function. The handle to the library is cached so that a module is compiled once per execution.

The current version is limited to the light FFI with a few more features. So there are extern methods, native type support and extern classes but does not support callbacks.

The interpreter can now run 7 out of the 24 tests for the FFI with C.

This version was kept "simple" by design, it lacks some not-so-hard to implement features as well as portability concerns:

  • Choosing the compiler (hard coded gcc) and the compile folder.
  • Recompile the extern library only when the source is updated.
  • Clear the compile folder.
  • Support shared library format on OS X.
  • Ensure call compatibility between the interpreter and extern libraries

The design was inspired by other interpreters and VM-based languages. The JNI expects the user to manually compile the extern code into a native shared library before executing the Java code. This FFI automates the same process.

  • Merge: Light FFI for the interpreter 5cf88d2
  • Merge: FFI clean up e9461f7
  • Merge: Activate C warnings in FFI code and revert NativeString to char * de34fbb
  • Merge: Fix FFI with private extern classes not needing a "C Header" block 59438ba
  • Merge: Use light FFI instead of the legacy NI in the lib 52ff3ec

New c_src

Last one was Fri Feb 6, five months ago

  • Merge: Regeneration of c_src 7ae4bc3

This generation includes a lot of new things and bug fixes that are now usable in the compiler and the standard library.

We also removed all remaining deprecated reference to nitg. The renaming of nitg to nitc and the deprecation of nitg was done Wed Dec 10 so 6 month ago. cf 1000

Language and Compiler Features

Literals

As a new addition to the parser, as requested in [!issue 1261], literal floating-point values in exponent notation.

  • Merge: Floats in exponent notation 186be29

Literal strings used with to_re are checked in order to find syntax errors in the regular expression. The question about a specific literal regular expression notation is still open.

  • Merge: Check for errors in regex within string literals at compilation 7a1c6f4

Improved JSON deserialization

This PR introduces better error management to the deserialization phase, the abstract engine and the JSON implementation. Deserialization of a malformed JSON string or a JSON serialized object from a different program version does not crash the program anymore, it will instead raise errors. When errors are raised, the deserialized object may be inconsistent with uninitialized attribute. Depending on the context, the object may have to be discarded or, usually with the user consent, it can be used with possible fatal failures later.

This PR also lightens the JSON format expected by the deserializer. This allow for nicer custom creation of Nit objects from JSON. For example, given a Opportunity-like Web app using the Nit class MeetupConfig declared like so:

class MeetupConfig
    serialize

    var description: String
    var max_participants: nullable Int
    var answers: Array[FlatString]
end

On the client-side JavaScript code, one can create a JSON object that will be converted to a Nit instance of MeetupConfig by the JSON deserialization services:

{"__class": "MeetupConfig", "description": "My Awesome Meetup",
 "max_participants": null, "answers": ["Pepperoni", "Chicken"]}
  • Merge: Deserialization manage errors & easily create Nit object from pure JSON 57021ef

Attribute and autoinit annotations

A big set of loosely coupled but related changes on the annotations on attributes and methods. Here is a summary of the changes on the user-side:

  • No more special case for the kind of classes, so you can use autoinit in interfaces.
  • Explicit annotation autoinit is understood on attributes and is the default on value-less concrete attribute.

  • Abstract attributes are noautoinit by default, if autoinit is given, it is applied on the setter. This way abstract attributes are basically just a couple of getter-setter.

  • Merge: Attribute and autoinit annotations 04a9a35

Faster Separate Compilation

Separate compilation implies that a module is compiled independently of its clients. This point ensures that the recompilation of programs is fast since only modified modules need to be recompiled and that modules used by multiples clients need to be compiled only once.

Unfortunately, right now an already compiled module still needs to be fully compiled in C on each compilation (avoiding doing it is a very complex matter). Fortunately, most of the compilation time is spend by the C compiler and this time is safely decimated by ccache.

However, the thing is that some modules are imported by programs (thus provide declarations, introductions, importations) but are only made of method definitions that will never be executed by the program. Such a module is called a dead module and our static analysis RTA can approximate them. So, now, the generation of C code for such a module is skipped (and its associated C compilation, obviously)

This will benefit mostly for small programs that import standard and other libraries but only use a small portion of them. Eg: programs in tests/ or in examples/rossetacode.

For numbers, let's try with time ./tests.sh ../examples/rosettacode/*.nit to test the 32 (currently) programs.

  • before: 30s

  • after: 21s

  • Merge: Do not compile dead modules b1fff49

loop

Warn useless do block, like

loop do
  print 1
end
  • Merge: Warn useless do block 8a1e5b7

Keep Going

This improve the robustness of tools when given --keep-going. Tools like nitpick, that have --keep-going by default, are more robust and collect more errors.

Moreover, the compiler can now compile simple programs with instructions that fail during the typing phase (most errors like unknown method or bad type).

These instructions are compiled with a run time error instead so the program is still expected to behave in an reliable way.

print 1
fail now
print 2

$ nitc kg.nit --keep-going
kg.nit:2,1--4: Error: method or variable `fail` unknown in `Sys`.
$ ./kg
1
Runtime error: FATAL: bad statement executed. (kg.nit:1)

One usage would be to force c_src to compile things even if it lags behind.

Portable stack-traces

Most detection of the availability of libunwind and the flags to use is moved at the C compilation time (in the Makefile) instead of the Nit compilation time.

Moreover, if libunwind is not available, then stacktraces are just disabled (instead of aborting).

The main advantage is that the generated C is more portable and can be compiled in a different system. Thus, this will help cross-compilation and should even enable the bootstrap on systems lacking libunwind, like Raspbian.

The option --stacktrace is also replaced with a simpler --no-stacktrace for people that want to disable it completely at C-compile time.

  • Merge: Portable stack-traces 84fe63c

Chars, UTF-8 and Bytes

Since we plan on conforming with UTF-8, chars can no longer be of C char type, these PR changes the internal representation of a Char to become capable of holding an arbitrary Unicode code point. The separation for Char and Byte is still in progress, stay tuned.

  • Merge: Native Types e8ce370
  • Merge: Fix support for Byte in the FFI 089c943
  • Merge: compiler: NativeString is a char* to please C library. fa62004
  • Merge: niti-ffi: NativeString is re-a char* 3a1e5ca

In the library

Reorganize the string modules

In v0.7.5, the string module is over 2500 loc long.

For the mental health of the maintainer(s), it was getting too big to understand what was going on in it. So, we introduced the text group in standard, and string is split in several modules inside of it:

  • native: Introduces NativeString
  • abstract_text: Introduction of basic types
  • flat: Any array-based string

  • ropes: Rope-based strings

  • Merge: Removed intern dependencies from abstract to flat strings 0f9fbcb

  • Merge: String split 879f535
  • Merge: Remove RopeString 910c6aa

A more POLA to_s on Collection

The default to_s on collections in now updated.

Before: [1, 2, 3].to_s == "123" After: [1, 2, 3].to_s == "[1,2,3]"

The main issue is for code that used to_s on array to get a fast concatenation. Those should now use plain_to_s (or join("")).

  • Merge: A more POLA to_s on Collection e5ae2fe

Better and up-to-date curl module

Changes:

  • Simplify callbacks so they do not break light FFI only engines.
  • Remove about 8 classes and 3 C structures.
  • Remove the custom implementation of NativeFile.
  • Update style to the latest best practices. Note that it was one of the first large module based on the FFI, a lot has changed since then.
  • Revamp and rename CurlMail, its API is now nicer to use:
import curl

var mail = new CurlMail("sen...@example.org", ["t...@example.org"], cc=["b...@example.org"])

mail.subject = "Hello From My Nit Program"
mail.body = "<h1>Here you can write HTML stuff.</h1>"

mail.headers_body["Content-Type:"] = """text/html; charset="UTF-8""""
mail.headers_body["Content-Transfer-Encoding:"] = "quoted-printable"

# Set mail server
var error = mail.set_outgoing_server("smtps://smtp.example.org:465",
    "us...@example.org", "mypassword")
assert error == null

# Send
error = mail.execute
assert error == null
  • Merge: Work on the Curl module 573e667

New DOM XML parser

As Nit programmers, we might have to parse XML sometimes for one use or another.

We already had a SAX parser in library for the cases when one wanted to parse a long XML document in a memory-conscious way, often at the expense of the simplicity of the end-program.

This PR adds a very simple DOM parser that will need to be improved and made more robust later, but which right now does the basics and should parse valid xml documents. There is still a long way to get a full spec-compliant XML parser, but this is one base.

Misc. in libraries

  • Merge: Better error handling in Socket 88cea7d
  • Merge: Manage errors on reading binary data 4992fca
  • Merge: Date Lib 2a539e2
  • Merge: file: fix simplify_path behavior for path starting with .. 4c6f0cc
  • Merge: Add some collections for java/collections.nit 6886542
  • Merge: Fix for Android 16f0634
  • Merge: lib/array: remove useless cast to comparable. d61cf02
  • Merge: Int/Byte Binops 0cdcbf0
  • Merge: Geometry fix 72053fd
  • Merge: lib/markdown: add some tests for xml parsing inside a markdown block. 5bd84f2
  • Merge: mongodb: auto generate ObjectId for inserted and saved documents 66f163e

State of contrib

One day, a package manager will simplify the installation and the maintenance of third-party projects, libraries and programs in Nit, so that import foo will import the project foo if installed.

Nowadys, contrib is used as a placeholder for some independent projects that are not mainly a library (or borderline). However, the nature of the Nit language, especially the refinement of class blurs the difference between libraries and programs and makes it conceivable to extend an existing program by refining it (or just reusing its base libraries). However, import foo will not work if foo is in contrib, this issue has two current workarounds:

  • the user use -I with some relative path location in the Makefile, that is not practical because it should be used on each tools thus broke basic manual commands (like nitc on the command line) and automatic commands (like vim+nitpick).
  • the project just move itself to lib/ so that client can easily import it, but this is hard to conceptualize for some project like nitiwiki

The easy solution to solve this issue is to add contrib to the libpath.

Improved Opportunity

There is now a partial i18n support to Opportunity with a french translation. Most strings from the UI are covered but not the error messages.

There are also templates for the Opportunity meetup creation page. The default software provides: 4 kinds of pizzas, 5 days work week and 7 days week. The software customized for xymus.net also provides: pizzas special LATECE and meeting hours for UQAM Sciences.

Try it at http://xymus.net/opportunity/

  • Merge: Opportunity: Adds total number of participants to a meetup d77539e
  • Merge: Opportunity i18n & traduction française 353444d
  • Merge: Opportunity templates and pizzas bffce43
  • Merge: contrib/opportunité: prevent useless xss exploit in meetup confirmation 5a6bb23

Improved Nitiwiki

Some fixes for nitiwiki in prevision for the nitlanguage.org site migration.

Feature summary:

  • ini accept array notation
  • customizable sidebar content
  • customizable sidebar position
  • wikilinks support external links
  • better search by title
  • better search by path
  • make breadcrumbs and summaries optional
  • add link to github for last_changes and edit mode
  • use relative links in generated documents to avoid hard_coding root_url.

Try the beta at http://nitlanguage.org/beta/

Sound in games

friendz is updated to use the new mnit abstract sound system. A placebo implementation is also provided for linux.

  • Merge: friendz with Android Audio API 9902c7a
  • Merge: Audio API implementation for linux 1c584fc

Misc. Contrib

  • Merge: nitrpg: remove points if a PR is closed without merge 0986e27
  • Merge: Use the DOM parser in the RSS Downloader contrib bd1e58a

Tools

Small features and bugfixes on tools. Nothing fundamental for the end user this month.

  • Merge: nitx: add hierarchy lookup commands 2827c89
  • Merge: nitunit: use the Markdown parser from lib/markdown 52ef622
  • Merge: nitdoc: fix things, add more tests and benches 2edb3cd
  • Merge: nitdoc: use svg instead of png 79ff83a
  • Merge: code: cleanup some docunits bc338fd
  • Merge: Nitunit fails bad code bb9e4e2

Internal

Some internal things and bugfixes for the developers of the compiler and other nit tools

  • Merge: Correct scope in assert else 9ab7aa2
  • Merge: Loose Tokens ea45ed8
  • Merge: nitvm: Basic blocks generation and SSA-algorithm bc11b95
  • Merge: Kill big vararg in parser ba3a053
  • Merge: Bug fixes in SSA module c72a6de
  • Merge: Adaptive typing: fix buggy warning messages 6805ff9
  • Merge: Minor Improvements to the Parallelization Phase 99a8e8b

Continuous integration

Some improvement in the tests/ directory with safer test execution and configurability

  • Merge: More limit on tests 0565337
  • Merge: Personalized tests 0b1ae88
  • Merge: Fix issues in tests introduced by previous series 9c43620
Reply all
Reply to author
Forward
0 new messages