Version 0.7 released

[Gaëtan de Menten]

I am pleased to announce that version 0.7 of Liam2 is now available.

The highlights of this release are:

* imports so that simulation files can be split and reused.
* improved performance and (peak) memory usage (for most models)
* nicer syntax for aggregate and one2many functions
* new features to help with debugging

More details and the complete list of changes are available below.

This new release can be downloaded on our website:
http://liam2.plan.be

As always, *any* feedback is very welcome, preferably on the liam2-users
mailing list: liam2...@googlegroups.com (you need to register to be
able to post).

New features:
-------------

* implemented imports so that simulation files can be split and reused.
This can be used to simply split a large model file into smaller files,
or (more interestingly) to create simulation variants without having to
duplicate the common parts. This feature was inspired by some code from
Alexis Eidelman.

* added new logit and logistic functions. They were previously used
internally but not available to modellers.

* added two new debugging features: autodump and autodiff. autodump will
dump all (non-scalar) variables (including temporaries) at the end of
each procedure in a separate hdf5 file. It can be used stand-alone for
debugging, or in combination with autodiff. Autodiff will gather all
variables at the end of each procedure and compare them with the values
stored previously by autodump in another run of the model (or a variant
of it). This can be used to precisely compare two versions/variants of a
model and see exactly where they start to differ.

* added new assert functions:

- assertIsClose to check that two results are "almost" equal
tolerating small value differences (for example due to rounding
differences).
- assertEquiv to check that two results are equal tolerating
differences in shape (though they must be compatible).
- assertNanEqual to check that two arrays are equal even in the
presence of nans (because normally nan != nan).

* added a new "timings" option to hide timings from the simulation log,
so that two simulation logs are more easily comparable (for example with
"diff" tools like WinMerge).

* added a menu entry in notepad++ to run a simulation in "debug mode".

Miscellaneous improvements:
---------------------------

* improved the performance and memory usage by changing the internal
memory layout. Most operations are now faster. new(), remove() and
"merging data" (for retrospective simulations) are now slower. In our
model, this translates to a peak memory usage 20% smaller and a 35%
speed increase.

* improved the performance and memory usage by changing the internal
memory layout. Most operations are now faster. new(), remove(), "merging
data" (for retrospective simulations) and writing data at the end of
each period are now slower. In our model, this translates to a peak
memory usage 20% smaller and a 35% overall speed increase. However, if
your model has a low processes/variables ratio, it may very well be
slower overall with this version. If it is your case, please contact us.

* changed the syntax for all aggregate functions: grpxxx(...) should now
be xxx(...). For example, grpsum(age) should now be: sum(age). The old
syntax is still valid but it is deprecated (it will be removed in a
later version). A special note for grpmin() and grpmax() which becomes
min() and max() respectively even though those functions already
existed. The meaning is deduced from the number of "non-keyword" arguments:

min(expr1, expr2)
minimum between expr1 and expr2 (for each individual)

min(expr)
(aggregate) minimum value of "expr" over all individuals

min(expr1, filter=expr2)
(aggregate) minimum value of "expr" over individuals satisfying the
filter

A tool to automatically upgrade models to the new syntax is provided. In
notepad++, you should use the **Liam2: upgrade model** command in the
**Macro** menu.

You can also run it via the command line: ::

main upgrade model.yml [output.yml]

see main upgrade --help for details.

* changed the syntax for all one2many link functions: xxxlink(link_name,
...) should now be link_name.xxx(...). For example, countlink(persons)
should now be: persons.count(). The old syntax is still valid but it is
deprecated (it will be removed in a later version). As for aggregate
functions, one can upgrade its models automatically with the "upgrade"
command.

* the "period" argument of value_for_period can now be a *scalar*
expression (it must have the same value for all individuals).

* when the output directory does not exist, Liam2 will now try to create it.

* when debug mode is on, print the position in the random sequence
before and after operations which use random numbers.

* entities are loaded/stored for each period in alphabetical order
instead of randomly. This has no influence on the results but produces
nicer log files.

* deprecated the "predictor" keyword. If you need several processes to
write to the same variable, you should use procedures instead.

Fixes:
------

* using invalid indexes in "global arrays" do not crash anymore if they
are properly enclosed in an if() expression. For example if you have an
array "by_age" with values for indices from 0 to 99, the following code
will now work as expected: ::

if(age < 50, by_age[age + 50], 0.5)

Periodic globals are unaffected (they always return "missing" when out
of bounds).

* fixed link expressions which span 3 (or more) *different* entities.

* fixed using show() on a scalar created by summing a "global array".

* fixed the progress bar of matching() when the number of individuals is
different in the two sets.


Changes since RC1
-----------------

* clarified the release notes concerning the performance changes. Thanks
to Howard Redway for his tests.

* fixed two regressions (compared to 0.6.2):

- fixed the "explore" command.
- fixed using link.get(expr) when there are several links to the same
entity.

--
Gaetan de Menten

Federal Planning Bureau
Economic Analyses & Forecasts
Avenue des Arts, 47-49 | 1000 Brussels
tel. +32 (0)2 507 7459
fax +32 (0)2 507 7373
email : g...@plan.be | www.plan.be


----------------------------------------------------------------------------

Disclaimer: please see "www.plan.be/disclaimer.html"

Please consider your environmental responsibility before printing this email

----------------------------------------------------------------------------