Preparing for new release 0.8

[David Moreno Montero]

Hi everybody,

it is really a long time without real releases (0.7 around 2 years ago), and as we approach 1000 stargazers (999 just now), and I just committed the commit 1000 (what a coincidence), I think its now good moment for a new release 0.8.

Highlights since 0.7 (in reverse chronological order):

1. Stability improvements on the poller (Thanks to Solar designer for help tracking it, idea by Rich Felker, fixed by David Moreno).

2. Performance improvements on onion_log (Zachary Grafton)

3. Many bugs found by clang thread sanitizer and fixed. (Zachary Grafton)

4. Improvements to JSON encode/decode (Thanks to Solar designer for help tracking it, fixed by David Moreno)

5. Undefined behaviour fixes (Zachary Grafton)

6. CPP bindings (David Moreno, Zachary Grafton)

7. Redis sessions (Zachary Grafton)

8. Rework on request parser to allow OCS_YIELD (David Moreno).

9. Overall API improvements (David Moreno)

10. Use of custom low level functions: onion_malloc, onion_free... (Basile Starynkevitch)

11. Bohem GC support (Basile Starynkevitch)

12. Many small fixes and improvements.

If something is missing, or wrongly attributed, please let me know. I will add a changelog to the repository with this data.

Before the release I would like:

1. to merge the fixtimeouts branch

2. make it work on mac osx -- But I dont have a mac, so I need help here.

And some community help:

3. what do you all want to have in a new onion release?

4. And in a future one?

Regards,

David.

--

David Moreno Montero 

dmo...@coralbits.com
+34 658 18 77 17

http://www.coralbits.com

[Solar Designer]

On Thu, Apr 14, 2016 at 03:36:36PM +0200, David Moreno Montero wrote:
> it is really a long time without real releases (0.7 around 2 years ago),
> and as we approach 1000 stargazers (999 just now), and I just committed the
> commit 1000 (what a coincidence), I think its now good moment for a new
> release 0.8.

Great!

> Highlights since 0.7 (in reverse chronological order):
>
> 1. Stability improvements on the poller (Thanks to Solar designer for help
> tracking it, idea by Rich Felker, fixed by David Moreno).

This is only available in a pull request that you have not merged:

https://github.com/davidmoreno/onion/pull/167

Your commit c80c46d5ff842291f0cce3917e7b8340c43d4315 is not it - I think
it's based on a misunderstanding of Rich's idea, so should not be
attributed to him, nor to me, please. (Of course, the existing commit
message has to stay as-is, but please don't propagate this to the
changelog.) Specifically, this commit does not literally use shutdown(),
postponing the close() call, but instead uses onion's own ->shutdown(),
which does a close() right away, so the problem persists.

> Before the release I would like:
>
> 1. to merge the fixtimeouts branch

Please don't. Instead, please consider merging the pull request above,
or let me know if you'd like me to resubmit it without those #if 0's
or/and split into two separate pull requests (for the two commits).

Alexander

[David Moreno Montero]

Hi Alex,

I can postpone the fixtimeouts to a later release (or just not do it), but I would need then the pull request without the #if 0's. If you think it deserves several commits, no problem for me.

Thanks.

Regards,

David.

[Basile Starynkevitch]

On 04/14/2016 03:36 PM, David Moreno Montero wrote:

Before the release I would like:

1. to merge the fixtimeouts branch

2. make it work on mac osx -- But I dont have a mac, so I need help here.

And some community help:

3. what do you all want to have in a new onion release?

At the very least, a well documented versioning API... I don't understand if and how onion has one.
I guess that many things have improved... If something exists, please put it in 0.8

See this question: http://programmers.stackexchange.com/a/315691/40065 ; my comment about lack of versioning is perhaps wrong (but I gave several links to interesting examples)
But at least, I need to understand more the versioning stuff.

Cheers.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***

[Solar Designer]

On Thu, Apr 14, 2016 at 04:07:03PM +0200, David Moreno Montero wrote:
> I can postpone the fixtimeouts to a later release (or just not do it), but
> I would need then the pull request without the #if 0's. If you think it
> deserves several commits, no problem for me.

OK, I committed the removal of those #if 0's. It's now part of the same
pull request, as a third commit. I hope this works for you (and I don't
have to start over without ever introducing the #if 0's).

To me, the code looks dirty either way. We're just patching it to make
it work, not cleaning it up yet.

A future change may be to move to timers that are unaffected by system
time adjustments, such as to CLOCK_MONOTONIC_RAW (which I am already
using in the app using libonion) since poller.c requires a non-ancient
Linux kernel anyway. But let's revisit this after the release.

Alexander

[Zachary Grafton]

I would have to +1 the comment about versioning. There is quite a bit different from the 0.7 release and 0.8 release. I'm not sure if it deserves only a minor release. There has been a few API changes, especially with the bindings. Also, maybe open a tracking issue on github so we all know what needs done to get to the next release.

Zack

[Zachary Grafton]

David,

Also, I think we should decide which compilers and language features we want to use and support for the next version. Things have been a little murky recently. I think Alexander was using Scentific Linux, which I believe uses gcc 4.4 out of the box. I believe he is using just the C API though. I'm thinking that the C++ bindings should be restricted to compilers that implement the large majority of C++11, just to avoid the complexity that was demonstrated by my one pull request. A compiler with C99 should be sufficient for the C API, unless we decide we want to use C11 atomics. If we decide to go the C99 route, any GCC over version 3 would be acceptable. If we go the C11 route, at least for atomics, GCC 4.9 would be the earliest version that we could support.

As a side note, we could probably clean up the documentation a bit before release as well. I'd also like to see a bit more testing with the Redis session support, which I think I might be working on this week.

Zack

[David Moreno Montero]

Hi,

about semantic versioning AFAIK we do it in the main C onion, as older functions from older releases are exactly the same. I double checked headers, and we only add functions. Change of types for compatible ones is ok (int -> onion_poller_slot_type_e). There is a real typo change from *_AVALIABLE to *_AVAILABLE. I think its ok to accept the corrected version and do not provide an old compatible way.

On top of that as everything is encapsulated (no direct access to internal structs, everything under onion_*_new), it should be ABI compliant, this means that even binaries from older versions should work.

And on top of that we version the libonion.so. It uses the git hash if compiling from a master branch, or the tag name if has a tag with a name (1). But this script is new so older 0.7 did not have it, was hardcoded into the CMakeLists.txt.

My idea is to keep this compatibility until 1.0 is released, and then maybe break it if necessary for 2.0. If its not necessary 1.1 should be compatible as well and so on. Although as now we are in pre 1.0, if its really necessary, we could break it, but stating it clearly.

If I miss something about semver, let me know.

The C++ bindings have changed, so maybe we should note it, even still mark them as experimental. In C++ its very dificult to achieve ABI compatibility, so I dont think we should aim for it. But I'm ok if somebody steps in to ensure it.

I will try to review the documentation.

About next version minimal compatibility, just as Zach said, C11 and C++11 sound good. But if somebody does the work for older compilers, and its not very intrusive, its ok.

1. Run ./git-version-gen should print 0.7.[hash], or copy git-version-gen to /tmp/, checkout v0.7 and run the /tmp/git-version-gen, it prints 0.7. This is used for the .so naming. The coping is needed as v0.7 did not have this script. Thanks to Ruediger Meier.

--
You received this message because you are subscribed to the Google Groups "onion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onion-dev+...@coralbits.com.
To post to this group, send email to onio...@coralbits.com.
Visit this group at https://groups.google.com/a/coralbits.com/group/onion-dev/.
For more options, visit https://groups.google.com/a/coralbits.com/d/optout.

[Zachary Grafton]

David,

I wouldn't mind working the C++ to be ABI compatible, but that would mean a lot of changes to the code to use the Pimpl idiom where necessary. I think marking the bindings as experimental would be best, maybe we should even split them out into a separate project. However, I do think it is possible to get them to a point where they could be ABI compatible with newer versions. Currently, all the internals of each class are completely visible to the end user. The 0.8 release is definitely not ABI compatible with 0.7 release of the C++ bindings. Maybe mark it as a project for the 1.0 release?

We should definitely document the compatibility under a contributors section of the wiki, and maybe list the known working compilers in the README.

I think in regards to Semver, we should probably be doing work on a develop branch and merging into master and tagging master. That way we can effectively use the patch level portion of Semver. I've been maintaining some Fedora packages in a COPR repo where I have to bump the package release everytime we merge a commit onto master, instead of just bumping the patch version. It's kind of misleading in my opinion.

Zack

[Basile Starynkevitch]

On 04/15/2016 04:43 PM, David Moreno Montero wrote:
> Hi,
>
> about semantic versioning AFAIK we do it in the main C onion, as older
> functions from older releases are exactly the same.

It would be nice to have an API function to query the version of the
library; at least something like

const char* onion_library_version_string();

and perhaps more, e.g.

int onion_library_version_major();
int onion_library_version_minor();

and probably some more macros in the headers. Glib from GTK could be
inspirational.

[Zachary Grafton]

I'm pretty sure we could accomplish this by using cmake and the configure_file directive to generate a single header with all the configuration options, including definitions like ONION_VERSION_MAJOR, ONION_VERSION_MINOR and ONION_VERSION_PATCH.

Zack

[Basile Starynkevitch]

On 04/15/2016 05:19 PM, Zachary Grafton wrote:
> I'm pretty sure we could accomplish this by using cmake and the
> configure_file directive to generate a single header with all the
> configuration options, including definitions like ONION_VERSION_MAJOR,
> ONION_VERSION_MINOR and ONION_VERSION_PATCH.

Perhaps, but I am familiar enough with cmake to understand how is that
possible.

[Zachary Grafton]

It would be something that we would have do on the library end. Basically, we'd create a file called something like config.h.in with contents like:

#ifndef ONION_CONFIG_H

#define ONION_CONFIG_H

#cmakedefine ONION_VERSION_MAJOR

#cmakedefine ONION_VERSION_MINOR

#cmakedefine ONION_VERSION_PATCH

#endif

Then, in our CMakeLists.txt file, we would just do something like:

set(ONION_VERSION_MAJOR 0)

set(ONION_VERSION_MINOR 8)

set(ONION_VERSION_PATCH 0)

configure_file(config.h.in config.h)

That's just a rough outline, but idea should be similar in the implementation. Then we'd have the macros available for the end user to test against. It would also probably require that we manually bump the versions when releases are done.

Zack

[David Moreno Montero]

Hi,

yes, after the 0.8 release, we will create a dev branch for development, and master should be always a tagged release.

Also I will add the section in the README with the compilers that I know currently works, please if you know it works on something older, do a pull request.

Regards,

David.

[David Moreno Montero]

I'm creating a bug report for adding the int onion_version_* functions and headers.

I also created a milestone set for 29 apr for the v0.8 release. Feel free to add issues with that milestone set.

Regads,

David.

--
You received this message because you are subscribed to the Google Groups "onion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onion-dev+...@coralbits.com.
To post to this group, send email to onio...@coralbits.com.
Visit this group at https://groups.google.com/a/coralbits.com/group/onion-dev/.
For more options, visit https://groups.google.com/a/coralbits.com/d/optout.

[Zachary Grafton]

David,

I think we should also revisit some of the open issues on github and close some of them out. I believe I fixed issue 118 a while ago but we haven't heard back from the original submitter. 116 could be closed out if Alexander's patch is merged. 110 is still waiting on a reply from the submitter and it's almost a year old. Issue 99 looks stale. 73 can be closed if you accept the pull request I submitted(assuming you think it's acceptable.) 37 isn't really a problem with onion itself, just distribution. 42 could probably be closed as well. 58 needs one of the maintainers to reply, if we have access to a Mac. I haven't seen any sign of life from 67 either.

When it comes to clib support, is that something we are interested in, and if so, which package managers are we prepared to support? There's about a half dozen competing package managers and none really seem to have reached a point where it's ubiquitous. Some even require pretty extensive changes to the cmake files.

Zack

On Thursday, April 14, 2016 at 9:36:56 AM UTC-4, David Moreno Montero wrote:

[David Moreno Montero]

I will review the issues and mark them for 0.8 if appropiate.

I also added a nwe "version" branch with the code for runtime version checking. If anybody has some time, please review it before i merge it.

Do anybody know if I as the owner of the project create a pull request, can you guys add comments?

I will create the merge request anyway and let me know if its possible for you to add comments.

Regards,

David.

--
You received this message because you are subscribed to the Google Groups "onion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onion-dev+...@coralbits.com.
To post to this group, send email to onio...@coralbits.com.
Visit this group at https://groups.google.com/a/coralbits.com/group/onion-dev/.
For more options, visit https://groups.google.com/a/coralbits.com/d/optout.

[Zachary Grafton]

David,

It appears I can comment on the pull request. Your approach was pretty similar to mine, but included a little bit extra. Also, my approach required manual intervention to bump versions, but it passed CI...

Zack

[David Moreno Montero]

Yes, Im working on CI right now :)

David.

[Zachary Grafton]

I'm glad our approaches were at least pretty similar :)