The Node project - take 2

[Aparajita Fishman]

Greetings,

The time has come to move Cappuccino to Node.js. Why move to Node? There are three main reasons:

#1. Cross-platform parity

Cappuccino's build system is built on top of narwhal + rhino (jsc on Mac OS X). Rhino/jsc is the Javascript engine, narwhal is a set of libraries that interface with the host system. rhino requires Java and is very slow. narwhal has been abandoned. In addition to these dependencies, our current build system relies on Posix binaries to overcome limitations in narwhal.

narwhal runs fully only on Mac OS X (using jsc), runs with difficulty on Linux, and doesn't run at all on Windows.

Node runs equally well on all three platforms, is very fast, and replaces narwhal, rhino, and jsc, with no other dependencies. By converting our Javascript code and Posix binary calls to Node, we can have parity on all platforms and a much simpler install.

#2. Ease of installation and upgrading

The single biggest point of friction for beginning users is the obscure incantations necessary to get Cappuccino installed and running. Updating is more difficult than it should be, with a frequent need to rebuild from scratch.

Our goal is to have the Cappuccino install be like this:

npm install cappuccino --dev

No bootstrap, no compiling. One line and you're done. Cappuccino will run fully from source, with no compiling necessary (other than themes). This will make updates much easier. Of course when it comes time to deploy, you will be able to create a compressed version.

#3. More modularity, better ecosystem

Currently Cappuccino is a very large, monolithic beast. Frequently when we change one small part of it, users have to clobber and rebuild the whole thing. Why should this be so?

We envision Cappuccino being split into modular packages that form a stack: Objective-J, Foundation, AppKit, Tools. Installing packages higher in the stack automatically installs the dependencies lower in the stack. 

Node (and thus Javascript) is very popular as a server-side scripting language. But wouldn't you rather use Objective-J + Foundation? In the future, you will be able to do that easily:

npm install Cappuccino-Foundation

More importantly, the modularity and ability to make easily installable packages will encourage Cappuccino developers to publish their extensions to Cappuccino. And we don't have to spend any time creating the infrastructure for finding and installing Cappuccino packages, npmjs.org already serves as a central repository and search engine for Node packages. There is a huge and growing Node ecosystem, and Cappuccino can be part of that.

Moving Forward

** Bringing parity to all platforms and making it easier to install and use is the single most important thing we can do to ensure wider adoption. **

More users means more support from the community, more funds for development, and thus an even better Cappuccino.

This is a big project -- we will be rebuilding Cappuccino from the ground up and rewriting many of the tools.

*** OUR FUNDRAISING GOAL ***

$5,000

A few months ago, we received almost half of that, and had to abandon the project at that time. But now we are ready to try again. Let's make it happen this time!

If you donated before, thank you! Please donate again. If you didn't donate the first time, now is your chance to contribute to a brighter future for Cappuccino. Any amount, great or small, is more than welcome and greatly appreciated.

You can donate here:

   https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=C2EG6R4LHBHT2

Thank you for using Cappuccino, and thank you for your support!

- The Cappuccino Core Team -

[Todd Freese]

Thank you Aparajita for taking on this very important task. I just sent in $1000 with high hopes that the Capp community will fill in the rest and support the future of an amazing framework. I'm sure you all are thinking that I've lost my mind today. Here is how I look at it. We use Capp in two large mission critical apps that run our business. So the future of Capp is very important to my business. This donation is small in comparison to what our company has invested already with 2 staff programmers and 3 years of work. I wish I had the time to work on the framework more but my schedule does not allow for it. But I can help the cause with a donation.

Any donation will help, regardless of the amount. Nothing is too small (or too large).

Todd Freese
The Filmworkers Club

[Michael Bach]

Dear Aparajita:

Here's to hoping we'll reach said goal! I've added too, though not with comparable impact to what Todd did, wow!

Best, Michael

[Aparajita Fishman]

Thanks, Todd. I'm hoping at least that everyone that donated before will donate again. This time it's for real, it simply has to be done.

[Aparajita Fishman]

Hi everyone,

An update on fundraising for the Node project.

Donators: 11

Donations: $3580 (after fees)

Goal: $5000

We're doing better than the first round in dollars, but in the first round 18 people donated. I have already started work -- this time there is no turning back -- so if you donated the first time but haven't yet this time, don't wait on the sidelines!

Thank you to everyone who has donated. I have a lot of work ahead of me, but I'm excited at the prospect of a better Cappuccino.

- Aparajita

[yvess]

hope the fundraising will happen this time! Donated three time as much as last time :-)

perhaps the owner of https://twitter.com/cappuccino can tweet it (1360 followers).

I think it's also really a problem of reaching the community. I don't check regularly the google group, so I could have

missed it this time. 

Yves

[Aparajita Fishman]

Good idea about twitter. Most of the group members don't receive emails.

[Aparajita Fishman]

Congratulations! We did it! Thanks to the generous donations by 25 people so far, we have now exceeded the $5K goal for funding the Node project.

This gives me great hope and encouragement, and I take the responsibility you have given me very seriously. I will do my utmost to deliver a solution that will delight everyone. The project is already under way, and I will keep everyone up to date on the progress both here and on the blog. But please be patient -- I do things very carefully and methodically, which takes time. Currently I'm studying Node, npm and grunt very intensely. I want to really understand the full range of their capabilities, so I can take maximum advantage of what these new tools have to offer. Actual coding will start next week.

If you haven't donated yet, it isn't too late and there is no such thing as giving too much. Any money I don't use for the Node project will go towards the many other projects that will follow: the documentation project, the tutorials project, the "cut the deployment size in half" project, the "fix the 100 outstanding bugs on github" project, the "fix my own private bug list" project, the Retina theme project, and so on. There is a mountain of stuff to do to take Cappuccino where I would like it to be.

Thank you again to everyone for their support!

- Aparajita

[Richard van Zon]

Good luck, Aparajita! 

We believe in you. The continuation of Cappuccino lays in your hands only...no pressure! ;-)

--
You received this message because you are subscribed to the Google Groups "Cappuccino & Objective-J" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objectivej+...@googlegroups.com.
To post to this group, send email to objec...@googlegroups.com.
Visit this group at http://groups.google.com/group/objectivej?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Szabolcs]

Take your time! Full trust!

[daboe01]

Any chance to get CPTextView and friends on your list?

Am Mittwoch, 29. Mai 2013 17:05:02 UTC+2 schrieb Aparajita Fishman:

...

[Aparajita Fishman]

It's on the list, I just didn't mention it.

[yvess]

Really great news! Thanks for not giving up and trying it a second time! 

yves 

[Torsten Luettgert]

On Fri, 31 May 2013 02:05:48 -0700 (PDT)
yvess <yves.s...@gmail.com> wrote:

Is this officially dead? There wasn't anything to see or hear since end
of may.

Regards,
Torsten

[Aparajita Fishman]

Hi,

It isn't dead! Life being what it is, it is just taking a lot longer than I had hoped. The first step was to get the compiler working on Node, and that required rewriting the preprocessor, which was a big job. I'm now working on a few changes to the compiler to make it work better with the new workflow.

It's a huge job, and all I can do is ask for patience. As soon as there is something presentable I will definitely announce it.

[Friedel Ziegelmayer]

Hey, 

we talked before and I just wanted to let you know that I'm still interested in helping out with coding/infrastructure changes on the node project.

I'm one of the maintainers of karma (aka testacular) and have been using npm and grunt for some time now. If you are interested let me know.

Cheers 

[Aparajita Fishman]

Thanks, I'll ping you when the time comes.

[Martin Carlberg]

Hi Friedel,

great that you want to help with this.

To my knowledge no work has been done for months on the node project. The parser and compiler work and exist as node modules:
http://npmjs.org/package/objj-acorn
http://npmjs.org/package/objj-compiler

I think the first step should be to get the Objective-J framework to compile with the new node compiler/parser without using gcc and jake.

Please feel free to jump in and work on this.

When that is done the next step should be to get the 'objj' and 'objjc' command line tools in the Objective-J framework to use the new node compiler.
Then it is time for the Foundation framework and later the Appkit framework and tools etc.

I think it is better to get some kind of working prototype to test the concept before we make all the decisions on how it should be done in the end. Just to get the Objective-J framework running should be a great start.


Regards,

- Martin

--
You received this message because you are subscribed to the Google Groups "Cappuccino & Objective-J" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objectivej+...@googlegroups.com.
To post to this group, send email to objec...@googlegroups.com.

Visit this group at http://groups.google.com/group/objectivej.

[Friedel Ziegelmayer]

Hey Martin, 

so I started reading and a very tiny setup here [1] to solve the include problem.
But now when all the files are combined and I try to use objjc build/Objective-J/Browser/Objective-J.js
I get this error


GLOBAL(CPLogRegisterRange) = function(aProvider, aMinLevel, aMaxLevel, aFormatter)
       ^
ERROR line 905 in build/Objective-J/Browser/Objective-J.js: Expected a semicolon (905:7)
Expected a semicolon (905:7)

which I'm a bit clueless about as the lines above compile fine but for some reason it breaks at this point,
maybe you have an idea? 

To reproduce you'll need to run

$ npm install
$ npm install -g objj-compiler
$ grunt 
$ objjc build/Objective-J/Browser/Objective-J.js

Cheers
Friedel


[1] https://github.com/Dignifiedquire/cappuccino/tree/grunt

--
dignifiedquire

> You received this message because you are subscribed to a topic
> in the Google Groups "Cappuccino & Objective-J" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/objectivej/OOzD5f-Nyrk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email

[Aparajita Fishman]

Contact me privately, no point in us duplicating effort. What is on npmjs.org is not the latest version of what I have been working on.

[Martin Carlberg]

Hi Friedel and Aparajita,

the latest I know of is on http://github.com/cappuccino/objj-compiler and that is the same as on http://npmjs.org/package/objj-compiler.

I think it is great if you two talk and combined your work. We really need to move forward on this again. If I can be of any assistance please let me know.

Regards,

- Martin

[daboe01]

does anybody know what aparajita actually leaves behind as a starting point for us to finish this important project?

best greetings,

daniel

[Benjamin FROLICHER]

Any news on that ?

Visit this group at https://groups.google.com/group/objectivej.
For more options, visit https://groups.google.com/d/optout.

[Martin Carlberg]

Hi!

Below is what we have got from Aparajita. There are some loose ends that needs to be adressed. Some of them are quite large like the preprocessor is gone and try to fit this into the current build system or create a new. The compiler does not handle framework @imports so that needs to be fixed. Also we will not be able to do compilation on the fly so everything need to be precompiled.

This is a lot of work so one way to go might be to use the compiler, create a new preprocessor and plug it into the current build and load system in Cappuccino. We get the ES6 syntax but everything else will be the same and it will still not run on Node.js.

Another alternative is to keep the current compiler and load system then create a new build system that will run on Node.js. When that is done we can continue and replace/expand the current compiler so it will include ES6 syntax etc.

- Martin

Message from Aparajita:

acorn-objj is done. objjc is done except for framework imports.

objjc is designed to be disk-only; no on-the-fly compiling. I think we talked about this a few years ago.

One of the main objectives of objjc was to have much better error reporting. To have good error reporting for framework imports means having detailed symbol info for the frameworks. So my plan was to add the ability to create a symbol file from a compilation, which would basically be a JSON dump of the various compiler symbol tables, but adding location info to each symbol (source path, line number, column number). When you encounter a framework import, you locate the symbol file, read it in, and merge it with the compiler symbol tables.

To support this, one small change will have to be made to acorn-issue-handler: support for creating and displaying an error that has no source code, because framework symbol files will have location data but no source.

Once you have framework imports working, then the steps to take are:

- There is no preprocessor any more, so macros have to be converted to functions, symbols to globals.

- Compile Objective-J.js.

- Compile Foundation.

- Compile AppKit.

As to how the new compiler will fit into the build system, I have no idea what changes will need to be made if you just want to drop it into the current build system.

BTW, I planned to namespace the packages by renaming them to cappuccino.acorn, cappuccino.compiler, etc., similar to the way lodash does it.

[daboe01]

i vote for keeping the current compiler and creating a new build system for Node.js

iirc this has already be done by a guy named nicolas goy.

@kuon are you still around?

 

best greetings,

daniel

[david.ri...@enquora.com]

On Saturday, October 15, 2016 at 9:28:41 AM UTC-6, mrcarlberg wrote:

Hi!

Below is what we have got from Aparajita. There are some loose ends that needs to be adressed. Some of them are quite large like the preprocessor is gone and try to fit this into the current build system or create a new. The compiler does not handle framework @imports so that needs to be fixed. Also we will not be able to do compilation on the fly so everything need to be precompiled.

This is a lot of work so one way to go might be to use the compiler, create a new preprocessor and plug it into the current build and load system in Cappuccino. We get the ES6 syntax but everything else will be the same and it will still not run on Node.js.

Another alternative is to keep the current compiler and load system then create a new build system that will run on Node.js. When that is done we can continue and replace/expand the current compiler so it will include ES6 syntax etc.

- Martin

Creating just a build system seems to lower the technical risk and effort substantially.

Can anyone restate (in a sentence or two) the need for any of this for latecomers to the discussion like myself?

I presume a driving factor is eliminating the dependance on Narwhal and Rhino?

I'm also assuming that 'make' isn't an option for any of this because of Windows?

d.r.

[Rosch]

Hi...

Am Donnerstag, 20. Oktober 2016 00:30:44 UTC+2 schrieb david.ri...@enquora.com:

 

Creating just a build system seems to lower the technical risk and effort substantially.

Can anyone restate (in a sentence or two) the need for any of this for latecomers to the discussion like myself?

I presume a driving factor is eliminating the dependance on Narwhal and Rhino?

I'm also assuming that 'make' isn't an option for any of this because of Windows?

 

Just my $0.02:

Generally I would say this whole thing should be broken into parts.

a) Buildsystem
b) Compiler
c) Other things
d) more other things

All this depending on the global picture. The aim of the project.

This way some things could run in parallel. In my eyes all this does not necessarily need to be done using node.js.
Other cross-platform (Mac,linux,Windows) systems should work, too.

Because of Windows. If Cappuccino would say as prerequisite on windows a user has to have cygwin or msys2 installed
to get the buildsystem running this is something everyone could IMHO live with. If you have these environments up and running
you have (most) unix known tools available: make, git, sed, awk, bash etc... So you can achieve a do it once and run everywhere...
I have done that here for our development department and it works quite well. Obviously we will not get XCodeCapp on Linux or windows,
but the buildsystem and core things could run everywhere.

Roland

[Martin Carlberg]

Hi!

Some background:

Cappuccino is a Javascript framework. It uses Objective-J as a language that is transpiled into Javascript. It actually runs in almost any Javascript environment like Browsers, Node.js, Narwal etc. Just use the prebuilt frameworks and you are up and running. Almost... check below.

To build Cappuccino we use a Javascript build tool that is called Jake. It is old and is not used any more except of us. We also have some homemade tools like ’press' and ’flatten' that tries to remove unused classes and joins everything together to one Javascript file. Today they need Rhino to run. Rhino is an old slow Javascript engine that uses Java. It is used when running from command line on any machine except Mac. On a Mac we can also use jsc (except for press and flatten) which is the Javascript engine for Safari. It is actually very fast, even faster then Node.js in many cases. Press and flatten are not necessary for using Cappuccino but is used when you do ’jake deploy’.

So the problem is that we can’t build Cappuccino on something modern like Node.js that is used by everyone else in the Javascript world. We could use Gulp, Grunt, Broccoli or any other build tool and create a new build system. It is a lot of work as we build Cappuccino in many steps with different versions like Debug, Release. And for many environments like Browser and command line. There are also many tools that should be built like capp, nib2cib etc. On top of that we have tons of unit tests that also need to be run as part of the build system.

Aparajita has done a great job on a new compiler that gives us ES6 support etc. but it has some shortcomings as described earlier.

Cappuccino does run very well today on Node.js with a slightly modified runtime (https://github.com/mrcarlberg/objj-runtime).

I use it for a webserver/backend (https://github.com/mrcarlberg/objj-backend) written in Objective-J together with a CoreData/EOF like Cappuccino framework called LightObjects (https://github.com/mrcarlberg/LightObject). It can be used with the CoreData Model Editor and Interface Builder in XCode or if you don't have a Mac directly from code. I made a demo with it at CappCon in Liege earlier this year. There is also a 5 minutes tutorial in the LightObject README file that even work if you don’t have Cappuccino installed.

If we want be able to build Cappuccino on Node.js I think the easiest path should be to create a new build system on top of our Node.js objj-runtime like described above.

When that is done we can start thinking about ES6 and other things.

- Martin

[daboe01]

take a look at: https://github.com/kuon/cappuccino/tree/node

[Martin Carlberg]

Yes, this branch can be used to make a new build system too. Not used for some years but can be a path we can walk.

- Martin

[Frank Rehwinkel]

I've forked cappuccino-base and added a narwhal-engine based on node.js, at https://github.com/FrankReh/cappuccino-base.

It presents a synchronous interface to narwhal, as do the jsc and rhino engines, so does not require any rewrite to Jake or the Objective-J compiler.  One minor change was needed to the cappuccino code and can be found at https://github.com/FrankReh/cappuccino.  Two minor changes were needed to narwhal scripts themselves.  All three are backwards compatible - jsc and rhino were tested on Darwin and shown to create the same CAPP_BUILD files as before, as was rhino tested on Debian Linux.  (jsc and rhino do not create the same files on Darwin with each other, interestingly enough.)  

Node is commonly used asynchronously but some synchronous functions exist and an npm package, fibers, was used to make the rest function synchronously from narwhal's perspective.  Most of the node/fibers async magic is handled by the narwhal-node/bootstrap.js script - for those interested.

Cappuccino can benefit from riding on the v8 coattails.  v8 is modern and supported on all major platforms.  This seemed like an easy way to get started.  From what I can tell, Darwin and Linux support is good and there is potential for this combination to work on Windows too.  I don't have a recent Windows license so wasn't tempted to try it yet.

[Martin Carlberg]

Hi Frank!

This is an excellent approach. I have looked at doing the
same thing but never had the time to do it.

I tried it on my Mac Book and it works perfectly.

Can someone who normally uses Rhino please try this on
Linux.

This is like Christmas all over again. :-)

Thanks Frank! This is really awesome work!

Can you please do pull requests for this so we can
eventually get it into the frameworks.

- Martin

>--
>You received this message because you are subscribed to
>the Google Groups "Cappuccino & Objective-J" group.
>To unsubscribe from this group and stop receiving emails
>from it, send an email to
>objectivej+...@googlegroups.com.

>To post to this group, send email to

[daboe01]

a docker image would be really cool ;-)

[Cacaodev cacaodev]

Great news, thanks a lot Franck !

[Frank Rehwinkel]

Please give the public docker image at the hub under frankreh/narwhal-node a try.

If it's useful, we can fine tune what an image should have in it - I can also just share the Dockerfile.  Just run the image and you will be placed in a bash shell under the user id of 'docker'.  No password.  sudo has been setup.  You may want to run narwhal with no args to see what the REPL is like.  Or cd to cappuccino and run jake build.  The image is still rather large but at least it works.  I tried various ways of shrinking it, including starting with a smaller debian image, but requiring java and gcc kept all my attempts so large that a slightly smaller image at this point didn't seem worthwhile posting.  Oh, its a debian 8 jessie distro.  Another approach for the image would be to provide a very small image (~60MB) with a script in it that then installed everything necessary via apt-get and git clone - my own tests show it just takes a few minutes to build up from scratch that way.

I haven't even tried to run it on a mac - should work.  I built it on a Linux machine and tested it there.

Had to install java into the docker image too since jake still uses it for a few things.  So you can build or run the unit tests with rhino too and compare the results.

Good luck!

[Martin Carlberg]

Hi!

This is really fantastic that we now can use Node.js as a engine. I have tested to build Cappuccino with ’jake install’ on all three engines. Here is a speed comparison:

Node: Build took 2 minute 2 seconds
JavascriptCore: Build took 1 minute 49 seconds
Rhino: Build took 5 minutes 31 seconds

This is done on a 2016 MacBook Pro.

- Martin

12 mars 2017 kl. 05:53 skrev Frank Rehwinkel <frankre...@gmail.com>:

Please give the public docker image at the hub under frankreh/narwhal-node a try.

If it's useful, we can fine tune what an image should have in it - I can also just share the Dockerfile.  Just run the image and you will be placed in a bash shell under the user id of 'docker'.  No password.  sudo has been setup.  You may want to run narwhal with no args to see what the REPL is like.  Or cd to cappuccino and run jake build.  The image is still rather large - maybe removing debian packages that were needed for the install but which aren't needed for running node would be useful in the future.  Oh, its a debian 8 jessie distro.

I haven't even tried to run it on a mac - should work.  I built it on a Linux machine and tested it there.

Had to install java into the docker image too since jake still uses it for a few things.  So you can build with rhino too and compare the results.

Good luck!

-Frank

-- 
You received this message because you are subscribed to a topic in the Google Groups "Cappuccino & Objective-J" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/objectivej/OOzD5f-Nyrk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to objectivej+unsubscribe@googlegroups.com.

To post to this group, send email to objec...@googlegroups.com.
Visit this group at https://groups.google.com/group/objectivej.
For more options, visit https://groups.google.com/d/optout.

[Марк Ланговой]

Hi, Frank!

Great work ;-) Could you open-source your Dockerfile?

воскресенье, 12 марта 2017 г., 21:15:10 UTC+3 пользователь Frank Rehwinkel написал:

[Frank Rehwinkel]

Glad to.  It was my first Dockerfile so be kind!  Hope you find it helpful.  I since learned that there are smaller debian images available but they would require some playing around with to see what else needed to be apt-get installed.

https://gist.github.com/FrankReh/66a687f85a5aa974b655145e2e0ef596

[Frank Rehwinkel]

I have. Also responded some days ago but my response didn't get through. It's a gist file under FrankReh.

[Марк Ланговой]

Thank you for Dockerfile!

I optimized it and I think now it more clear and lighter (359MB - my, 417MB - your). You can find Dockerfile here - https://gist.github.com/langovoi/2feb108e047ca1c1f88e8e593d15e3ae and docker image at Docker Hub as «marklangovoi/narwhal-node».

Based on node:7 image and latest commits from your forks of cappuccino and cappuccino-base.

Bigger part of image is node (264MB)

11 мая 2017 г., в 0:36, Frank Rehwinkel <frankre...@gmail.com> написал(а):

Glad to.  It was my first Dockerfile so be kind!  Hope you find it helpful.  I since learned that there are smaller debian images available but they would require some playing around with to see what else needed to be apt-get installed.

https://gist.github.com/FrankReh/66a687f85a5aa974b655145e2e0ef596

--
You received this message because you are subscribed to a topic in the Google Groups "Cappuccino & Objective-J" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/objectivej/OOzD5f-Nyrk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to objectivej+...@googlegroups.com.