NPK - Node.js packaging Utility

[Fred Chien]

Hi All,

I am glade to introduce NPK, which is a packaging utlity for Node.js. :-)

NPK aims to package project which is based on Node.js, and bundle all JavaScript files to make all in one file, even protect source code.

Here is repository at Github:

https://github.com/cfsghost/npk

You also can install it directly via npm:

npm install npk -g

BTW, We are now trying to support a feature that precompile JavaScript files to be binary to speed up loading time of program of Node.js.

Cheers,

Fred

[Mark Hahn]

>  feature that precompile JavaScript files to be binary

What does this mean?  Do you mean to minify the js?  That is not binary.

[Fred Chien]

It means "WE ARE TRYING" to do that, we're just writing some code which is using V8 API to precompile JavaScript files and generate binary things. THAT IS BINARY :-)

Fred

2013/5/11 Mark Hahn <ma...@hahnca.com>

>  feature that precompile JavaScript files to be binary

What does this mean?  Do you mean to minify the js?  That is not binary.

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Fred Chien]

We hope to have a feature to make a Binary but NPK doesn't support yet. That's our goal.

2013/5/11 Fred Chien <cfsg...@gmail.com>

[mscdex]

On May 11, 2:03 am, Fred Chien <cfsgh...@gmail.com> wrote:
> NPK aims to package project which is based on Node.js, and bundle all
> JavaScript files to make all in one file, even protect source code.

How does it protect the source code? What is to stop someone from
simply calling toString() on a function?

[Fred Chien]

Currently, NPK is using Uglify-JS to do code protection, something's like minimizing and obfuscation.

We are implementing a feature that putting JavaScript in C++ code and make it become Node.js native module. I think it could do more things that's like converting JavaScript to pure C++ code in future. :-S

BTW, another idea is, compiling JavaScript to bytecode and store it in native module. Convert it to JavaScript for V8 in runtime. I'm not sure whether it works or not.

Fred

2013/5/11 mscdex <msc...@gmail.com>

[Saleem Abdul Hamid]

If you convert it back to javascript for V8, then anyone else could do the same so you won't have really gained anything- no performance, because you're giving V8 js anyway and no code protection, because anyone can take your code and use it to convert the binary to javascript themselves.

It seems to me like the only way this could work is if you could get V8 to accept and run something other than JavaScript. Not sure if it's possible to give V8 bytecode for execution instead of Javascript? Maybe you could fork V8 and use a custom implementation with your packages?

Converting JavaScript to a self-contained, small, native binary would be very cool, but quite a bit of a project, obviously. Doing something like including the whole V8 in the binary would not be very useful.

[Fred Chien]

It's not easy to make V8 to run something other than JavaScript. V8 has no implementation of bytecode, therefore the only way I think is that converting JavaScript to C++ code. :-S

Fred

2013/5/11 Saleem Abdul Hamid <mee...@gmail.com>

[Mark Hahn]

Correct me if I'm wrong, but however you do it, converting to binary is going to be very hard with no conceivable advantage.

[Alex Kocharin]

 

Well, if NPK will be able to join all dependencies in one javascript file, it will already be a big deal. On my system "npm" executable file takes 1.153s to load from disk and 0.153s to load from cache. All binary executables like "gpg" or "dpkg" take 0.004s to load. It's really annoying sometimes...

 

--
// alex

 

 

11.05.2013, 10:03, "Fred Chien" <cfsg...@gmail.com>:

--

[Tim Caswell]

Alex, I share your pain.  That is why I made a version/port of node that didn't use V8, but rather, the much lighter-weight luajit engine.  It's at luvit.io. (warning, not compatible with node.js code or ecosystem, just the same idea / API style)

[Alex Kocharin]

 

If we are talking about non-compatible solutions, I might as well use python instead ;)

 

--
// alex

 

 

14.05.2013, 01:04, "Tim Caswell" <t...@creationix.com>:

[Tim Caswell]

If you ever come across a JS engine that boots as fast as Lua, I'll port node to it.  On my raspberry pi and my various webos smart-phones, node process booting takes over 1000ms, but lua is nearly instant.

Actually, I think a new JS engine who's goal is to be light-weight would be really neat actually.  As long as it doesn't run too terribly slow.

[Alex Kocharin]

 

Another implementation will go terribly slow.

 

V8 is a browser thingy, and I'm sure it is as fast as it could possibly be, because every millisecond counts there.

 

So there is nothing wrong with V8. But there could be something wrong with node.js itself. Think about hundreds of .js files it needs to read on startup... "npm" itself do 964 system calls to various */node_modules/*.js files when it's starting (use strace to check that).

 

--
// alex

 

 

14.05.2013, 01:17, "Tim Caswell" <t...@creationix.com>:

[Matt]

Here's the list of syscalls on a mac running just process.exit():

 150 stat64

  52 mmap

  33 mprotect

  29 ioctl

  24 munmap

  19 __semwait_signal

  13 madvise

   8 getattrlist

   4 sigaction

   4 lstat64

   4 __sysctl

   3 thread_selfid

   3 pipe

   3 open

   3 issetugid

   3 geteuid

   3 close

   2 write

   2 readlink

   2 read

   2 getuid

   2 getpid

   2 fstat64

   2 csops

   2 bsdthread_create

   2 __pthread_sigmask

   1 shared_region_check_np

   1 proc_info

   1 pread

   1 pathconf

   1 open_nocancel

   1 kqueue

   1 getrlimit

   1 getegid

   1 getaudit_addr

   1 fcntl_nocancel

   1 close_nocancel

   1 bsdthread_register

   1 audit_session_self

   1 __mac_syscall

   1 SYSCALL

[Tim Caswell]

V8 is as fast as it can possibly be at running hot.  Which for long-running node servers is great.  In fast, in my testing, javascript in node is generally slightly faster than lua in luajit.

V8, however is not optimized for process startup.  It uses a lot of ram and does a lot of up-front optimizing to get said runtime speed.  All software has to make tradeoffs and my experience with V8 and the V8 team is they will sacrifice startup time of the vm itself and memory usage, if it makes the scripts run significantly faster.

Yes there is some blame in node as well.  Node is under the assumption that most uses are for servers who startup rarely and run for a long time.  So a hundred ms of extra startup time on your server doesn't matter at all.

I'm not convinced that combining all your js files into a single file will significantly affect process startup time.  I'm happy to be proven wrong, but in my experience this is not the bottleneck.

One of my responsibilities when I worked on webOS was to make node startup fast-enough on older smart-phones that we could start node processes on-demand in reaction to user-input.  We tried creating a fork-server, using snapshots, composing and minifying js code.  But in the end, V8 and node were just too heavy and waiting on an on-demand node process affected user experience and introduced significant lag.  We would have just kept the node services running all the time, but then they used too much ram.  10mb each was simply too much for a smart-phone.

My solution was to use another Vm and it solved all my problems, but then it wasn't javascript anymore, and not many people were interested.  (Also the webOS project was killed by the then CEO)

If NPK can somehow remove the need for a JavaScript engine, then it will be fast, but if you end up needing all of V8 and js code to run, it will never help memory-usage or startup time, no matter how much you combine everything into a single file.  Just starting a node repl on these devices can take over 1000ms.

[Azer Koçulu]

OneJS also does the samething; http://github.com/azer/onejs

I myself use it for bundling my stuff for browsers, some others use it
for other platforms...