(on engine branch) Using npm modules in smart packages

[Avital Oliver]

Smart packages can now depend on npm modules. Prior to this change, certain core npm modules were included with Meteor itself (as part of our "dev bundle"), and you had to resort to hacks to use others. We've leveled the playing field so that now your smart package can define dependencies directly. Each smart package now has the same ability. To try this out, make sure you're running against the engine branch.

To declare a dependency on an npm module, add a call to Npm.depends in package.json, such as:

Npm.depends({

  "awssum": "0.12.2",

  "underscore.string": "2.3.1"

});

One of the goals of our npm integration is to ensure that users of a package get replicable results. To ensure that, we make sure to pin dependency versions all the way down. These recursive dependency versions are captured in a '.npm/npm-shrinkwrap' file automatically generated within your package directory. All of this works well with hot code reload, just like the rest of Meteor. When you make changes, the bundler will automatically download missing npm packages and re-pin its dependencies. You never have to manually edit npm-shrinkwrap.json.

To use a module within server code, use `Npm.require` as you would normally use plain `require`. Notably, `__meteor_bootstrap__.require` has been eliminated and all of its uses have been converted to `Npm.require`.

Meteor code is generally not written in the typical npm async style, since we think synchronous APIs supply a better user experience. Fortunately, in many cases it's fairly straight-forward to transform async functions into synchronous ones using `Future.wrap`. We'll be doing some more work on best practices, including a helper, perhaps named `Meteor.wrapAsync` that will supply an API that accepts an optional callback. This way you can use either synchronous or asynchronous styles in smart packages wrapping npm modules. This is similar to the server API we have in the 'http' package.

We'll be making some more changes before we merge to devel, after which we'll have a more official release, but we're pretty confident this part of the API won't be changing.

Examples can be found in core packages such as mongo-livedata and less. I've also ported Mike Bannister's 'meteor-awssum' package to use our new conventions. You can find it here: https://github.com/avital/meteor-awssum

[Avital Oliver]

Yes, that's right. :)

On Fri, Feb 22, 2013 at 11:02 AM, Patrick Coffey <patrick...@gmail.com> wrote:

I believe you mean package.js, not package.json. :)

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Andrew Wilcox]

using `Future.wrap`

Oh, by the way, fyi I ran into an issue with Future.wrap the other day.  The source looks like this:

Future.wrap = function(fn, idx) {
  idx = idx === undefined ? fn.length - 1 : idx;

fn.length is the declared number of function arguments (MDC), not the actual length of the arguments.  I had some other function wrapper I had to use between the async API method and Future.wrap, and it was returning a vararg style function:

function anotherWrapper (fn) {

  ...

  return function (/* arguments */) { return fn.apply(...) };

}

so inside of Future.wrap, fn.length was coming out 0.

So an edge case to watch out for when implementing Meteor.wrapAsync is when you're looking for the callback at the end of the argument list, make sure you're using arguments.length instead fn.length :)

Andrew

[Blake Miner]

When will this functionality be available?  Which Meteor release?

[Blake Miner]

Also... what if the smart package `package.js` file needs to `require` a file to properly perform `register_extension` calls?  A good example is the CoffeeScript smart package.  If CoffeeScript wasn't a core Meteor package, how could this be modified to `require("coffeescript")` within the smart package???

[Avital Oliver]

Hi Blake,

We're on the final stretch of work (you can follow the engine branch if you're curious), but I can't commit to an exact date. It will most likely be part of a 0.6.0 release, but there will be at least one release on the 0.5.x channel first.

The coffeescript package on the engine branch is already ported to the new scheme: https://github.com/meteor/meteor/blob/engine/packages/coffeescript/package.js. Does this answer your question?

Thanks,

Avital.

[Blake Miner]

Yes.  Thanks very much for the clarification.  `Npm.depends` is called in `package.js`.  `Npm.require` can then be used later.  This is awesome!  Thanks again!

--
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/b6zQrgk8lYo/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.

[Ken Yee]

On Thursday, February 7, 2013 7:22:20 PM UTC-5, Avital Oliver wrote:

Smart packages can now depend on npm modules

Can you do this for your app as well?  I.e., in the startup() code, do the Npm.require to pull it in automatically?
Or would we have to create our own smart package that's added to our app, then that smart package declares all the requires?
 

[Avital Oliver]

Apps can't directly use npm packages at the moment. What you'd do is create a smart package in your app's packages/ directory. On the engine branch, these are always added to your app. Then you can easily use Npm.depends in those packages and expose an interface to your app.

--

[Blake Miner]

One more question.  Does the "engine" branch address the following issues?

• https://github.com/meteor/meteor/issues/181 - ability to load certain JavaScript files before others
  
• https://github.com/meteor/meteor/issues/180 - bundler options are not available to smart packages
  

Thanks!

[Avital Oliver]

Neither of them are addressed per se (as we are not yet done with the full packaging work), but you can now put packages directly in your app (in the packages/ directory within the app directory). Each of those packages have a 'package.js' file, in which you can specify file load order.