help get component to v1 with a component rewrite!

[Jonathan Ong]

i just spent some time rewriting component. it still needs a lot of work, but there are a lot of improvements. i made a blog post with ways you can help: http://jongleberry.com/component-rewrite.html

let's get component to v1 and kill all those issues in the repos!

[vision media [ Tj Holowaychuk ]]

kill! stab stab! 

I should note that we should be able to get rid of require() entirely, with our newly pre-computed dep resolution it should be pretty straight-forward, which will be super nice for --standalone, no extra cruft! 

I got a bit more of the registry done today, only had a few hours to spend but I'll get back on it tomorrow

On Sat, Jan 11, 2014 at 10:22 PM, Jonathan Ong <jonathanr...@gmail.com> wrote:

i just spent some time rewriting component. it still needs a lot of work, but there are a lot of improvements. i made a blog post with ways you can help: http://jongleberry.com/component-rewrite.html

let's get component to v1 and kill all those issues in the repos!

--
You received this message because you are subscribed to the Google Groups "component" group.
To unsubscribe from this group and stop receiving emails from it, send an email to componentjs...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jonathan Ong]

how would that implementation look like? can't think of a solution that would significantly save bytes and still work the same

[vision media [ Tj Holowaychuk ]]

my plan was to use https://github.com/visionmedia/node-requires and replace the require() calls with a resolved reference, but with require() being so small it wouldn't really matter much, I guess that wouldn't work well in script tag boot related stuff too so we would need to auto-require the initial bootstrap component

--

[Ian Storm Taylor]

nice! super happy to hear things moving towards 1.0 cuz i really want some solid building, versioning action :)

amir has a cool concept for what builder could look like here: https://github.com/yields/builder.js/tree/rework that is pretty sweet. handles the plugins system nicely. i also really like the ability to build scripts, styles easily from jonathan’s fork since that seems like a common use case and with the plugins we could have shorthands that would do that for you… wrote up a gist of what that might look like: https://gist.github.com/ianstormtaylor/8409952

that gist actually has a slight tweak to the api, in that the state is inside a “build” object that all the plugins get, which i also wrote up in a gist here: https://gist.github.com/ianstormtaylor/8409798

and then maybe those plugins should be pulled out and made into their own repos as well…

haven’t dived deep into the resolver/remotes stuff yet, but hopefully i can get some time this week to check that stuff out too. would definitely be good to have a way to resolve these things across component-* CLI tools without rewriting that faulty logic all the time

[John Syrinek]

Hey Jonathan, TJ, Ian, etc. Great job so far, and news of version 1.0 is really exciting. I started a new job about 6 months ago, and the first thing I did was "npm install -g component". Out of all the JavaScript libraries/frameworks out there, I really think Component.io gets it right (Ian - thanks for showing me the light in #segmentio). For this reason, I obviously want to see higher adoption, so here's some input from my short time in the corporate world. Forgive me if I'm ignorant of existing solutions to these problems. I've done lots of searching, but haven't come up with much.

1. Better documentation for `component build`

I'm happy to see the builder2.js overhaul. We have a crazy Makefile (which can't even be run on Windows without first installing MingW or Cygwin). Only recently have I realized you can use `component build --use ...` to extend the build process, but the documentation is still really lacking. I think you should at least allude to `--use` in the component/component Readme and find some alternative to GNU make for all the UI developers (and coworkers) out there using Windows. Here are some of the things our current setup support:

• Building
• Converting all .html files in our component (sub)directories to .js
• Unit testing: make test
  
• Acceptance testing: karma start karma.conf.js (I'd like to use component for this, but can't. See node/npm require() below)
• Getting a list of third-party licenses for our legal department
• Copying files to our development box via SSH (we build our own NIC's, so virtualization is near impossible)
• Updating third-party components

2. Revision records

"which is why I think we need component pin, component update, and component outdated commands" - YES! It also made me really happy to see this, as I just recently incorporated component build into our CI system. As I'm sure you know, this isn't reliable without some record of versions/revisions. Ideally, I'd like to always stay at the latest versions of third-party dependencies, so I don't care so much about semver or pinning dependencies at specific versions; however, I do need some way to be consistent across all working copies of our code. I think a simple lock file would be useful.

3. node/npm require()

I know the npm guys aren't budging on their current dependency resolver. Regardless, this causes a lot of confusion in the Component.io community I'm sure. At first glance, component seems like a way to run code in either the browser or a server. In practice, this is tricky, and it all boils down to very subtle differences in the require() implementations of npm and component. Some clear documentation on these two common use cases would be really helpful: 1) Using components in node, 2) Using npm packages in the browser alongside components. In the case of 2, I think it's fine to say "you can't" as long as the answer is clear. As far as I know, the only solution is using a try/catch clause with different require()'s. A better solution might be to support an (optional) list of dependencies and their paths (I know you don't like this idea). Maybe implement something like RequireJS's shim, but discourage its use?

4. Lazy loading

`component build` outputs a single file - well, one JS, one CSS, etc. We're expected to simply include this one file in our HTML. I think all of us prefer this over loading multiple files for performance reasons; however, on a large project like mine, build.js can become pretty large. It would be nice to be able to lazy load these files only when they're needed. This is critical for breaking the 1000ms mobile barrier. I consider this advanced, but I wanted to mention it anyways.

Again, thanks for devoting your time to such a great project. I've been contributing to some of the non-core libraries, and would love to help out more.

[Jonathan Ong]

build is going to have better documentation because we're probably going to remove a lot of features from the CLI like `-u`. people should just use the JS API (so make would be unnecessary).

component already converts .html files to .js as of 0.17 or something, so there's no need for you to do it. you have to store these files in .templates

what is required for licenses? component.json includes a license field - is that sufficient? or do you need the entire license?

we can push for client/browser usage by having people publish and use modules as <user>-<repo> i.e. require('component-emitter'). i'm also thinking of optional aliases so you can `require('emitter')` outside of the build (which you can't with this rewrite unless you do `require('component/emi...@1.1.1')`).

with the resolver, it will be easy to make multiple bundles. 

[Ian Storm Taylor]

good stuff. i’m definitely +1 having better docs. as soon as we get a builder/installer rewrite push i’m super down to be an ocd doc writer one of these weekends. the plugin system amir’s got is so badass it would be fun to write examples for anyways :D

component-license is a cool idea. i assume it just iterates the tree and prints out a name + license from the component json for every component. that would be a sweet third-party plugin

i like the user-name for getting them to work node side too. but honestly i’m either waiting for an npm alternative or for npm to adopt user/name semantics. both sound impossible :)

for the build, you might also want to look in to splitting up your build on your end too. we’ve moved to having N builds, one for each “page” of our app. some contain a few pages (login contains forgot password logic) but thats the general idea.

--

[John Syrinek]

build is going to have better documentation because we're probably going to remove a lot of features from the CLI like `-u`. people should just use the JS API (so make would be unnecessary).

Hell yes, just what I wanted to hear :D

i assume it just iterates the tree and prints out a name + license from the component json for every component

Yeah, I suppose I should move this to a builder.js plugin and publish it!

 we can push for client/browser usage by having people publish and use modules as <user>-<repo> i.e. require('component-emitter'). i'm also thinking of optional aliases so you can `require('emitter')` outside of the build (which you can't with this rewrite unless you do `require('component/emitter@1.1.1')`).

From what I've read, component has avoided using author names in require()'s to make swapping out the component repository more straight forward (e.g. when you fork a component). I've experienced the benefits of this first-hand on multiple occasions. Unfortunately, I don't think there's any way around npm's lack of namespace outside of being explicit with include paths. 

i’m either waiting for an npm alternative or for npm to adopt user/name semantics. both sound impossible :)

I couldn't agree more, but unfortunately, I feel this is an issue we should try to address for new users, even if it's something like "Trying to do this will only lead to confusion, but it's cool, check out all these other benefits...". 

I feel that thinking through this is one of the best things we can do to improve adoption, but to be honest, all of the direct/indirect dependency, npm-in-component, components-in-node confuses me. I almost wonder if it would be better to make require() pluggable with a require.use() or something similar. Users could address the include-path issue themselves on a case-by-case basis. Regardless, this has been my experience to date: 

1. Ooh, a package manager that works in the browser AND in node? TJ started it, and Ian from #segmentio recommends it?! (intrigued)
2. Start using it in browser
3. Try to load a simple npm package (confusion)
4. Give up and just avoid using npm packages in the browser
   
5. Try to use a component in npm (less confusion, but still confusion)
6. Avoid using components in node because it's a lot of work to fork a bunch of repos just to create component.json files
7. Look for alternatives to component that play better with npm
8. Realize all this confusion isn't so bad because component's other strengths make up for it (and then some)

 

with the resolver, it will be easy to make multiple bundles. 

Awesome! I'm going to dig deeper.

we’ve moved to having N builds, one for each “page” of our app

Cool, that's the route I was considering. Thanks for the validation! I wonder though, how do you deal with this scenario:

1. Component A is included in initial page load
2. Component A requires component B, which was not included in the initial page load

Do you use AJAX to load the bundle with component B in it? If so, is there any way to do this with plain ole require(), or do you use some sort of AJAX wrapper? This may be another justification for something like require.use(). I guess implementing this behavior in core would expand components scope from a dependency manager to a dependency manager PLUS loader (like LABjs). Hmm, I should document all this in the wiki somewhere.

Oh btw, I'm johntron on Freenode. I used to hang out in #segmentio, but haven't been on much lately.

[Ian Storm Taylor]

yeah i think sharing between node and component is cool, but it’s not component’s primary goal. it would be nice to have a simple guide for the ways to make the sharing between the two easy. there are four different ways i know of:

1. don’t publish to npm. instead just name the package.json the exact same as component and use the lesser known npm feature that lets you reference github repositories directly. you can see this in action at ianstormtaylor/is which installs by npm install ianstormtaylor/is — the downside is just that it means you have to hard pin deps with the `#x.x.x` syntax, and your versions in your package.json aren’t ideal

2. publish to npm with the owner-repo name like jonathan suggested, which means you can require it in component with the same owner-repo name because of how the aliases work

3. publish it to npm with whatever name you want, and then have a try/catch in your file requiring the component by both of its names, one for component and one for npm. you can see this in action at segmentio/validator

4. use a special module that augments require and lets you pass in two different names, and tries to resolve each of them internally. basically a nicer api for the try/catch method. https://github.com/CamShaft/require-component

5. we write an npm alternative that uses component style owner/repo namespacing, and has a fallback where if you don’t have a `/` then it hits the npm servers :) 

—

as for apps, we don’t load anything async after the page has loaded. every app is smaller so it just contains everything it needs on page load. actually i’ve never checked the size of the bundles… lets see… before bundling into apps our filesizes were..

css - 220kb

js - 1.1mb :)

and after bundling into apps they vary a lot per app, but generally…

css - 50kb - 100kb

js - 50kb - 500kb

all of that is unminified. and the ones that are big probably still have jquery deps that we haven’t weeded out yet. and then tons of require aliasing that will hopefully soon be un-reverted and cut down on size again

haven’t really cared about size too much though since we got other things that are more pressing. its a tradeoff cuz the homepage is a small app, so it loads much faster now, but then theres less caching since they are all separate. but we’ve done it mostly for code debt reasons, which are more important imo

[Jonathan Ong]

i hope to support this simple bundling case natively in component. it's pretty easy with a resolved dependency tree.

you have your /lib folder like this:

    /lib

      /boot

      /bundle-a

      /bundle-b

      /bundle-c

so each bundle will in fact be its own local component, just an entry point to the bundle just like boot. 

then the following files will be created:

    /build/boot.js

    /build/common.js

    /build/bundle-a.js

    /build/bundle-b.js

    /build/bundle-c.js

where common.js holds the common dependencies of all the non-boot bundles. if there's only one additional bundle, you can concatenate it with it with either bundle.

so on page load, you'll just have to do <script src="/build/boot.js"></script>. then when you load page "a", you'll have to do <script src="/build/common.js"></script><script src="/build/bundle-b.js"></script>. any subsequent pages shouldn't reference /build/common.js again.

[ericgj]

On Tuesday, January 14, 2014 10:52:24 PM UTC-5, Ian Taylor wrote:

yeah i think sharing between node and component is cool, but it’s not component’s primary goal. it would be nice to have a simple guide for the ways to make the sharing between the two easy. there are four different ways i know of:

Just to add to this #6: use forbeslindesay/is-browser , possibly my favorite component of all time ;) ... I prefer it over the ugly try/catch:

var isBrowser = require('is-browser')

var foo = isBrowser ? require('foo') : require('npm-foo')

 

[John Syrinek]

That IS a good component, Eric; however, it's not a good solution when you're trying to use third-party components/packages. You'd have to fork and patch any package you'd like to use. Since many package maintainers don't care about supporting component, forking would mean your repo will diverge from the original.

--
You received this message because you are subscribed to a topic in the Google Groups "component" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/componentjs/MJZ3P53PzYA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to componentjs...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

John Syrinek

@johntron

www.johntron.com

[John Syrinek]

1. don’t publish to npm. instead just name the package.json the exact same as component and use the lesser known npm feature that lets you reference github repositories directly. you can see this in action at ianstormtaylor/is which installs by npm install ianstormtaylor/is — the downside is just that it means you have to hard pin deps with the `#x.x.x` syntax, and your versions in your package.json aren’t ideal

2. publish to npm with the owner-repo name like jonathan suggested, which means you can require it in component with the same owner-repo name because of how the aliases work

3. publish it to npm with whatever name you want, and then have a try/catch in your file requiring the component by both of its names, one for component and one for npm. you can see this in action at segmentio/validator

4. use a special module that augments require and lets you pass in two different names, and tries to resolve each of them internally. basically a nicer api for the try/catch method. https://github.com/CamShaft/require-component

5. we write an npm alternative that uses component style owner/repo namespacing, and has a fallback where if you don’t have a `/` then it hits the npm servers :) 

Great info! I'll add a wiki page for this :D 

[John Syrinek]

See Component and npm. Also, the revised wiki homepage.

[ericgj]

On Wednesday, January 15, 2014 11:27:35 AM UTC-5, John Syrinek wrote:

That IS a good component, Eric; however, it's not a good solution when you're trying to use third-party components/packages. You'd have to fork and patch any package you'd like to use. Since many package maintainers don't care about supporting component, forking would mean your repo will diverge from the original.

True, the general problem remains with dependencies that don't have a component.json, or that themselves have dependencies that don't have component.json.  But otherwise I think it works fine (i.e. using component libraries from node) since as Ian pointed out you can reference the github repo directly in package.json.

[Ian Storm Taylor]

Nice! Add the is-browser method instead of the kill npm method, since that’s probably not a “component” stance haha that’s just me ranting :)

--

You received this message because you are subscribed to the Google Groups "component" group.

To unsubscribe from this group and stop receiving emails from it, send an email to componentjs...@googlegroups.com.

[Jonathan Ong]

btw you can check if you're in the browser environment by checking module.client or module.component https://github.com/component/require/blob/master/lib/require.js#L32

[Jonathan Ong]

btw guys if you have suggestions, feel free to open up an issue here: https://github.com/component/spec. comments on open and closed issues would be nice as well. 

On Saturday, January 11, 2014 10:22:42 PM UTC-8, Jonathan Ong wrote:

[John Syrinek]

I just got builder2.js working. The example had some flaws. I used the tests as reference and updating the README accordingly. PR here: https://github.com/component/builder2.js/pull/6 Now to build a plugin to compile test suites :D

On Sunday, January 12, 2014 12:22:42 AM UTC-6, Jonathan Ong wrote:

[Jonathan Ong]

yeah i just got my app building. saved 40% minified space from aliases and around 15% in the gzip build. 

i need to fix an issue with using the local + github remote, then i'll add a bunch of docs. 

[Jonathan Ong]

everything pretty much works right now. i updated the builder docs if you want to take a stab at it: https://github.com/component/builder2.js

here are some things that still need to be done before i'm going to bother making a PR to component(1):

- work without the harmony flag. i already precompile all the code, so once my PR for the regenerator runtime gets pulled, everything should start working on node >= 0.8 without any compilation steps. for now, just create a build script in JS and run with node --harmony-generators build.js

- windows. i need to normalize a lot of paths. if you have a windows machine, help would be nice, otherwise i'm just going to guess my way through everything.

- messages. i've added some debug statements, but i still want a good messaging API so end-users can know when stuff gets installed and what not. 

- validator. i need to validate and normalize component.jsons so older components will work. 

right now my resolve step is taking around ~90ms and my build step is ~900ms (1/2 is rework + autoprefixer + html-minifier) with a 31kb .min.js.gz and 12kb min.css.gz. it's nice since the builder is split into three, you won't see that ~400ms rework/autoprefixer overhead when just building js. 

[John Syrinek]

Yeah, it's working for me too, but having a few issues. builder2 wasn't including "local" dependencies in the output when I used Github and Local (explicitly instantiated) - only "dependencies" and "scripts". I had to switch gears, but when I have time I'll try to reproduce/fix/test. Regardless, it feels close. I'm super busy this week with work, but I'd like to try to help out with source maps.

[Jonathan Ong]

we changed the spec from "local" to "locals", so you have to change your component fields. i haven't worked on backwards compatibility yet, but there will be a validator.js that handles this.