New 0.6.5 release candidate ; please test your apps

[Nick Martin]

Hey all,

We're getting really close to the 0.6.5 release. I've pushed a new RC: 0.6.5-rc15. Please give it a whirl in your app with:

   meteor update --release 0.6.5-rc15

Changes since the last RC:

- Don't allow package names with dots or special characters (dots were already broken, disallow them earlier in the codepath)

- fixup oauth popup window sizing and scrollbars

- include err.response on oauth errors

- upgrade mongodb driver to 1.3.17

- correctly show crash log in browser in development mode

- print path of failed match on `check` failure

- add `Match.Integer`

- fix less package error messages

Below is a first-cut at the History.md for this release, detailing all the changes since 0.6.4.

Cheers,

-- Nick

## v0.6.5

* New package system with package compiler and linker:

  * Each package now has it own namespace for variable

    declarations. Global variables used in a package are limited to

    package scope.

  * Packages must explicitly declare which symbols they export with

    `api.export`.

  * Apps and packages only see the exported symbols from packages they

    explicitly use. For example, if your app uses package A which in

    turn depends on package B, only package A's symbols will be

    available in the app.

  * Package names can only contain alphanumeric characters and

    dashes. Packages with dots, spaces, and underscores must be renamed.

  * Remove hardcoded list of required packages. New default

    `standard-app-packages` package adds dependencies on the whole

    Meteor stack. This package can be removed to make an app with only

    parts of the Meteor stack.

  * Custom app packages in the `packages` directory are no longer

    automatically used. They must be explicitly added to the app with

    `meteor add <packagename>`.

  * New "unipackage" format for built packages. Compiled packages are

    cached and rebuilt only when their source or dependencies change.

  * Add "weak" and "unordered" package dependency modes to allow

    circular package dependencies and conditional code inclusion.

  * New `registerBuildPlugin` API for declaring compilers,

    preprocessors, and file extension handlers. The old

    `register_extension` API is deprecated.

  * Move HTTP serving out of core Meteor into the `webapp` package. This

    allows building Meteor apps that are not web servers (eg. command

    line tools, DDP clients, etc.). Connect middlewares can now be

    registered on the new `WebApp.connectHooks` instead of the old

    `__meteor_bootstrap__.app`

* Add JavaScript sourcemaps in development mode. This includes files

  compiled by CoffeeScript. XXX how to use.

* Add new `Assets` API and `private` subdirectory for including and

  accessing static assets on the server. http://docs.meteor.com/#assets

* Add `Meteor.disconnect`. Call this to temporarily disconnect from the

  server and stop all live data updates. #1151

* Add `Match.Integer` to `check` for 32-bit signed integers.

* New `Log` function which prints with timestamps, color, filenames and

  linenumbers.

* `Meteor.connect` has been renamed to `DDP.connect` and is now fully

  supported on the server. Server-to-server DDP connections use

  websockets, and can be used for both method calls and subscriptions.

* Rename `Meteor.default_connection` to `Meteor.connection` and

  `Meteor.default_server` to `Meteor.server`.

* Rename `Meteor.http` to `HTTP`.

* `ROOT_URL` may now have a path part. This allows serving multiple

  Meteor apps on the same domain.

* Allow creating named unmanaged collections with

  `new Meteor.Collection("name", {connection: null})`

  XXX note on why useful XXX doc update!

* Include http response in errors from oauth providers. #1246

* The `observe` callback `movedTo` now has a fourth argument `before`.

* Move NPM control files for packages from `.npm` to

  `.npm/package`. Also, when removing the last NPM dependency, clean up

  the `.npm` dir.

* Remove deprecated `Meteor.is_client` and `Meteor.is_server` variables.

* Implement "meteor bundle --debug" #748

* Add `forceApprovalPrompt` option to `Meteor.loginWithGoogle`. #1226

* Make server-side Mongo `insert`s, `update`s, and `remove`s run

  asynchronously when a callback is passed.

* Improve memory usage when calling `findOne()` on the server.

* Delete login tokens from server when user logs out.

* Rename package compatibility mode option to `add_files` from `raw` to

  `bare`.

* Fix Mongo selectors of the form: {$regex: /foo/}.

* Fix Spark memory leak.  #1157

* Fix EPIPEs during dev mode hot code reload.

* Fix bug where we would never quiesce if we tried to revive subs that errored

  out (5e7138d)

* Fix bug where `this.fieldname` in handlebars template might refer to a

  helper instead of a property of the current data context. #1143

* Fix submit events on IE8. #1191

* Handle `Meteor.loginWithX` being called with a callback but no options. #1181

* Work around a Chrome bug where hitting reload could cause a tab to

  lose the DDP connection and never recover. #1244

* Upgraded dependencies:

  * Node from 0.8.18 to 0.8.24

  * MongoDB from 2.4.3 to 2.4.4, now with SSL support

  * CleanCSS from 0.8.3 to 1.0.11

  * Underscore from 1.4.4 to 1.5.1

  * Fibers from 1.0.0 to 1.0.1

  * MongoDB Driver from 1.3.7 to 1.3.17

Patches contributed by GitHub users btipling, mizzao, timhaines and zol.

[Ken Yee]

On Friday, August 9, 2013 10:21:26 PM UTC-4, Nick Martin wrote:

  * Packages must explicitly declare which symbols they export with

    `api.export`.

Is there any way to flag packages that are missing this?  i.e., old ones that haven't updated to the new API?
Or is it a matter of trying your app and seeing what blows up? :-)

[Gadi Cohen]

I second this :)  Maybe an api.export(null) to explicitly specify that there are no exports, and then Meteor could give a warning if the package doesn't call api.export() at all?  I think this could result in clearer package code too.

[Andrew Wilcox]

Another idea: in Atmosphere allow a package author to annotate which versions of Meteor a particular version of a package has been tested and is expected to work with.

For example, I could say that my package version 1.0.1 works with Meteor 0.6.4, 0.6.4.1, and 0.6.5-rc15.

One place to put this would be in the smart.json file, maybe something like this:

{

  "name": "foobar",

  "version": "1.0.1",

  "supports": {

    "meteor": ["0.6.4", "0.6.4.1", "0.6.5-rc15"]

  }

}

[David Glasser]

Indeed, one of the main things separating the current implementation
of Atmosphere from an eventual fully-supported package repository
would be solving problems like this one.

> --
> You received this message because you are subscribed to the Google Groups
> "meteor-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to meteor-talk...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[David Glasser]

I don't think this makes much sense. A package doesn't have to have
code: it can just be images or CSS. Or it can just *do* something
when loaded instead of defining an API. Or it can be an empty package
which is checked for by other packages with weak dependencies on it
(like insecure and autopublish).

[David Glasser]

On Fri, Aug 9, 2013 at 10:21 PM, Nick Martin <n...@meteor.com> wrote:
> Hey all,
>
> We're getting really close to the 0.6.5 release. I've pushed a new RC:
> 0.6.5-rc15. Please give it a whirl in your app with:
>
> meteor update --release 0.6.5-rc15
>
> Changes since the last RC:
>
> - Don't allow package names with dots or special characters (dots were
> already broken, disallow them earlier in the codepath)

After auditing Atmosphere, we decided to allow dots in package names.
(The package/slice separator character will change from '.' to ':'.
Note that we don't yet actually have a good way to declare your own
slices in packages, but the infrastructure is in place to do that in a
later release.) I'm working on the change now.

[Ken Yee]

On Monday, August 12, 2013 3:20:09 PM UTC-4, Andrew Wilcox wrote:

Another idea: in Atmosphere allow a package author to annotate which versions of Meteor a particular version of a package has been tested and is expected to work with. 

I like this better if it's possible.  That'd make it work like other packaging systems...
 

[James Wilson]

Is there a solid upgrade path for all of the packages that will break? Does there need to be?

How much is too much?

When I do meteor update, can it map that accounts-url internal package was rolled into accounts-base and

doesn't exist any more?

Thank you everyone for their hard work!

James Wilson

Shameless plug: I made a script to generate a project structure following the unofficial faq:

http://gist.github.com/jamesdwilson/6136770

James Wilson

--

[David Glasser]

All the package built in to Meteor no longer depend on accounts-urls.

Third party packages will have to upgrade themselves. Yes, this isn't
great, which is why one of the main things that needs to go into the
design of a fully supported package system is making upgrades easy.

[Nick Martin]

Aaaand one more RC =). New release is: 0.6.5-rc16. Give it a whirl with:

    meteor update --release 0.6.5-rc16

Only a couple changes from -rc15:

- re-allow dots in package names (by changing the slice separator to ':' instead of '.')

- clean up error message in match failures in the check package.

-- Nick

On Fri, Aug 9, 2013 at 10:21 PM, Nick Martin <n...@meteor.com> wrote:

[Mitar]

Hi!

On Mon, Aug 12, 2013 at 4:20 PM, Andrew Wilcox <andrew...@gmail.com> wrote:
> Another idea: in Atmosphere allow a package author to annotate which
> versions of Meteor a particular version of a package has been tested and is
> expected to work with.

Oh, no. Not that. This is what Mozilla had with their add-ons and
means that everytime they released a new version of Firefox, add-ons
become "incompatible" and you as a developer had to go and bump the
versions of Firefox supported. Initially, you had to release a new
version (like in your proposed case changing the smart.json), later,
you could just update this data over web interface. But this still
meant that you as a developer had to maintain the package all the
time. Even if there were no changes really necessary. I think this is
not a good approach. Frankly, I don't have time for it.

If something, then packages should depend on APIs. If that API change,
then the package manager should complain or something. Meteor versions
are just to coarse.

But I think the easiest approach is simply to have Travis CI tests and
some way that when a new release of Meteor is done, package is
retested against it. I think Atmosphere would be the best place to
support that. So every time a new Meteor is released, for all packages
on Atmosphere, which have Travis CI enabled, tests are run. Atmosphere
could even display tests results (green/red/gray) next to each
package. Even on the index page - this would make developers want to
be "green".

So you could have one color for Meteor stable, and one color for
Meteor development. This would help Meteor developers as well, because
Matt could immediately see, after new RC, how many packages on
Atmosphere still work and how much they broke the ecosystem.

And then, if the developer really wants, he or she can always make a
test which checks if current Meteor version is 0.6.4, 0.6.4.1, or
0.6.5-rc15. :-)

So I believe that would be a great continuous integration based
packaging system. Forget about versions and dependencies between
versions. This is so old-school and doesn't really work. Why we have
tests? Because you cannot trust versions. :-)


Mitar

> For example, I could say that my package version 1.0.1 works with Meteor
> 0.6.4, 0.6.4.1, and 0.6.5-rc15.
>
> One place to put this would be in the smart.json file, maybe something like
> this:
>
> {
> "name": "foobar",
> "version": "1.0.1",
> "supports": {
> "meteor": ["0.6.4", "0.6.4.1", "0.6.5-rc15"]
> }
> }
>

> --
> You received this message because you are subscribed to the Google Groups
> "meteor-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to meteor-talk...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

http://mitar.tnode.com/
https://twitter.com/mitar_m

[Mitar]

Hi!

Is is already possible to have somewhere in the project code
repository some JavaScript code which should not be bundled? For
example, I would like to have a background worker code in the same
repository, but there is no real suitable place for it. "test" and
"packages" is not really that. And everything else goes immediately
into the bundle.

Why does Meteor take all the files from the repository anyway? Why it
is not just "client", "server", "lib" and root directories which it
uses? I think it is better if this would be more strict. It is easier
to reason about and easier to have custom directories in the
repository.

The best current approach I can think of is just to create a directory
under packages and misuse it for custom code?

Is it possible to run Meteor in console-only mode, like how it is
possible to run tests? Just to run management/background commands in
Meteor environment?


Mitar

[Andrew Wilcox]

For example, I could say that my package version 1.0.1 works with Meteor 0.6.4, 0.6.4.1, and 0.6.5-rc15.

One place to put this would be in the smart.json file, maybe something like this:

{

  "name": "foobar",

  "version": "1.0.1",

  "supports": {

    "meteor": ["0.6.4", "0.6.4.1", "0.6.5-rc15"]

  }

}

Actually, come to think of it, smart.json isn't a good place to put this information.  Suppose I release version 1.0.1 of my packages saying that it now works with Meteor 0.6.5... but then later I realize that 1.0.1 no longer works with 0.6.4.  If the "X works with Y" information is stored in the package itself, then I can't update the "works with" information without releasing a new version 1.0.2 of the package... which still doesn't fix the problem that 1.0.1 claims to work with 1.0.1.

So it would be better for the "supports" or "works with" information to be stored separately in Atmosphere, where it can be updated as needed.

But I think the easiest approach is simply to have Travis CI tests and 
some way that when a new release of Meteor is done, package is 
retested against it.

 

These aren't incompatible.  If you're lucky enough that automated tests can determine that X works with Y reliably for you, then you can have the test results feed automatically into the "X works with Y" data; otherwise, for packages where that isn't possible or haven't had automated tests written for, one can update that info manually.

[David Glasser]

This sounds like a static asset, so "public" should do the trick, or
if this is in a package, passing {isAsset:true} (new in 0.6.5) to
add_files.

[Andrew Wilcox]

Minor complaint:  is logging the timeInexact status necessary in dev?  I'm finding it irritating to have all my debug messages prefixed by "?"...

[Mitar]

Hi!

So all non-Meteor files in repository should go into public? But what I want to keep them private to the server?


Mitar

[Andrew Mao]

With regard to 

* Allow creating named unmanaged collections with

  `new Meteor.Collection("name", {connection: null})`

  XXX note on why useful XXX doc update!

The reason this is useful is here: https://groups.google.com/forum/#!topic/meteor-talk/Ze9U9lEozzE

On Friday, August 9, 2013 10:21:26 PM UTC-4, Nick Martin wrote:

[Mitar]

Hi!

On Tue, Aug 13, 2013 at 9:08 AM, Andrew Wilcox <andrew...@gmail.com> wrote:

But I think the easiest approach is simply to have Travis CI tests and 
some way that when a new release of Meteor is done, package is 
retested against it.

 

These aren't incompatible.  If you're lucky enough that automated tests can determine that X works with Y reliably for you, then you can have the test results feed automatically into the "X works with Y" data; otherwise, for packages where that isn't possible or haven't had automated tests written for, one can update that info manually.

Lucky enough? You are saying that versions can be better at that? If all developers would use versions correctly and write bug-less code, maybe. But this will never happen.

And as I said, the main reason is that I don't want to be bothered with package maintenance until some test fails. On me (as a developer) is that I make good tests. On ecosystem is that it works until it breaks. Not that I have to update versions every time some dependency updates the version. Or that thing breaks but nobody notices until somebody installs on a fresh system and gets a newer version that was last tested by.

So this is coming from my experience with Python libraries. If you defined maximum version of a dependency, you had to update it manually every time (it does not matter so much if it is on the web interface or in repository). If you didn't, then things sometimes broke and you discovered this when somebody else tried to use your library. Or when you updated the code and Travis CI installed some newer version of dependency to run tests and tests fail on something completely different, because dependency changed.

Tests should provide complete coverage anyway and there should then be no luck involved. As I said, you can always create a test to test versions, if you are lazy (some boilerplate could be provided for that as well).


Mitar

[Tom Coleman]

I don't see any fundamental reason why you couldn't change the package metadata of an already released package?

Am I missing something?

--
You received this message because you are subscribed to the Google Groups "meteor-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-core...@googlegroups.com.
To post to this group, send email to meteo...@googlegroups.com.
Visit this group at http://groups.google.com/group/meteor-core.

[Tom Coleman]

Some thoughts:

1. If we restrict the discussion to fixing Meteor version compatibility in packages (as opposed to _other package_ version support [1]), then keep in mind that Meteor is "unstable" at the moment and likely to be released less often once 1.0 has been reached.

2. I feel your pain Mitar, but keep in mind that the tools can ease it.

Suppose that we already had this feature as of now, and tomorrow 0.6.5 was released. Then you might be annoyed that I haven't yet released a version of the Router that explicitly supports 0.6.5, so you couldn't update your app.

What you could do is fork me on github, test, and add 0.6.5 to the smart.json (and send me a PR!). Then you could change your app's smart.json to:

 'packages': {

   'router': {git: 'https://github.com/mitar/meteor-router'}

 }

And use it that way until I get my act together.

3. If we had the other "missing" feature from meteorite/atmosphere, which is "github-style" namespaced packages, then you could publish your own fork of the router to atmosphere, and the above would be even easier. 

[hint; something like

  packages: {

    'mitar:router': {}

  }

]

4. If anyone is feeling keen and wants to implement either of those features, that would be super ;)

Tom

[1] The "dependency hell" issue that fixed version dependencies introduces does worry me too. To some degree it's mitigated by packages being fixed to Meteor versions, ala Debian. Plus 2+3. above will probably help too.

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.

[Andrew Wilcox]

On Wednesday, August 14, 2013 12:10:00 AM UTC-4, Tom Coleman wrote:

I don't see any fundamental reason why you couldn't change the package metadata of an already released package?

Right, you do want to be able to change the metadata (of "X works with Y") of an already released package.

But, for the files that make up a particular release of a package, we also want those files to be immutable.  That is, if someone installs version 1.2.3 of my package on Monday, and someone else installs version 1.2.3 of my package on Tuesday, the files that they get in their packages directory should be the same.  (It gets way too confusing to talk about whether version "1.2.3" has a bug in it or not otherwise).

The solution is for the information "1.2.3 works with Meteor 0.6.5" to be stored in the database about 1.2.3 instead of being stored inside of 1.2.3.

Suppose that we already had this feature as of now, and tomorrow 0.6.5 was released. Then you might be annoyed that I haven't yet released a version of the Router that explicitly supports 0.6.5, so you couldn't update your app.

There's no reason to prevent someone from using Router version 0.5.3 with Meteor 0.6.5, even if you haven't yet said whether 0.5.3 works with 0.6.5 or not.  If someone explicitly lists "router": "0.5.3" as the version they want to use, mrt should go ahead and install that version.  At most, print a warning.

What you could do is fork me on github, test, and add 0.6.5 to the smart.json (and send me a PR!).

No no no :)  If which versions of Meteor a particular version of a package works with is part of the version of a package, this has unpleasant consequences:

• If a new version of Meteor comes out, I have to release a new version of the package, even if it hasn't changed.  But I want to keep track of when my package has actually changed: e.g. to be able to say things like "no change was needed for 0.6.5".
  
  
• If I make a mistake, and release a package version 1.2.3 that claims to work with Meteor 0.6.1 when in fact it doesn't, the only way for me to fix that mistake is to release a new version of my package.  But having a new version of the package doesn't change 1.2.3 mistakenly claiming to work with 0.6.1!  So now I can't reliably say "Still using Meteor 0.6.1?  The last version that supported 0.6.1 was 1.2.3".

It's natural for me to ask "so, does Router version 0.5.3 work with Meteor 0.6.4.2?"  The reason why it's natural for me to ask that is because 0.5.3 could work with 0.6.4.2 if such a patch release came out: we just don't know yet.

So, put the "works with" metadata information in the database, not inside the package.

As I said, you can always create a test to test versions

This is backwards.  Tests are an automated way to help tell me whether versions work together or not.   Whether "1.2.3 works with Meteor 0.6.5" is information that comes out of a test, not input to a test.

If all developers would use versions correctly and write bug-less code, maybe.

I think you're confusing having a database about which versions work together with what Python does.

If I run tests with version 1.2.3 of my package against Meteor 0.6.5 and the tests pass, this adds to my knowledge of the world: "1.2.3 works with 0.6.5" (as far as the tests can tell).

I can either accept that as-is; write a failing test to prove that 1.2.3 doesn't work with 0.6.5; or state that "I know that 1.2.3 doesn't work with 0.6.5" (even if I don't know how or don't have time to write a test to show that).

[Tom Coleman]

On Wednesday, 14 August 2013 at 6:47 PM, Andrew Wilcox wrote:

On Wednesday, August 14, 2013 12:10:00 AM UTC-4, Tom Coleman wrote:

I don't see any fundamental reason why you couldn't change the package metadata of an already released package?

Right, you do want to be able to change the metadata (of "X works with Y") of an already released package.

But, for the files that make up a particular release of a package, we also want those files to be immutable.  That is, if someone installs version 1.2.3 of my package on Monday, and someone else installs version 1.2.3 of my package on Tuesday, the files that they get in their packages directory should be the same.  (It gets way too confusing to talk about whether version "1.2.3" has a bug in it or not otherwise).

The solution is for the information "1.2.3 works with Meteor 0.6.5" to be stored in the database about 1.2.3 instead of being stored inside of 1.2.3.

I think we are talking in the same direction, except that perhaps we disagree about the purpose of the smart.json for a package -- in my mind it's exactly how you communicate with atmosphere's database.

Perhaps the list of package dependencies listed in smart.json needs to be immutable for a single package version however. We could make a second file, but this is kind of sucky; anyway this is a technical detail that won't survive the merge into core I'd imagine.

 

Suppose that we already had this feature as of now, and tomorrow 0.6.5 was released. Then you might be annoyed that I haven't yet released a version of the Router that explicitly supports 0.6.5, so you couldn't update your app.

There's no reason to prevent someone from using Router version 0.5.3 with Meteor 0.6.5, even if you haven't yet said whether 0.5.3 works with 0.6.5 or not.  If someone explicitly lists "router": "0.5.3" as the version they want to use, mrt should go ahead and install that version.  At most, print a warning.

What if they(do what most people do and) don't specify a version? Does it stop you from upgrading to 0.6.5 ?

What you could do is fork me on github, test, and add 0.6.5 to the smart.json (and send me a PR!).

No no no :)  If which versions of Meteor a particular version of a package works with is part of the version of a package, this has unpleasant consequences:

• If a new version of Meteor comes out, I have to release a new version of the package, even if it hasn't changed.  But I want to keep track of when my package has actually changed: e.g. to be able to say things like "no change was needed for 0.6.5".
  
  
• If I make a mistake, and release a package version 1.2.3 that claims to work with Meteor 0.6.1 when in fact it doesn't, the only way for me to fix that mistake is to release a new version of my package.  But having a new version of the package doesn't change 1.2.3 mistakenly claiming to work with 0.6.1!  So now I can't reliably say "Still using Meteor 0.6.1?  The last version that supported 0.6.1 was 1.2.3".

It's natural for me to ask "so, does Router version 0.5.3 work with Meteor 0.6.4.2?"  The reason why it's natural for me to ask that is because 0.5.3 could work with 0.6.4.2 if such a patch release came out: we just don't know yet.

So, put the "works with" metadata information in the database, not inside the package.

Again, see point above.

I was imagining you'd just update smart.json to list "meteor: 0.6.4.2" and run `mrt release .` without bumping your package version.

I definitely think a file in the git repo has advantages here, cf. forking / PRs etc.

Tom

[Gadi Cohen]

Ok, I have a better proposal this time :)

I think it's a mistake to constantly require package authors to have to modify their packages (source or repository metadata) after every platform upgrade.

If Meteor is the platform of the future, I think it needs to implement API compatibility layers like Android does.  e.g:

Meteor 0.6.4 uses API 5, automatically exports global variables;
Meteor 0.6.5 uses API 6, requires explicit api.export() calls.

App X specifies a { targetAPI: 5 } in package.js.  Meteor notes this while loading, and runs a compatibility layer, which will still export global variables automatically.

Hence, every time Meteor changes the API, ALL apps will continue to run as expected, with NO changes required by every single developer.

Obviously this might be irrelevant for pre-1.0, but I think for 1.0 this should be a priority.  My 2 cents :)

For reference, the Android SDK reference on this strategy:
http://developer.android.com/guide/topics/manifest/uses-sdk-element.html

Gadi

[Gadi Cohen]

Obviously (as noted in the Android SDK), you should also be able to specify a minimum API version that has all the features your package requires.

[Mitar]

Hi!

On Wed, Aug 14, 2013 at 1:14 AM, Tom Coleman <t...@thesnail.org> wrote:

What you could do is fork me on github, test, and add 0.6.5 to the smart.json (and send me a PR!). Then you could change your app's smart.json to:

Isn't it easier if Travis CI tests just run, they check that your package works with latest Meteor, and I know everything works. No PR needed, no forking, no smart.json changes. Nothing? Only when something breaks, we have and should do some changes. Not when everything works.

It is really simple for me: if something can be done automatically by computers, it should be done. If computers can test and see that everything works with new version, then let's just make it so we can then all use that new version, automatically. And you anyway needs tests which cover all your use cases. So you provide everything needed for automation already.

Even more, if we all, who are using a given packages, write our tests against that package, this is really a great tool for that package develop to have a good test suite against its APIs. If no dependent package breaks, develop can know that everything still works as it should. If something break, there is something wrong: in the package or in the dependent package. The result is that whole ecosystem is improved.

I agree that this might not be used instead of versions. But I really believe that Atmosphere is the best place for such a feature.

And use it that way until I get my act together.

"Use that" means I have to do some work, when it could be done automatically.

3. If we had the other "missing" feature from meteorite/atmosphere, which is "github-style" namespaced packages, then you could publish your own fork of the router to atmosphere, and the above would be even easier.

Why would I have to publish anything in most cases where the upgrade didn't break anything?

Mitar

[Mitar]

Hi!

On Wed, Aug 14, 2013 at 6:05 AM, Gadi Cohen <dra...@wastelands.net> wrote:

If Meteor is the platform of the future, I think it needs to implement API compatibility layers like Android does.  e.g:

Meteor 0.6.4 uses API 5, automatically exports global variables;
Meteor 0.6.5 uses API 6, requires explicit api.export() calls.

No! Please, really, stop thinking about versions as numbers. This is definitely not the future. Trying to serialize all changes in consequent versions is always problematic. If API 5 has some feature you are not using and then they upgrade to API 6 because that feature you are not using changed, what do you care about that?

Or we will have for each function in the API its own version? Crazy.

I really suggest you all check this video, it is really great in my opinion:

http://vimeo.com/71278954

The point I am trying to make here is that API should be discoverable: if my application does tests against your API and everything passes, this means API is compatible and this is all I care about. I don't care about versions, I care about my application being able to speak to your library. And this should really be a way of self-negotiating protocol. We have all components for such protocol: integration tests, automation tools, Travis CI, central package repositories, git repositories with code versioning, always online cloud services able to communicate to each other.

So why future development would not be like this: I read documentation of your library, I write code and tests (I need them anyway) in the way how I understand that your library works for me. If it works for me, great, I use that. If not, I debug. Then once it works for me, I am using it. When it fails with new versions. I debug. Or keep using the old version if I don't have time. This is it.

App X specifies a { targetAPI: 5 } in package.js.  Meteor notes this while loading, and runs a compatibility layer, which will still export global variables automatically.

I think the idea is that Meteor will not change so much after 1.0 version. This is why they are changing things now, to try different ideas.

But I doubt all library developers will have time to keep all possible compatibility layers (sometimes they are even not possible!) up to date. I am worried about that. Not so much about changes in Meteor. I am worried more about changes in packages my applications depend on.

Mitar

Hence, every time Meteor changes the API, ALL apps will continue to run as expected, with NO changes required by every single developer.

Obviously this might be irrelevant for pre-1.0, but I think for 1.0 this should be a priority.  My 2 cents :)

For reference, the Android SDK reference on this strategy:
http://developer.android.com/guide/topics/manifest/uses-sdk-element.html

Gadi

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Ken Yee]

On Wednesday, August 14, 2013 5:00:01 AM UTC-4, Tom Coleman wrote:

What if they(do what most people do and) don't specify a version? Does it stop you from upgrading to 0.6.5 ?

If you're dependent on a package that hasn't been updated for 0.6.5, yes.
That's how Debian's package manager works.  If your package is dependent on another package that hasn't been updated, then your package is kept back.
That's better than upgrading a package partially only to find out that it's really broken because it won't work.

[Andrew Wilcox]

I definitely think a file in the git repo has advantages here, cf. forking / PRs etc.

Right, if you look at GitHub, there are commits which are immutable, and then there are things tied to commits, such as tags.  Then there are other things which turn out better not to be tied to commits, such as issues which are tied to repositories instead.  (If issues were tried to commits, then I wouldn't be able to add a new issue without making a new commit).

So commits are a good place to start, but then sometimes we may think it through and realize that tying something to a commit isn't the best choice.  In terms of "X works with Y", the analogy with issues is actually pretty close.  I make a release, and then realize that my new version doesn't work with Meteor 0.6.1 anymore.  If "works with 0.6.1" is part of the commit, then I don't have a way to say "no wait, that commit actually doesn't work with 0.6.1".

I was imagining you'd just update smart.json to list "meteor: 0.6.4.2" and run `mrt release .` without bumping your package version.

Well, that can be confusing in another way... I publish my package 1.2.3, and on GitHub the v1.2.3 tag contains a smart.json that says it works with 0.6.1 but not 0.6.4.2; but if you look at Atmosphere it says that 1.2.3 doesn't work with 0.6.1 and does with 0.6.4.2.  The source of the confusion is my putting information in the commit that isn't immutable with the commit, so it'd be better if I didn't put it in the commit in the first place.

What if they(do what most people do and) don't specify a version? Does it stop you from upgrading to 0.6.5 ?

Perhaps if I type "meteor update" it could say "of your current packages, the following haven't been certified to work with 0.6.5; do you want proceed?"  But for today the simplest thing to do as a first step would be just to be able to see on Atmosphere which packages work with which Meteor versions... :)

[Tom Coleman]

On Friday, 16 August 2013 at 8:11 AM, Andrew Wilcox wrote:

I definitely think a file in the git repo has advantages here, cf. forking / PRs etc.

Right, if you look at GitHub, there are commits which are immutable, and then there are things tied to commits, such as tags.  Then there are other things which turn out better not to be tied to commits, such as issues which are tied to repositories instead.  (If issues were tried to commits, then I wouldn't be able to add a new issue without making a new commit).

So commits are a good place to start, but then sometimes we may think it through and realize that tying something to a commit isn't the best choice.  In terms of "X works with Y", the analogy with issues is actually pretty close.  I make a release, and then realize that my new version doesn't work with Meteor 0.6.1 anymore.  If "works with 0.6.1" is part of the commit, then I don't have a way to say "no wait, that commit actually doesn't work with 0.6.1".

I don't think atmosphere needs to know about [1] every commit of your repository, just the ones that you've tagged as releases.

So if you can move the tag for a given release, this isn't actually a problem. I think we can agree that it's a bad idea to change the code for a given release (although there's nothing stopping you doing that right now), but given smart.json isn't actually code [2], this isn't a fair comparison.

I understand the point about not being able to make an issue without making a commit, but I'm not sure that's a fair comparison either as these compatibility changes are going to be pretty rare. Right now I often have to make a commit to do a version bump for release anyway, which is kind of "impure" in the same way if you think about it.

The other big advantage of this information being in the repo is that my app doesn't need to talk to atmosphere to make statements about whether my packages are going to work with a given Meteor version. 

[1] i.e. have an understanding of compatibility with Meteor.

[2] except for the "packages" part.

 

I was imagining you'd just update smart.json to list "meteor: 0.6.4.2" and run `mrt release .` without bumping your package version.

Well, that can be confusing in another way... I publish my package 1.2.3, and on GitHub the v1.2.3 tag contains a smart.json that says it works with 0.6.1 but not 0.6.4.2; but if you look at Atmosphere it says that 1.2.3 doesn't work with 0.6.1 and does with 0.6.4.2.  The source of the confusion is my putting information in the commit that isn't immutable with the commit, so it'd be better if I didn't put it in the commit in the first place.

I can't see how this would come about. When I `mrt release` it will move the v1.2.3 tag on github to the commit I just made.

Is it fundamentally a bad thing to move such tags on GH? I can see there could be an argument that it is, which case this approach would be bad.

What if they(do what most people do and) don't specify a version? Does it stop you from upgrading to 0.6.5 ?

Perhaps if I type "meteor update" it could say "of your current packages, the following haven't been certified to work with 0.6.5; do you want proceed?"  But for today the simplest thing to do as a first step would be just to be able to see on Atmosphere which packages work with which Meteor versions... :)

Yeah, fair enough.

Tom 

[Gadi Cohen]

Hey

On Thursday, August 15, 2013 8:22:14 PM UTC+3, Mi Tar wrote:

Hi!

On Wed, Aug 14, 2013 at 6:05 AM, Gadi Cohen <dra...@wastelands.net> wrote:

If Meteor is the platform of the future, I think it needs to implement API compatibility layers like Android does.

No! Please, really, stop thinking about versions as numbers. This is definitely not the future. Trying to serialize all changes in consequent versions is always problematic.

I strongly disagree.  Clearly it's not problematic, if it's working for 900 million Android devices, which has become the most widely used smartphone OS in the world.  I've been using Android since it's first release, and every time  I upgrade the OS, all my apps stay working.  And that's not because every time a new release comes out, every single app developer is updating their code.  It's because the app developer writes their code according to the docs of a specific API, and even if Android upgrades, it still runs older apps exactly how they expect according to this API contract.  And that's because of the API compatibility layers.

 

If API 5 has some feature you are not using and then they upgrade to API 6 because that feature you are not using changed, what do you care about that?

No, I don't, nor should I.  I write my app/package according to an API level, and I know that even if Meteor is upgraded, my app/package is guaranteed to continue working with no changes on my part.  But yes, if I change to the new API, I will have to make changes to my code, exactly like a Meteor upgrade now, except, I decide when to make those changes without everything breaking on upgrade.

 

The point I am trying to make here is that API should be discoverable: if my application does tests against your API and everything passes, this means API is compatible and this is all I care about. I don't care about versions, I care about my application being able to speak to your library.

These ideas aren't mutually exclusive.  I specify a minimum and target API that I want to use.  Between that range, I use API discoverability to use features that are available.  BUT... it's absolutely impossible to rely on API discoverability for FUTURE updates.  A perfect point is the api.export().  Yes, AFTER the new Meteor release, I can update my package to use api.export() if it exists.  But, without updating my code FIRST, it will BREAK on the Meteor upgrade.  Just look at meteor-talk now to see how much havoc is going on now, because package authors had to update their code BEFORE the update, to avoid breakages.

So yes, I can use API discoverability to accomodate a wide range of versions within a finite range.  But this only works when I know all the APIs in this range.  It's impossible to future-proof your code, only to update it before new releases, and I think this is an unrealistic expectation, to expect every package author to update all their packages every time there is a breaking API change.  Better to guarantee that your package will work on all APIs 3+.  Guaranteed.  You can target API 5, and use feature discoverability in your code.   But then the API changes to 6, you know that your code, which is requesting API 5, will continue to work perfectly without any changes on your part.

I think the idea is that Meteor will not change so much after 1.0 version. This is why they are changing things now, to try different ideas.

Forgive me, but I think this is very naive.  Meteor will continue to develop after 1.0.  Breaking changes will occur less often, definitely, but I don't think you can promise that code you write today is guaranteed to still work with every future version of Meteor.

But I doubt all library developers will have time to keep all possible compatibility layers (sometimes they are even not possible!) up to date. I am worried about that. Not so much about changes in Meteor. I am worried more about changes in packages my applications depend on.

 

Two things:

1) You're worried about packages your app depends on, but those packages depend on Meteor.  If Meteor introduces a change that breaks a single package, either used directly by your app or in the dependency chain of any other package used by your app, your app will still break.  So you should definitely still be concerned about this.

2) I hope it's clear that the compatibility layer is in Meteor.  The whole idea is to save developers unnecessary work.

To conclude, I think it's definitely fair that I know then when I "meteor upgrade", my whole app won't fall flat on it's face, leaving me to either try downgrade, or remove packages, or wait for those authors to upgrade their packages, or send PR requests for those packages and run a fork and stop using the fork when the fix is implemented, for every single one of my packages and the packages they depend on, on every breaking upgrade.  Especially when this could all be made completely unnecessary.

Gadi

[Andrew Wilcox]

Is it fundamentally a bad thing to move such tags on GH? I can see there could be an argument that it is, which case this approach would be bad.

Yes, changing tags for releases is fundamentally bad.  It means that if I download "v1.2.3" and you download "v1.2.3" we can get different files.  This leads to awesome levels of confusion ("no, no, not the 05a23b commit of v1.2.3, the 94d8af commit!").  Even if it's only allowed for "non-code" changes, eventually someone will write a tool that parses the data in a commit and confusion ensues.

If you have code that is evolving, you should use a branch for that, because branches keep a history of changes.  Resetting a tag erases the information of what the tag used to point at.

When using tags for releases, you should never ever change the tag after making a release.  (e.g. https://github.com/meteor/meteor/commit/d27a3b4aa6beae7e51d7a8c35b85ec02c4487b44)

The other big advantage of this information being in the repo is that my app doesn't need to talk to atmosphere to make statements about whether my packages are going to work with a given Meteor version. 

Right, GitHub is a data store, and has various API's for accessing that data (including using plain git to clone a repo)... and the question is, is that data model a good fit for this particular case?  If it's not a good fit, we may be better off giving the API to Atmosphere instead of trying to cobble something on top of GitHub.

[Mitar]

Hi!

On Fri, Aug 16, 2013 at 7:30 AM, Gadi Cohen <dra...@wastelands.net> wrote:

I strongly disagree.  Clearly it's not problematic, if it's working for 900 million Android devices, which has become the most widely used smartphone OS in the world.

Maybe we talk about different things. I can agree that for the host platform (Android, Meteor) API versions can work. And that developers of the host platform can try to make backwards compatible layers (but this does require additional engineering effort).

But where I believe things break are in 3rd party libraries. Which can have custom dependencies between libraries and where different developers use versions in different ways, not always according to best practices. So you sooner or later discover that some library you dependent on, released new version and now you have a question: do you test manually and update to the new version manually, or do you allow automatic testing and it updates automatically to the new version of dependent library, if tests succeed. Or inform you, if tests fail. (And then you can update manually, when and if you want.) I am arguing that in most cases new versions of libraries don't break things (or keep backwards compatible behavior) and then it is good that those new versions are used without much effort. On the other hand, it is useful for a library developer to see how many things he or she might have broken with the new version, where he or she might not even intended that.

> A perfect point is the api.export().

Indeed it is. If my library is not exporting any symbols, I don't have to use api.export() so nothing broke in my library so why would I have to spend time testing this manually. I would want that when a new release candidate is made, my library is tested automatically and if it fails, I know that there was some change and I have to go and add api.export(). But if there was a change in API, in the part I am not using, things should be marked as supported under the new version automatically.

> It's impossible to future-proof your code,

No, you don't future-proof the code. You just encode with tests expecting behaviors. What I am saying is that any big enough system has multiple parts you might or might not be using. If you are using a part which changed - you have to update. But if you are not using a part which changed - why should you be bothered by having to update some number in your package (and even release a new version)? If this can be done automatically.

> Better to guarantee that your package will work on all APIs 3+.  Guaranteed.

Of course. If all developers of all libraries on the world would have time to maintain backwards compatible changes (and even if all changes could be backwards compatible). While Meteor after 1.0 release might try to maintain that, I doubt that all packages on Atmosphere have resources to maintain that for their users. In ideal world it would be of course the best, but sadly this is not how it is.

> You can target API 5, and use feature discoverability in your code.   But then the API changes to 6, you know that your code, which is requesting API 5, will continue to work perfectly without any changes on your part.

In theory yes. In practice I experienced many times my app breaking because of minor changes with minor version increases. Of course you can say that this was developer's fault and major version change should be made, but this does not help me at the end: I still have to spend time to manually run all tests every time and learn to not trust developers with how they are increasing versions. Or sometimes things break because the library was fixed, API stays the same (so major number of the version is OK not to change), but expected behavior (encoded in my tests) changes. So it is great that you are notified of this.

What I am saying that sadly you cannot relay on versions. And breaking changes sometimes do happen even inside one major version. Assuming that breaking changes will happen only with new major versions is nice hope, but not realistic. Tests are one way which can help here.

But yes, this can be done independently of Atmosphere. But I believe it would be really awesome if it would be integrated into the Atmosphere. So when there is a new version of a package (or Meteor) released, all other packages which depend on this package and are using Travis CI, would have their tests rerun.

And yes, this idea can be implemented in addition to other proposals.

Mitar