RubyGems/TUF Hack Week Project at Square

[Tony Arcieri]

Square's Hack Week starts tomorrow, and we'll be doing a project to add
security to RubyGems. We have been looking at the TUF work that is already
being done on PyPI/pip as a sort of design document for how we might apply
these same sorts of ideas to RubyGems:

https://github.com/theupdateframework/pep-on-pypi-with-tuf

I'm thinking we could even fork this document and create a derived one
that's applicable to RubyGems.

There are at least 17 interested developers on this project, so I hope we
can accomplish something within a week!

I just wanted to touch base with the RubyGems people/TUF people so you know
1) this is happening 2) can give us some feedback as far as whether we're
doing a good job ;)

This project will focus on looking at the RubyGems ecosystem end-to-end and
applying the TUF design principles to the respective parts of this system.
It's expected to leverage the existing digital signature system that's
already in place in RubyGems, but add additional security around things
like Gemcutter, bundler-api, and RubyGems mirrors, per TUF's
separation-of-responsibilities principles.

One of the design principles of TUF is for users to not see an impact in
their experience *unless* the system has been compromised and we certainly
hope to attain that too. The only additional step this project would add to
the workflow would be mandatory gem signing using the standard RubyGems
commands for doing so as they exist today.

--
Tony Arcieri
_______________________________________________
RubyGems-Developers mailing list
http://rubyforge.org/projects/rubygems
RubyGems-...@rubyforge.org
http://rubyforge.org/mailman/listinfo/rubygems-developers

[Nick Quaranto]

This is awesome. Is there any way for someone outside of Square to observe
what's going on?

[Tony Arcieri]

I've made the main project mailing list public in hopes that if we screw
something up, someone else will notice ;)

https://groups.google.com/forum/#!forum/rubygems-tuf

[Nick Quaranto]

Cool. Good luck!

[Tony Arcieri]

My main question to the TUF team would be how we apply the TUF design to something like bundler-api:

https://github.com/bundler/bundler-api

bundler-api is an interactive service designed to tell you the subsets of dependencies you might be interested in to satisfy a given set of input dependencies. This seems different from the ideas of Complete File Snapshots (CFS) and Differential File Snapshots (DFS) as seen in TUF. Perhaps I'm misunderstanding, but I'm not sure how something like bundler-api fits into the overall TUF design. It would seem to be a sort of "Partial File Snapshot" generated dynamically on-the-fly for a particular client, and was introduced as an optimization as fetching RubyGems' equivalent of a "CFS" negatively impacted Bundler's performance. 

On Sun, Nov 17, 2013 at 4:44 PM, Tony Arcieri <bas...@gmail.com> wrote:

Square's Hack Week starts tomorrow, and we'll be doing a project to add security to RubyGems. We have been looking at the TUF work that is already being done on PyPI/pip as a sort of design document for how we might apply these same sorts of ideas to RubyGems:

https://github.com/theupdateframework/pep-on-pypi-with-tuf

I'm thinking we could even fork this document and create a derived one that's applicable to RubyGems.

There are at least 17 interested developers on this project, so I hope we can accomplish something within a week!

I just wanted to touch base with the RubyGems people/TUF people so you know 1) this is happening 2) can give us some feedback as far as whether we're doing a good job ;)

This project will focus on looking at the RubyGems ecosystem end-to-end and applying the TUF design principles to the respective parts of this system. It's expected to leverage the existing digital signature system that's already in place in RubyGems, but add additional security around things like Gemcutter, bundler-api, and RubyGems mirrors, per TUF's separation-of-responsibilities principles.

One of the design principles of TUF is for users to not see an impact in their experience *unless* the system has been compromised and we certainly hope to attain that too. The only additional step this project would add to the workflow would be mandatory gem signing using the standard RubyGems commands for doing so as they exist today.

--
Tony Arcieri

--
Tony Arcieri

[Tony Arcieri]

On Sun, Nov 17, 2013 at 6:50 PM, Justin Cappos <jca...@nyu.edu> wrote:

Is the output of this meant for machine or human consumption?

It's meant for machine consumption by the bundler tool, which is a dependency resolver that does something akin to topological sorting to produce a single answer as to the best set of dependencies which fit a given set of constraints:

http://bundler.io/

Is the goal to have it tell you "the one true answer" to resolve the needed dependencies or should it give options?

My perhaps naive understanding is that the local Bundler application is trying to produce "the one true answer" and uses bundler-api to query the set of potential dependencies which may fit into that answer, but what bundler-api returns is effectively a subset of the "CFS" relevant to the original query.

--
Tony Arcieri

[Tony Arcieri]

On Sun, Nov 17, 2013 at 7:09 PM, Justin Cappos <jca...@nyu.edu> wrote:

Here are a couple of papers that discuss some of the issues and how we addressed them in Stork, if you are interested:

http://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf

http://isis.poly.edu/~jcappos/papers/cappos_pmsec_tr08-02.pdf

What if the digital signature of a Complete File Snapshot were derived from a Merkle Tree such that it could selectively be pruned down in an automated manner by a "Dependency Information as a Service" service like bundler-api? This would work by omitting the details for most of the packages and including only a content hash per package. It'd be slower than how bundler-api works now because it'd still need to return a content hash for everything in the original CFS, but it would be a more succinct version with only the requested details.

--
Tony Arcieri

[Jordi Massaguer Pla]

That is very cool! I am looking forward to see your results :) !

On 11/18/2013 01:44 AM, Tony Arcieri wrote:
> Square's Hack Week starts tomorrow, and we'll be doing a project to add
> security to RubyGems. We have been looking at the TUF work that is already
> being done on PyPI/pip as a sort of design document for how we might apply
> these same sorts of ideas to RubyGems:
>
> https://github.com/theupdateframework/pep-on-pypi-with-tuf
>
> I'm thinking we could even fork this document and create a derived one
> that's applicable to RubyGems.
>
> There are at least 17 interested developers on this project, so I hope we
> can accomplish something within a week!
>
> I just wanted to touch base with the RubyGems people/TUF people so you know
> 1) this is happening 2) can give us some feedback as far as whether we're
> doing a good job ;)
>
> This project will focus on looking at the RubyGems ecosystem end-to-end and
> applying the TUF design principles to the respective parts of this system.
> It's expected to leverage the existing digital signature system that's
> already in place in RubyGems, but add additional security around things
> like Gemcutter, bundler-api, and RubyGems mirrors, per TUF's
> separation-of-responsibilities principles.
>
> One of the design principles of TUF is for users to not see an impact in
> their experience *unless* the system has been compromised and we certainly
> hope to attain that too. The only additional step this project would add to
> the workflow would be mandatory gem signing using the standard RubyGems
> commands for doing so as they exist today.
>

[Justin Cappos]

As I understand bundler.io and the metadata it uses, the problem may be deeper than this.   Let me explain how I think this works first...

Okay, so as I understand it, any user can upload a package and list any dependencies they want for their package.   When the user requests package A with dependencies B and C, bundler.io determines that B and C are needed (along with whatever it takes to satisfy their dependencies).   

The party that is responsible for knowing who is allowed to update package A, is the RubyGEMS repository.   This happens when the package is registered.   bundler.io uses the dependency information from the developers (via the repository) to provide dependency resolution services on the behalf of clients.

Do you have a notion of 'virtual dependencies' where many different packages with different names can provide a resolution for a dependency like 'mail' or 'crypto'?   For example, can a package rely on an one of a set of other packages and if so, how is this specified?

Is there a way to specify conflicts for packages?   Can it be that two cannot be installed together?

Once I understand a little more, I will point out some potential issues I see here...

Thanks,

Justin

--
You received this message because you are subscribed to the Google Groups "rubygems-developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubygems-develo...@googlegroups.com.
To post to this group, send email to rubygems-...@googlegroups.com.
Visit this group at http://groups.google.com/group/rubygems-developers.
For more options, visit https://groups.google.com/groups/opt_out.

[Tony Arcieri]

We found this somehow and it seems interesting:

http://mirror1.poly.edu/test-rubygems/

This looks like an example of how TUF's metadata formats could live
side-by-side with the existing RubyGems formats. Is that the case? Any idea
where this came from?

On Sun, Nov 17, 2013 at 4:44 PM, Tony Arcieri <bas...@gmail.com> wrote:

[Justin Cappos]

Four of the students in my App Sec class built this. They are trying to
get an end-to-end integration of TUF with gem going.

I'll forward the email they sent a few days ago to the lists.

Thanks,
Justin

[Tony Arcieri]

Aha! Is that this project?

https://github.com/PoppySeedPlehzr/gemsontuf

[Trishank Karthik Kuppusamy]

On 11/18/2013 02:47 PM, Tony Arcieri wrote:
> Aha! Is that this project?
>
> https://github.com/PoppySeedPlehzr/gemsontuf
>

Yes :) See also:

https://github.com/PoppySeedPlehzr/gemsontuf/wiki/Getting-Started-with-GEMs-on-TUF

[Justin Cappos]

Yes, they are one and the same.

Justin

[Eric Hodel]

On 17 Nov 2013, at 16:44, Tony Arcieri <bas...@gmail.com> wrote:
> Square's Hack Week starts tomorrow, and we'll be doing a project to add
> security to RubyGems. We have been looking at the TUF work that is already
> being done on PyPI/pip as a sort of design document for how we might apply
> these same sorts of ideas to RubyGems:
>
> https://github.com/theupdateframework/pep-on-pypi-with-tuf

Is there a place (besides the mailing list mentioned below in this thread) that I can hang out and answer RubyGems integration questions? I would love to help out with that.

[Tony Arcieri]

On Mon, Nov 18, 2013 at 3:58 PM, Eric Hodel <drb...@segment7.net> wrote:

> Is there a place (besides the mailing list mentioned below in this thread)
> that I can hang out and answer RubyGems integration questions? I would
> love to help out with that.
>

#rubygems on Freenode? ;)

--
Tony Arcieri

[Eric Hodel]

On 18 Nov 2013, at 16:30, Tony Arcieri <bas...@gmail.com> wrote:
> On Mon, Nov 18, 2013 at 3:58 PM, Eric Hodel <drb...@segment7.net> wrote:
> Is there a place (besides the mailing list mentioned below in this thread) that I can hang out and answer RubyGems integration questions? I would love to help out with that.
>
> #rubygems on Freenode? ;)

Excellent!

[Jon]

What testing are you doing on Windows systems or Windows-on-a-VM this week?

[Benjamin Fleischer]

Looks like the TUF work has stopped. Is there still
interest? https://github.com/rubygems/rubygems.org/pull/626