A distributed and secure hackage sounds like a dream, I really hope this
comes to life!
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/20150413121848.GA3834%40x60s.casa.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/4487776e-b862-429c-adae-477813e560f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
CTO | Capital Match
CapitalMatch
71 Ayer Rajah Crescent | #06-16 | Singapore 139951
(FR) +33 617 121 978 / (SG) +65 8408 7973 | arn...@capital-match.com | www.capital-match.com
Disclaimer:
Capital Match Platform Pte. Ltd. (the "Company") registered in Singapore (Co. Reg. No. 201501788H), a subsidiary of Capital Match Holdings Pte. Ltd. (Co. Reg. No. 201418682W), provides services that involve arranging for multiple parties to enter into loan and invoice discounting agreements. The Company does not provide any form of investment advice or recommendations regarding any listings on its platform. In providing its services, the Company's role is limited to an administrative function and the Company does not and will not assume any advisory, fiduciary or other duties to clients of its services.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhaskell+unsubscribe@googlegroups.com.
To post to this group, send email to commercialhaskell@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/CAKA2JgL4MviHic52_S3P8RqxyJndkj3oFA%2BPVG11AAgMhMJksw%40mail.gmail.com.
any use of cryptographic primitives of any form NEEDS to articulate what the trust model is, and what the threat model islikewise, i'm trying to understand who the proposed feature set is meant to serve.Several groups are in the late stages of building prototypes at varying points in the design space for improving package hosting right now for haskell, and I'm personally inclined to let those various parties release the tools, and then experiment with them all, before trying to push heavily for any particular design that hasn't had larger community experimentation.
I actually care most about being able to have the full package set be git cloneable, both for low pain on premise hackage hosting for corporate intranets, and also for when i'm on a plane or boat and have no wifi. At my current job, ANY "host packages via s3" approach is totally untenable, and i'm sure among haskell using teams/organizations, this isn't a unique problem!
The Author authentication/signing model question in an important one, but I"m uncomfortable with just saying "SHA512 and GPG address that". Theres A LOT of subtlety to designing a signing protocol thats properly audit-able and secure! Indeed, GPG isn't even a darn asymmetric crypto algorithm, its a program that happens to IMPLEMENT many of these algorithms. If we are serious about having robust auditing/signing, handwaving about the cryptographic parts while saying its important is ... kinda irresponsible. And frustrating because it makes it hard to evaluate the hardest parts of the whole engineering problem! The rest of the design is crucially dependent on details of these choices, and yet its that part which isn't specified.
to repeat myself: there is a pretty rich design space for how we can evolve future hackage, and i worry that speccing things out and design by committee is going to be less effective than encouraging various parties to build prototypes for their own visions of future hackage, and THEN come together to combine the best parts of everyones ideas/designs. Theres so much diversity in how different people use hackage, i worry that any other way will run into failing to serve the full range of haskell users!
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/4487776e-b862-429c-adae-477813e560f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Arnaud BaillyCTO | Capital Match
CapitalMatch
(FR) +33 617 121 978 / (SG) +65 8408 7973 | arn...@capital-match.com | www.capital-match.com
Disclaimer:
Capital Match Platform Pte. Ltd. (the "Company") registered in Singapore (Co. Reg. No. 201501788H), a subsidiary of Capital Match Holdings Pte. Ltd. (Co. Reg. No. 201418682W), provides services that involve arranging for multiple parties to enter into loan and invoice discounting agreements. The Company does not provide any form of investment advice or recommendations regarding any listings on its platform. In providing its services, the Company's role is limited to an administrative function and the Company does not and will not assume any advisory, fiduciary or other duties to clients of its services.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/33c89d4a-12b9-495b-a151-7e317177b061%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/etPan.552de40d.3d1b58ba.f2%40mbp.local.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/CAKRanNCnSV%3Ddds4ZDmacNO8WMxSgDmEh6acc0StMh%2Btgz%3D09hA%40mail.gmail.com.
Yes, I think you've summarized the security aspects of this nicely. There's also the reliability and availability guarantees we get from a distributed system, but that's outside the realm of security (unless you're talking about denial of service).
On Tue, Apr 14, 2015 at 9:50 PM, Michael Snoyman <mic...@snoyman.com> wrote:Yes, I think you've summarized the security aspects of this nicely. There's also the reliability and availability guarantees we get from a distributed system, but that's outside the realm of security (unless you're talking about denial of service).Is it possible to separate out the concept of trusted revisions from a distributed hackage (into 2 separate proposals) then?If Hackage wanted to it could implement trusted revisions. Or some other (distributed or non-distributed) package service could implement it (as long as the installer tool knows to check for revisions there, perhaps this would be added to Chris's signing tooling).
Ok, to narrow it down, you are concerned about the ability to
> * Properly authenticate users
> * Keep authorization lists of who can make uploads/revisions (and who can grant those rights)
and more specifically:
> * Currently, authorized uploaders are identified by a user name and a
> password on Hackage. How do we correlate that to a GPG key? Ideally, the
> central upload authority would be collecting GPG public keys for all
> uploaders so that signature verification can happen correctly.
> * There's no way for an outside authority to vet the 00-index.tar.gz file
> downloaded from Hackage; it's a completely opaque, black box. Having the
> set of authorization rules be publicly viewable, auditable, and verifiable
> overcomes that.
On 1) now you have the problem “what if the central upload authority’s store of GPG keys is violated”. You’ve just kicked the can. “Web of Trust” is not a tractable answer. My answer is simpler: I can verify that the signer of version 1 of a package is the same as the signer of version 0.1. This is no small trick. And I can do so orthogonal to hackage. Now, if I really want to verify that the signer of version 1 is the person who is “Michael Snoyman” and is in fact the exact Michael Snoyman I intend, then I need to get your key by some entirely other mechanism. And that is my problem, and, definitionally, no centralized store can help me in that regard unless I trust it absolutely — which is precisely what I don’t want to do.
On 2) I would like to understand more of what your concern with regards to “auditing” is. What specific information would you like to know that you do not? Improved audit logs seem again orthogonal to any of these other security concerns, unless you are simply worried about a “metadata only” attack vector. In any case, we can incorporate the same signing practices for metadata as for packages — orthogonal to hackage or any other particular storage mechanism. It is simply an unrelated question. And, honestly, compared to all the other issues we face I feel it is relatively minor (the signing component, not a better audit trail).
In any case, your account of the first two points reveals some of the confusion I think that remains:
> * Allow safe uploads of packages and metadata
> * Distribute packages and metadata to users safely
What is the definition of “safe” here? My understanding is that in the field of security one doesn’t talk about “safe” in general, but with regards to a particular profile of a sort of attacker, and always only as a difference of degree, not kind.
So who do we want to prevent from doing what? How “safe” is “safe”? Safe from what? From a malicious script-kid, from a malicious collective “in it for the lulz,” from a targeted attack against a particular end-client, from just poorly/incompetently written code? What are we “trusting”? What concrete guarantees would we like to make about user interactions with packages and package repositories?
While I’m interrogating language, let me pick out one other thing I don’t understand: "creating a coherent set of packages” — what do you mean by “coherent”? Is this something we can specify? Hackage isn’t supposed to be coherent — it is supposed to be everything. Within that “everything” we are now attempting to manage metadata to provide accurate dependency information, at a local level. But we have no claims about any global coherence conditions on the resultant graphs. Certainly we intend to be coherent in the sense that the combination of a name/version/revision should indicate one and only one thing (and that all revisions of a version should differ at most in dependency constraints in their cabal file) — but this is a fairly minimal criteria. And in fact, it is one that is nearly orthogonal to security concerns altogether.
What I’m driving at is — it sounds like we _mainly_ want new decentralized security mechanisms, at the cabal level, but we also want, potentially, a few centralized mechanisms. However, centralization is weakness from a security standpoint. So, ideally, we want as few centralized mechanisms as possible, and we want the consequences of those mechanisms being broken to be “recoverable” at the point of local verification.
Let me spell out a threat model where that makes sense. An adversary takes control of the entire hackage server through some zero day linux exploit we have no control over — or perhaps they are an employee at the datacenter where we host hackage and secure control via more direct means, etc. They have total and complete control over the box. They can accept anything they want, and they can serve anything they want. And they are sophisticated enough to be undetected for say a week.
Now, we want it to be the case that _whatever_ this adversary does, they cannot “trick” someone who types “cabal install warp” into instead cabal installing something malicious. How do we do so? _Now_ we have a security problem that is concrete enough to discuss. And furthermore, I would claim that if we don’t have at least some story for this threat model, then we haven’t established anything much “safer” at all.
This points towards a large design space, and a lot of potential ideas, all of which feel entirely different than the “strawman” proposal, since the emphasis there is towards the changes to a centralized mechanism (even if in turn, the product of that mechanism itself is then distributed and git cloneable or whatever).
Ok, let me counter that with a simpler idea: every Hackage edit action has an explanation field that the trustee can choose to optionally write some text in
And additonally: there Is a globally visible feed / log of all Hackage edits.
I believe some folks are working to add those features to hackage this spring.
I am emphatically against stronger security things being tacked on top without a threat model that precisely jusrifies why. Recent experience has shown me that organizations which mandate processes in the the name of a nebulous security model counter intuitively become less secure and less effective.
Let me repeat myself, enterprise sounding security processes should only be adopted in the context of a concrete threat model that actually specifically motivates the applicable security model. Anything else is kiss of death. Please be concrete. Additonally, specificity allows us to think of approaches that can be both secure and easy to use.
Ok. Let's get https support into cabal.
How do we best go about doing that?
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/CAHYVw0xbNQPZ%2Bockbn1Zve69eQoZ4OOeUKt-bqa72vn-N_FQPg%40mail.gmail.com.
A cryptographcially unforgable Hackage log is an interesting idea. I'll have to think about what that means though.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/1429176843.25663.31.camel%40dunky.localdomain.
I never claimed nor intended to imply that range requests are non-standard. In fact, I'm quite familiar with them, given that I implemented that feature of Warp myself! What I *am* claiming as non-standard is using range requests to implement an incremental update protocol of a tar file. Is there any prior art to this working correctly? Do you know that web servers will do what you need and server the byte offsets from the uncompressed tar file instead of the compressed tar.gz? Where are you getting the signatures for, and how does this interact with 00-index.tar.gz files served by non-Hackage systems?
On the security front: it seems that we have two options here:
1. Use a widely used piece of software (Git), likely already in use by the vast majority of people reading this mailing list, relied on by countless companies and individuals, holding source code for the kernel of likely every mail server between my fingertips and the people reading this email, to distribute incremental updates. And as an aside: that software has built in support for securely signing commits and verifying those signatures.
2. Write brand new code deep inside two Haskell codebases with little scrutiny to implement a download/update protocol that (to my knowledge) has never been tested anywhere else in the world.
Have I misrepresented the two options at all?
I get that you've been working on this TUF-based system in private for a while, and are probably heavily invested already in the solutions you came up with in private. But I'm finding it very difficult to see the reasoning to reinventing wheels that need to reinventing.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/1429185521.25663.103.camel%40dunky.localdomain.
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/mhouaf%24it2%241%40ger.gmane.org.
Many of you saw the blog post Mathieu wrote[1] about having more composable community infrastructure, which in particular focused on improvements to Hackage. I've been discussing some of these ideas with both Mathieu and others in the community working on some similar thoughts. I've also separately spent some time speaking with Chris about package signing[2]. Through those discussions, it's become apparent to me that there are in fact two core pieces of functionality we're relying on Hackage for today:* A centralized location for accessing package metadata (i.e., the cabal files) and the package contents themselves (i.e., the sdist tarballs)* A central authority for deciding who is allowed to make releases of packages, and make revisions to cabal filesIn my opinion, fixing the first problem is in fact very straightforward to do today using existing tools. FP Complete already hosts a full Hackage mirror[3] backed by S3, for instance, and having the metadata mirrored to a Git repository as well is not a difficult technical challenge. This is the core of what Mathieu was proposing as far as composable infrastructure, corresponding to next actions 1 and 3 at the end of his blog post (step 2, modifying Hackage, is not a prerequesite). In my opinion, such a system would far surpass in usability, reliability, and extensibility our current infrastructure, and could be rolled out in a few days at most.However, that second point- the central authority- is the more interesting one. As it stands, our entire package ecosystem is placing a huge level of trust in Hackage, without any serious way to vet what's going on there. Attack vectors abound, e.g.:* Man in the middle attacks: as we are all painfully aware, cabal-install does not support HTTPS, so a MITM attack on downloads from Hackage is trivial* A breach of the Hackage Server codebase would allow anyone to upload nefarious code[4]* Any kind of system level vulnerability could allow an attacker to compromise the server in the same wayChris's package signing work addresses most of these vulnerabilities, by adding a layer of cryptographic signatures on top of Hackage as the central authority. I'd like to propose taking this a step further: removing Hackage as the central authority, and instead relying entirely on cryptographic signatures to release new packages.I wrote up a strawman proposal last week[5] which clearly needs work to be a realistic option. My question is: are people interested in moving forward on this? If there's no interest, and everyone is satisfied with continuing with the current Hackage-central-authority, then we can proceed with having reliable and secure services built around Hackage. But if others- like me- would like to see a more secure system built from the ground up, please say so and let's continue that conversation.[4] I don't think this is just a theoretical possibility for some point in the future. I have reported an easily trigerrable DoS attack on the current Hackage Server codebase, which has been unresolved for 1.5 months now
--
You received this message because you are subscribed to the Google Groups "Commercial Haskell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to commercialhask...@googlegroups.com.
To post to this group, send email to commerci...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/CAKA2JgL4MviHic52_S3P8RqxyJndkj3oFA%2BPVG11AAgMhMJksw%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/commercialhaskell/CANUq-hHWexCeL%2Bp%2BWSU7PNgpdpWo_j0uPJ47-ui7QRjdktZ9sg%40mail.gmail.com.