Tarball uploads

[Volker Braun]

In order to streamline updating third-party tarballs I've written a small web app where you can directly upload them. That way you don't need to host files yourself. Plus, the files can be retrieved by sha1 so with a little bit more scripting I won't always forget to manually copy them to the mirrors. Its a bit on the cutting-edge side (Python 3 aiohttp and Polymer) but should work on all current browsers, so its ready to beta-test:

    http://fileserver.sagemath.org:8080/

Source code is at https://github.com/vbraun/SageDevApp

[mmarco]

I think that, since it involves sending login information, it should use https instead of http

[Volker Braun]

Yes, though we don't have a certificate for *.sagemath.org. Besides the cost, you also need to periodically renew etc. Though I'm hoping that Let's Encrypt (https://letsencrypt.org) will fix that. Launching this September...

[Michael Orlitzky]

On 08/09/2015 07:09 AM, Volker Braun wrote:
> Yes, though we don't have a certificate for *.sagemath.org. Besides the
> cost, you also need to periodically renew etc. Though I'm hoping that
> Let's Encrypt (https://letsencrypt.org) will fix that. Launching this
> September...

Just use a self-signed cert and post the fingerprint. If we trust you
enough to click that URL, we trust you enough to post the fingerprint.
For anyone who cares, that process is more secure than using a CA cert.
Anyone who doesn't care or who believes the browser warning can use
plain http.

[mmarco]

Even without checking the certificate, https is mre secure than plain http. Of course you are vulnerable to MITM attacks (just as you are with http), but at least you are secure from pasive attacks.

I really don't understand why browsers show a scary warning when you try to connect a web page by https with an untrtusted certificate... but show no warning at all when you connect to a much less secure plain http page.

[Vincent Delecroix]

On 10/08/15 11:32, mmarco wrote:
> I really don't understand why browsers show a scary warning when you try to
> connect a web page by https with an untrtusted certificate...

Do you? A certificate authority makes money from selling certificates.
They use a lot of energy in forcing browser developers to trust their
services... It the same story why there is still a lot of salt, sugar
and fat in all the prepared food that you buy in supermarket (this is
very bad for the health but "tasty" for cheap, hence attracting for the
food industry).

I agree with you: from a technical point of view this is stupid.

[Volker Braun]

On Monday, August 10, 2015 at 11:42:16 AM UTC+2, vdelecroix wrote:

I agree with you: from a technical point of view this is stupid.

It is not. There is no security without the chain of trust. Maybe in a parallel universe where everybody is so far on the autistic spectrum that they religiously verify finger prints over a second channel, but not in the real world.

[Vincent Delecroix]

It is. There is no security without the chain of trust. And I do not
trust certificate authority. If a trusted certificate authority X
provides me a certificate for google.com then I can be a man in the
middle. What prevent them for doing so? This is a very weak design.
Moreover, who can be a certificate authority?

Let me propose something less stupid: the first time you access to a
website you have to accept the certificate manually (if you wish you can
have a look at the fingerprint). Then, until it changes nothing happens
(the very same way ssh works). It does not prevent certificate authority
to keep signing the certificate if they wish.

Right now, if a certificate changes but it is certified, you browser
will not alert you. But it definitely should.

[mmarco]

What I meant is that it doesn't make any sense to show a scary warning in the case of "encrypted but not verified" pages, but don't show any warning in the case of "neither encrypted nor verified" plain http pages. The second is strictly less secure than the first... yet the browser induces the user to think that it is actually more secure.

About the security model of CA certificates, I recommend this talk:
http://m.youtube.com/watch?v=pDmj_xe7EIQ

[mmarco]

[Volker Braun]

On Monday, August 10, 2015 at 5:34:49 PM UTC+2, vdelecroix wrote:

Moreover, who can be a certificate authority?

There is always google if you want to know the requirements for a CA

Your proposal would result in daily "new certificate" warnings as you browse to new web pages and/or certificates expire. How long until your mom/gf/so/non-technical friend would stop caring and click on "OK" out of reflex? How to revoke compromised self-signed certs? You are essentially expanding your trust from hundreds of CAs to millions of small websites.

[Volker Braun]

On Monday, August 10, 2015 at 7:10:43 PM UTC+2, mmarco wrote:

What I meant is that it doesn't make any sense to show a scary warning in the case of "encrypted but not verified" pages, but don't show any warning in the case of "neither encrypted nor verified" plain http pages. The second is strictly less secure than the first... 

The former is a tell-tale sign of a MITM attack, the latter is just how http usually works. 

[mmarco]

Precisely. Tee way http works is strictly less secure than the most insecure HTTPS scenario.

If I wanted to mitm some HTTPS connection, I wouldn't do so by redirecting the victim to a fake HTTPS web page, but to a fake http one. The lack of warnings from the browser would make such an attack go unnoticed in many cases.

That is, the lack of a warning from the browser in plain http makes the protection of ssl certificates much less effective.

In the video I linked before moxie marlinspike proposes an alternative method to check the authenticity of a web site that is not based on CAs. I see some problems to his approach, but I agree with him that we need to look for something different than what we have right now.

[Dima Pasechnik]

an alternative might be to use github - package maintainers can create tarballs
via github release creation.

[Michael Orlitzky]

You don't need to verify the fingerprint. If you don't have to pay for
certs, you can make them valid for eternity, and only warn the user when
they change.

There's only one opportunity for a MITM -- when you first "save" the
cert -- and that's no different than the CA model (where did you get
your root certs?).

[Michael Orlitzky]

On 08/10/2015 11:34 AM, Vincent Delecroix wrote:
>
> Let me propose something less stupid: the first time you access to a
> website you have to accept the certificate manually (if you wish you can
> have a look at the fingerprint). Then, until it changes nothing happens
> (the very same way ssh works). It does not prevent certificate authority
> to keep signing the certificate if they wish.
>

This is called "certificate pinning" and it's a great idea. Mozilla and
Google pin their own certs for mozilla.org and google.com (and a few
others, now) in Firefox/Chrome. But it's not available generally because
it doesn't make anyone money.

Once you have certificate pinning, you can just get rid of the CAs and
use self-signed certs that last forever. Works better, for free.

[Dan Drake]

On Saturday, August 8, 2015 at 12:22:18 PM UTC-7, Volker Braun wrote:

In order to streamline updating third-party tarballs I've written a small web app where you can directly upload them. That way you don't need to host files yourself. Plus, the files can be retrieved by sha1 so with a little bit more scripting I won't always forget to manually copy them to the mirrors. Its a bit on the cutting-edge side (Python 3 aiohttp and Polymer) but should work on all current browsers, so its ready to beta-test:

    http://fileserver.sagemath.org:8080/

Looks very nice. I uploaded the new sagetex tarball there. Everything seems to work; my only suggestion is to somehow allow someone to download a file with the original file name, instead of its SHA1. I made an issue on your repo: https://github.com/vbraun/SageDevApp/issues/1.

[John Cremona]

I will try this out with the new eclib tarball for trac #19091 (later today).

John

 

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[John Cremona]

On 27 August 2015 at 09:16, John Cremona <john.c...@gmail.com> wrote:

On 27 August 2015 at 03:42, Dan Drake <ddr...@math.wisc.edu> wrote:

On Saturday, August 8, 2015 at 12:22:18 PM UTC-7, Volker Braun wrote:

In order to streamline updating third-party tarballs I've written a small web app where you can directly upload them. That way you don't need to host files yourself. Plus, the files can be retrieved by sha1 so with a little bit more scripting I won't always forget to manually copy them to the mirrors. Its a bit on the cutting-edge side (Python 3 aiohttp and Polymer) but should work on all current browsers, so its ready to beta-test:

    http://fileserver.sagemath.org:8080/

Looks very nice. I uploaded the new sagetex tarball there. Everything seems to work; my only suggestion is to somehow allow someone to download a file with the original file name, instead of its SHA1. I made an issue on your repo: https://github.com/vbraun/SageDevApp/issues/1.

I will try this out with the new eclib tarball for trac #19091 (later today).

I don't know whether this worked or not.  When I first went to that link, the main window was empty and the links on the left column did not do anything.  Later I found a page inviting me to register, which I did, and went to the upload page and selected a local file to upload.  But nothing seems to have happened, and now the links on the left side do nothing.

Where would the file I uploaded have ended up?

John

PS Firefox on Suse Linux enterprise.

[Volker Braun]

The problem with the file name is that it can be wrong, so you'd have to be able to remove/overwrite previously-used names. Whereas the SHA1 is always correct and really is all that is needed to download the file from the Sage build scripts.

[Volker Braun]

Hmm it did not work for you. Maybe you didn't wait long enough for the upload to finish? The site will show you the SHA1 and download link (http://fileserver.sagemath.org:8080/api/v1/pkg/download/77f404be91fd605f6220a1411912f578c8947c50) when its finished.

I just tried with Firefox and it worked for me.

[John Cremona]

On 27 August 2015 at 15:13, Volker Braun <vbrau...@gmail.com> wrote:

Hmm it did not work for you. Maybe you didn't wait long enough for the upload to finish? The site will show you the SHA1 and download link (http://fileserver.sagemath.org:8080/api/v1/pkg/download/77f404be91fd605f6220a1411912f578c8947c50) when its finished.

How would I know that it had finished?  After selecting the file it goes back to the "choose file" page with no apparant change and nothing happening.

I just worked out that you have to double-click the links on the left to get the new page, which explains some of the strangeness (and why?  that is not normal).  But I see no evidence of any upload actually happening.

John

[Volker Braun]

On Thursday, August 27, 2015 at 10:22:23 AM UTC-4, John Cremona wrote:

How would I know that it had finished?  After selecting the file it goes back to the "choose file" page with no apparant change and nothing happening.

You can upload multiple files, so its by design that you can choose more files.

The grey progress bar under the filename should fill with blue as the upload works its way. And the upload is finished when the progress bar is full, as usual.

I just worked out that you have to double-click the links on the left to get the new page

Yes, the menu requiring double-click when on Firefox is a bug:  https://github.com/PolymerElements/polymer-starter-kit/issues/166

[John Cremona]

On 27 August 2015 at 15:33, Volker Braun <vbrau...@gmail.com> wrote:

On Thursday, August 27, 2015 at 10:22:23 AM UTC-4, John Cremona wrote:

How would I know that it had finished?  After selecting the file it goes back to the "choose file" page with no apparant change and nothing happening.

You can upload multiple files, so its by design that you can choose more files.

The grey progress bar under the filename should fill with blue as the upload works its way. And the upload is finished when the progress bar is full, as usual.

There is no filename and no progress bar!

Never mind, on the ticket I put a link to where I normally put the file and Jeroen has given me a positive review, so this was just an exercise.

John

 

I just worked out that you have to double-click the links on the left to get the new page

Yes, the menu requiring double-click when on Firefox is a bug:  https://github.com/PolymerElements/polymer-starter-kit/issues/166

[Jeroen Demeyer]

On 2015-08-08 21:22, Volker Braun wrote:
> http://fileserver.sagemath.org:8080/

Seems down...

[Volker Braun]

Port 8080 seems firewalled now, I moved it to the standard port:

http://fileserver.sagemath.org

[Jeroen Demeyer]

On 2015-08-08 21:22, Volker Braun wrote:

> In order to streamline updating third-party tarballs I've written a
> small web app where you can directly upload them. That way you don't
> need to host files yourself. Plus, the files can be retrieved by sha1 so
> with a little bit more scripting I won't always forget to manually copy
> them to the mirrors.

It seems that the files can *only* be retrieved by SHA1, not by
filename, or am I missing something?

For example, is there a way to download

http://fileserver.sagemath.org/api/v1/pkg/download/880b92213fb7dc1fb4a8dc5313af5560418e4cb3

in such a way that the filename becomes pathpy-7.1.tar.bz2

[Volker Braun]

Yes, thats intentional. Filenames might be wrong (requiring some further UI to change it) but the sha1 is tautologically correct.

[Volker Braun]

PS: I'm working on a patch for sage to fall back to downwload by sha1 after the mirrors were checked, this is http://trac.sagemath.org/ticket/19102

[Jeroen Demeyer]

On 2015-09-07 18:25, Volker Braun wrote:
> Yes, thats intentional. Filenames might be wrong (requiring some further
> UI to change it) but the sha1 is tautologically correct.

So, can you remind us what exactly is the use case for this server?

[Volker Braun]

On Monday, September 7, 2015 at 7:29:54 PM UTC+2, Jeroen Demeyer wrote:

So, can you remind us what exactly is the use case for this server?

The goal is no manual steps when handling third-party tarballs

[Volker Braun]

PS: The SAGE_ROOT/upstream content can of course be programatically generated from the git repo + downloading files by sha1

On Monday, September 7, 2015 at 7:29:54 PM UTC+2, Jeroen Demeyer wrote:

[Jeroen Demeyer]

Sorry, but I am lost. How are we, Sage developers, supposed to use this
new tarball upload service in practice? Or is it meant to be used only
once #19102 has been merged?

[Volker Braun]

Right now you can try it out and check that you can upload tarballs. Its only truly useful with  #19102