Merge with Convergence?

[Jakub Warmuz]

Convergence [1] is currently unmaintained, but it has much cleaner code
base (architecture, interfaces, etc.) and a couple of really interesting
solutions (c.f [2]). That would be an opportunity to clean up our code,
adapt tools they're using (e.g. twisted [3]) as well as unify our
protocols. What do you think?


[1] https://github.com/moxie0/Convergence
[2] https://github.com/danwent/Perspectives-Server/issues/40
[3] https://twistedmatrix.com/trac

--
Yours virtually,
Kuba

[Gerold Meisinger]

On 2014-11-14 19:59, Jakub Warmuz wrote:
> Convergence [1] is currently unmaintained, but it has much cleaner code
> base (architecture, interfaces, etc.) and a couple of really interesting
> solutions (c.f [2]). That would be an opportunity to clean up our code,
> adapt tools they're using (e.g. twisted [3]) as well as unify our
> protocols. What do you think?

Hi Jakub,

good idea actually. We could strip out the parts we need.

> That would be an opportunity to ... unify our protocols.
> What do you think?

Do you know What the current state of the Convergence network is? Are
there still any servers out there? If yes, I think we should do that.


I saw the Blackhat video [1] a few months ago and found it very
informative. It's funny how his three points about Perspectives
limitations are still true three years later:
1. No embedded content support
2. Privacy issues
3. Unresponsive servers

@1
I implemented a first draft for embedded content support, see [2].
Unfortunately I got stuck on how to restart a connection for sub-content.

@2
Local caching is already implemented.
Queries via HTTPS still need to be implemented, see [3].
Notary bouncing (a form of onion routing) sounds like a good concept.
Foremost however I think we need more servers :) This would also allow
us to query notaries randomly.

@3
Dave currently tackles this.


Still a lot of work to do :)

[1] https://www.youtube.com/watch?v=Z7Wl2FW2TcA
[2] https://github.com/danwent/Perspectives/issues/14#issuecomment-46087643
[3] https://github.com/danwent/Perspectives/issues/81

Sincerely,
Gerold

[Gerold Meisinger]

> Convergence [1] ... has much cleaner code base (architecture,
interfaces, etc.)
> ... That would be an opportunity to clean up our code ...

That's true (although I'm not a fan of OOP style). It's also true that
Perspectives could use some code cleanup. However I don't like the idea
of completely refactoring everything (see Second system syndrome
[1][2]). The main functionality is located in notaries.js which is ~1000
LOC but I'm splitting it up right now. Besides this, I don't think that
the Perspectives code base is particular complex. In the short- and
mid-term there are more urgent basic functionality issues [3]. If we are
ever going to cleanup Perspectives as a whole I would much rather vote
for a more "secure" base (like a statically typed language) but still
only evolve it gradually.


I just installed the original Convergence but wasn't able to connect to
a HTTPS site anymore. It ships with four notary servers but they also
appear to be down. The settings look a bit cleaner and I like the idea
of a cache manager (see attached images).

Looking at the fork network [4] there a two active forks:

1. Convergence "Extra" [5]

The latest version 0.11 on AMO [6] was updated on 2014-04-09 but suffers
from the same problem and ships with the same public notaries.

From github:
"... I don't use "bounce notaries", as all notaries I use are private
anyway and there aren't any useable public ones (due to compatibility
things outlined above), so that feature might be broken."

Using the version from the git repo I get the following error:
"Convergence Certificate Verfication Failure" -> Details -> "Location:
https://undefined"

@Jakub: Did you get it Convergence to work? Do you know any public notaries?

2. FreeSpeechMe [7][8]

"is a modification to Moxie Marlinspike's tool Convergence, modified to
implement the Namecoin .bit specification. It can resolve .bit domains
to IPv4 addresses, and verify .bit TLS certficates via the Namecoin
blockchain. This allows safe usage of self-signed certificates, without
trusting any third party."

Didn't try.

[1] https://en.wikipedia.org/wiki/Second-system_effect
[2] http://www.joelonsoftware.com/articles/fog0000000069.html
[3] https://github.com/danwent/Perspectives/wiki/Perspectives-Roadmap
[4] https://github.com/moxie0/Convergence/network
[5] https://github.com/mk-fg/convergence
[6] https://addons.mozilla.org/en-us/firefox/addon/convergence-extra/
[7] https://github.com/namecoin/Convergence
[8] https://github.com/JeremyRand/Convergence

[Stefano Fornari]

thank for sharing the video, it's very informative! it also sheds some
light on the issue of using HTTPS to query the notaries as per the
other thread[1].

[1] https://groups.google.com/forum/#!topic/perspectives-dev/9UhtEyeuT3M
Ste

[Dave Schaefer]

Hey Jakub, interesting idea. So are you proposing to merge and close the entire project? Only the scanner component? Scanner and server?

Perspectives currently has around 7,000 daily users, and I wouldn't want to leave them in the dirt by simply shutting things down. Are you suggesting porting some Convergence components over here rather than us moving over there?


Gerold said:
>It's funny how [moxie's] three points about Perspectives

>limitations are still true three years later:
>1. No embedded content support
>2. Privacy issues
>3. Unresponsive servers

Thanks for summarizing this Gerold. If nothing else I think it is very important to fix these issues as part of our short-term goals. As you note we have some work going on this.


Gerold, thanks for the great research and overview of the state of things. If the current Convergence is not actively maintained/there are bugs with the default installation/there are not many notaries, then I would prefer to try to fix some of that first before making any big changes or moves.

What are everyone's thoughts if we move for a combined approach of:
* Continue to fix the most glaring Perspectives issues on this side: unresponsive notaries, new hashes, HTTPS
* Merge in/convert/update some components if we can show they are better (e.g. the scanner component)

I am definitely open to patches or changes that improve the project. I also want to keep the project stable and running for the people that rely on it. I'm biased because I have spent several years and hundreds of hours of my spare time trying to keep Perspectives running, keep it stable, and working to improve it, but I feel that incremental, tested, and proven changes are the best way to improve.

That said, I am not in the habit of refusing patches, and I don't want to start. If you have a strong feeling about what could be improved, the best way to make that happen is to write the code, test or prove that it's better, and send a pull request.


It might be useful to have our notaries implement all of the Perspectives, Convergence, and Namecoin APIs so the servers could be used by any client. Then they would be useful and could serve as additional notaries regardless.

[Stefano Fornari]

Hi Gerold and All,
Convergence is quite interesting as well. There is one thing I did not
get from the video and your current discussion:

> @1
> I implemented a first draft for embedded content support, see [2].
> Unfortunately I got stuck on how to restart a connection for sub-content.

If I well understand it, this is to handle resources inside a page
(e.g. images, stylesheets, scripts, etc.). If that is the case, why
the current implementation does not take care of it? what the add-on
does today is to add the trusted certificate to the ff trustdb; from
that point on all requests to the same service will not require a new
validation any more, therefore the content will be retrieved with no
issues. The only problem I see is that the content may refer to a
different HTTPS service, which then requires an additional validation.
But this is a problem even today. For example, let say I have the page
below served by https://service1.com/content.html:

<html>
<body>
<img src="https://service2.com/diskone/HomeCabiNet/product/components.jpg">
</body>
</html>

If service2.com provides a certificate not yet trusted, FF will not
show the picture.
Am I missing anything?

>
> @2
> Local caching is already implemented.
> Queries via HTTPS still need to be implemented, see [3].
> Notary bouncing (a form of onion routing) sounds like a good concept.
> Foremost however I think we need more servers :) This would also allow
> us to query notaries randomly.

I am not convinced by the bouncing notaries because, if 1 is true, I
am not sure the effort would pay off. Given that a service certificate
is added to the trustdb, apart from the first call that triggers the
about:certerror page, no other calls will interact with the notaries
so no browser history will be tracked by the notaries.
What instead I thought more interesting is the concept of being able
at any time to change our trust of a service. In other words, if I
decide to trust a service today, it does not mean I want to do it
forever. How is this handled by perspectives?

ste

[Carl Antuar]

Perspectives currently has around 7,000 daily users, and I wouldn't want to leave them in the dirt by simply shutting things down. Are you suggesting porting some Convergence components over here rather than us moving over there?

I know I would prefer that. Convergence is an interesting idea, with some good features, but it aims to *replace* PKI - eg by using its own certificate for everything - whereas Perspectives allows you to *augment* it.

Gerold said:
>It's funny how [moxie's] three points about Perspectives limitations are still true three years later:
>1. No embedded content support
>2. Privacy issues
>3. Unresponsive servers

 

I'm not clear on #1; does this mean that Perspectives doesn't validate certificates for resources eg images in the page, but just leaves them to regular PKI?

#3 is obviously a technical issue with the current notaries, not part of the design.

But I think it's worth taking on the notary bounce feature for those who want it, to improve #2.

 

What are everyone's thoughts if we move for a combined approach of:
* Continue to fix the most glaring Perspectives issues on this side: unresponsive notaries, new hashes, HTTPS
* Merge in/convert/update some components if we can show they are better (e.g. the scanner component)

That makes sense to me. I think Convergence should have been created as extra Perspectives features, instead of a whole separate project, in the first place.

 

It might be useful to have our notaries implement all of the Perspectives, Convergence, and Namecoin APIs so the servers could be used by any client. Then they would be useful and could serve as additional notaries regardless.

Can anyone explain why Convergence needed a separate API in the first place? Was it just for notary bounce? Yeah, supporting them all makes sense.

[Gerold Meisinger]

On 2014-11-17 23:42, Stefano Fornari wrote:
> If I well understand it, this is to handle resources inside a page
> (e.g. images, stylesheets, scripts, etc.). If that is the case, why
> the current implementation does not take care of it? what the add-on
> does today is to add the trusted certificate to the ff trustdb; from
> that point on all requests to the same service will not require a new
> validation any more, therefore the content will be retrieved with no
> issues.

What you said is correct for the large part except Perspectives doesn't
add the certificate as trusted but rather makes a security exception.
Pretty much the same what you can do as a user.

And the difference of the main request versus embedded content is that
we don't get the "certificate not trusted"-page for embedded content
which automatically restarts the request for us. So in my
"embedded_content" branch there are actually exceptions installed but
they only work if the user refreshes the page because I don't know how
to restart them yet.

> I am not convinced by the bouncing notaries because, if 1 is true, I
> am not sure the effort would pay off. Given that a service certificate
> is added to the trustdb, apart from the first call that triggers the
> about:certerror page, no other calls will interact with the notaries
> so no browser history will be tracked by the notaries.

The exception is applied everytime.

> What instead I thought more interesting is the concept of being able
> at any time to change our trust of a service. In other words, if I
> decide to trust a service today, it does not mean I want to do it
> forever. How is this handled by perspectives?

In Perspectives you currently have the option to clear the cache or
whitelist a domain. We thought about adding ignore list as well.

If we would use a Trustdb approach you could delete them in Firefox ->
Properties -> Advanced -> Certificates -> View Certificates -> Delete or
Distrust...

[mic...@yanovich.net]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 11/17/2014 10:08 PM, Carl Antuar wrote:
> Gerold said:
>>> > >It's funny how [moxie's] three points about Perspectives limitations are
>> > still true three years later:
>>> > >1. No embedded content support
>>> > >2. Privacy issues
>>> > >3. Unresponsive servers
>> >
>
> I'm not clear on #1; does this mean that Perspectives doesn't validate
> certificates for resources eg images in the page, but just leaves them to
> regular PKI?

One of the reasons I originally found the Perspectives Project was my frustration
with how Convergence would wrap everything in *their* SSL certificate, thereby
limiting your technical view to the SSL certificate in Firefox at the time.
Though, I think accounting for mixed content should also be accounted for.

What if there was an option in the Perspectives bubble to list all third party
resources and their notary statuses as an "advanced view" off of the current
notary summary in the extension?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJUa1ebAAoJEKBpQL3CDq4dD3gP/2JTKRazxCqvt5jJ5f1EiO/2
qhdi1xPhGce6XS8iCBmAXX0EsjbHKGG+qUjldIDpQCOVSbFWAFizBUOs/Q8Cad5c
3haX8YLjEeiXx4I4raaibSohVWPH4v1j/i/sI7Vhb9FDNmvOPltSS7lddrGckcJ+
cANfNS743sOvTRlsOyHKxu/Y2EIr+ijhXJlvUkz3lt7tDmVIUPFn5QjtpIW+8mQa
q1gNEqboZUuXbY65D8fDqdo7/TbLcZ73OG5mOAkm0Avr5IW32Z1LMOeomSG99SmZ
OCUEVBiRdOeNA9d95QCbdN7+Zc/7hZJpNbgJK6IsnQyyyPZ2r19bklu0XE6c8bGD
KJLCaTpwZZKmon15TC34acX0RlBa+0qreEWBQJQXkM4Xubc/DJM2UmYDpwoNwOE5
/5w7Ufpqg7GN5CafE8Z9HUPfpOxYUzNxfmB3Qj3EQNQriBh9P8KtDKpF2fM4VRHb
fbh0SarhMiG4RlihgEtM/qv/lfOiFiejjyPfd3wXlAQggLtdxfCGgf6TIxUFgJDh
pAtFO7/6BVR1pkkb7P/b8Tv11M4dYF4/pQtxWbszSFfLfRhtc/5z5hp/ZlyHs5oa
qXNIfQl7eAWjWOpYfa3JR3M49jVgOClQD9g4J+zbuiBDwM3s6mpSAQU2FQ8Z411v
9Ef/K+IyvdX1lCVtTRdT
=IFzm
-----END PGP SIGNATURE-----

[Stefano Fornari]

On Tue, Nov 18, 2014 at 10:11 AM, Gerold Meisinger
<gerold.m...@gmail.com> wrote:
> On 2014-11-17 23:42, Stefano Fornari wrote:
>> If I well understand it, this is to handle resources inside a page
>> (e.g. images, stylesheets, scripts, etc.). If that is the case, why
>> the current implementation does not take care of it? what the add-on
>> does today is to add the trusted certificate to the ff trustdb; from
>> that point on all requests to the same service will not require a new
>> validation any more, therefore the content will be retrieved with no
>> issues.
>
> What you said is correct for the large part except Perspectives doesn't
> add the certificate as trusted but rather makes a security exception.
> Pretty much the same what you can do as a user.

yep. I was over simplifying :)

>
> And the difference of the main request versus embedded content is that
> we don't get the "certificate not trusted"-page for embedded content
> which automatically restarts the request for us. So in my
> "embedded_content" branch there are actually exceptions installed but
> they only work if the user refreshes the page because I don't know how
> to restart them yet.

Yep, I got that point. But my point is that once the exception is
installed, the browser works well for the content unless it requests a
resource on another server for which you do not have an exception yet.
But this is not different from how the browser works without
perspective therefore to me this is not a real issue. => better to
focus on other areas probably (like the expiration as per the below).

>
>> I am not convinced by the bouncing notaries because, if 1 is true, I
>> am not sure the effort would pay off. Given that a service certificate
>> is added to the trustdb, apart from the first call that triggers the
>> about:certerror page, no other calls will interact with the notaries
>> so no browser history will be tracked by the notaries.
>
> The exception is applied everytime.

Exactly. Since the exception is applied every time (by the browser) no
notaries are actually contacted, therefore no real browsing history is
recorded (apart of course the host names the first time the service is
contacted - it seems acceptable).

>
>> What instead I thought more interesting is the concept of being able
>> at any time to change our trust of a service. In other words, if I
>> decide to trust a service today, it does not mean I want to do it
>> forever. How is this handled by perspectives?
>
> In Perspectives you currently have the option to clear the cache or
> whitelist a domain. We thought about adding ignore list as well.
>
> If we would use a Trustdb approach you could delete them in Firefox ->
> Properties -> Advanced -> Certificates -> View Certificates -> Delete or
> Distrust...

True, but this is not very user friendly and would require a user to
remember to do it (very unlike) and also to identify which certificate
is associated to which service, which in case of self-signed
certificates will not be easy. Keep in mind that differently from the
CA approach, we potentially install a certificate/exception for each
site we browse). What about:

1. clear the cache automatically once in a while; or
2. keep a validity lifespan for each certificate; or
3. use certificates validity (I guess this may not improve the
situation much for very long certificate validity).

Ste

[Gerold Meisinger]

> On 11/17/2014 10:08 PM, Carl Antuar wrote:

> Convergence is an interesting idea, with some good features, but it
> aims to *replace* PKI - eg by using its own certificate for everything

On 2014-11-18 15:28, mic...@yanovich.net wrote:
> One of the reasons I originally found the Perspectives Project was my
> frustration with how Convergence would wrap everything in *their* SSL
> certificate, thereby limiting your technical view to the SSL
> certificate in Firefox at the time.

Could someone elaborate on this please. I also read that on the web
everywhere but I don't understand the difference to Perspectives.
Perspectives and Convergence notaries both fetch the certificate
fingerprints, the client compares it and installs a security exception
override. The only difference I found is that Convergence wouldn't allow
me to surf to a site at all if it couldn't verify the certificate. This
should be easy to fix though.

> On 11/17/2014 10:08 PM, Carl Antuar wrote:
> I'm not clear on #1; does this mean that Perspectives doesn't validate
> certificates for resources eg images in the page, but just leaves
> them to regular PKI?

Yes, that is correct.

On 2014-11-18 15:28, mic...@yanovich.net wrote:
> What if there was an option in the Perspectives bubble to list all third party
> resources and their notary statuses as an "advanced view" off of the current
> notary summary in the extension?

Good point, we probably need something like that for more sophisticated
embedded content support.

[Gerold Meisinger]

On 2014-11-18 22:22, Stefano Fornari wrote:
> But my point is that once the exception is
> installed, the browser works well for the content unless it requests a
> resource on another server for which you do not have an exception yet.
> But this is not different from how the browser works without
> perspective therefore to me this is not a real issue.

I see. Does anyone else have an opinion on the priority of "restarting
verified embedded content requests"? If you don't this is an issue we
could integrate the embedded content support as is.

Personally I found it annoying to have to reload the whole page and I
was expecting the some behaviour as having all CAs trusted.

> Exactly. Since the exception is applied every time (by the browser) no
> notaries are actually contacted, therefore no real browsing history is
> recorded (apart of course the host names the first time the service is
> contacted - it seems acceptable).

But the first query is already a huge privacy issue :)

>> If we would use a Trustdb approach you could delete them in Firefox ->
>> Properties -> Advanced -> Certificates -> View Certificates -> Delete or
>> Distrust...
> True, but this is not very user friendly and would require a user to
> remember to do it (very unlike) and also to identify which certificate
> is associated to which service, which in case of self-signed
> certificates will not be easy. Keep in mind that differently from the
> CA approach, we potentially install a certificate/exception for each
> site we browse). What about:
>
> 1. clear the cache automatically once in a while; or
> 2. keep a validity lifespan for each certificate; or
> 3. use certificates validity (I guess this may not improve the
> situation much for very long certificate validity).

Currently the cache is cleared when you close the browser. Otherwise the
lifespan is ~3 days.

I filed a feature request for a cache manager, see #158.

[Gerold Meisinger]

On 2014-11-15 16:09, Gerold Meisinger wrote:
> 1. Convergence "Extra"

I found a list of public notaries here [1]. Of those notaries I was only
able to download: thoughtcrime.org, ccsl.carleton.ca, void.gr, wsg.no,
dc585.info, hsbp.org, intrepidusgroup.com, khjk.org, netomatic.de,
schuurman.com, secyoure.com. Of those, using the scheme descripted in
[2], I only got reponses from:
intrepidusgroup.com => error 500
secyoure.com => error 503
hsbp.org => 200

Try: https://notary.hsbp.org:8443/target/github.com+443

Debugging the extension also revealed that Convergence suffers from the
same getRecentBadCerts removal. Using the hsbp.org notary and Firefox 32
thus then successfully validated my connection. I am missing the result
view of Perspectives though. So much for the current state of the
network and extension of Convergence.


I also did some more research on the meta-issue of the CA system and
looked for other similar projects. I'm going to write up my findings in
the wiki soon. There is at least one similar project called
"Monkeysphere" [5] which uses the OpenPGP Web of Trust to authenticate
connections. However I couldn't even get it to show up in the browser.
In the talk from Next Hope 2010 [6] the authors mention it should show
up in the status bar. The status bar was replaced by the addon bar in
Firefox 4, which in turn was entirely removed in Firefox 29 >_<. I tried
the "add addon bar extension" with Firefox 35, I tried the original
addon bar in Firefox 28, I tried Firefox 22 which is the last officially
supported version [7]. I also tried Firefox 3.6 but they didn't support
Linux x64 back then so I gave up. Besides that, the actual extension
code is very minimalistic and there is nothing to loot, except perhaps
for the overall approach, which is very interesting.


I still need to look into Convergence's codebase and we should also get
in contact with the current maintainer to discuss what the best approach
for both projects might be.

[1] https://github.com/moxie0/Convergence/wiki/Notaries
[2] https://github.com/moxie0/Convergence/wiki/Notary-Protocol
[3] https://github.com/danwent/Perspectives/issues/143
[4] I filed a bug report at https://github.com/mk-fg/convergence/issues/6
[5] http://web.monkeysphere.info
[6] https://www.youtube.com/watch?v=i535XVqjJco
[7] https://lists.riseup.net/www/arc/monkeysphere/2013-09/msg00003.html

[Gerold Meisinger]

On 2014-11-23 18:07, Gerold Meisinger wrote:
> I also did some more research on the meta-issue of the CA system and
> looked for other similar projects. I'm going to write up my findings in
> the wiki soon.

You can find the first version of the wiki article here:
https://github.com/danwent/Perspectives/wiki/Related-work

If you know additional projects or have more information please feel
free to add them!