Cryptography with Prolog: How do I begin?

[Markus Triska]

Hi all,

the upcoming stable release, SWI-Prolog 7.6, will provide cryptographic functionality that has never before been available in any Prolog system.

To encourage the use of these features, I have uploaded a document that explains how you can call and combine the new predicates in typical cases:

https://www.metalevel.at/prolog/cryptography

The examples include: securely storing user passwords, verifying digital signatures, and encrypting data with a password.

You can try all included queries and code samples already today, using the current SWI-Prolog development release (7.5.15) and OpenSSL 1.1.0.

I hope that the newly available features help you to implement Prolog applications that achieve very high security standards.

If you need any additional features, or if you have any questions or comments, please let me know any time!

Thank you and all the best,

Markus

[Jan Burse]

If you want to make crypo transparent, you should extend the support
for URLs in SWI-Prolog. This is like the end-user view of crypto, and
not the developer view of crypto.

So it would address the need for those who for example just want
to establish a connection to a HTTPS web site without much husse.
I doubt that we need an extra open/[3,4] for that.

I have aready promoted extending ISO core Prolog streams to also
support all kind of connections for a while. I remember discussions
with Logtalk, why we would then drop the requirement that

stream_property really keeps a table of all connections. But maybe
different design considerations are possible. Last week I figured out,
that absolute_file_name/[2,3] can be used to let a web site normalize
paths, and make visible when a HTTPS connection is requested.

This does not (yet?) work in SWI-Prolog I only get:

?- absolute_file_name('http://blick.ch/robots.txt', X).
X = 'c:/users/jan burse/documents/prolog/http:/blick.ch/robots.txt'.

On the other hand for the upcoming release of Jekejeke Prolog I have:

?- absolute_file_name('http://blick.ch/robots.txt', X).
X = 'https://www.blick.ch/robots.txt'

Note the magic that happend (by virtue of the HTTP 301 code):
    - The host was automatically prefixed by 'www.'
    - The protocol was automatically changed from 'http:' to 'https:'

This is a very nice canonization without the use of an other infrastructure
from the internet, namely the domain name server infrastructure. Because
this infrastructure usually yields nonsense for a web server:

I guess the nonsense happens because of modern CDN technique,
a fact that I had to learn last weak also.

?- use_module(library(system/domain)).
% 1 consults and 0 unloads in 16 ms.
Yes
?- host_lookup('blick.ch', X).
X = '37.58.62.60'
?- host_lookup(X, '37.58.62.60').
X = '37.58.62.60'
?- host_lookup('www.blick.ch', X).
X = '23.211.6.46'
?- host_lookup(X, '23.211.6.46').
X = 'a23-211-6-46.deploy.static.akamaitechnologies.com'

For the end-user, extending the absolute_file_name to URLs
is very comfortabke. For example then reading a HTTPS web
site works as easily as:

?- open('http://blick.ch/robots.txt', read, X, [alias(test)]).
X = 0r4bc68116
?- read_line(test, X).
X = 'User-agent: * '
?- close(test).
Yes

See also my blog post here:
https://plus.google.com/+JekejekeCh/posts/7f3Av1mruKw

Disclaimer: This realization wasn't planned. I was first
researching into Puny code. But this web server based to
find normal URLs is very welcome.

By the approach of just using the ISO core Prolog API, I
spare additional predicates such as for example http_open/3.
I could add all the http_open/3 options to open/[3,4] as well.

I did the following test here (providing a HTTP URL which
will be redirected to a HTTPS URL):

Welcome to SWI-Prolog (threaded, 64 bits, version 7.6.0-rc1)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

?- use_module(library(http/http_open)).
true.

ᄀ?- use_module(library(http/http_ssl_plugin)).
ERROR: c:/program files/swipl/library/crypto.pl:68:
        c:/program files/swipl/library/crypto.pl:68: Initialization goal raised exception:
        '$open_shared_object'/3: Das angegebene Modul wurde nicht gefunden.

?- http_open('http://blick.ch', X, []).
ERROR: Undefined procedure: ssl:'_ssl_context'/4
ERROR:   However, there are definitions for:
ERROR:         ssl:ssl_context/3

But after the first error it probably didn't make any sense to
continue testing. Is the SSL plugin not supported on Windows?
Or is this a smal problem of the release  version 7.6.0-rc1 I was using?

I was using this recommendation here:
http://www.swi-prolog.org/pldoc/man?section=ssl-https-client

But I don't know whether there is a recommendation to
get the redirected URL before actually establishing the connection.
Is this possible?

[Markus Triska]

Hi Jan,

On Sunday, September 17, 2017 at 7:31:39 PM UTC+2, Jan Burse wrote:

But after the first error it probably didn't make any sense to
continue testing. Is the SSL plugin not supported on Windows?
Or is this a smal problem of the release  version 7.6.0-rc1 I was using?

It is a small problem with 7.6.0-rc1 and Jan pushed a fix so that as of tomorrow's daily build (and then with 7.6.0-rc2), it should work!

Once it works, you get for example:

?- http_open('http://blick.ch', Stream, [final_url(Final)]).

Stream = <stream>(0x7fc6ed020c00),

Final = 'https://www.blick.ch/'.

This shows that the HTTP --> HTTPS redirect does indeed happen, and a TLS session is created.

Thank you for trying this, and I hope it works for you with tomorrow's build!

It is an interesting idea to use open/3 for this: I frequently need to open HTTP and HTTPS streams, and I always wondered why http_open/3 is not at least autoloaded:

https://github.com/SWI-Prolog/packages-http/issues/61

All the best,

Markus

[Jan Burse]

Cool!

Didn't pay attention to this fine detail, the final_url/1 option.

BTW do you support puny code (under the hood)?
This one gives also a nice test case (German city with umlaut):

http://nürnberg.de/robots.txt

[Markus Triska]

Hi Jan,

On Monday, September 18, 2017 at 9:11:24 PM UTC+2, Jan Burse wrote:

Cool!

Didn't pay attention to this fine detail, the final_url/1 option.

In this context, the option redirect/1 (or, more generally, max_redirect/1) is also extremely useful: Together with the status_code/1 option, it can be used to prevent redirects. It is sometimes important to rewrite redirects, for example in (reverse) proxy servers, and in such cases we want to perform the redirect ourselves instead of relying on the HTTP framework. This feature is used for example in Proloxy to rewrite redirects so that the correct target service is used even though the service itself is not aware of the proxy.

 

BTW do you support puny code (under the hood)?
This one gives also a nice test case (German city with umlaut):

http://nürnberg.de/robots.txt

With SWI, I get:

?- http_open('http://nürnberg.de/robots.txt', Stream, [redirect(false)]).

ERROR: Socket error: Unknown error 0

ERROR: In:

ERROR:   [16] throw(error(socket_error('Unknown error 0'),_8894))

This arises already at the HTTP layer, so the SSL bindings are not (yet) involved in this error.

It is, however, also a security issue to support such URLs, because they give rise to a certain class of attacks:

https://en.wikipedia.org/wiki/IDN_homograph_attack

Therefore, this should definitely also be considered in connection with HTTPS.

Are you considering ways to mitigate such attacks?

All the best!

Markus

[Jan Burse]

Try any web browser, type in http://nürnberg.de/robots.txt , it
will do the redirect for you. No warnings no nothing:

Chrome: http://nürnberg.de/robots.txt  --> https://www.nuernberg.de/robots.txt
Edge: http://nürnberg.de/robots.txt- -> https://www.nuernberg.de/robots.txt
Seamonkey: http://nürnberg.de/robots.txt --> https://www.nuernberg.de/robots.txt

DNS does only understand ASCII, so you have to ACE the URL first.

?- uri_puny('http://nürnberg.de/robots.txt', X).
X = 'http://xn--nrnberg-n2a.de/robots.txt'

I am using puny coding during all connections, because in Java there
is no other way (so far my testing shows), to access Unicode domain

names. The libary I am using is this:
https://docs.oracle.com/javase/6/docs/api/java/net/IDN.html

You need to puny code the authority part of an URL. The library, through
open/[3,4] also allows to open homographic similar domain names.
There is no restriction what ever built-in. Maye you want to contact
the server and ask for further credentials, an imposter might not

give you the correct information, so its necessary that you can
connect to homographic similar domain names, you might anyway
have further validation steps in the application layer. Since puny code
works, it seems that nowhere in the stack any such checks based
 
on the domain name are done. And for example browsers, agains
the usual myth, done use puny code when something is not ASCII.
They do the contrary, they convert puny code back to Unicode.
Try this, and error in the nürnberg.de web site URL,

I changed the country form "de" to "com":

http://xn--nrnberg-n2a.com/robots.txt

Here is what the browsers do with this beast (Some
browsers try the prefix www. on their own, but all
browser do a Un-Puny code):

Chrome:

http://xn--nrnberg-n2a.com/robots.txt -->

http://nürnberg.com/robots.txt and Error Page

Edge:

http://xn--nrnberg-n2a.com/robots.txt -->

http://nürnberg.com/robots.txt and Error Page

Seamonkey:

http://xn--nrnberg-n2a.com/robots.txt -->

http://www.nürnberg.com/robots.txt and Error Page

In my opinion this is fine. You could do more as
described here, but maybe the whole domain name
system needs a different approach:
https://www.xudongz.com/blog/2017/idn-phishing/

Like some lexical backend in the domain servers,
with world wide application. Most likely domain servers
will look different in 2525. But today is only 2017.

[Jan Burse]

The recommendation goes to the domain owner:

"If you are buying a domain in a registry which does not have proper
anti-spoofing protections (like .com), it is sadly the responsibility of
domain owners to check for whole-script homographs and register them."

From the same article:
https://www.xudongz.com/blog/2017/idn-phishing/

[Jan Burse]

I guess I am mocking Paul V. Mockapetris.

[Jan Burse]

The following paper sums up the current DNS:

Development of the Domain Name System
Paul V. Mockapetris, Kevin J. Dunlap - 1988
http://bnrg.eecs.berkeley.edu/~randy/Courses/CS268.F08/papers/31_dns.pdf

You find some fundamental decisions here: 

2.2 The name space
Labels are variable-length strings of octets, and each
octet in a label can be any 8-bit value. The zero length
label is reserved for the root. Name space searching
operations (for operations defined at present) are done
in a case-insensitive manner (assuming ASCII). Thus
the labels “Paul”, “paul”, and “PAUL”, would match
each other.

And why it might be important to either access the
DNS even for strange domain names. The DNS itself
could give further hints:

2.2 The name space
"but the default assumption is that
the only way to tell definitely what a name represents
is to look at the data associated with the name."

But if the DNS is corrupted, either through the end zones or
intermediary zoones, because they cannot be trusted.

I guess this doesn't help either...

[Jan Burse]

There was also the idea to use the DNS to censor the internet:
https://de.wikipedia.org/wiki/Zugangserschwerungsgesetz

[Jan Burse]

We might begin with data types. In Java, most of crypto is done
with byte[] (array of primitive byte), and since in Java byte is fixed

as octet its kind of internet compatible... Same in ISO core Prolog
byte is fixed as octet. And this already looks good:

library(crypto): Cryptography and authentication library

https://eu.swi-prolog.org/pldoc/man?section=crypto

Bytes, bytes, bytes, etc... Cool!

And this probably due to Markus Triska:

:- use_module(library(clpfd)).

bytes_integer(Bs, N) :-
        foldl(pow, Bs, 0-0, N-_).

pow(B, N0-I0, N-I) :-
        B in 0..255,
        N #= N0 + B*256^I0,
        I #= I0 + 1.

What happens if Bs is just a variable:

Welcome to SWI-Prolog (threaded, 64 bits, version 7.5.8)

SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

?- bytes_integer(L,X).

L = [],
X = 0

L = [X],
X in 0..255

L = [_3136, _3142],
_3136 in 0..255,
_3136+_3204#=X,
_3204 in 0..65280,
_3142*256#=_3204,
_3142 in 0..255,X in 0..65535 .

And then you need:

crypto_data_encrypt(+PlainText, +Algorithm, +Key, +IV, -CipherText, +Options)

encoding(+Encoding)

Encoding to use for PlainText. Default is utf8. Alternatives are utf8 and octet.

Nice!

Am Freitag, 15. September 2017 16:37:46 UTC+2 schrieb Markus Triska: