[Dillo-dev] Delay loading some sites

[Dennis New]

Is anyone else experiencing a long delay (many seconds) before some
sites load, like:

http://craigslist.org

_______________________________________________
Dillo-dev mailing list
Dill...@dillo.org
http://lists.dillo.org/cgi-bin/mailman/listinfo/dillo-dev

[James C]

Yes. For me, meetup.com is often bad.

[Johannes Hofmann]

Hm, for me both sites seem to load reasonably fast here, but I just
tested it once right now.
You could try something like this (with show_msg=YES):

dillo http://craigslist.org/ | gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }'

Maybe that gives a hint what is taking up time.

Cheers,
Johannes

[Dennis New]

On Fri, 7 Jul 2017 18:22:43 +0200, Johannes Hofmann wrote:
> Hm, for me both sites seem to load reasonably fast here, but I just
> tested it once right now.
> You could try something like this (with show_msg=YES):
>
> dillo http://craigslist.org/ | gawk '{ print strftime("[%Y-%m-%d %H:%
> M:%S]"), $0 }'
>
> Maybe that gives a hint what is taking up time.

It's something in the Nav_open_url process. I don't have this delay
using other browsers, so I don't think it's a DNS issue.

[2017-07-07 12:30:58] Nav_open_url: new url='http://craigslist.org/'
[2017-07-07 12:30:58] Dns_server [0]: craigslist.org is 208.82.237.129
[2017-07-07 12:30:58] Connecting to 208.82.237.129
[2017-07-07 12:30:58] ** ERROR **: [Dpi_read_comm_keys] No such file or directory
[2017-07-07 12:30:58] Dpi_blocking_start_dpid: try 1
[2017-07-07 12:30:58] Nav_open_url: new url='http://www.craigslist.org/'
[2017-07-07 12:30:58] Dns_server [0]: www.craigslist.org is 208.82.237.1
[2017-07-07 12:30:58] Connecting to 208.82.237.1
[2017-07-07 12:30:58] Nav_open_url: new url='https://www.craigslist.org/'
[2017-07-07 12:30:59] Nav_open_url: new url='https://geo.craigslist.org/'
[2017-07-07 12:32:30] Nav_open_url: new url='https://montreal.craigslist.ca'
[2017-07-07 12:32:30] Cookies: not sent for request by 'geo.craigslist.org' for 'montreal.craigslist.ca'

[Jorge Arellano Cid]

Hi,

On Fri, Jul 07, 2017 at 06:22:43PM +0200, Johannes Hofmann wrote:
> Hm, for me both sites seem to load reasonably fast here, but I just
> tested it once right now.
> You could try something like this (with show_msg=YES):
>
> dillo http://craigslist.org/ | gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }'
>
> Maybe that gives a hint what is taking up time.

That example has plenty of redirections and DNS queries. This is, the delay
depends on network latency and DNS server response time (which is often
measured in seconds for some popular providers).

Just count the number of connections and DNS queries!

jcid@devuan:~$ dillo http://craigslist.org/

Nav_open_url: new url='http://craigslist.org/'

Dns_server [0]: craigslist.org is 208.82.237.129

Connecting to 208.82.237.129:80

Nav_open_url: new url='http://www.craigslist.org/'

Dns_server [0]: www.craigslist.org is 208.82.237.225
Connecting to 208.82.237.225:80

Nav_open_url: new url='https://www.craigslist.org/'

Connecting to 208.82.237.225:443

Nav_open_url: new url='https://geo.craigslist.org/'

Dns_server [0]: geo.craigslist.org is 208.82.237.225
Connecting to 208.82.237.225:443
Nav_open_url: new url='https://santiago.craigslist.org/'
Dns_server [0]: santiago.craigslist.org is 208.82.237.146
Connecting to 208.82.237.146:443

HTH.

--
Cheers
Jorge.-

[Dennis New]

On Fri, 7 Jul 2017 13:30:04 -0400, Jorge Arellano Cid wrote:

> Well, there's clearly a time taken by dpid to start. Fire a new
> dillo, ask for the bookmarks (this will get dpid running), then go to
> craigslist.org and see if the delay persists.

The delay persists.

> If yes, try with another DNS. Other browsers seem to default to
> google ones.

I tried with only google's 8.8.8.8 DNS server, same thing. I think this
could be something else.

The delay happens after the Nav_open_url https://geo.craigslist.org
line. I noticed that with the links terminal browser, the DNS activity
is four pairs of A and AAAA queries:

-- dns with links --
06:36 : query[A] craigslist.org from 10.0.0.1
06:36 : query[AAAA] craigslist.org from 10.0.0.1

06:36 : query[A] www.craigslist.org from 10.0.0.1
06:36 : query[AAAA] www.craigslist.org from 10.0.0.1

06:36 : query[A] geo.craigslist.org from 10.0.0.1
06:36 : query[AAAA] geo.craigslist.org from 10.0.0.1

06:36 : query[A] montreal.craigslist.ca from 10.0.0.1
06:36 : query[AAAA] montreal.craigslist.ca from 10.0.0.1
--

While dillo has an extra www.craigslist.org query, and none of the last
three have a corresponding AAAA query:

-- dns with dillo
06:46 : query[A] craigslist.org from 10.0.0.1
06:46 : query[AAAA] craigslist.org from 10.0.0.1

06:47 : query[A] www.craigslist.org from 10.0.0.1
06:47 : query[AAAA] www.craigslist.org from 10.0.0.1

06:47 : query[A] www.craigslist.org from 10.0.0.1

06:47 : query[A] geo.craigslist.org from 10.0.0.1

08:18 : query[A] montreal.craigslist.ca from 10.0.0.1
--