Re: [Dillo-dev] Duckduckgo error and want to test latest Dillo 3.1

[Dillo]

Hi,

On Thu, Feb 02, 2017 at 11:09:30PM -0500, John Doe wrote:
> I have found an issue involving the use of Ctrl+s to do a quick
> https Duckduckgo search. The search engine does show results via
> this method; however, the links are screwed up. Duckduckgo is
> using some kind of prefix for each of the links and only browsers
> like Midori or Firefox know how to handle them. To fix this, I
> use the Ctrl+s shortcut and then I have to press search once more
> after the results page has loaded.

Good trick!

There're two problems, URI escaping and a bad HTML redirection
page by duckduckgo. I've some code for it and hope to upload a
solution in a week or so.

> Also, need a guy to help test Dillo 3.1? I use Mac and Linux.
> Don't know if you got my MacOS fix from the last email I sent.

Probably yes, but please join dillo-dev. There're Mac users
there. BTW, I'm CC'ing this to dillo-dev.

@all: sorry for my lack of activity in the past months, nothing
technically related with dillo, but with my personal life and my
beloved little daughter.

--
Cheers
Jorge.-

_______________________________________________
Dillo-dev mailing list
Dill...@dillo.org
http://lists.dillo.org/cgi-bin/mailman/listinfo/dillo-dev

[Jeremy Henty]

Dillo wrote:

> On Thu, Feb 02, 2017 at 11:09:30PM -0500, John Doe wrote:
> > I have found an issue involving the use of Ctrl+s to do a quick
> > https Duckduckgo search. The search engine does show results via
> > this method; however, the links are screwed up. Duckduckgo is
> > using some kind of prefix for each of the links and only browsers
> > like Midori or Firefox know how to handle them. To fix this, I use
> > the Ctrl+s shortcut and then I have to press search once more
> > after the results page has loaded.
>
> Good trick!
>
> There're two problems, URI escaping and a bad HTML redirection page
> by duckduckgo. I've some code for it and hope to upload a solution
> in a week or so.

Experiments with wget suggest to me that it depends on whether you GET
or POST your query. A GET request returns a page where each result
link points to a DuckDuckGo URL that returns some HTML containing
nothing but a Javascript redirect to the real search result. Dillo
does not support Javascript so it displays a blank page. Browsers
that support Javascript immediately load the search result. A POST
request returns a page where each result link points straight to the
result. Clicking the Search button in the form re-POSTs the result,
which is why the "Ctrl-s, then Search button" trick works.

I have no idea why DuckDuckGo might want GET and POST to differ in
this way, but it seems clear that they do. You can test this yourself
by comparing the results of

wget 'https://duckduckgo.com/lite/?q=shawarma'
wget --post-data='q=shawarma' -- 'https://duckduckgo.com/lite/'

Perhaps we could allow dillorc to tell Dillo whether to GET or POST
the result? Maybe implement a "search_url_post=..." option that
works just like "search_url=..." except that it POSTs the query? Or
maybe support an optional keyword "POST" at the end of the
"search_url=..." option?

Regards,

Jeremy Henty