smart_urlquote fails with some URLs - is this unfixable?

[Mike Dewhirst]

Django 1.11.15 Python 2.7/3.6 Ubuntu 16.04 (staging and production)
Windows 10 development. Postgres 9.6. Haven't tried this beyond
development yet.

My app relies on the Admin and uses a widget which adjusts the clicked
link to open in a new tab instead of the current tab like this ...

class New_Tab_AdminURLFieldWidget(AdminURLFieldWidget):
    """ Adjust target to open in a new browser tab """
    def render(self, name, value, attrs=None):
        html = super(AdminURLFieldWidget, self).render(name, value, attrs)
        if value:
            value = force_text(self._format_value(value))
            smart_url = smart_urlquote(value)
            if smart_url.endswith("="):
                smart_url = smart_url[:-1]
            final_attrs = {'href': smart_url}
            html = format_html(
                '<p class="url">{} <a{} target="_blank">{}</a><br />{}
{}</p>',
                'Click here:', flatatt(final_attrs), value,
                '', html
            )
        return html

This works nicely with most URLs but I have found one which barfs

working url with smart_urlquote disabled
http://gestis-en.itrust.de/nxt/gateway.dll/gestis_en/011230.xml?f=templates$fn=document-frame.htm$3.0$GLOBAL=G_&G_DIEXSL=gestis.xsl$q=%5Bfield,schnellsuche%3A%5Borderedprox,0%3Aacetone%5D%5D%20$uq=$x=server$up=1#LPHit1

Output from enabled smart_urlquote which fails
http://gestis-en.itrust.de/nxt/gateway.dll/gestis_en/011230.xml?f=templates%24fn%3Ddocument-frame.htm%243.0%24GLOBAL%3DG_&G_DIEXSL=gestis.xsl%24q%3D%5Bfield%2Cschnellsuche%3A%5Borderedprox%2C0%3Aacetone%5D%5D+%24uq%3D%24x%3Dserver%24up%3D1#LPHit1

I can get the above widget working by disabling smart_urlquote() but
when I do that ... a previously working url stops working ...

previously working url (ie smart_urlquote() output)
https://www.echemportal.org/echemportal/substancesearch/substancesearch_execute.action?numberType=CAS&amp=&number=67-64-1&amp=&allParticipants=true

input url pre-smartquote
https://www.echemportal.org/echemportal/substancesearch/substancesearch_execute.action?numberType=CAS&amp;number=67-64-1&amp;allParticipants=true

not working url (smart_urlquote() disabled)
https://www.echemportal.org/echemportal/substancesearch/substancesearch_execute.action?numberType=CAS&amp;number=67-64-1&amp;allParticipants=true

input url pre-smartquote
https://www.echemportal.org/echemportal/substancesearch/substancesearch_execute.action?numberType=CAS&amp;number=67-64-1&amp;allParticipants=true

Perhaps the only way I can fix this is to detect the ugly bits and
replace them with nice bits?

That would be seriously fragile

Any ideas?

Thanks

Mike

[Mike Dewhirst]

On 19/08/2018 7:09 PM, Mike Dewhirst wrote:
> Django 1.11.15 Python 2.7/3.6 Ubuntu 16.04 (staging and production)
> Windows 10 development. Postgres 9.6. Haven't tried this beyond
> development yet.
>
> My app relies on the Admin and uses a widget which adjusts the clicked
> link to open in a new tab instead of the current tab like this ...
>
> class New_Tab_AdminURLFieldWidget(AdminURLFieldWidget):
>     """ Adjust target to open in a new browser tab """
>     def render(self, name, value, attrs=None):
>         html = super(AdminURLFieldWidget, self).render(name, value,
> attrs)
>         if value:
>             value = force_text(self._format_value(value))

What I ended up doing is to detect a word in the URL which gets
corrupted and skip the smart_urlquote() step - which for the moment
seems to work. Feels quite fragile though.

                if 'gestis' in value:
                    smart_url = value
                else: