Encoding problems for jsmol context menu

[Andrey Novoseltsev]

Ingo Dahn reported the following problem, but I have no idea how to fix. Can anyone give some pointers? Or even better - pull request ;-)

---------- Forwarded message ----------
I just noted an encoding problem with the jmol context menu when called from my own web page https://netmath.vcrp.de/downloads/Systeme/Sage/SpatX.html which is utf-8 encoded.

As you can see it correctly detects that the language is German but the item in the menu should read “Vergrößerung”.

However, when I call it directly from SageCell using https://sagecell.sagemath.org/?z=eJyNkk0KwyAQhfeF3iNhXMxo87fwJKWLEAItDQlkgr1-ndiIXRl08Xx-Mz5Rh9aNw7asxR2VH4_yenEUPYqeTjj6eeavlsTrCdH267p8CofKkRqWaVltsTcqBUBKAX0Ae9cAICWA2QP5GkfgtGi_Ldp4jeS1Dpoo-GHJZKfXPBZ3H0HKH2p7vob3PDLbNg1Fcibrg9ZnaHPQRkmEDH1Lk2COrpLeebpLcueT1AftUblnjm9icjzFt5EnPMGH3wLyI0BeHdhPDWyAb8AVcA3cALfA3RfMccC8&lang=sage

It shows the correct menu:

[Paul Masson]

The "correct" menu has a spelling error...

This is clearly an encoding problem. I'm also seeing differences between browsers: in Safari the non-ASCII characters are not shown, but in Firefox they're replaced with boxes.

The foreign language menus as stored in .po files in the jsmol/idioma directory. I downloaded the German one and it looks fine when opened locally. Is there some server configuration that would change the encoding when serving the same file in a browser? What's in the .htaccess file regarding encoding?

[Andrey Novoseltsev]

On Saturday, 5 November 2016 16:24:49 UTC-6, Paul Masson wrote:

The foreign language menus as stored in .po files in the jsmol/idioma directory. I downloaded the German one and it looks fine when opened locally. Is there some server configuration that would change the encoding when serving the same file in a browser? What's in the .htaccess file regarding encoding?

I am not installing any .htaccess files that I am aware off, so some defaults are used. Any ideas what should they be and where that file should be placed?

As a possibly related issue: when used cross-domain, jsmol tends to use its PHP script for transmitting files, the menu file may or may not be affected. Configuration file for nginx that serves static content and proxies PHP is https://github.com/sagemath/sagecell/blob/master/contrib/vm/compute_node/etc/nginx/conf.d/sagecell.conf

[Jonathan Gutow]

1) This appears to be an instance of different encodings on the web page and
within Sage-Cell. But I cannot see the version where it does not work.

2) JSmol sometimes has trouble when things are passed to it through too many
layers because extra quotes need to be added to prevent mangling by higher
levels. That could be the cause if the encodings are actually consistent.
The specific fix would depend on the details of the particular web page.  If
examples can be provided of the broken page, I might be able to get some
input from the Jmol team's translation expert.


Jonathan

[Paul Masson]

Andrey, I was reading too many old links about Apache server, which used to default to iso-8859-1 encoding but now defaults to utf-8 as I'm sure all modern server software does. Please ignore my comment on that.

[Paul Masson]

Johnathan, if I go to http://netmath.vcrp.de/downloads/Systeme/Sage/SpatX.html, right click on the rendered JSmol and hover over "Language", I see lots of encoding problems for all non-ASCII characters.

[Andrey Novoseltsev]

On Monday, 7 November 2016 15:41:22 UTC-7, Paul Masson wrote:

Johnathan, if I go to http://netmath.vcrp.de/downloads/Systeme/Sage/SpatX.html, right click on the rendered JSmol and hover over "Language", I see lots of encoding problems for all non-ASCII characters.

Indeed, I have the same issue. Do you know in general what can cause some letters appear OK and others not? Say for Russian and Ukranian Cyrillic letters are mostly OK except for the one that looks (or rather should look) exactly like Latin "c".

[Jonathan Gutow]

OK, I can see the problem. However, I cannot see where it is coming from. I will forward the link to the jmol-developers list and see if any of our translation people have ideas.

Jonathan

--
You received this message because you are subscribed to the Google Groups "sage-cell" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-cell+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cell/8cace745-7377-45a3-ac27-8e3330776548%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
                        Dr. Jonathan H. Gutow
Chemistry Department                                 gu...@uwosh.edu
UW-Oshkosh                                           Office:920-424-1326
800 Algoma Boulevard                                 FAX:920-424-2042
Oshkosh, WI 54901
                http://www.uwosh.edu/facstaff/gutow/ 

[Jonathan Gutow]

Here are some ideas...


-------- Forwarded Message --------
Subject: Re: [Jmol-developers] Fwd: [sage-cell] Re: Encoding problems
for jsmol context menu
Date: Wed, 9 Nov 2016 10:13:06 +0100
From: Angel Herráez <angel....@uah.es>
Reply-To: jmol-de...@lists.sourceforge.net
Organization: Universidad de Alcalá
To: jmol-de...@lists.sourceforge.net

Hi Jonathan

I've seen this kind of a problem occasionally (my system is configured
for Spanish so the JSmol popup menu opens in Spanish, and has accented
characters), but have not been able to track it down.

I think it may be a bad combination of charsets in the several files
involved.
I cannot be certain, but the page
http://netmath.vcrp.de/downloads/Systeme/Sage/SpatX.html
declares UTF-8
while https://sagecell.sagemath.org/embedded_sagecell.js seems to be
encoded as "windows-1252"
I am assuming this is the JS file that will call JSmol.min.js (not
obvious though)

Summary: all JS files called from the page should also be encoded as
UTF-8 and maybe they need to be declared as such or the server
configured to serve them as such. I am not familiar with the particular
details.

Please try to make sure of that and let's see if that fixes the problem





------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Jmol-developers mailing list
Jmol-de...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jmol-developers

[Andrey Novoseltsev]

Not much progress, but some test results:

1) As in the original report, I can reproduce correctly shown system language in JSmol menu if I go to the root page.
2) As in the original report, this menu is garbled if I use a web-page which embeds cells.
3) embedded_sagecell.js is indeed served as windows-1252 in both cases, changing nginx settings to utf-8 was detected in the browser, but did not affect behaviour in 1) and 2). The reason is likely that encoding problems are related to PHP script which is used for cross-domain requests in JSmol - browser console shows menu file with "charset=x-user-defined" independent of nginx settings.
4) "Native language names" are "a bit off" no matter what I have tried, e.g. "с" in "Русский" is not rendered correctly.

Any suggestions what else to try to make things work?..