Widget to import .txt-File as Tiddler: How do I import umlaute ö ä ü ß an other strange German characters

[Jan]

Hello everybody!
With the help of Jed I was able to create this http://szenio.de/Test/
widget, which imports a txt-file and writes it to a tiddler.
This is already a great help for me... but as I am a german teacher
things are a little complicated because I need Umlaute and other strange
German things like this here ß. Does anyone know how to import them
correctly?

As we say in german: Tschüss ;-)

[Jed Carty]

I am not sure what I did but I am glad I was able to help.


The problem is almost certainly with encoding, but I am not certain where. One unlikely place is using

var urltext = unescape(client.responseText);

in the javascript macro, changing that to

var urltext = decodeURI(client.responseText);

may help. Even if it doesn't help here it is good practice to use decodeURI instead of unescape.

I think that a more likely place for the problem is on the server side. If the text file isn't being read as a utf8 string than any characters that aren't in the basic ascii set won't be sent correctly. That includes any letters with accents or language specific variants, like
ß.

What sort of server are you using on the computer that contains the text file?

[Mark S.]

If you paste the address into the address bar, it shows the umlauts. To me that suggests that the server knows how to serve them from the text file.

[Jed Carty]

It may be at least partially a browser thing, safari doesn't display the characters, Firefox does.

But if it isn't a server thing I am not sure what else to check.

[Jan]

Hi Jed, Hi Mark,
thanks for your help, I tried some things before answering:

stackoverflow says this could help:

   exports.run = function(url) {
      var client = new XMLHttpRequest();
      client.open('GET', url, false);
      client.setRequestHeader('Content-type', 'application/x-www-form-urlencoded; charset=ISO-8859-1');

https://stackoverflow.com/questions/11906904/how-to-force-xmlhttprequest-to-use-iso-8859-1-charset-only


alas as visible here szenio.de/Test/ , it does not

The server is a commercial provider. nginx - I assume. 

Jan

Am 17.07.2019 um 18:30 schrieb Jed Carty:

It may be at least partially a browser thing, safari doesn't display the characters, Firefox does.

But if it isn't a server thing I am not sure what else to check.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/20de66bc-6129-495f-9fc4-d02a4f23e33d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jan]

This at least has an effect...though i doubt it is the right direction...
      client.overrideMimeType('text/plain; charset=x-user-defined');

[Jan]

Hi Mark,
of course I did.  Jed's line of conde is still in the widget on szen.io/test.
But I thing what is going wrong is happening before that step. At that point the data already seems to be unreadable.

Tschö (another German word with rockdots to say 'see you') Jan
   

Am 18.07.2019 um 03:04 schrieb 'Mark S.' via TiddlyWiki:

But did you try Jed's suggestion:

var urltext = decodeURI(client.responseText);

?

On Wednesday, July 17, 2019 at 4:58:15 PM UTC-7, Jan wrote:

Hi Jed, Hi Mark,
thanks for your help, I tried some things before answering:

stackoverflow says this could help:

   exports.run = function(url) {
      var client = new XMLHttpRequest();
      client.open('GET', url, false);
      client.setRequestHeader('Content-type', 'application/x-www-form-urlencoded; charset=ISO-8859-1');

https://stackoverflow.com/questions/11906904/how-to-force-xmlhttprequest-to-use-iso-8859-1-charset-only


alas as visible here szenio.de/Test/ , it does not

The server is a commercial provider. nginx - I assume. 

Jan

Am 17.07.2019 um 18:30 schrieb Jed Carty:

It may be at least partially a browser thing, safari doesn't display the characters, Firefox does.

But if it isn't a server thing I am not sure what else to check.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/20de66bc-6129-495f-9fc4-d02a4f23e33d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/bbed96e9-5044-49f9-a93b-d5a492ebb22e%40googlegroups.com.

[Jan]

Hi Mark, hi Jed...hi anyone interested.
the charset issue is quite a drag, I was only able to solve it taking a detour over a .php which does the work.

<?php
function breakkeeper($text) {
   return strtr($text, array("\n" => "&#10;&#10;"));
}

$url = $_GET['url'];
$posturl = $_POST['url'];
if (!empty($posturl)) {
        $url = $posturl ;
        }
$text = file_get_contents($url);
$such_array  = array ('ä', 'ö', 'ü', 'A', 'Ö', 'Ü', 'ß');

$ersetzen_array = array ('&auml;', '&ouml;', '&uuml;','&Auml;', '&Ouml;', '&Uuml;', '&szlig;');

$text  = str_replace($such_array, $ersetzen_array, $text);
$text  = breakkeeper($text)
?>
<?=$text?>

My Interaction mechanism is gaining shape. If you want to see the way to post to the Wiki check out the posttester on szenio.de/test.

Tschö Jan

[TonyM]

Jan

I could not help you character issue but I am very keen on you import text file.

Could I please ask when you are ready to document it a little more clearly so us English speaking dummies can understand and make use of it.

Thanks in advance
Tony

[Jan]

Hi Tony,
I am happy that you are interested ( by the way, did you check the PHPs
i sent to you?).
I guess will need some time and help, until the handling is intuitive.
At the moment the use is hard to explain because it has still
workarrounds which are not really easy.
Have you tried to send me a text via the Posttester? This part allready
should be ituitive.

Yours Jan