encoding issues in fetch urls

[Yael Netzer]

Hi,

In many cases when I do 'add column by fetching urls' -- in order to receive jsons that contain Hebrew I get in return the Hebrew in wrong encoding.

Any suggestions?

The API call is

https://www.sefaria.org/api/index/Abarbanel%20on%20Torah

( "https://www.sefaria.org/api/index/" + value )

 

Thanks in advance!

Yael

[Owen Stephens]

Hi Yael,

That call works for me OK and retrieves and displays the Hebrew correctly. Can you give some more information on which version of OpenRefine you are using and on which platform (Windows/Linux/Mac)?

One option with mis-encoded data is to use the `reinterpret` transformation - this will try to reinterpret the data using an encoding scheme.

e.g. value.reinterpret("utf-8")

You could see if that is a solution, although it would be good (from the point of view of solving the underlying problem) to understand why it comes through incorrectly for you and correctly for me

Best wishes,

Owen

[Yael Netzer]

Thanks for the quick reply

value.reinterpret("utf-8") does not work here (no change)

I use windows 10 Home --

OpenRefine Version 3.4.1 [437dc4d]

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/006fa504-d83b-458c-a48a-09f89e1602a8n%40googlegroups.com.

[Thad Guidry]

Hi Yael,

Very sorry you are having problems here with Fetching.

This was a bug in the 3.4.1 release.

It has been fixed now and will work once we release 3.5

reinterpret() won't fix this since the bug was how we stored the value during Fetch (we've switched HTTP clients and missed a few things that are now fixed)

As a quick fix, you can use the latest snapshot (3.5 forthcoming and only lightly tested) which can be downloaded from here:

https://github.com/OpenRefine/OpenRefine-snapshot-releases/releases/tag/3.4-beta-983-g804e3a3-dirty

I tried that latest snapshot version on your URL and it fixes the issue, so rest assured that 3.5 will work for that.

Thad

https://www.linkedin.com/in/thadguidry/

https://calendly.com/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/CAPAg932VSaSGSfVAJinHXb1UEsg2%3DP%3DXU_2HofNtyYFY4LFH-Q%40mail.gmail.com.

[Yael Netzer]

Thad, thanks! It works excellent.

Looking forward the full and stable 3.5!

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/CAChbWaN_-M2wTz8cT0L_%3Dc4w3dt0OSX25SGyvqJFjMoLHVsnWA%40mail.gmail.com.