[Pharo-project] looking for Unicode testers
Michael Rueger
finally I have something to look at :-)
http://impara.de/michael/unicode.zip
It is the pharo one click with a modified (slightly outdated) image and
an updated Unix VM.
The image is not cleaned up at all, just so you are warned ;-)
Please try out this version on different platforms and check if unicode
keyboard input, clipboard copy and paste as well as filenames are working.
There are probably still plenty of bugs in there, that's why I need
extra testers :-)
Please also point out any existing patches I missed to include in this
image.
Enjoy
Michael
_______________________________________________
Pharo-project mailing list
Pharo-...@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Lukas Renggli
> an updated Unix VM.
The one-click thing does not work on my Mac, nothing happens. I can
open the image with another VM though.
Is this a special VM that comes with the one-click image? I might miss
the context.
> Please try out this version on different platforms and check if unicode
> keyboard input, clipboard copy and paste as well as filenames are working.
Pressing any key-combination with the apple-key raises an error
"MessageNotUnderstood UndefinedObject>>#<".
Pasting most of the texts from this website
<http://www.columbia.edu/kermit/utf8.html> raises an error: "Invalid
utf8 input detected".
Cheers,
Lukas
--
Lukas Renggli
http://www.lukas-renggli.ch
Igor Stasenko
When i run image, it asks me multiple times to insert a disc in a drive :)
(Its a well known issue of freetype, which trying to look a fonts in
well-known locations)
- typing a russian text works (of course, to see letters i need to set
a proper font, otherwise i see only '?????' )
- copy/paste works fine in both directions
what is not very convenient is that copy/paste (as well as the rest of
keyboard shortcuts ) is not working when keyboard layout not english.
2009/2/6 Lukas Renggli <ren...@gmail.com>:
--
Best regards,
Igor Stasenko AKA sig.
Chris Kassopulo
>> It is the pharo one click with a modified (slightly outdated) image and
>> an updated Unix VM.
>
> The one-click thing does not work on my Mac, nothing happens. I can open
> the image with another VM though.
>
> Is this a special VM that comes with the one-click image? I might miss
> the context.
>
>> Please try out this version on different platforms and check if unicode
>> keyboard input, clipboard copy and paste as well as filenames are
>> working.
>
> Pressing any key-combination with the apple-key raises an error
> "MessageNotUnderstood UndefinedObject>>#<".
>
> Pasting most of the texts from this website
> <http://www.columbia.edu/kermit/utf8.html> raises an error: "Invalid
> utf8 input detected".
>
> Cheers,
> Lukas
On linux pasting ä or é from that same webpage gives
"Error: Invalid utf8 input detected"
How do I input those characters from the keyboard?
Chris
Mariano Martinez Peck
I test áéúíó ñ and all works well :)
On the other hand, I test changing the fonts for code, and also works ok.
Good news!
thanks,
mariano
Hilaire Fernandes
My system is LANG=fr_FR.UTF-8
My keyboard keying is us_intl, (accented character are composed), this
set up is used to work with standard pharo and ecoding to latin1
In the workspace when I type in 'e, I get e?
When I open the FileBrowser on a folder 'ici et là' it is ok. Hoever
there are some random error related to #pathParts message sent to an
undefined object.
Next, in pharo.sh, the encoding is set to latin1, why ? I removed it
before the test.
Hilaire
2009/2/6 Michael Rueger <m.ru...@acm.org>:
--
http://blog.ofset.org/hilaire
Michael Rueger
> Does not work here.
>
> My system is LANG=fr_FR.UTF-8
>
> My keyboard keying is us_intl, (accented character are composed), this
> set up is used to work with standard pharo and ecoding to latin1
Are you using ubuntu? I have a hard time setting up the keyboard for
compositing.
> When I open the FileBrowser on a folder 'ici et là' it is ok. Hoever
> there are some random error related to #pathParts message sent to an
> undefined object.
If you could send me some error logs that would be great.
> Next, in pharo.sh, the encoding is set to latin1, why ? I removed it
> before the test.
Forgot to take it out, I invoked the VM and image "manually"...
Michael Rueger
>> It is the pharo one click with a modified (slightly outdated) image and
>> an updated Unix VM.
>
> The one-click thing does not work on my Mac, nothing happens. I can
> open the image with another VM though.
>
> Is this a special VM that comes with the one-click image? I might miss
> the context.
I simply took the one click I had downloaded, so I wonder if it ever
worked on Mac?
According to the system log it always tries to launch "squeak", even
though I changed the entry to point to the newer VM from the Squeak.org
download.
> Pressing any key-combination with the apple-key raises an error
> "MessageNotUnderstood UndefinedObject>>#<".
It seems that slot six is nil when any modifier is pressed? John?
> Pasting most of the texts from this website
> <http://www.columbia.edu/kermit/utf8.html> raises an error: "Invalid
> utf8 input detected".
Hmm, is the clipboard encoding UTF-16 on Mac? Thought it was UTF-8...
Michael
Michael Rueger
> On linux pasting ä or é from that same webpage gives
> "Error: Invalid utf8 input detected"
That's probably due to the left-over encoding setting in Pharo.sh. The
encoding line needs to be removed.
> How do I input those characters from the keyboard?
You need to switch to a different keyboard layout.
Compositing should also work, but I currently don't know how to activate
that either ;-)
Michael
John M McIntosh
(a) by default the macintosh squeak VM is shipped setup for macintosh
encoding of file names.
I know other vertical app suppliers have set that to UTF8. However I'm
wondering if Pharo is
UTF8 aware or uses the macintosh subset thus not supporting all valid
file names?
http://code.google.com/p/pharo/issues/detail?id=513
(b) The macintosh VM always supplies slot six with unicode data, the
unix vm, check with your support person?
(c) The clipboard logic in the macintosh VM is using the 1984
clipboard logic.
We would need to bring in the extended clipboard support from Sophie
to enable
nice things like UTF-16 and ability to move pictures on the clipboard.
http://code.google.com/p/pharo/issues/detail?id=514
--
=
=
=
========================================================================
John M. McIntosh <joh...@smalltalkconsulting.com>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================
Michael Rueger
> Well there are a number of questions here
>
> (a) by default the macintosh squeak VM is shipped setup for macintosh
> encoding of file names.
> I know other vertical app suppliers have set that to UTF8. However I'm
> wondering if Pharo is
> UTF8 aware or uses the macintosh subset thus not supporting all valid
> file names?
> http://code.google.com/p/pharo/issues/detail?id=513
UTF-8, at least with the unicode changes I'm currently working on.
Where is this set?
> (b) The macintosh VM always supplies slot six with unicode data, the
> unix vm, check with your support person?
It's the Mac VM from Squeak.org.
If you put a transcript show of the event buffer into the handleKeyboard
logic you can see that slot six is nil on meta keys.
>
> (c) The clipboard logic in the macintosh VM is using the 1984
> clipboard logic.
> We would need to bring in the extended clipboard support from Sophie
> to enable
> nice things like UTF-16 and ability to move pictures on the clipboard.
> http://code.google.com/p/pharo/issues/detail?id=514
Oh, I thought the new Mac VMs supported Unicode in the clipboard logic
as the other VMs now do.
How much effort would that be to add UTF-x support to the current
clipboard logic?
Michael Rueger
> Hilaire Fernandes wrote:
>> Does not work here.
>>
>> My system is LANG=fr_FR.UTF-8
>>
>> My keyboard keying is us_intl, (accented character are composed), this
>> set up is used to work with standard pharo and ecoding to latin1
>
I simply get 'e
> Are you using ubuntu? I have a hard time setting up the keyboard for
> compositing.
ok, figured it out, but the VM seems to ignore it or the compositing
sequence gets lost somewhere else...
Michael Rueger
> what is not very convenient is that copy/paste (as well as the rest of
> keyboard shortcuts ) is not working when keyboard layout not english.
You mean you are typing the equivalent of alt-c etc on the Russian
keyboard/layout? Or how do the ASCII short cuts map?
Didn't someone create a mapping that would work?
Michael
John M McIntosh
On 9-Feb-09, at 8:52 AM, Michael Rueger wrote:
> John M McIntosh wrote:
>> Well there are a number of questions here
>> (a) by default the macintosh squeak VM is shipped setup for
>> macintosh encoding of file names.
>> I know other vertical app suppliers have set that to UTF8. However
>> I'm wondering if Pharo is
>> UTF8 aware or uses the macintosh subset thus not supporting all
>> valid file names?
>> http://code.google.com/p/pharo/issues/detail?id=513
>
> UTF-8, at least with the unicode changes I'm currently working on.
>
> Where is this set?
http://smalltalkconsulting.com/html/squeakinfoplist.html
SqueakEncodingType
>
>
>> (b) The macintosh VM always supplies slot six with unicode data,
>> the unix vm, check with your support person?
>
> It's the Mac VM from Squeak.org.
>
> If you put a transcript show of the event buffer into the
> handleKeyboard logic you can see that slot six is nil on meta keys.
meta-keys not sure what you are refering to? How about an example?
>
>> (c) The clipboard logic in the macintosh VM is using the 1984
>> clipboard logic.
>> We would need to bring in the extended clipboard support from
>> Sophie to enable
>> nice things like UTF-16 and ability to move pictures on the
>> clipboard.
>> http://code.google.com/p/pharo/issues/detail?id=514
>
> Oh, I thought the new Mac VMs supported Unicode in the clipboard
> logic as the other VMs now do.
> How much effort would that be to add UTF-x support to the current
> clipboard logic?
Well that *is* the extended clipboard support. Just need the smalltalk
code to make the proper primitive calls to the extended clipboard
plugin.
You need the ClipboardExtendedPlugin.bundle
>
>
> Michael
--
=
=
=
========================================================================
John M. McIntosh <joh...@smalltalkconsulting.com>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================
Igor Stasenko
> Igor Stasenko wrote:
>
>> what is not very convenient is that copy/paste (as well as the rest of
>> keyboard shortcuts ) is not working when keyboard layout not english.
>
> You mean you are typing the equivalent of alt-c etc on the Russian
> keyboard/layout? Or how do the ASCII short cuts map?
> Didn't someone create a mapping that would work?
>
I think that shortcuts should be bound to meta-keys (or hardware key
scan code), so then regardless of keyboard layout, pressing alt-c
should be understood as a 'copy' action.
I'm not sure if VM passing enough information in keyboard event to
image side to use keyboard scan codes, instead of converted to
ASCII/Unicode character values.
Of course one could always create a mapping for own keyboard layout..
But this leads to situation that it will require to provide mappings
for all existing languages in the world :)
> Michael
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-...@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Michael Rueger
> 2009/2/9 Michael Rueger <m.ru...@acm.org>:
>> Igor Stasenko wrote:
> I think that shortcuts should be bound to meta-keys (or hardware key
> scan code), so then regardless of keyboard layout, pressing alt-c
> should be understood as a 'copy' action.
That's actually what the key even logic (at least on Windows) does.
But, looking beyond Squeak, how are things like common shortcuts (cut,
copy, paste, print, quit etc) handled for all the different keyboard
layouts?
I remember early Smalltalk code using the scan code only, so ctrl-"
translated to ctrl-u-Umlaut on German keyboards, not a really
intuitive thing for a user ;-)
michael
Igor Stasenko
> On Mon, Feb 9, 2009 at 10:56 PM, Igor Stasenko <sigu...@gmail.com> wrote:
>> 2009/2/9 Michael Rueger <m.ru...@acm.org>:
>>> Igor Stasenko wrote:
>
>
>> I think that shortcuts should be bound to meta-keys (or hardware key
>> scan code), so then regardless of keyboard layout, pressing alt-c
>> should be understood as a 'copy' action.
>
> That's actually what the key even logic (at least on Windows) does.
>
> But, looking beyond Squeak, how are things like common shortcuts (cut,
> copy, paste, print, quit etc) handled for all the different keyboard
> layouts?
>
- in FF3 i pressing same keys on keyboard for copy/paste/find
regardless of current layout.
- notepad too
- wordpad too
i think its a good practice, saving you from extra typing compare when
you typing text in non-english lang and need to copy part of it:
switch-copy-paste-swith back vs just copy-paste
and most of people in Ukraine using even 3 keyboard layouts:
- english
- russian
- ukrainian
so, imagine how painful it could be when you need to copy text, but
shortcuts working only in english layout..
you need to press languag switch keys 1 or 2 times (depending on
settings), and look at a language indicator to check that now you in
english. Very distracting and very inconvenient.
> I remember early Smalltalk code using the scan code only, so ctrl-"
> translated to ctrl-u-Umlaut on German keyboards, not a really
> intuitive thing for a user ;-)
>
keyboards having different layouts (i mean physical) , and placing
shortcut on keys which may differ on different keyboards is not good
practice.
But i think, for any keyboard which allows entering latin letters,
using latin alphabet keys for shortcuts is safe.
P.S. it seems that VM does not transfers key scan codes to image side :(
> michael
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-...@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Michael Rueger
>> Where is this set?
>
> http://smalltalkconsulting.com/html/squeakinfoplist.html
>
> SqueakEncodingType
Ah, thanks!
>> If you put a transcript show of the event buffer into the
>> handleKeyboard logic you can see that slot six is nil on meta keys.
>
> meta-keys not sure what you are refering to? How about an example?
alt/ctrl
so if you hit alt-c (apple-c) slot six is nil
>> Oh, I thought the new Mac VMs supported Unicode in the clipboard logic
>> as the other VMs now do.
>> How much effort would that be to add UTF-x support to the current
>> clipboard logic?
>
> Well that *is* the extended clipboard support. Just need the smalltalk
> code to make the proper primitive calls to the extended clipboard plugin.
> You need the ClipboardExtendedPlugin.bundle
OK, will do for now.
Guess in future versions we should remove the clipboard support from the
core VM and replace it by the (extended) clipboard plugin.
Michael
Michael Rueger
>> Well that *is* the extended clipboard support. Just need the smalltalk
>> code to make the proper primitive calls to the extended clipboard plugin.
>> You need the ClipboardExtendedPlugin.bundle
Hmm, you still need FFI despite the plugin:
readUTF8StringClipboardData
| bytes |
"utf8"
bytes := self readClipboardData: 'public.utf8-plain-text'.
bytes := MacCFString callMakeUTF8NormalizedStringViaByteString: bytes
asString.
^bytes ifNil: [bytes] ifNotNil:
[bytes asString convertFromWithConverter: (UTF8TextConverter new)]
John M McIntosh
On 11-Feb-09, at 6:39 AM, Michael Rueger wrote:
>
> alt/ctrl
> so if you hit alt-c (apple-c) slot six is nil
Well you have 3 different events that occur on tapping that key(s)
This follows the rules set by the windows VM.
on the down stroke, we give you EventTypeKeyboard->EventKeyDown, the
macintosh keycode (some virtual magic number), and zero for the
unicode value
After the operating system determines what that plastic or rubber key
was,
we send EventTypeKeyboard-> EventKeyChar, the macintosh macRoman
character (if possible, may be zero), and the unicode value
Once the key is released,
we send EventTypeKeyboard> EventKeyUp , the macintosh keycode (some
virtual magic number), and zero for the unicode value
If the key is held down it repeats, and according to windows rules it
doesn't send the keyup (I believe check the archives), just key down,
key char
I had offered to return the unicode value for keydown/keyup but was
told it was too hard for the windows VM to provide that data at the
point where
key down/up occurs
http://lists.squeakfoundation.org/pipermail/squeak-dev/2003-December/071701.html
As you know some flavours of the Squeak UI look for character input on
keyChar or keyUp...
So is this on key up/down or key char?
>
>
>>> Oh, I thought the new Mac VMs supported Unicode in the clipboard
>>> logic as the other VMs now do.
>>> How much effort would that be to add UTF-x support to the current
>>> clipboard logic?
>> Well that *is* the extended clipboard support. Just need the
>> smalltalk code to make the proper primitive calls to the extended
>> clipboard plugin.
>> You need the ClipboardExtendedPlugin.bundle
>
> OK, will do for now.
> Guess in future versions we should remove the clipboard support from
> the core VM and replace it by the (extended) clipboard plugin.
>
> Michael
>
--
=
=
=
========================================================================
John M. McIntosh <joh...@smalltalkconsulting.com>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================
John M McIntosh
be translated into bytes according to
"Normalizes the string into the specified form as described in Unicode
Technical Report #15."
and ensure we return the bytes as UTF8.
The FFI calls let the macintosh operating system deal with the messy
details.
http://unicode.org/unicode/standard/reports/tr15/tr15-18.html
also see
http://developer.apple.com/qa/qa2001/qa1235.html
Say is that a precomposed and decomposed Unicode you have there,
really what does squeak want? want does the operating system want.
Choices choices.
PS beware the utf8 to macroman or macroman to utf8 text converts in
squeak, I don't think the one that came from etoys does the right thing
can't remember, it's named wrong, does utf8 to ascii or something...
On 11-Feb-09, at 7:29 AM, Michael Rueger wrote:
> Michael Rueger wrote:
>
>>> Well that *is* the extended clipboard support. Just need the
>>> smalltalk code to make the proper primitive calls to the extended
>>> clipboard plugin.
>>> You need the ClipboardExtendedPlugin.bundle
>
> Hmm, you still need FFI despite the plugin:
>
> readUTF8StringClipboardData
> | bytes |
> "utf8"
>
> bytes := self readClipboardData: 'public.utf8-plain-text'.
> bytes := MacCFString callMakeUTF8NormalizedStringViaByteString:
> bytes asString.
> ^bytes ifNil: [bytes] ifNotNil:
> [bytes asString convertFromWithConverter: (UTF8TextConverter new)]
>
>
>
> Michael
--
=
=
=
========================================================================
John M. McIntosh <joh...@smalltalkconsulting.com>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================
Michael Rueger
> No, it's just more smalltalk coding, the utf8-plain-text bytes need to
> be translated into bytes according to
Somewhere in the call chain it is finally FFI calls as you illustrate below.
How much would it take to have the primitive return the normalized UTF-8
string directly?
Michael
John M McIntosh
On 12-Feb-09, at 5:12 AM, Michael Rueger wrote:
> John M McIntosh wrote:
>> No, it's just more smalltalk coding, the utf8-plain-text bytes need
>> to be translated into bytes according to
>
> Somewhere in the call chain it is finally FFI calls as you
> illustrate below.
>
> How much would it take to have the primitive return the normalized
> UTF-8 string directly?
>
> Michael
Er, case of beer?
Seriously tho you need to define if the UTF8 you expect squeak to work
with is precomposed and decomposed Unicode.
Then we can alter the plugin, also having a test case would be
helpful, move data into squeak, move data out of squeak.
Well and enabling the ability to copy/paste PNG or JPEG would be
helpful, again the plugin can help with this, but you'll
need a frame work (ala sophie) to front the existing logic, that and
the windows VM support.
--
=
=
=
========================================================================
Michael Rueger
<joh...@smalltalkconsulting.com> wrote:
>> How much would it take to have the primitive return the normalized UTF-8
>> string directly?
> Er, case of beer?
What kind? :-)
Duckstein?
> Seriously tho you need to define if the UTF8 you expect squeak to work with
> is precomposed and decomposed Unicode.
I would say precomposed, basically what is returned by the extended
clipboard logic.
> need a frame work (ala sophie) to front the existing logic, that and the
> windows VM support.
Windows already returns UTF-8 in the default Squeak clipboard, as does Linux.
michael
Michael Rueger
> what is not very convenient is that copy/paste (as well as the rest of
> keyboard shortcuts ) is not working when keyboard layout not english.
If you could be so kind to tell me what comes back in slot 3 of the
input event on your Russian keyboard if you simply type the equivalent
of 'xcv' (without alt or ctrl)?
I'm interested in what comes back on the keyDown and the keystroke event.
Thanks
Michael
Igor Stasenko
> Igor Stasenko wrote:
>
>> what is not very convenient is that copy/paste (as well as the rest of
>> keyboard shortcuts ) is not working when keyboard layout not english.
>
> If you could be so kind to tell me what comes back in slot 3 of the
> input event on your Russian keyboard if you simply type the equivalent
> of 'xcv' (without alt or ctrl)?
>
> I'm interested in what comes back on the keyDown and the keystroke event.
>
I tried something like this
[ | event |
event := Sensor peekEvent.
event ifNotNil: [
((event first = 2) and: [event fourth = 1] ) ifTrue: [
Transcript show: (event third printStringRadix: 16); cr.
] ] ] repeat
but it not shows anything :(
Have a snippet which i can run to show key codes?
> Thanks
>
> Michael
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-...@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Adrian Lienhard
HTH,
Adrian
Igor Stasenko
> Sensor kbdTest
>
"There is no implementors of kbdTest"
:(
What image might contain this method?
Adrian Lienhard
Adrian
Igor Stasenko
Here what prints #kbdTest
c - '99 0'
z - '122 0'
x - '120 0'
in russian layout
я (z key) - '79 0'
ч (x key) - '71 0'
с (c key) - '65 0'
Now i wrote own test method:
testKbd
"This test routine will print the unmodified character, its keycode,
and the OR of all its modifier bits, until the character x is typed"
| char event |
char := nil.
[true] whileTrue: [
event := self peekEvent.
(event notNil and: [ (event at: 1) = EventTypeKeyboard and: [
(event at: 4) = EventKeyDown] ]) ifTrue: [
(String streamContents: [:s |
event do: [:each | s nextPutAll: each printString; space].
s nextPutAll: ' ']) displayAt: 10@10
].
(Delay forMilliseconds: 10) wait.
]
----
And it works if you do:
[Sensor testKbd] fork
And! Wonderfull!
Regardless of active keyboard layout (en/ru)
it always prints same events for these keys, as following:
z - 2 <timestamp> 90 1 0 90 0 1
x - 2 <timestamp> 88 1 0 88 0 1
c - 2 <timestamp> 67 1 0 67 0 1
so, if we use these events for shortcuts, everything will work similar
regardless of keys layout.
2009/2/24 Adrian Lienhard <a...@netstyle.ch>:
Michael Rueger
> Regardless of active keyboard layout (en/ru)
> it always prints same events for these keys, as following:
>
> z - 2 <timestamp> 90 1 0 90 0 1
> x - 2 <timestamp> 88 1 0 88 0 1
> c - 2 <timestamp> 67 1 0 67 0 1
>
> so, if we use these events for shortcuts, everything will work similar
> regardless of keys layout.
Great, thanks!
For which shortcuts does that usually work?
cut x
copy c
paste v
undo z
open o
print p
all a
save s
anything else?
Igor Stasenko
> Igor Stasenko wrote:
>
>> Regardless of active keyboard layout (en/ru)
>> it always prints same events for these keys, as following:
>>
>> z - 2 <timestamp> 90 1 0 90 0 1
>> x - 2 <timestamp> 88 1 0 88 0 1
>> c - 2 <timestamp> 67 1 0 67 0 1
>>
>> so, if we use these events for shortcuts, everything will work similar
>> regardless of keys layout.
>
> Great, thanks!
>
> For which shortcuts does that usually work?
> cut x
> copy c
> paste v
> undo z
> open o
> print p
> all a
> save s
>
> anything else?
>
do-it, inspect-it
> Michael
>
> _______________________________________________
> Pharo-project mailing list
> Pharo-...@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________