Ruby M17N meeting summary

[Martin Duerst]

This is a rough translation of the Japanese meeting summary
(see [ruby-dev:33825]) by NARUSE, Yui.

There was a Ruby M17N (multilingualization) meeting on 2008/02/15.

(attending: Matz, ko1, ark, nobu, naruse and duerst)

== Looking for a successor for cgi.rb
Conditions are as follows:
* different name than cgi.rb
* using MVC separation

== String#gsub(regexp, hash)
Proposal to allow the following syntax:
String#gsub(regexp, {"ä"=>"\u00C4", ..})
-> accepted

== Information regarding "replica" information
-> write this down

== Difference between CP949 and GBK
-> They differ, separate

== Concatenating empty strings and with UTF-16
Proposal: empty strings can be concatenated with any string, even if
not ASCII Compatible
-> accepted

== Slowness of String#length, String#at,...
Speed up using search_nonascii
Speed up additionally for VALID UTF-8

== String#getbyte, String#setbyte
-> accepted

== Indexer
-> on hold

== about inspect
Consider adding obj.inspect_acumulate(enc)

== Direction for strftime
strftime is locale-independent
If and when a locale-dependent version becomes necessary,
it will be added separately.

== Introducing require_relative
Proposal to add require_relative, require'ing a file relative to the
file where it is used.
-> accepted

== IO.copy_stream
Proposal to introduce IO.copy_stream(filename, filename) and IO.copy_stream(stream, stream) to copy a file or a stream.
However, because there is a system call when using a filename,
Ruby can do something differently.
-> on hold, think about names

== Not sure about Hash#compare_by_identity
Would like to change to Hash#compared_by(:equal?, :obj_id)
-> accepted
-> Hash#identifed_by may be better (see [ruby-dev:33817])

== Unicode Normalization
Proposals for API:
* String#encode("utf-8 nfc")
* String#normalize("nfc")
* Encoding::UTF_8.nfc(str)
Because one might want to use Unicode Normalization e.g. when
reading a file, we also need need to consider what to do for example
for open(fn, "r:utf-8", ...).

#-#-# Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-# http://www.sw.it.aoyama.ac.jp mailto:due...@it.aoyama.ac.jp

[James Gray]

On Feb 18, 2008, at 4:33 AM, Martin Duerst wrote:

> This is a rough translation of the Japanese meeting summary
> (see [ruby-dev:33825]) by NARUSE, Yui.

Thank you for providing this.

> == Looking for a successor for cgi.rb
> Conditions are as follows:
> * different name than cgi.rb
> * using MVC separation

Was there any discussing of what's desired from the new version,
beyond the new architecture?

Are there encoding issues with cgi.rb? (I ask because of the meeting
it was raised at.)

> == String#gsub(regexp, hash)
> Proposal to allow the following syntax:
> String#gsub(regexp, {"ä"=>"\u00C4", ..})
> -> accepted

Can you explain how this works?

Does the Hash provide sequences of translations that are applied
before the match? Are they reversed after the match?

> == Introducing require_relative
> Proposal to add require_relative, require'ing a file relative to the
> file where it is used.
> -> accepted

This sounds nice.

James Edward Gray II

[ts]

>>>>> "J" == James Gray <ja...@grayproductions.net> writes:

J> Can you explain how this works?

vgs% ./ruby -e 'p "abcdabcd".gsub(/./, "a" => 1, "b" => 2, "c" => 3)'
"123123"
vgs%

when ruby find a match it look if it has a key with this value.

Guy Decoux

[James Gray]

I see. So this translates matches. Neat.

Thanks for the clarification, Guy.

James Edward Gray II

[Daniel DeLorme]

Martin Duerst wrote:
> This is a rough translation of the Japanese meeting summary
> (see [ruby-dev:33825]) by NARUSE, Yui.

Thank you very much, these are interesting news to hear about.

> == String#gsub(regexp, hash)
> Proposal to allow the following syntax:
> String#gsub(regexp, {"&auml;"=>"\u00C4", ..})
> -> accepted

This is very nice; I've often had to use a pattern roughly like
string.gsub(/#{hash.keys.join("|")}/){ h[$&] }
But shouldn't the syntax be more like this?
String#gsub({"&auml;"=>"\u00C4", ..})
I can't really see a use case for keeping the regex separate from the
hash... what happens in a case like this?
"abc".gsub(/a/,{})
"a" gets deleted?

> == String#getbyte, String#setbyte
> -> accepted

Great! Will this method take/return integers or 1-byte-long strings?

> == Introducing require_relative
> Proposal to add require_relative, require'ing a file relative to the
> file where it is used.
> -> accepted

This is great news too.

--
Daniel

[Richard Kilmer]

On Feb 18, 2008, at 5:33 AM, Martin Duerst wrote:

> == Introducing require_relative
> Proposal to add require_relative, require'ing a file relative to the
> file where it is used.
> -> accepted

Will this include a load_relative to keep things balanced?

-rich

[Gonzalo Garramuño]

Martin Duerst wrote:
> == Introducing require_relative
> Proposal to add require_relative, require'ing a file relative to the
> file where it is used.
> -> accepted
>

Is this a good idea at all? Often times, libraries need to be modified
due to bugs or enhanced features. By being able to change LOAD_PATH,
this is trivial. Any use of require_relative in a library module will
screw that up completely and prevent files from being overridden.

--
Gonzalo Garramuño
gga...@advancedsl.com.ar

AMD4400 - ASUS48N-E
GeForce7300GT
Xubuntu Gutsy

[NARUSE, Yui]

Hi,

James Gray wrote:
>> == Looking for a successor for cgi.rb
>> Conditions are as follows:
>> * different name than cgi.rb
>> * using MVC separation
>
> Was there any discussing of what's desired from the new version, beyond
> the new architecture?

Only two requirement were discussed. We are still considering what
architecture is good for Ruby. Advices about that is welcome, for
exmaple we need this feature or that feature is good for web programers
and so on.

> Are there encoding issues with cgi.rb? (I ask because of the meeting it
> was raised at.)

I didn't check but, if locale is set other than C, encoding differ
LOCALE and ASCII-8BIT may happen. If locale is C and process strings
with ASCII-8BTI encoding, I don't know any critical issues yet. Anyway
the fact that cgi.rb doesn't think about encoding must be a critical
issue XD

>> == String#gsub(regexp, hash)
>> Proposal to allow the following syntax:
>> String#gsub(regexp, {"&auml;"=>"\u00C4", ..})
>> -> accepted
>
> Can you explain how this works?
>
> Does the Hash provide sequences of translations that are applied before
> the match? Are they reversed after the match?

Longer exmaple is following.

s = "&lt;foo&gt;a&auml; &amp; &infin;&lt;/foo&gt;"
TABLE_HTMLREF = {
'&lt;' => '<',
'&gt;' => '>',
'&amp;' => '&',
'&auml;' => "\u00C4",
'&infin;' => "\u221E",
}
s.gsub(/&\w+;/, TABLE_HTMLREF)

This code equals following.
s.gsub(/&\w+;/){|k| TABLE_HTMLREF[k]}

But new gsub with hash is faster than this.

--
NARUSE, Yui <nar...@airemix.com>
DBDB A476 FDBD 9450 02CD 0EFC BCE3 C388 472E C1EA

[NARUSE, Yui]

Hi,

Daniel DeLorme wrote:
>> == String#gsub(regexp, hash)
>> Proposal to allow the following syntax:
>> String#gsub(regexp, {"&auml;"=>"\u00C4", ..})
>> -> accepted
>
> This is very nice; I've often had to use a pattern roughly like
> string.gsub(/#{hash.keys.join("|")}/){ h[$&] }
> But shouldn't the syntax be more like this?
> String#gsub({"&auml;"=>"\u00C4", ..})

/&auml;|&aring;|.../ may be slower than /&\w+;/. String#gsub seems
performance conscious.

> I can't really see a use case for keeping the regex separate from the
> hash... what happens in a case like this?
> "abc".gsub(/a/,{})
> "a" gets deleted?

You can use default value of hash.

s = "&lt;foo&gt;a&auml; &amp; &infin;&lt;/foo&gt;"

TABLE_HTMLREF = Hash.new{|hash, key| key}
TABLE_HTMLREF.update(

'&lt;' => '<',
'&gt;' => '>',
'&amp;' => '&',
'&auml;' => "\u00C4",
'&infin;' => "\u221E"

)
s.gsub(/&\w+;/, TABLE_HTMLREF)

>> == String#getbyte, String#setbyte
>> -> accepted
>
> Great! Will this method take/return integers or 1-byte-long strings?

Integer. This feature is already implemented, so you can try following.

s = "ABCDE"
s.getbyte(0) #=> 65
s.setbyte(0, 97)
s #=> "aBCDE"

[Tanaka Akira]

In article <CF75B3D1-7DF7-483C...@infoether.com>,
Richard Kilmer <ri...@infoether.com> writes:

> Will this include a load_relative to keep things balanced?

No. Only require_relative is discussed and accepted.
--
Tanaka Akira

[Tanaka Akira]

In article <47BA6B64...@advancedsl.com.ar>,
Gonzalo Garramuño <gga...@advancedsl.com.ar> writes:

> Is this a good idea at all? Often times, libraries need to be modified
> due to bugs or enhanced features. By being able to change LOAD_PATH,
> this is trivial. Any use of require_relative in a library module will
> screw that up completely and prevent files from being overridden.

I think require_relative should be used in a library or
test.

For example, I used require_relative in
test/ruby/test_file.rb and test/ruby/test_pipe.rb
to require test/ruby/ut_eof.rb.

I guess no one want to use modified ut_eof.rb by changing
load path.
--
Tanaka Akira

[Michal Suchanek]

On 19/02/2008, Gonzalo Garramuño <gga...@advancedsl.com.ar> wrote:
> Martin Duerst wrote:
> > == Introducing require_relative
> > Proposal to add require_relative, require'ing a file relative to the
> > file where it is used.
> > -> accepted
> >
>
> Is this a good idea at all? Often times, libraries need to be modified
> due to bugs or enhanced features. By being able to change LOAD_PATH,
> this is trivial. Any use of require_relative in a library module will
> screw that up completely and prevent files from being overridden.
>

You would have to copy (or symlink) the whole library then, not just a
single file.

However, not having a corresponding load feature makes this really unbalanced.

Also I would think of this as another tool for overriding single files
but I do not know how it would interact with normal require then.

Using this in a library that has multiple nested directories might
look convenient but it would turn the library into a monolith. You
cannot move any files around without unexpected consequences.

And again the semantics of mixing require_relative with require are
unclear. As they look up files differently it is no longer easy to
tell if certain file is supposed to be loaded on require or it is
already present.

Thanks

Michal

[Radosław Bułat]

My favorite is require_relative. Thanks fot that change!

> And again the semantics of mixing require_relative with require are
> unclear. As they look up files differently it is no longer easy to
> tell if certain file is supposed to be loaded on require or it is
> already present.

Ruby 1.9 expands all paths keeped in $" so if require_relative would
do the same (I think it should) there is no problem.

--
Radosław Bułat

http://radarek.jogger.pl - mój blog

[Ezra Zygmuntowicz]

On Feb 18, 2008, at 2:33 AM, Martin Duerst wrote:

> == Looking for a successor for cgi.rb
> Conditions are as follows:
> * different name than cgi.rb
> * using MVC separation

May I suggest we use a stripped down version of rack for the cgi.rb
replacement? http://rack.rubyforge.org/

I am using this as the basis for my merb web framework. I know that
rails is thinking of switching to use it also. Many of the other newer
ruby web platforms re using it as well. We plan to use rack as the
interface to rubyland in mod_rubinius.

I think this would make a very suitable replacement for cgi.rb. I
would be happy to volunteer to spearhead the work on getting this into
the stdlib.

Thoughts?

Thanks-

- Ezra Zygmuntowicz
-- Founder & Software Architect
-- ez...@engineyard.com
-- EngineYard.com

[Jeremy Kemper]

On Feb 20, 2008 11:35 AM, Ezra Zygmuntowicz <ezmo...@gmail.com> wrote:
> On Feb 18, 2008, at 2:33 AM, Martin Duerst wrote:
>
> > == Looking for a successor for cgi.rb
> > Conditions are as follows:
> > * different name than cgi.rb
> > * using MVC separation
>
>
> May I suggest we use a stripped down version of rack for the cgi.rb
> replacement? http://rack.rubyforge.org/
>
> I am using this as the basis for my merb web framework. I know that
> rails is thinking of switching to use it also. Many of the other newer
> ruby web platforms re using it as well. We plan to use rack as the
> interface to rubyland in mod_rubinius.
>
> I think this would make a very suitable replacement for cgi.rb. I
> would be happy to volunteer to spearhead the work on getting this into
> the stdlib.
>
> Thoughts?

Seconded, though I wish Rack would use a more Rubyish API than
big-hash-in/conventional-array-out.

Recent discussion on the rails-core list:
http://groups.google.com/group/rubyonrails-core/browse_thread/thread/02db664ff5cfed3a

Best,
jeremy

[Luis Lavena]

On Wed, Feb 20, 2008 at 5:35 PM, Ezra Zygmuntowicz <ezmo...@gmail.com> wrote:
>
> May I suggest we use a stripped down version of rack for the cgi.rb
> replacement? http://rack.rubyforge.org/
>

Even Ruby build isn't as "opinionated software" as is Rails, I second
Ezra suggestion.

CGI is old. Is doesn't feel right that we must depend on gems like
cgi_multipart_eof_fix until the fixes for CGI got merged... switching
to something modern and more update bridge between webserver and ruby
creation of "services" will be excellent.

> Thoughts?
>

I like the idea, even that my "thought" don't have commit rights :-)

Regards,

--
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams

[Michal Suchanek]

On 19/02/2008, Radosław Bułat <radek...@gmail.com> wrote:
> My favorite is require_relative. Thanks fot that change!
>
>
> > And again the semantics of mixing require_relative with require are
> > unclear. As they look up files differently it is no longer easy to
> > tell if certain file is supposed to be loaded on require or it is
> > already present.
>
>
> Ruby 1.9 expands all paths keeped in $" so if require_relative would
> do the same (I think it should) there is no problem.

So if I understand this correctly ruby stores the full path to the
file that is loaded and modifying $: could make the same require
command load a file with the same name from a different place again.

Such as

require 'cgi' # loads cgi.rb somewhere from ruby stdlib

$:.unshift "/my/directory"

require 'cgi' # loads /my/directory/cgi.rb - it is not loaded yet

This would also mean that require_relative would seldom mark the
standard file as loaded.

Thanks

Michal

[Trans]

On Feb 20, 4:37 pm, "Jeremy Kemper" <jer...@bitsweat.net> wrote:

> On Feb 20, 2008 11:35 AM, Ezra Zygmuntowicz <ezmob...@gmail.com> wrote:
>
>
>
> > On Feb 18, 2008, at 2:33 AM, Martin Duerst wrote:
>
> > > == Looking for a successor for cgi.rb
> > > Conditions are as follows:
> > > * different name than cgi.rb
> > > * using MVC separation
>
> > May I suggest we use a stripped down version of rack for the cgi.rb

> > replacement?http://rack.rubyforge.org/

>
> > I am using this as the basis for my merb web framework. I know that
> > rails is thinking of switching to use it also. Many of the other newer
> > ruby web platforms re using it as well. We plan to use rack as the
> > interface to rubyland in mod_rubinius.
>
> > I think this would make a very suitable replacement for cgi.rb. I
> > would be happy to volunteer to spearhead the work on getting this into
> > the stdlib.
>
> > Thoughts?
>
> Seconded, though I wish Rack would use a more Rubyish API than
> big-hash-in/conventional-array-out.

I think it's only a good idea if API is much improved. I've use Rack
too and it works well, but the API is rather painful.

T.

[Ezra Zygmuntowicz]

Agreed, I have some ideas for a better api that I am implementing in
merb and will post for you guys to review. The current rack interface
makes streaming responses a bit tough but I think I have a good
workaround that should make the api less painful.

Cheers-