change subreddit names to another language

Showing 1-13 of 13 messages
change subreddit names to another language Benjamin Golding 10/11/10 2:10 PM
hey guys,
i'm trying to setup reddit in my language, i think a lot of people are
trying this also so maybe this thread will help them too. so for now
i've successfully translated all the phrases (including those that
aren't presented in admin tools) to my language.

The final step for me is to figure out a way to let users create a
subreddit name in my language and not only in english, or even only in
my language. this is major because without it i won't be able to
attract 99% of the surfers in my country.

i'll point out some facts that have to be cleared:
A user can only create a subreddit name in english right now.
This subreddit can contain a Title & Description in my language.
The subreddit title that was created is shown perfectly when entering
it.

So the title and description are contained in the DB and are presented
with no problems.
My question is why is it a problem changing the class:
VSubredditName(VRequired) in validator.py to something else like the
description class: VSubredditDesc(Validator) ?

why is it that after i change the class,  the end user gets a "problem
posting 500" when trying to create a subreddit name that is not ascii.
Is there a workout? please help
Re: [reddit-dev] change subreddit names to another language David King 10/11/10 2:17 PM
> i'm trying to setup reddit in my language, i think a lot of people are trying this also so maybe this thread will help them too. so for now i've successfully translated all the phrases (including those that aren't presented in admin tools) to my language.

You didn't have to do that. We have an i18n repo which, while not complete, is probably at least 50% translated for the majority of the languages it represents.

> My question is why is it a problem changing the class: VSubredditName(VRequired) in validator.py to something else like the description class: VSubredditDesc(Validator) ?

It might not be, but we don't really support reddits named by non-ASCII characters. It would probably require some testing of a lot of the URL-generation code if nothing else.

> why is it that after i change the class,  the end user gets a "problem posting 500" when trying to create a subreddit name that is not ascii. Is there a workout? please help

That depends on the stack-trace behind the 500 ("the user gets an error" is *never* enough information; please always accompany that by the error itself), but we haven't tested this case so you're likely to have introduced a number of bugs

Re: change subreddit names to another language Benjamin Golding 10/11/10 2:46 PM
Thanks David for the quick reply.

Here is the original code for the VSubredditName class:

class VSubredditName(VRequired):
    def __init__(self, item, *a, **kw):
        VRequired.__init__(self, item, errors.BAD_SR_NAME, *a, **kw)

    def run(self, name):
        name = chksrname(name)
        if not name:
            return self.error()
        else:
            try:
                a = Subreddit._by_name(name)
                return self.error(errors.SUBREDDIT_EXISTS)
            except NotFound:
                return name

Here is my version of the class:

class VSubredditName(Validator):
    def run(self, name):
        return name

Okay so don't mind for now that i didn't check chksrname & that there
is a subreddit already with the same name. when i try to create a
subreddit name in my language the end users gets "an error occurred
while posting (status: 500)" and the traceback is:

File '/usr/local/src/reddit/r2/r2/controllers/validator/validator.py',
line 166 in newfn
  simple_vals, param_vals, *a, **kw)
File '/usr/local/src/reddit/r2/r2/controllers/validator/validator.py',
line 204 in validatedForm
  val = self_method(self, form, responder, *a, **kw)
File '/usr/local/src/reddit/r2/r2/controllers/api.py', line 1228 in
POST_site_admin
  **kw)
File '/usr/local/src/reddit/r2/r2/models/subreddit.py', line 81 in
_new
  with g.make_lock('create_sr_' + name.lower()):
File '/usr/local/src/reddit/r2/r2/lib/lock.py', line 64 in __enter__
  while not self.cache.add(self.key, my_info, time = self.time):
File '/usr/local/src/reddit/r2/r2/lib/cache.py', line 171 in add
  return mc.add(key, val, time=time)
File '/usr/lib/python2.6/contextlib.py', line 34 in __exit__
  self.gen.throw(type, value, traceback)
File '/usr/local/lib/python2.6/dist-packages/pylibmc-1.0_reddit_04-
py2.6-linux-i686.egg/pylibmc.py', line 215 in reserve
  yield mc
File '/usr/local/src/reddit/r2/r2/lib/cache.py', line 171 in add
  return mc.add(key, val, time=time)
TypeError: argument 1 must be string, not unicode

so basically in the function  mc.add(key, val, time=time) the 'key'
value must be a string and not unicode. why dosen't this happen for
the description? and how do i workaround this? (in my early post i
wrote workout lol)
Re: [reddit-dev] Re: change subreddit names to another language David King 10/11/10 2:53 PM
> Okay so don't mind for now that i didn't check chksrname & that there is a subreddit already with the same name.

It also doesn't check to see that it doesn't contain a slash, or any number of other things that would break it. If you go through with implementing this you'll have to rethink that regex, not just remove it.

> File '/usr/local/src/reddit/r2/r2/lib/cache.py', line 171 in add
>  return mc.add(key, val, time=time)
> TypeError: argument 1 must be string, not unicode

Yes, that's one of a number of bugs that will be introduced in trying to support unicode sr names

> so basically in the function  mc.add(key, val, time=time) the 'key'
> value must be a string and not unicode. why dosen't this happen for
> the description? and how do i workaround this? (in my early post i
> wrote workout lol)

Because the sr name is used as a key in memcached (in Subreddit._by_name, if you're curious), and the description is not. Like I said, you can "work around" this by actually supporting unicode sr names, which is a lot of work. It really is a matter of writing and testing the code to implement it, not just changing some configuration parameters.

Re: change subreddit names to another language Benjamin Golding 10/11/10 3:25 PM
Thanks again David.

what would happen if i were to encode and decode the variable to a
8bit string using xml’s character references?
like for example do:

class VSubredditName(Validator):
    def run(self, name):
        name = name.encode('ascii', 'xmlcharrefreplace') //encoding
        return name

Re: [reddit-dev] Re: change subreddit names to another language David King 10/11/10 4:26 PM
> what would happen if i were to encode and decode the variable to a 8bit string using xml’s character references?

I don't know, but we don't use XML anywhere internally so I can't imagine it would help much. I'm trying to tell you that there's no simple fix here, implementing unicode sr names is going to be some real development work.

Re: change subreddit names to another language Benjamin Golding 10/12/10 6:13 AM
David can you help me figure how to solve this step:

> File '/usr/local/src/reddit/r2/r2/lib/cache.py', line 171 in add
>  return mc.add(key, val, time=time)
> TypeError: argument 1 must be string, not unicode

maybe i'll get the hang of it and be successful altering all the code
Re: [reddit-dev] Re: change subreddit names to another language David King 10/12/10 8:47 AM
> David can you help me figure how to solve this step:
>> File '/usr/local/src/reddit/r2/r2/lib/cache.py', line 171 in add
>> return mc.add(key, val, time=time)
>> TypeError: argument 1 must be string, not unicode

That's not enough stack trace to solve any problem, but even if it were I honestly don't have time to walk you through programming, you just going to have to do some debugging. But it looks like there's unicode where there should be a string.

> maybe i'll get the hang of it and be successful altering all the code

If you do, please post your patch, it may be something that we can all make use of

Re: change subreddit names to another language miopa 10/12/10 11:18 AM
I did all this almost a year ago, but at the time I was too busy to
extract only the language related changes from the code (and then
forgot about it). Now I got Benjamin request and decided to dig in a
bit to identify the changes related to this issue and add them here
for reference to others who might need it. Have in mind that the
changes are on a year old reddit source.

First, Python should be set to work by default with unicode. I did
this by adding

sys.setdefaultencoding('utf8')

to sitecustomize.py of mu app.

Next, unicode names should be decoded to utf-8 strings

grep -rin 'name.decode("utf-8")' *

r2/models/subreddit.py:87:        q = cls._query(lower(cls.c.name) ==
name.decode("utf-8").lower(),
r2/models/subreddit.py:97:        name = name.decode("utf-8").lower()
r2/models/link.py:228:            res = "/r/%s/%s" %
(sr.name.decode("utf-8"), p)
r2/models/account.py:184:        uid =
cls._by_name_cache(name.decode("utf-8").lower(), allow_deleted,
_update = _update)
r2/lib/subreddit_search.py:17:        name =
sr.name.decode("utf-8").lower()
r2/lib/subreddit_search.py:39:        name =
sr.name.decode("utf-8").lower()


I also know that I modified this in filters.py, can't recall why

def _force_unicode(text):
    try:
        text = unicode(text, 'utf-8')
    except UnicodeDecodeError:
        text = unicode(text, 'cp1251') # from latin1 to cyrillic code
page
    except TypeError:
        text = unicode(text)
    return text


Please try this, I'll keep digging if there are further issues.



On Oct 20 2009, 2:16 am, miopa <mio...@gmail.com> wrote:
> I can't publish it right now, the site is not yet officially launched
> and the code is still in development/debug (and I have to much todo).
>
> I'll post it here as soon as It's done, should be in a week or two.
>
> On Oct 19, 6:10 pm, Jeremy Edberg <jedb...@gmail.com> wrote:
>
> > Could you send out the link to your source code so we can all take a look?
>
> > Thanks!
>
> > j
>
> > On Mon, Oct 19, 2009 at 07:06, miopa <mio...@gmail.com> wrote:
>
> > > I'm running a cyrillic reddit on http://r.ping.mk
>
> > > There are few tweaks to the code that must be made for this to work
> > > properly.
>
> > > Unfortunately I've made quite a lot changes to the code so I can't
> > > extract the language related that easy right now. If you're interested
> > > I'll have more time next week and I'll look it up.
>
> > > On Oct 16, 1:11 pm, Victor <vicr...@gmail.com> wrote:
> > >> When I try to create a Reddit in another language, I get the error
> > >> message "that name isn't going to work"
>
> > >> I assume it's because the Reddit name goes into the URL?
>
> > >> Would there be a way to have different languages for the Reddit name?
Re: change subreddit names to another language Benjamin Golding 10/12/10 1:53 PM
OMG

it actually works!!! i can't believe it's working!
You see it's because of people like miopa that open source projects
rocks.
A guy from Macedonia make changes to the open source and enables us to
configure our code to work in dutch, arabic etc.

There are more changes you must do in subreddit.py because the code
was changed a few times in revision updates, so the code is in
different places in that file and
has new names like "lnames".

I'm still checking my app to see if everything is working right.
after that i'll post exactly where to change the code in each file so
these
instruction will apply to the current reddit code.

miopa you didn't have to do all this work but you choose to help other
people
here, not only today but in the future. So this is a HUGE THANKS!
Re: change subreddit names to another language miopa 10/12/10 4:13 PM
You're welcome, glad that I helped.
Re: change subreddit names to another language lvv...@gmail.com 12/4/12 1:04 AM
do you know how to change subreddit names to another language?like chinese...
thanks hope your reply...

在 2010年10月13日星期三UTC+8上午4时53分17秒,Benjamin Golding写道:
Re: change subreddit names to another language Ahmet Novalić 1/19/13 8:42 AM
Hey, 

I know this was a long time ago, but do you maybe know can the same process be applied to the current reddit version?


On Wednesday, October 13, 2010 1:13:00 AM UTC+2, miopa wrote:
You're welcome, glad that I helped.