true unicode slugs (blogofile 0.8dev, python3)

[Wasil Sergejczyk]

_config.py:

from markupsafe import Markup
import re
_punct_re = re.compile(r'[\t !"#$%&\'()*\-/<=>?@\[\\\]^_`{|},.]+')
#from unidecode import unidecode # if you are OK with non-ASCII
symbols, comment this out, else install unidecode library
unidecode_func = lambda s: s # if you set this to unidecode, non-ASCII
symbols will be slugified into ASCII
# e.g. unidecode("北亰") == "Bei Jing "
str_func = unicode if sys.version_info < (3,) else str
def custom_create_slug(title, delim='-'):
# Get rid of any html entities
slug = Markup(title).unescape()

result = []
for word in _punct_re.split(slug):
result.extend(unidecode_func(word).split())
slug = str_func(delim.join(result)).lower()
return slug
blog.post.slugify = custom_create_slug

also, i had to modify _controllers/blog/post.py/create_permalink() so
it now uses config.slugify, if exists (it must be a bug).
works with both posts and categories.
btw, i didn't find any bugtracker on github, am i missing something?

[Wasil Sergejczyk]

ps i used this snippet: http://flask.pocoo.org/snippets/5/

[Mike Pirnat]

On Mon, May 28, 2012 at 9:38 AM, Wasil Sergejczyk <szelg...@gmail.com> wrote:
> btw, i didn't find any bugtracker on github, am i missing something?

Yes, github has a bug tracker:

https://github.com/blog/831-issues-2-0-the-next-generation

And if you have patches you'd like to submit, pull requests make it really easy:

http://help.github.com/send-pull-requests/

On top of that, sending a pull request automatically opens an issue in
the tracker. :-)


--
Mike Pirnat
mpi...@gmail.com
http://mike.pirnat.com/

[Doug Latornell]

On Mon, May 28, 2012 at 6:38 AM, Wasil Sergejczyk <szelg...@gmail.com> wrote:
> _config.py:
>
> from markupsafe import Markup
> import re
> _punct_re = re.compile(r'[\t !"#$%&\'()*\-/<=>?@\[\\\]^_`{|},.]+')
> #from unidecode import unidecode # if you are OK with non-ASCII
> symbols, comment this out, else install unidecode library
> unidecode_func = lambda s: s # if you set this to unidecode, non-ASCII
> symbols will be slugified into ASCII
> # e.g. unidecode("北亰") == "Bei Jing "
> str_func = unicode if sys.version_info < (3,) else str
> def custom_create_slug(title, delim='-'):
>    # Get rid of any html entities
>    slug = Markup(title).unescape()
>
>    result = []
>    for word in _punct_re.split(slug):
>        result.extend(unidecode_func(word).split())
>    slug = str_func(delim.join(result)).lower()
>    return slug
> blog.post.slugify = custom_create_slug

You're right that Unicode handling in slug creation is not what it
should be. I think this snippet could be worked into the create_slug()
function in blogofile_blog/site_src/_controllers/blog/post.py

I'd welcome a pull request for that.

> also, i had to modify _controllers/blog/post.py/create_permalink() so
> it now uses config.slugify, if exists (it must be a bug).
> works with both posts and categories.

I'm pretty sure that if you make the change in create_slug() it will
flow through to permalink pages and categories too.

[Wasil Sergejczyk]

> You're right that Unicode handling in slug creation is not what it
> should be. I think this snippet could be worked into the create_slug()
> function in blogofile_blog/site_src/_controllers/blog/post.py
>
> I'd welcome a pull request for that.

as soon as i learn git a little bit, which will be slowed down by RL-
stuff.

> > also, i had to modify _controllers/blog/post.py/create_permalink() so
> > it now uses config.slugify, if exists (it must be a bug).
> > works with both posts and categories.
>
> I'm pretty sure that if you make the change in create_slug() it will
> flow through to permalink pages and categories too.

of course, but blog.post.slugify setting doesn't work as intended (at
least, it doesn't affect path).

[Wasil Sergejczyk]

> > I'd welcome a pull request for that.
>
> as soon as i learn git a little bit, which will be slowed down by RL-
> stuff.

also, i'd like to add slugging for categories and avoiding doubling of
categories' names and posts' paths first.
in order to implement the latter feature i should keep track of Post
and Category instances. i'm planning to do it like this:
http://stackoverflow.com/questions/328851/python-printing-all-instances-of-a-class
(see "Use a mixin and weakrefs" section). is that OK?

[Wasil Sergejczyk]

ok, nevermind my pull request. i think, slugging would be better moved into blogofile core, because other plugins (there isn't any, but i can think of a couple and definitely will be making them after this) might want to use that too. so, will i redo this?

[Doug Latornell]

Don't be too hasty to abandon that pull request!

I'm not sure where you are thinking about putting the slugify code in
blogofile core, but don't plan on it going anywhere under site_init/.
If the work I'm doing now on refactoring the `blogofile init` command
goes as I hope site_init/ should disappear from blogofile core.
Anything that we want to keep from there should go into one or more
plugins.

Have a look at https://groups.google.com/d/topic/blogofile-discuss/GO6_q29zZpk/discussion
both the Google doc in Ryan's post that heads the thread, and the
discussion. That will give you some idea of the design that the
plugins branch is aiming for.

I can understand that if you see multiple uses for the slugify code
among plugins, you don't want to repeat it. I just don't know right
now what the right mechanism is to facilitate that.

[Wasil Sergejczyk]

i thought something like that: in plugin you can write, say,

from blogofile.tools import create_slug

but the exact place is yours to decide.

2012/5/31 Doug Latornell <d...@douglatornell.ca>

--
You received this message because you are subscribed to the Google Groups "blogofile-discuss" group.

To post to this group, send email to blogofil...@googlegroups.com.
To unsubscribe from this group, send email to blogofile-disc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/blogofile-discuss?hl=en.

--
Best regards.

[Mike Pirnat]

Was just about to advocate the same sort of thing, though we should take care that this "junk drawer" doesn't get too cluttered over time.

--

Mike Pirnat / mpi...@gmail.com

"I am a leaf on the wind; watch how I soar."

[Doug Latornell]

On Thu, May 31, 2012 at 10:32 AM, Wasil Sergejczyk <szelg...@gmail.com> wrote:
> i thought something like that: in plugin you can write, say,
> from blogofile.tools import create_slug
> but the exact place is yours to decide.

Yeah, okay, that works for me.

We already have a "junk drawer" (as Mike calls it) - blogofile.util.
So, I guess that's where create_slug() should go.

I'd prefer to call it blogfile.tools, but I don't think we need util
and tools. And I don't have a good sense yet for what kind of
backwards compatibility mess would result from moving the non-junk
from util into tools.

[Wasil Sergejczyk]

ok, i'm on it. will make a branch in a few days.

пятница, 1 июня 2012 г., 11:23:49 UTC+6 пользователь Doug Latornell написал:

>> To post to this group, send email to blogofile-discuss@googlegroups.com.

>> To unsubscribe from this group, send email to

>> blogofile-discuss+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/blogofile-discuss?hl=en.
>>
>
>
>
> --
> Best regards.
>
> --
> You received this message because you are subscribed to the Google Groups
> "blogofile-discuss" group.

> To post to this group, send email to blogofile-discuss@googlegroups.com.

> To unsubscribe from this group, send email to

> blogofile-discuss+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/blogofile-discuss?hl=en.

пятница, 1 июня 2012 г., 11:23:49 UTC+6 пользователь Doug Latornell написал:

>> To post to this group, send email to blogofile-discuss@googlegroups.com.

>> To unsubscribe from this group, send email to

>> blogofile-discuss+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/blogofile-discuss?hl=en.
>>
>
>
>
> --
> Best regards.
>
> --
> You received this message because you are subscribed to the Google Groups
> "blogofile-discuss" group.

> To post to this group, send email to blogofile-discuss@googlegroups.com.

> To unsubscribe from this group, send email to

> blogofile-discuss+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/blogofile-discuss?hl=en.

пятница, 1 июня 2012 г., 11:23:49 UTC+6 пользователь Doug Latornell написал:

>> To post to this group, send email to blogofile-discuss@googlegroups.com.

>> To unsubscribe from this group, send email to

>> blogofile-discuss+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/blogofile-discuss?hl=en.
>>
>
>
>
> --
> Best regards.
>
> --
> You received this message because you are subscribed to the Google Groups
> "blogofile-discuss" group.

> To post to this group, send email to blogofile-discuss@googlegroups.com.

> To unsubscribe from this group, send email to

> blogofile-discuss+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/blogofile-discuss?hl=en.

пятница, 1 июня 2012 г., 11:23:49 UTC+6 пользователь Doug Latornell написал:

>> To post to this group, send email to blogofile-discuss@googlegroups.com.

>> To unsubscribe from this group, send email to

>> blogofile-discuss+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/blogofile-discuss?hl=en.
>>
>
>
>
> --
> Best regards.
>
> --
> You received this message because you are subscribed to the Google Groups
> "blogofile-discuss" group.

> To post to this group, send email to blogofile-discuss@googlegroups.com.

> To unsubscribe from this group, send email to

> blogofile-discuss+unsubscribe@googlegroups.com.

[Wasil Sergejczyk]

i'm very sorry, but better late, than never. anyway, here are two pull requests from me on github.

[Doug Latornell]

No need to apologize, Wasil. There are lots of things in life that can
and should take precedence over writing code for blogofile!

Thanks for the pull requests. I'll take a look at them sometime this week.

On Sat, Jun 16, 2012 at 11:14 PM, Wasil Sergejczyk <szelg...@gmail.com> wrote:
> i'm very sorry, but better late, than never. anyway, here are two pull
> requests from me on github.
>

> --
> You received this message because you are subscribed to the Google Groups
> "blogofile-discuss" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/blogofile-discuss/-/yUZOqSPGCpUJ.
>
> To post to this group, send email to blogofil...@googlegroups.com.

> To unsubscribe from this group, send email to

> blogofile-disc...@googlegroups.com.