Character n-grams in NLTK

[Correa Denzil]

Hi,

Does NLTK have a provision to extract character n-grams from text? I would like to extract character n-grams (instead of traditional unigrams,bigrams) as features to aid my text classification task.

--Regards,
Denzil

[JAGANADH G]

On Thu, Jan 6, 2011 at 9:04 PM, Correa Denzil <mce...@gmail.com> wrote:

Hi,

Does NLTK have a provision to extract character n-grams from text? I would like to extract character n-grams (instead of traditional unigrams,bigrams) as features to aid my text classification task.

Writing a character n-gram package is straight forward and easy in Python. You can try that !!
--
**********************************
JAGANADH G
http://jaganadhg.freeflux.net/blog

[Steven Bird]

On 7 January 2011 06:13, JAGANADH G <jaga...@gmail.com> wrote:
>
> Writing a character n-gram package is straight forward and easy in Python.
> You can try that !!

True, though we've also provided it:

http://nltk.googlecode.com/svn/trunk/doc/api/nltk.util-module.html#ngrams

And if anyone is wondering whether NLTK supports certain
functionality, you might try consulting the documentation index here:

http://nltk.googlecode.com/svn/trunk/doc/api/identifier-index.html

-Steven

[JAGANADH G]

With nltk.util you can create character n-gram in this way

>>> t = "Does NLTK have a provision to extract character n-grams from text? I would like to extract character n-grams (instead of traditional unigrams,bigrams) as features to aid my text classification task."
>>> chrs = [c for c in t]
>>> from nltk.util import ngrams
>>> ngrams(chrs,3)
It will give you trigrams like

[('D', 'o', 'e'),
 ('o', 'e', 's'),
 ('e', 's', ' '),
 ('s', ' ', 'N'),
 (' ', 'N', 'L'),
 ('N', 'L', 'T'),
 ('L', 'T', 'K'),
 ('T', 'K', ' '),
 ('K', ' ', 'h'),
 (' ', 'h', 'a'),
 ('h', 'a', 'v'),
 ('a', 'v', 'e'),
 ('v', 'e', ' '),
 ('e', ' ', 'a'),
 (' ', 'a', ' '),
 ('a', ' ', 'p'),
...]