Nltk error

Kantajit Shaw

unread,

Jul 12, 2015, 8:35:13 PM7/12/15

to nltk-...@googlegroups.com

I am new to nltk. I was trying some basics.

import nltk
nltk.word_tokenize("Tokenize me")

gives me this following error

Traceback (most recent call last):
File "<pyshell#27>", line 1, in <module>
nltk.word_tokenize("hi im no onee")
File "C:\Python27\lib\site-packages\nltk\tokenize\__init__.py", line 101, in word_tokenize
return [token for sent in sent_tokenize(text, language)
File "C:\Python27\lib\site-packages\nltk\tokenize\__init__.py", line 85, in sent_tokenize
tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
File "C:\Python27\lib\site-packages\nltk\data.py", line 786, in load
resource_val = pickle.load(opened_resource)
AttributeError: 'module' object has no attribute 'defaultdict'

Please someone help. Please tell me how to fix this error.

Vardaan Bhat

unread,

Jul 12, 2015, 9:33:57 PM7/12/15

to nltk-...@googlegroups.com

You've got to POS tag first.

Regards,

Vardaan

--
You received this message because you are subscribed to the Google Groups "nltk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nltk-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Vardaan Bhat

"There is no such thing as a little freedom. Either you are all free, or you are not free.”
– Walter Cronkite

Alessandro Gagliardi

unread,

Jul 13, 2015, 4:06:43 PM7/13/15

to nltk-...@googlegroups.com

That's an odd error. Contrary to Vardaan, I don't think it has anything to do with POS tagging. (POS tagging usually comes after tokenization.) It looks like it can't find defaultdict which is weird because defaultdict is part of collections which is part of the standard library.

Looks like you're using Windows. Any Windows users out there seen a problem like this?

Alessandro D. Gagliardi

Lead Professor

alessandro@galvanize.com

44 Tehama Street | San Francisco, CA 94105

Twitter: @MadDataScience

www.galvanize.com | www.galvanizeU.com

On Thu, Jul 9, 2015 at 7:13 AM, Kantajit Shaw <kantaj...@gmail.com> wrote:

--

Alexis Dimitriadis

unread,

Jul 14, 2015, 4:12:36 AM7/14/15

to nltk-...@googlegroups.com

You've got to POS tag first.

No you don't.

AttributeError: 'module' object has no attribute 'defaultdict'

This is rather a shot in the dark, but the easiest way to get this error during unpickling is if your script includes a module called collections.py. Could that be the case? (If so, rename it!)

Alexis

Reply all

Reply to author

Forward