Using nltk.ne_chunk DOESN'T WORK

[Vardaan Bhat]

Hello,

The nltk.ne_chunk function does not work.

I have tried it on several sentences.

I get the following error whenever I run the function in idle 3.4.1:

Traceback (most recent call last):

  File "<pyshell#4>", line 1, in <module>

    nltk.ne_chunk(sent)

  File "C:\Python34\lib\site-packages\nltk\chunk\__init__.py", line 176, in ne_chunk

    chunker = load(chunker_pickle)

  File "C:\Python34\lib\site-packages\nltk\data.py", line 779, in load

    resource_val = pickle.load(opened_resource)

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8d in position 1: ordinal not in range(128)

Can anyone please help me?

This is urgent.

Sorry but thanks

Vardaan

[Nigel Legg]

Can you show the sentences you are using?  Looks like there is a problem with encoding of the text.

Cheers, Nigel
07914 740972

--
You received this message because you are subscribed to the Google Groups "nltk-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nltk-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vardaan Bhat]

Oh, thanks.

I used the sentence:

“Jon likes potato pie in New York.”

Thank you for the prompt reply

-- 

Vardaan Bhat

Sent with Sparrow

[Vardaan Bhat]

I used the sentence: "Jon likes potato pie in New York."

Thanks for the prompt reply.

[Nigel Legg]

Can you post your code as well? Have you run a PoS tagger on this?

Cheers, Nigel
07914 740972

[Vardaan Bhat]

 sent="Jon likes potato pie in New York."

print(nltk.ne_chunk(sent))

On Mon, Nov 10, 2014 at 11:41 PM, Vardaan Bhat <varda...@gmail.com> wrote:

No. I have not. Sorry, I skipped some parts of the book. I did not realize I had to run a PoS tagger on this.

All I did was define a sentence and then run the chunk function.

Let me show you the code

-- 

Vardaan Bhat

Sent with Sparrow

[Nigel Legg]

nltk.ne_chunk() takes PoS tagged tokens, so you need to preprocess the text. 

Please read all of the book, and work through it methodically, that way you will get fewer errors.

Cheers, Nigel
07914 740972

[Vardaan Bhat]

Sorry. I thought this was a separate part aside from the previous chapters.

Thanks so much. 

By the way, just to clarify, this method does NER, right?
That's what the book said, but I wanted to clarify.

One last question.

My project is taking text, summarizing it using a simple python algorithm I found online, and then removing names and dates at random from the key sentences in the summary, and creating a 'cloze' test.

Would the chunk method suffice for the NER required?

Thanks again

[Vardaan Bhat]

And as clarification, my project is due in early Dec., but I have the cloze test code and summarization code, I'm just working on NER. Is completion of the project feasible given my time constraints, the amount I have finished, given I am going to continue reading the book thoroughly?

Thanks again

[Nigel Legg]

It's not a method I've ever used, you'll have to try it out and see.

Cheers, Nigel
07914 740972

[Vardaan Bhat]

Does the method sound logical though?

My project doesn't need to be flawless, just a rough model/prototype.

[Nigel Legg]

Not if you skip parts of the book and don't learn how to use nltk.

Cheers, Nigel
07914 740972

[Vardaan Bhat]

But if I do learn to?

by learn how to use nltk do you mean practicing the exercises from the book?

-- 

Vardaan Bhat

Sent with Sparrow

[Nigel Legg]

I don't know.

Cheers, Nigel
07914 740972

[Vardaan Bhat]

but about learning to use NLTK, that’s what you mean, right?

-- 

Vardaan Bhat

Sent with Sparrow