LSA on word co-occurrence matrix

[Emre]

Hi there,

David very thank you, I constructed my word co-occurrence matrix with you help. My thought was to transfer this matrix to another environment(R) and make SVD on it. But I got some problems. 

So I think that maybe I can do LSA operation with S-space package too. I know that LSA normally works on word-document matrises but I want to use it on a word co-occurrence matrix.

As i said, I have a huge word co-occurrence matrix(approximately 50000x50000) and I want to do SVD operation on it. After this operation I will take cosine differences of words. I have an English synonmy test(TOEFL).I hope that after these operations I will get a good score on this test.

David are these operations possible in s-space package? If yes, how can I do them?

Thanks... 

[David Jurgens]

Hi Emre,

 

David very thank you, I constructed my word co-occurrence matrix with you help. My thought was to transfer this matrix to another environment(R) and make SVD on it. But I got some problems. 

It should be possible to export the data to a format recognizable by R, so if that's the issue, please let me know.

 

So I think that maybe I can do LSA operation with S-space package too. I know that LSA normally works on word-document matrises but I want to use it on a word co-occurrence matrix.

As i said, I have a huge word co-occurrence matrix(approximately 50000x50000) and I want to do SVD operation on it. After this operation I will take cosine differences of words. I have an English synonmy test(TOEFL).I hope that after these operations I will get a good score on this test.

David are these operations possible in s-space package? If yes, how can I do them?

Yes, this should be possible.  It's not implemented already but it should only take a bit of extra code.  You have a few options.  You could either extend the GenericWordSpace class with your own subclass and then override the processSpace method so that when called, it will perform the SVD.  

Alternatively, you could use the code from our earlier discussion and just call the SVD directly on the Matrix object you constructed. 

If you need more details on the implementation, please let me know.

  Thanks,

  David

 

Thanks... 

--

---
You received this message because you are subscribed to the Google Groups "S-Space Package Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to s-space-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.