New build released

[Tim Jurka]

Hi team,

The new build was rolled out on ( install.rtexttools.com ). I am also disabling the old repository so as to not create any confusion with version numbers.

Changelog:

1. Re-wrote dtm_to_sparsem() function to run faster on larger matrices, and eliminate several bugs that were causing a "invalid class "matrix.csr" object: ia has wrong number of elements" error.
2. Corrected code in create_corpus that was causing the error in ( 1 ).
3. Fixed the topic code truncation bug for boosting, bagging, random forests, glmnet, decision trees, and neural nets.
4. Added Wouter's wizard functions- train/classify_models, wizard_read_data, and wizard_train_test.

Errors:

1. cross_validate() is giving 0 accuracy all the time for maximum entropy. I'll have to look into what's causing this over the next few days.

Unfortunately, I did not get the time to create a demo file for this version. I plan to release that tomorrow ( 6/9 ), so perhaps we should hold off on sending this to the others until that's done. Is there a dataset we can use for demonstration purposes- Amber, perhaps your truncated NYT dataset?

Best,
Tim

[Wouter van Atteveldt]

Hey Tim, all,

Great news, looking forward to playing with the new release, probably this
weekend.

> 1. cross_validate() is giving 0 accuracy all the time for maximum entropy.
> I'll have to look into what's causing this over the next few days.

Is this the "old" cross_validate or my partial code? Because I can have a look
at it again and see what it going wrong?

> Unfortunately, I did not get the time to create a demo file for this
> version. I plan to release that tomorrow ( 6/9 ), so perhaps we should
> hold off on sending this to the others until that's done. Is there a
> dataset we can use for demonstration purposes- Amber, perhaps your
> truncated NYT dataset?

Also, I would like to update the documentation to reflect the new wizards, that
should make it much easier to start. However, I cannot do that until this
weekend. Shall we aim at release on Monday?

-- Wouter

[Tim Jurka]

No, this is the "old" cross_validate. I'm not sure I got any "new" code on your end, perhaps I missed it? I still haven't isolated the source but I'll look into it tomorrow... all 7 work except maxent. Weird, huh?

I'll try to polish the documentation tomorrow because I think Amber was targeting a Friday release at latest. Most of us will be leaving by 6/20, so I want to give at least 10 days for the testers to give us some comments about the software.

Tim

[Loren Collingwood]

Hi, I was testing this out today and ran into a few issues.

1. "train_models" models = train_models(corpus, algorithms=c("SVM","MAXENT")) did not work

2. > maxent_model <- train_model(corpus,MAXENT=TRUE) #LC NOTE: train_maxent_sparse not there

Error in train_model(corpus, MAXENT = TRUE) : 

  could not find function "train_maxent_sparse"

Evidently, some of the functions did not make it into the build. Joy!

-Loren

--
Loren Collingwood
loren.co...@gmail.com

[Tim Jurka]

I'm unable to reproduce the problem with a clean install in R 2.12.2. Is it possible an old maxent installation is getting mixed in? Did you try removing RTextTools and maxent (using remove.packages()) and reinstalling?

Tim

[Tim Jurka]

I also tried a clean install in 2.13 from the repository and was unable to replicate it. The maxent implementation in the repository has the train_maxent_sparse function (double check for me... http://install.rtexttools.com/src/contrib/ )

Tim

[Loren Collingwood]

Yes. That is what happened. I had removed them, but somehow the new versions must not have loaded (probably because I didn't reboot R). What a rookie!

-Loren

On Jun 13, 2011, at 11:05 PM, Tim Jurka wrote:

Loren Collingwood
Ph.D. Candidate
Department of Political Science
University of Washington
http://staff.washington.edu/lorenc2
lor...@uw.edu

[Tim Jurka]

This happened to me so many damn times.