Using Ngrams

[Dina Bavli]

Hi,
Great package and GUI- Thank you!
Excellent documentation and tutorials! 
I'm new to stylo and R (I worked with python until now).

I'm requested to use stylo and R to analyze historical journals in Hebrew. 

When I try to analyze using 2 or 4 ngrams, I receive this message:
"Error in make.ngrams(sample, ngram.size = ngram.size) : 

  something wrong with your sample: unable to make 2-grams out of 0 element(s)"

Does this function need minimum words to work? 
How can I make this function work?

Thank you so much,
Have a wonderful day,
Dina Bavli. 

[Maciej Eder]

Hi Dina,

yes, this message sounds familiar. In a vast majority of cases, it is caused by codepage issues. To be more precise: whenever you go beyond basic Latin alphabet (e.g. Hebrew, Greek, but also Czech, French or German for that matter), it really matters what operating system you use, what is the encoding of your input texts, and what is the language option chosen on Stylo's gui. Assuming that your texts are compliant with the Unicode (UTF-8), please simply check the option "Other" on Stylo's gui (rather than "English" or "English All"). If this does not work, then try to repeat you procedure with 1-grams, and then open the resulting file "wordlist.txt": you should be able to recognize Hebrew words. It it doesn't happen, try to check your input texts if they are UTF-8-encoded. I hope this helps!

Best wishes,

Maciej

[Dina Bavli]

Hi  Maciej,
For 1- grams, it works perfectly. The error message is just for 2 grams and up.
Great weekend,
Dina