ANNOUNCE: Beta-version of a new Lojban corpus search system

123 views
Skip to first unread message

Dan Rosén

unread,
Apr 5, 2014, 6:08:47 PM4/5/14
to loj...@googlegroups.com
ju'i jbopli

I have made a new Lojban corpus searching system.  The idea is to enhance the study of the usage and development of the language.  The search tool supports, apart from searching for a single word, also searching for selma'o, place structures of bridi, seltau, date, irc nick, and more.

Good old jbofi'e has attempted to parse all sentences, and its terbri information is extracted from successful parses. When the parse fails, cmafi'e is used for word segmentation and selma'o-tagging.

The system is kindly hosted by durka at:

https://www.alexburka.com/~danr

Here are some examples of what you can do with the extended search:
(Please be patient when clicking the links, it takes a little while to render the pages.)

Searching for usages of traji3:
https://www.alexburka.com/~danr/#?stats_reduce=word&cqp=%5Btags%20_%3D%20%22traji3%22%5D&search_tab=1&within=sentence&hpp=100&search=cqp

Please note that you can click on the words in the search results to get more information in the right-hand side sidebar.

Searching for self-greetings, COI + mi:
https://www.alexburka.com/~danr/#?cqp=%5Bpos%20%3D%20%22COI%22%5D%20%5Bword%20%3D%20%22mi%22%5D&stats_reduce=word&search_tab=1&within=sentence&search=cqp

Searching for irc messages authored by Robin:
https://www.alexburka.com/~danr/#?stats_reduce=word&cqp=%5B_.text_nick%20%3D%20%22rlpowell%22%20%26%20lbound(sentence)%5D&search_tab=1&within=sentence&search=cqp&page=855

Usages of pi'o as terminal rafsi (or zi'evla):
https://www.alexburka.com/~danr/#?stats_reduce=word&cqp=%5Bword%20%26%3D%20%22pi'o%22%20%26%20pos%20%3D%20%22BRIVLA%22%5D&search_tab=1&within=sentence&search=cqp

Examples of statistics:

Statistics of lo + BRIVLA:
https://www.alexburka.com/~danr/#?cqp=%5Bword%20%3D%20%22lo%22%5D%20%5Bpos%20%3D%20%22BRIVLA%22%5D&stats_reduce=word&search_tab=1&within=sentence&search=cqp&result_tab=1

The most common seltau (the info is set to "end with q"):
https://www.alexburka.com/~danr/#?cqp=%5Btrans%20%26%3D%20%22q%22%5D&search_tab=1&within=sentence&page=0&search=cqp&stats_reduce=word&result_tab=1

Popular selbri in gadri (info ends with n):
https://www.alexburka.com/~danr/#?cqp=%5Btrans%20%26%3D%20%22n%22%5D&search_tab=1&within=sentence&page=0&search=cqp&result_tab=1

There is also a comparison mode, which requires you to save to searches, by pressing the down arrow next to the search button. Then you can search for statistically significant differences between them. Try yourself by comparing used words without vs with only the irc corpus!

The system is an adaptation of the Swedish corpora search system Korp:
http://spraakbanken.gu.se/korp
If you are interested to help out, please do contact me.

Happy exploring!

mi'e .danr. ko banli mu'o

selpa'i

unread,
Apr 5, 2014, 6:19:45 PM4/5/14
to loj...@googlegroups.com
Dan, this is amazing and one of the greatest things I have ever seen
created in Lojbanistan. You have my utmost respect. Thank you very much
for making something this awesome for everyone of us to use.

I can't say much more right now, as I'm too awestruck to form coherent
speech that properly honors your accomplishment.

io sai sai .i'o sai sai
doi je'a banli

mi'e la selpa'i mu'o

Jacob Errington

unread,
Apr 5, 2014, 7:13:17 PM4/5/14
to loj...@googlegroups.com
.i je'u doi la .dan. do banli .i sei lakne lo do se finti cu balrai lo ro lojbo tutci

This will be incredibly useful to anyone wanting to do research on words and their frequencies, which will of course be of the utmost importance in the furthering of Lojban.

ki'e sai .dan.
.i mi'e la tsani mu'o


--
You received this message because you are subscribed to the Google Groups "lojban" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lojban+un...@googlegroups.com.
To post to this group, send email to loj...@googlegroups.com.
Visit this group at http://groups.google.com/group/lojban.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages