Token (Word) Definition Settings

[ReedD]

In Global Settings - Token Definition Settings, what does each setting
do?

Letter Token Classes
[x] Letter
[ ] Uppercase [ ] Lowercase [ ] Modifier [ ] Other

Number Token Classes
[ ] Number
[ ] Decimal [ ] Letter [ ] Other

Punctuation Token Classes
[ ] Punctuation
[ ] Connector [ ] Dash [ ] Open [ ] Close [ ] Initial [ ] Final
[ ] Other

Symbol Token Classes
[ ] Symbol
[ ] Math [ ] Currency [ ] Modifier [ ] Other

[LAnthony]

Hi,
This is a great question. But the answer is not so easy. The classes
refer to Unicode character classes. Each one is quite large and spans
different languages. So, it's not so easy to just list up the
characters in each class. If you look on the Unicode homepage, you can
find out more information about them, but I just checked but could not
find a good page that listed each class separately. Does anybody else
on the list know a good link to Unicode character classes?

I'll try and find a good page and upload it later.
Regards,
Laurence.

[LAnthony]

I've finally found a nice page showing all the character classes that
AntConc uses.

Here is the link: http://www.fileformat.info/info/unicode/category/index.htm

If you click on one of the categories, you'll get a description and
below that a <List> mark. If you click on that, you'll be able to see
all the characters in each class, and even be able to see a visual
representation of each.

So, for example, if you were looking for all the letter (lowercase)
characters, you'd click on <[Ll] Letter, Lowercase> to go to the
link:
http://www.fileformat.info/info/unicode/category/Ll/index.htm

Then, you'd click on the <List> link to go to:
http://www.fileformat.info/info/unicode/category/Ll/list.htm

From here, you can see all the characters. You can also click on <View
all images> to see them all appear at one.

I hope this helps.
Laurence.