escaping special characters

[Kvetch]

Hello, what is the proper way to escape certain characters within the
yaml? I seem to be getting different results on different options and
with different special characters and have tried adding one by one but
I get stuck .
For instance if I want to use the following character set.
(a, b, c, 1, 2, 3, backtick, backslash, exclamation, semicolon, single
quote, comma, period, forwardslash, colon, space character and a
double quote)
abc123`\!;',./: "

For the characters option I believe I only need to escape the double quote so
characters: "abc123`\!;',./: \""
But for max_char_occurs it seems that the exclamation point, colon and
double quote do not escape right. I have tried escaping the just
certain characters but I still can't seem to get it working nicely,
max_char_occurs: abc123:20,`\\!;',./\: \":20

I cannot seem to get the colon and comma to be recognized in any
option and the double quote only works in certain options as does the
exclamation point.
The min and max_char_list_occurs options do not seem to need escaping
of the backslash sometimes.

I tried following the http://www.yaml.org/spec/current.html#id2517668
guide for escaping special characters which seems like just using
Unicode but even then I couldn't get it to recognize the ascii hex
value.

Anyone with any suggestions as to how to utilize all special characters?
Thanks,
Nick

[Teo]

ciao

I did some tests.

for chars like ;,! it is ok

ex:
characters: ;.!'

if you add "tab" (\t) you have to double quote all the string after :
ex:
characters: ";.!'\t"

full example

characters: ";.!'\t"
min_length: 1
max_length: 3
max_char_occurs: a:100
min_char_occurs: a:0
min_char_list_occurs: ".:1,!\t:1"
max_char_list_occurs: a:1000
max_run_iterations: 1000
prefix_string:
postfix_string:
max_consecutive_chars: 2
include_regex: "."
exclude_regex: "^$"


The problem occurs with the characters : and , I use as separator in
min/max options. see split(':') and split(',')
in the code of wg.rb

I have added an issue for that

http://code.google.com/p/ruby-words-generators/issues/detail?id=11


In the meanwhile you can change the code or propose a solution. I can
also give you write permission to svn repository if you want to solve
the issue.

thanks
matteo

On Jul 22, 9:23 pm, Kvetch <kve...@gmail.com> wrote:
> Hello, what is the proper way to escape certain characters within the
> yaml?  I seem to be getting different results on different options and
> with different special characters and have tried adding one by one but
> I get stuck .
> For instance if I want to use the following character set.
> (a, b, c, 1, 2, 3, backtick, backslash, exclamation, semicolon, single
> quote, comma, period, forwardslash, colon, space character and a
> double quote)
> abc123`\!;',./: "
>
> For the characters option I believe I only need to escape the double quote so
>     characters: "abc123`\!;',./: \""
> But for max_char_occurs it seems that the exclamation point, colon and
> double quote do not escape right.  I have tried escaping the just
> certain characters but I still can't seem to get it working nicely,
> max_char_occurs: abc123:20,`\\!;',./\: \":20
>
> I cannot seem to get the colon and comma to be recognized in any
> option and the double quote only works in certain options as does the
> exclamation point.
> The min and max_char_list_occurs options do not seem to need escaping
> of the backslash sometimes.
>

> I tried following thehttp://www.yaml.org/spec/current.html#id2517668

[Kvetch]

Thanks for looking into it Matteo. I will mess around with the code
and see if I can come up with anything and let you know.

Nick

[Kvetch]

Hi Matteo, I just went to look at the code and noticed you updated the
code with new splitter values and options. I also noticed you added
the dump_results_file too. Thank you so much.

I just tested it with the character set of
characters: "abcd:,_"
And since I wanted to include underscore in my characters I just set
the key_value to something I wasn't going to include such as the F8
key.
splitter_options: ",,"
splitter_key_value: "<F8>"
dump_results_file: /tmp/results.txt
max_run_iterations: 10000
wordlist:
characters: "abcd:,_"
min_length: 2
max_length: 4
max_char_occurs: "abcd:,_<F8>5"
min_char_occurs: "b<F8>1,,a<F8>1,,c<F8>1,,d<F8>1,,:<F8>1,,,<F8>1,,_<F8>2"
min_char_list_occurs: "abd:,<F8>1,,c<F8>0,,_<F8>0"
max_char_list_occurs: "abcd:,_<F8>5"

All seems good when running the init
$ ruby wg.rb settings.yaml init
I, [2009-07-27T11:14:40.147206 #14043] INFO -- : Logging to JMS
broker localhost:61613
I, [2009-07-27T11:14:40.150157 #14043] INFO -- : ..logged!
#<Stomp::Connection:0x7f246c17b230>
W, [2009-07-27T11:14:40.150254 #14043] WARN -- : dump_results_file:
/tmp/results.txt.blah.17034
W, [2009-07-27T11:14:40.150306 #14043] WARN -- : splitter_options: ,,
W, [2009-07-27T11:14:40.150354 #14043] WARN -- : splitter_key_value: <F8>
D, [2009-07-27T11:14:40.150418 #14043] DEBUG -- : characters: abcd:,_
D, [2009-07-27T11:14:40.150470 #14043] DEBUG -- : max_run_iterations: 10000
D, [2009-07-27T11:14:40.150534 #14043] DEBUG -- : include_regex: (?-mix:)
D, [2009-07-27T11:14:40.150583 #14043] DEBUG -- : exclude_regex: (?-mix:^$)
D, [2009-07-27T11:14:40.150723 #14043] DEBUG -- : min_char_occurs:
,1a1b1c1:1d1_2
D, [2009-07-27T11:14:40.150806 #14043] DEBUG -- : max_char_occurs:
,5a5b5c5:5d5_5
D, [2009-07-27T11:14:40.150880 #14043] DEBUG -- :
min_char_list_occurs: c0abd:,1_0
D, [2009-07-27T11:14:40.150945 #14043] DEBUG -- : max_char_list_occurs: abcd:,_5
I, [2009-07-27T11:14:40.150994 #14043] INFO -- : Adding empty string
as candidate

What does the "Adding empty string as candidate" mean? I would assume
it means that the space character is going to be used in the candidate
results but I do not see that happening? All of my results do not
include a space character.

Also I am still having trouble getting the dump_results to write out
the file. I assume with the addition of the dump_results_file I do
not need to pipe the output like before but I am not seeing this file
being created when I run
$ ruby wg.rb settings.yaml dump_results
and still isn't being created when I pipe the output out like so
$ ruby wg.rb settings.yaml dump_results >> /tmp/results.txt

Thanks again,
Nick

[Teo]

ciao

try using the latest wg.rb, the previos one had a bug. now the results
would be dumped in the file yo set in config file


about your question "Adding empty string as candidate means?"

the generated words can be a "candidate" string (taken from candidate
queue) and for sure they are generated from candidate words

if XXX is a candidate string, a new candidate string is generating
appending to it one of valid characters..