Configfile parser rewrite, try 2

[Thue Janus Kristensen]

Ok, configfile parser rewrite, take 2.

Uwe suggested that I use lexer states to support naked string. That was an excellent suggestion, and IMO the resulting configfile parser it is much prettier than the one in the current SVN.

Unlike the previous rewrite i posted, this one handles naked strings.

It should be functionally equivalent to the one in current SVN, except for
-only booleanizes naked strings, not a="true".
-The simpler code of the rewritten parser is easier to understand, less likely to have bugs.
-ints and floats are translated to PHP ints and floats. Is this OK?
-includes my quoted string fixes from the last rewrite. Quoted strings use php escape syntax for chars like ', ", \, \t and \n.

The rewrite passes the unit tests in SVN, except for the one which assumes that a=123.4 returns a string and not a float. But it could use more tests.

The rewrite needs the fix for lookahead in the lexer generator, which I posted earlier.

We might want to remove """-strings, since both ' and "-strings now handle multiline just fine.

The manual has the example
   bodyBgColor = #000000
So I made naked strings include leading #'s. But that is really quite ugly, and you could consider changing that so that # always denotes comment start (and changing the manual for smarty 3 accordingly...).

Some corner cases will have a result different from smarty 2. For example
  num_persons = 2 # Uwe and mohrt
This would (I think without checking) be interpreted by smarty 2 as num_persons="2 # Uwe and mohrt", while smarty 3 would see it as num_persons=2. But I think that is a reasonable break of backwards compatibility in a major version release.

Regards, Thue

[uwe.tews]

Thue

I think we should keep """ strings for backward compatibility.

By the way, where is your code?

I plan to update the SVN tomorrow.

Uwe

[Thue Janus Kristensen]

Now with code!

I was too focused on writing the email itself to remember to attach the patch...

Regards, Thue

--

You received this message because you are subscribed to the Google Groups "Smarty Developers" group.
To post to this group, send email to smarty-d...@googlegroups.com.
To unsubscribe from this group, send email to smarty-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/smarty-developers?hl=en.

[Thue Janus Kristensen]

I forgot to recompile the parser after the last change, so I accidentally sent a non-working patch. Sorry :(.

Here is a working patch, as well as the full files.

Btw, the documentation in LexerGenerator.php says
 * Lastly, if you wish to cycle to the next matching rule, return any value other than
 * true, false or null:
 *
 * <code>
 * /*!lex2php
 * "{@" ALPHA {
 *     if ($this->value == '{@internal') {
 *         return 'more';
 *     }
 *     ...
 * }

But I could not get that to work. Hence the extra new state NAKED_STRING_VALUE in the latest patch, when it would have been much more elegant to just return 'more' in the maybe_bool lexer rule.

Regards, Thue

[uwe.tews]

Hi Thue

Some remaining problems:

Naked strings do return an unwanted extra space at the end.

You should not use &$something at the parameter of function
definitions.

I got
Warning: Call-time pass-by-reference has been deprecated in C:\wamp\www
\_Smarty3Work\distribution\libs\sysplugins
\smarty_internal_configfileparser.php on line 170

Warning: Call-time pass-by-reference has been deprecated in C:\wamp\www
\_Smarty3Work\distribution\libs\sysplugins
\smarty_internal_configfileparser.php on line 179

when E_STRICT is enabled.

And sorry, I did disable the handling of

* <code>
* /*!lex2php
* "{@" ALPHA {
* if ($this->value == '{@internal') {
* return 'more';
* }
* ...
* }

It did create lots of code and very huge preg_match arrays even if you
don't use this functionallity.
Disabling it did reduce the size of the lexer.php files dramatically.

Uwe

> >> smarty-develop...@googlegroups.com<smarty-developers%2Bunsu...@googlegroups.com>

> >> .
> >> For more options, visit this group at
> >>http://groups.google.com/group/smarty-developers?hl=en.
>
>
>

>  smarty_rewrite_configfile_compiler.patch
> 17KViewDownload
>
>  smarty_internal_configfilelexer.plex
> 5KViewDownload
>
>  smarty_internal_configfileparser.y
> 6KViewDownload

[Thue Janus Kristensen]

On Sun, Dec 27, 2009 at 12:04 AM, uwe.tews <uwe....@googlemail.com> wrote:

Hi Thue

Some remaining problems:

Naked strings do return an unwanted extra space at the end.

The regexp for naked_string in smarty_internal_configfilelexer.plex should be changed from
  naked_string = /[^\n]+?(?=[ \t\r]*\n)/
To
  naked_string = /[^\n]+?(?=[ \t\r]*\n)/
Ie + should be +?

You should not use &$something at the parameter of function
definitions.

I got
Warning: Call-time pass-by-reference has been deprecated in C:\wamp\www
\_Smarty3Work\distribution\libs\sysplugins
\smarty_internal_configfileparser.php on line 170

Warning: Call-time pass-by-reference has been deprecated in C:\wamp\www
\_Smarty3Work\distribution\libs\sysplugins
\smarty_internal_configfileparser.php on line 179

when E_STRICT is enabled.

OK. The & should just be removed from the two calls to set_var in smarty_internal_configfileparser.y, since the & is also already in the function definition. Because I use my distributions slightly older PHP version, I did not get this strict error, even though i have strict errors on.
 

And sorry, I did disable the handling of

 * <code>
 * /*!lex2php
 * "{@" ALPHA {
 *     if ($this->value == '{@internal') {
 *         return 'more';
 *     }
 *     ...
 * }

It did create lots of code and very huge preg_match arrays even if you
don't use this functionallity.
Disabling it did reduce the size of the lexer.php files dramatically.

Uwe

Ok, fair enough.
Let us just keep it that way for my sake. I was mostly just curious.

I have attached a new version of the patch (or you can just make the suggested changes yourself)

Regards, Thue