Jira (PUP-6875) [Spike] Investigate usage of ReGex \w character class in Puppet

[Ethan Brown (JIRA)]

Ethan Brown created an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet Issue Type: Bug Assignee: Unassigned Created: 2016/11/03 2:20 PM Priority: Normal Reporter: Ethan Brown The \w character class in Ruby does not support Unicode and will only match on [a-zA-z0-9_] which is limiting. Instead, we should generally be using [[:word:]] where Unicode is important within a regex. This ticket involves auditing the existing code to identify critically important areas where this may be a problem in types / providers, gems, etc. Note that the Unicode compliant word character class will be slower than the ASCII version, so it would be good to keep an eye on performance as changes are proposed / made This could also extend to supported modules, but should probably be a new ticket. Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols

The {{\w}} character class in Ruby does not support Unicode and will only match on {{[a-zA-z0-9_]}} which is limiting. Instead, we should generally be using {{[[:word:]]}} where Unicode is important within a regex.  This ticket involves auditing the existing code to identify critically important areas where this may be a problem in types / providers, gems, etc.  Note that the Unicode compliant word character class will be slower than the ASCII version, so it would be good to keep an eye on performance as changes are proposed / made This could also extend to supported modules, but should probably be a new ticket.

h5. In scope for Agent + Platform Team - Identify areas of concern, file tickets, assess impact and priority. - Analysis of gems should limit scope (using {{--without development}} flag to bundler).

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP 2016- 11 12 - 30 28

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP  Grooming  2016-11-30

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Story Points: 3

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6875

Re: [Spike] Investigate usage of ReGex \w character class in Puppet Please note that simply changing \w to :word: is not doable for elements of the puppet language since names are case independent and for many characters in the :word: class there is no way to do up/down case without also having a Locale. We cannot make the language be locale dependant. To work well we would need to make the language either be case dependant, or use Ruby's wonky "ascii is up/down-cased, but not others"-mode.

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875

[Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP  2016  2017 - 12 01 - 28 11

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP 2017-01- 11 25

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP 2017- 01 02 - 25 08

Add Comment

[Ethan Brown (JIRA)]

Ethan Brown commented on PUP-6875

Re: [Spike] Investigate usage of ReGex \w character class in Puppet Henrik Lindberg thanks for the note - when you say "elements of the puppet language", can you be a bit more specific on the scope there? Do you mean keywords / identifiers? We would definitely have you review anything in the parsing / lexing / etc should changes eventually be proposed there (which I don't believe we were anticipating). This ticket was filed because of a few errors encountered / fixed for end users - and is meant to identify any more similar issues proactively.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-6875

Re: [Spike] Investigate usage of ReGex \w character class in Puppet

Ethan Brown I used that vague term "elements of the language" to describe that there are various regular expressions in the lexer that match various parts of the language - it is not as simple as one regexp == identifier; maybe "the regular expressions used by the lexer to identify the tokens of the language then recognized as syntactical elements of the language by the parser" is more accurate. Several of those tokens are for "elements of the language" (tokens put together in the parser to have a specific meaning). Or put differently: Since we cannot do up/down case of arbitrary unicode as that requires both a LOCALE and a gem to compute (Ruby on its own cannot do this) and the gems for this are written in Ruby and have severe performance implications, we should not alter the regular expressions in the lexer or parser. It is simply a can of worms where it is best to keep the lid on.

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875

[Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP 2017-02- 08 22

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP 2017- 02 03 - 22 08

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: AP  2017-03-08  Ready for Engineering

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Sprint: Agent Ready for Engineering  1  0

Add Comment

[Geoff Nichols (JIRA)]

Geoff Nichols updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Geoff Nichols Fix Version/s: PUP 5.y

Add Comment

[John Duarte (JIRA)]

John Duarte updated an issue

Puppet / PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

Change By: John Duarte Labels: triaged

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-6875

Re: [Spike] Investigate usage of ReGex \w character class in Puppet One note, the unicode version is a much larger set of characters and has caused performance problems with tags. So we have to be careful about which regexps we change.

Add Comment

This message was sent by Atlassian JIRA (v7.5.1#75006-sha1:7df2574)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-6875

[Spike] Investigate usage of ReGex \w character class in Puppet

Change By: Josh Cooper Team: Coremunity Night's Watch

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)