Regex Group in Perl: how to capture elements into array from regex group that matches unknown number of/multiple/variable occurrences from a string?

Means that the parentheses content won't be captured as group (we don't need the spaces, only key and value) S matches the variable name. Then we skip any amount of spaces and an equal g quoted would result in EDIT: Since you really want to get the whole assignment, and not the single keys/values, here's a one-liner that extracts those: my @list = $string =~ /(?:^|\s+)((?:\S+)\s*=\s*(?:"^"*"|\S*))/g.

My $string = "var1=100 var2=90 var5=hello var3=\"a, b, c\" var7=test var3=hello"; while($string =~ /(?:^|\s+)(\S+)\s*=\s*("^"*"|\S*)/g) { print " => \n"; } Prints: => => => => => => Explanation: Last piece first: the g flag at the end means that you can apply the regex to the string multiple times. The second time it will continue matching where the last match ended in the string. Now for the regex: (?:^|\s+) matches either the beginning of the string or a group of one or more spaces.

This is needed so when the regex is applied next time, we will skip the spaces between the key/value pairs. The? : means that the parentheses content won't be captured as group (we don't need the spaces, only key and value).

\S+ matches the variable name. Then we skip any amount of spaces and an equal EDIT: Since you really want to get the whole assignment, and not the single keys/values, here's a one-liner that extracts those: my @list = $string =~ /(?:^|\s+)((?:\S+)\s*=\s*(?:"^"*"|\S*))/g.

The OP said one regex group was desired, and this captures into 2 regex groups... – drewk Aug 12 '10 at 1:19 Right, my fault. You can fix this be adding more parens around the key/value part of the regex. – jkramer Aug 12 '10 at 7:35 So you could do: ideone.Com/7EQgz :- my $string = "var1=100 var2=90 var5=hello var3=\"a, b, c\" var7=test var3=hello"; my @array = (); while($string =~ /(?:^|\s+)(\S+)\s*=\s*("^"*"|\S*)/g) { push( @array, $1.

"=". $2 ); my @array = (); } for ( my $i = 0; $i \n"; } – Rob Aug 12 '10 at 9:59 1 Updated the post with a one-liner that extracts the complete var=value assignments. – jkramer Aug 12 '10 at 10:19.

I'm not saying this is what you should do, but what you're trying to do is write a Grammar. Now your example is very simple for a Grammar, but Damian Conway's module Regexp::Grammars is really great at this. If you have to grow this at all, you'll find it will make your life much easier.

I use it quite a bit here - it is kind of perl6-ish. Use Regexp::Grammars; use Data::Dumper; use strict; use warnings; my $parser = qr{ + =(?:""|) var\d+ ** (,) \S+ }xms; qvar1=100 var2=90 var5=hello var3="a, b, c" var7=test var3=hello =~ $parser; die Dumper {%/}; Output: $VAR1 = { '' => 'var1=100 var2=90 var5=hello var3="a, b, c" var7=test var3=hello', 'pair' => { '' => 'var1=100', 'value' => '100', 'key' => 'var1' }, { '' => 'var2=90', 'value' => '90', 'key' => 'var2' }, { '' => 'var5=hello', 'value' => 'hello', 'key' => 'var5' }, { '' => 'var3="a, b, c"', 'key' => 'var3', 'list' => 'a', 'b', 'c' }, { '' => 'var7=test', 'value' => 'test', 'key' => 'var7' }, { '' => 'var3=hello', 'value' => 'hello', 'key' => 'var3' } .

2 +1 because I like the idea of the grammar concept (having studied them to an extent in Computer Science) though I haven't tried this answer. I like the grammar concept because this approach could be applied to solve even more complex problems, particularly in parsing code/data from a legacy obsolete language, for migration into a new language or data driven system/database -- which was actually the reason my original question (though I didn't mention it at the time. ) – Rob Aug 12 '10 at 10:22 1 I'd welcome you you to check out this module.

Too often Regexs blur into Grammar -- and if you're going to write a Grammar with a Regex (not a bad idea) then this module is really dead on. Check out my application of it to parse the COPY command in my psql shell. – Evan Carroll Aug 12 '10 at 15:01 +1 for your link, I'll check it out.

– Rob Aug 12 '10 at 23:00.

With regular expressions, use a technique that I like to call tack-and-stretch: anchor on features you know will be there (tack) and then grab what's between (stretch). In this case, you know that a single assignment matches \b\w+=. + and you have many of these repeated in $string.

Remember that \b means word boundary: A word boundary (\b) is a spot between two characters that has a \w on one side of it and a \W on the other side of it (in either order), counting the imaginary characters off the beginning and end of the string as matching a \W. The values in the assignments can be a little tricky to describe with a regular expression, but you also know that each value will terminate with whitespace—although not necessarily the first whitespace encountered! €”followed by either another assignment or end-of-string.To avoid repeating the assertion pattern, compile it once with qr// and reuse it in your pattern along with a look-ahead assertion (?=...) to stretch the match just far enough to capture the entire value while also preventing it from spilling into the next variable name.

Matching against your pattern in list context with m//g gives the following behavior: The /g modifier specifies global pattern matching—that is, matching as many times as possible within the string. How it behaves depends on the context.In list context, it returns a list of the substrings matched by any capturing parentheses in the regular expression. If there are no parentheses, it returns a list of all the matched strings, as if there were parentheses around the whole pattern.

The pattern $assignment uses non-greedy . +? To cut off the value as soon as the look-ahead sees another assignment or end-of-line.

Remember that the match returns the substrings from all capturing subpatterns, so the look-ahead's alternation uses non-capturing (?:...). The qr//, in contrast, contains implicit capturing parentheses. #!

/usr/bin/perl use warnings; use strict; my $string = +? /x; my @array = $string =~ /$assignment (?= \s+ (?: $ | $assignment))/gx; for ( my $i = 0; $i $array$i. "\n"; } Output: 0: var1=100 1: var2=90 2: var5=hello 3: var3="a, b, c" 4: var7=test 5: var3=hello.

Thanks for your contribution. Tried your solution, it works for me too -thanks! +1.

Also thanks for suggesting your systematic approach/technique to regex building: "tack-and-stretch: anchor on features you know will be there (tack) and then grab what's between (stretch). " I'll read your answer more deeply when I've more time and feedback later. – Rob Aug 12 '10 at 13:47 @Rob I'm glad it helps.

Enjoy! – Greg Bacon Aug 12 '10 at 15:06 +1 That is a really great explanation of how you approached this problem. – drewk Aug 13 '10 at 0:20.

! /usr/bin/perl use strict; use warnings; use Text::ParseWords; use YAML; my $string = "var1=100 var2=90 var5=hello var3=\"a, b, c\" var7=test var3=hello"; my @parts = shellwords $string; print Dump \@parts; @parts = map { { split /=/ } } @parts; print Dump \@parts.

1 I think this is better done with Text::ParseWords rather than Text::Shellwords. Text::ParseWords has similar functionality but is part of the Perl core. – drewk Aug 12 '10 at 1:32 1 @drewk Thanks for the reminder.

I always confuse the two. I'll update the example to use Text::ParseWords. – Sinan Ünür Aug 12 '10 at 1:58 Works fine for me.

See output further on in this comment. This depends on a module - I was lucky on my machine that this is present but for some Perl modules this is not always guaranteed on every distribution/platform. Here's the output: --- - var1=100 - var2=90 - var5=hello - 'var3=a, b, c' - var7=test - var3=hello --- - var1: 100 - var2: 90 - var5: hello - var3: 'a, b, c' - var7: test - var3: hello – Rob Aug 12 '10 at 10:36 1 @Rob: I think that Text::ParseWords has been part of the core distribution since 5.00.

The shellwords functionality is very useful and prior to 5.00 many used a shell eval to get that even with the security isk. Don't need to do that anymore since 5.00. – drewk Aug 12 '10 at 16:27 1 @Rob: Ask yourself which one is more maintainable: A complicated patter, a custom parser or a core module dependency.

– Sinan Ünür Aug 12 '10 at 16:43.

A bit over the top maybe, but an excuse for me to look into p3rl.org/Parse::RecDescent. How about making a parser? #!

/usr/bin/perl use strict; use warnings; use Parse::RecDescent; use Regexp::Common; my $grammar = $item{VALUE}\n"; } startrule: assignment(s) _EOGRAMMAR_ ; $Parse::RecDescent::skip = ''; my $parser = Parse::RecDescent->new($grammar); my $code = q{var1=100 var2=90 var5=hello var3="a, b, c" var7=test var8=" haha \" heh " var3=hello}; $parser->startrule($code); yields: var1 => 100 var2 => 90 var5 => hello var3 => "a, b, c" var7 => test var8 => " haha \" heh " var3 => hello PS. Note the double var3, if you want the latter assignment to overwrite the first one you can use a hash to store the values, and then use them later. PPS.My first thought was to split on '=' but that would fail if a string contained '=' and since regexps are almost always bad for parsing, well I ended up trying it out and it works.

Edit: Added support for escaped quotes inside quoted strings.

Thanks for your answer. I'll need to install Parse module on my particular system to try it out though. I would therefore favour a solution without this dependency.

– Rob Aug 12 '10 at 10:04.

This one will provide you also common escaping in double-quotes as for example var3="a, \"b, c". @a = /(\w+=(?:\w+|"(?:^\\"*(?:\\. ^\\"*)*)*"))/g; In action: echo 'var1=100 var2=90 var42="foo\"bar\\" var5=hello var3="a, b, c" var7=test var3=hello' | perl -nle '@a = /(\w+=(?:\w+|"(?:^\\"*(?:\\.

^\\"*)*)*"))/g; $,=","; print @a' var1=100,var2=90,var42="foo\"bar\\",var5=hello,var3="a, b, c",var7=test,var3=hello.

Yes that works. +1 – Rob Aug 12 '10 at 10:19.

You asked for a RegEx solution or other code. Here is a (mostly) non regex solution using only core modules. The only regex is \s+ to determine the delimiter; in this case one or more spaces.

Use strict; use warnings; use Text::ParseWords; my $string="var1=100 var2=90 var5=hello var3=\"a, b, c\" var7=test var3=hello"; my @array = quotewords('\s+', 0, $string); for ( my $i = 0; $i "\n"; } Or you can execute the code HERE The output is: 0: var1=100 1: var2=90 2: var5=hello 3: var3=a, b, c 4: var7=test 5: var3=hello If you really want a regex solution, Alan Moore's comment linking to his code on IDEone is the gas!

It is possible to do this with regexes, however it's fragile. My $string = "var1=100 var2=90 var5=hello var3=\"a, b, c\" var7=test var3=hello"; my $regexp = qr/( (?:\w+=\w\,+) | (?:\w+=\"^\"*\") )/x; my @matches = $string =~ /$regexp/g.

Might need to add something missing or correct something here, as I get an error message when I run it: ideone. Com/4bR1b and also on my own machine too. – Rob Aug 12 '10 at 10:24 Bareword found where operator expected at .

/regex_solution. Pl line 8, near "qr/( (?:\w+=\w\,+) | ( syntax error at . /regex_solution.

Pl line 8, near "qr/( (?:\w+=\w\,+) | (?:\w+=\"^\"*\") )/xg" Execution of . /regex_solution. Pl aborted due to compilation errors.

– Rob Aug 12 '10 at 10:24.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.