Pre-compiled patterns retain their settings if they are interpolated

The qr// operator precompiles a regular expression and returns a reference to that compiled pattern. You can use that reference with the binding operator to perform a match, interpolate the reference into the match or substitution operators, or combine the reference with other references or literal patterns to make a larger pattern. We cover this in Chapter 9, “Regular Expression References”, but don’t emphasize what happens with regex flags when you combine patterns.

The perlop documentation says (since v5.10 and up to at least v5.22 with adjustment for flags):

If a precompiled pattern is embedded in a larger pattern then the effect of “msixpluadn” will be propagated appropriately.

This sentence isn’t good because it doesn’t say what “appropriately” means or what “propogate” means. Does that mean that the flags of the larger pattern extend into the subpattern that is already compiled or that the flags set in the subpattern leak out into the larger pattern?

“Propogate” means that the effect will spread, but that’s not really true. It’s the wrong word to use. The compiled pattern has already decided how that portion works. Larger patterns that include it won’t re-compile the subpattern and will not impose new or different flags on the subpattern. And, the regular engine won’t look at the compiled pattern to set flags in the larger pattern. Instead, the behavior of the pre-compiled pattern is determined once and does not change even if it is interpolated.

Here’s a demonstration of that. I’ve constructed the several different patterns and combine them in several different ways. The only ones that work are those where the original subpattern sets the flags:

use v5.22;
use Test::More;

$_ = 'abcFREDdef';

my $p    = qr/Fred/;
my $p_i  = qr/Fred/i;     # Works
my $p_i2 = qr/(?i:Fred)/; # Works
my $p_i3 = qr/$p/i;       # does not change $p
my $p_i4 = qr/(?i:$p)/;   # does not change $p

ok( ! m/abc${p}def/,  "No /i anywhere did not match" );

note "--- /i in subpattern but not larger operator ---";
ok( m/$p_i/,  "/i from qr// matched"            );
ok( m/$p_i2/, "(?i:) from qr// matched"         );
ok( m/$p_i3/, "/i from qr/\$p/ matched"         );
ok( m/$p_i4/, "(?i:) from qr/(?i:\$p)/ matched" );

note "--- /i in pattern but not subpattern ---";
ok( m/$p/i,     "/i from m// matched"    );
ok( m/$p/i,     "/i from m// matched"    );
ok( m/(?i:$p)/, "(?i:) from m// matched" );
ok( m/$p_i3/,   "qr/\$p/i matched"       );
ok( m/$p_i4/,   "qr/(?i:\$p)/ matched"   );

note "--- /i in subpattern and leaking out ---";
ok( m/ABC${p_i}def/,  "/i from qr// matched"   );
ok( m/ABC${p_i2}def/, "(?i:) from m// matched" );

note "--- turning off /i in larger pattern ---";
ok( ! m/(?-i:$p_i)/,  "(?-i:) turned off /i from qr//"    );
ok( ! m/(?-i:$p_i2)/, "(?-i:) turned off (?i:) from qr//" );

done_testing();

Let’s go through some of these more closely. First, if the pre-compiled pattern isn’t case insensitive, it doesn’t matter
that the larger pattern is. This test fails because the portion of the pattern in $p didn’t have the /i flag when it was compiled:

my $p    = qr/Fred/;

ok( m/$p/i,             "/i from m// matched"    );

Going the other way, the larger pattern could not turn off flags already set in the pre-compiled patterns. You can turn off flags for an uncompiled subpattern with (?-flags:PATTERN). That unsetting doesn’t affect parts of the group that you’ve already compiled. In both of these the “Fred” portion of the pattern stays case-insensitive because that’s how that part was compiled:

my $p_i  = qr/Fred/i;     # Works
my $p_i2 = qr/(?i:Fred)/; # Works

ok( ! m/(?-i:$p_i)/,  "(?-i:) turned off /i from qr//"    );
ok( ! m/(?-i:$p_i2)/, "(?-i:) turned off (?i:) from qr//" );

All of those examples use case-insensitivity, but this applies to any of the pattern-affecting flags (see Know the difference between regex and match operator flags).

Here’s an example that uses /x, which allows you to use insignificant whitespace and Perl comments in a pattern. In the pre-compiled portion, there’s what looks like a comment but is a literal string to match. That the larger pattern that includes the pre-compiled subpattern uses the extended pattern doesn’t matter:

use v5.10;

   $_ = '# Fred';
my $p = qr/# Fred/;

say "Matched!" if /$p/x;

This part matches because the /x in the larger pattern cannot affect the pre-compiled $p:

Matched!

You can go the other way by setting /x in pre-compiled portion. Now, this is a bit of a trick but if you’ve taken our Learning Perl course you have already seen the trick:

use v5.10;

   $_ = '# Fred';
my $p = qr/# Fred/x;

say "Matched!" if /$p/;

This is a different pattern but it still matches!

Matched!

This final example doesn’t deal with propagation (or lack of propagation), but it’s still cute.

In the end, the simple thing to say is that once a pattern is compiled, including it in larger patterns does not change it or its settings. It’s better to say that it retains rather than propagates its behavior.

Leave a Reply

Your email address will not be published. Required fields are marked *