<> is magic :) 
07:14
But I don't think that SO answer was wrong (except in the one way I had said before). Instead, I believe the problem is that, even though the Perl interpreter acts on
use directives at compile time, I had failed to realize that it would still apply only to invocations of open that appear in code that is scoped to it (i.e., in its lexical scope, and probably below it). I am still not 100% sure that's what's going wrong, but I think it is.
Perl scripts compile to bytecode before being run, as in many other interpreted languages (e.g., Python), but in Perl it is further from just an implementation detail or optimization than many others, because you can write code that runs during that phase.
use has this effect, as do BEGIN blocks when -n or -p has not been passed. With -n or -p, BEGIN blocks have more awk-like semantics, I have heard. I've used BEGIN with -n and -p but I'm far from 100% on the details. 07:28
I know it is my above implementation, and not that answer, that is wrong in the way that matters, because I can write a more cumbersome one-liner that avoids using
-p (or -n) and instead code a similar while (<>) loop myself. (I could code the exact equivalent while (<>) loop, too, which invokes print in a continue block, checks if it succeeded, and bails if it didn't. But in many situations it is not really necessary, and I consider this one of them.) When I do this, it works: 08:35
@Zanna The slightly more complicated task of grouping the code points by the lexeme in which they appear is, in my opinion, far more amusing.
09:15
I want to say that this is something that is more compact in Perl than in Python! But actually I don't know, because I don't know how to match a lexeme in Python. Python's regex dialect does not suppport
\X, which in Perl does that. That is, \X is doing the magic of matching the two-code-point sequence ழ் but also matching just single code points when they are displayed as a single symbol. I do not presume Python can't also do it easily and automatically, only that I don't know how. -0 doesn't change the output record separator $/ (which is undef by default in Perl anyway), only the input record separator $\. The more intriguing question is why there is a newline in the output of that one-liner.
Yeah. That's why the trailing
(10) is what went away from the non-chomping version to the current, chomping version.
@Zanna Correct. Though in situations where there are non-terminal newlines,
. in a Perl regex without any flags set won't match them, either: "m/PATTERN/msixpodualngc"
"/PATTERN/msixpodualngc"
Searches a string for a pattern match, and in scalar context
returns true if it succeeds, false if it fails. If no string is
specified via the "=~" or "!~" operator, the $_ string is
searched. (The string specified with "=~" need not be an
lvalue--it may be the result of an expression evaluation, but
remember the "=~" binds rather tightly.) See also perlre.
@EliahKagan I copied the later version to easily check that I understood your joke correctly, and then I used that command to find out what character
10 represents (I don't like guessing) ek@Io:~$ perl -wE 'use open qw(:std :utf8); say join "\n", map { join "", map { chr } /(\d+)/g } <>' <<'EOF'
(2996 3021) (1488) (120)
(2996 3021) (1488) (120)
EOF
ழ்אx
ழ்אx
ek@Io:~$ ek@Io:~$ perl -wE 'use open qw(:std :utf8); say join "\0", map { join "", map { chr } /(\d+)/g } <>' <<'EOF' | xargs -0
(2996 3021) (1488) (120)
(121) (32) (10) (122)
EOF
ழ்אx y
z
ek@Io:~$
(That seems like the most readable of all easy ways to view the result of reversing the middle of those examples.)
You'll notice that
<> has a different effect here, because it appears in list context rather than scalar context, so it reads to the end of input rather than reading a single line of input. I used <> once in list context, rather than letting the interpreter automatically evaluate it repeatedly in scalar context through the implied while (<>) of -p or -n, so that it wouldn't produce any output until the input was done. 10:52
What an expression means can differ depending whether it appears in void, scalar, or list context. I do not know of other languages that work this way. It seems to be a source of many bugs, but I guess it is also handy. The idea is that human language works this way, supposedly.
With here documents, as shown above, it doesn't matter that much, nor would it if you were reading from a file or redirecting stdin, or redirecting stdout, but otherwise you'd get partial output before you were done entering input. This way, you don't:
ek@Io:~$ perl -wE 'use open qw(:std :utf8); say join "e", map { join "", map { chr } /(\d+)/g } <>'
(97) (98)
(99)
(100) (97) (114) (105) (97) (110)
abecedarian
ek@Io:~$
I've removed the prompt so it's easier to see where the output actually ends. Similarly, when it is entered while
perl runs, instead of beforehand as a shell here document: ek@Io:~$ perl '-Mopen qw(:std :utf8)' -wnE 'BEGIN { $\ = "e" } print join "", map { chr } /(\d+)/g'
(97) (98)
abe(99)
ce(100) (97) (114) (105) (97) (110)
dariane
Anyway this must be horribly confusing, since I realize that, in addition to neither saying much of anything about void/scalar/list context before and not linking to anything on it, I also never actually said specifically what
-n and -p do, even though that was the one thing you had actually wanted to know! :)
2 hours later…
14:11
@Zanna In Perl, you would have to take an array with you when you go shopping for groceries (or you could take a scalar that is a reference to an array), because lists are ephemeral.
for example: In Perl, for and foreach mean the same thing. There are two forms of the for loop, but you can use either keyword for either one. It's common, but not universal, to just use for for both of them, which is what I prefer. I am about to describe the "foreach" style loop. This is the same style of "for" loop as Bourne-style shells all have. It is exactly the same for loop we discussed earlier. There we had:
In all of these cases, the items that are actually being looped over -- that get bound to the loop variable (which is
$_ if not declared explicitly) -- are in a list. In those loops, (1.. 5) is a list. You can also write stuff like:
This creates an array called
@vals and initialized it with the a list of numbers from 1 to 10. It is thus an array of those numbers. (1..10) is a list @vals is an array.
In general, Perl uses eager evaluation, but lists can be evaluated (or, well, used) lazily. To observe this, notice first that you can bail out of a
for loop before iterating through each item. One way to do so is with last, which is like break in most other languages. Here's a contrived example:
That works just fine, while, in contrast, this causes the Perl interpreter to quit immediately with an out-of-memory error:
When you write multiple values and separate them with commas, that usually makes a list. Ranges make a list. Arrays, when you actually use them, frequently make a list. Sorry about this imprecise description. What I am talking about is this:
0> my @vals = (10..14); $res[0] = [ 10, 11, 12, 13, 14 ] 1> my @morevals = (1..3, @vals, 17, 21); $res[1] = [ 1, 2, 3, 10, 11, 12, 13, 14, 17, 21 ]
(1..3, @vals, 17, 21) is not a list that contains the array @vals. It is a list that contains all the element of the array @vals, as well as some other elements. This list is used to initialize the second array @morevals.
This is quite dissimilar to many languages. In most languages, arrays are not automatically expanded like that.
The
@ sigil means "array." The $ sigil means "scalar." Conceptually, arrays are plural, while scalars are singular. @x and $x are completely separate. Both can exist at once, or either, or neither. The elements of arrays are scalars.
As that blog post about arrays and lists says, arrays can be multidimensional... but what this actually means is that you have an array of references to arrays. Those references are themselves actually scalars. There is no special language construct for a multidimensional array. Furthermore, you can have an array where some of its elements are references to other arrays and some are not. That's usually a bad idea, but you can do it.
The
[ ] make it a list literal. They do not have the same meaning as ( ), which are really just doing grouping. (You can make lists without the enclosing parentheses, in some situations where it would not be ambiguous.)
@Zanna Yes. Though you can make an array in Perl without the square brackets, as you've seen. Also, the same words mean different things for different languages. Lists in Python are like arrays in Perl. So really the
[ ] notation has a more similar meaning in the two languages than might be immediately apparent just by listening to people talk about it.
You can assign an array to an array. That copies all the elements. It does not alias the arrays; if you then modify one of the arrays, the other is not modified.
0> my @original = qw(foo bar baz quux); $res[0] = [ 'foo', 'bar', 'baz', 'quux' ] 1> my @copy = @original; $res[1] = [ 'foo', 'bar', 'baz', 'quux' ] 2> push @copy, 'foobar'; $res[2] = 5 3> @copy $res[3] = [ 'foo', 'bar', 'baz', 'quux', 'foobar' ] 4> @original $res[4] = [ 'foo', 'bar', 'baz', 'quux' ]
In contrast, in Python, assigning a list to a variable just binds the variable to the same list as before:
>>> original = ['foo', 'bar', 'baz', 'quux']
>>> copy = original
>>> copy.append('foobar')
>>> copy
['foo', 'bar', 'baz', 'quux', 'foobar']
>>> original
['foo', 'bar', 'baz', 'quux', 'foobar']
@Zanna Well it's a copy of the variable, which in Python stores a reference to the actual object. The objects referenced by the
original and copy variables in that Python example are the same object. It is not actually a copy of the list. There are ways to do that. In Python, for lists, the idiomatic ("Pythonic") way is to slice the array... but include all of it:
When the number to the left of the
: is omitted, it's 0. When the number to the right is omitted, it is the length of the list. If you omit both, you slice the whole list. Anyway, that's Python. Arrays in Perl also allow slicing, and it is basically the same way -- by indexing them with a range -- but there is no need, in Perl, to use slicing to copy the elements.
@Zanna Right. In scalar context, the array becomes its size (i.e., its number of elements). You might say it's not really counting them, because arrays in Perl know their own size; obtaining the size of an n-element array is an O(1) operation, not an O(n) operation.
Notice that there are other situations where scalar context applies, besides assignment to a scalar variable:
You can force scalar context with the built-in
scalar function (really, I think it is technically a named unary operator): 0> my @vals = (3..7); $res[0] = [ 3, 4, 5, 6, 7 ] 1> @vals; $res[1] = [ 3, 4, 5, 6, 7 ] 2> scalar @vals; $res[2] = 5
Using an array in scalar context -- with or without
scalar -- is actually the typical and idiomatic way to take the size of an array in Perl!
However, ranges in Perl contain their upper bound, so if the purpose is to write a range of indices, then the actual size may not really be what you want. For example, this would be a mistake if I used it for any purpose besides to demonstrate this sort of mistake:
4> for my $i (0..@vals) { print "$i: $vals[$i]\n" }
0: 3
1: 4
2: 5
3: 6
4: 7
Use of uninitialized value in concatenation (.) or string at reply input line 1.
5:
$res[3] = ''
Instead, I should use the highest index as my upper bound. I could obtain this by subtracting 1 from the size, but I don't need to, because Perl provides syntax for this:
When you declare an array variable but don't initialize or otherwise assign to it, it is the default value for an array, which is the empty array. This has the same effect as if you had initialized it with an empty list.
In contrast, this reads to end-of-input (I pressed Ctrl+D after typing
quux and pressing Enter) and puts each line into a separate element of the array:
Because the items looped over by a
for loop are supplied as a list, writing for (<>) causes <> to be evaluated in list context, which causes all input to be read into memory before any of it is used. Sometimes this is fine; occasionally it is exactly what you require; but often it is preferable to read and operate on one line at a time instead. 17> for (<>) { chomp; say "$_ squared is ", $_ * $_ }
2
9
55
91
6
2 squared is 4
9 squared is 81
55 squared is 3025
91 squared is 8281
6 squared is 36
$res[15] = ''
18> while (<>) { chomp; say "$_ squared is ", $_ * $_ }
2
2 squared is 4
9
9 squared is 81
55
55 squared is 3025
91
91 squared is 8281
6
6 squared is 36
$res[16] = '' 15:47
@Zanna One of the reasons this can happen is that the meaning of
+ is not ambiguous. The + symbol in Perl is not also being used to indicate string concatenation, for example. In Python: >>> a = [10, 20, 30] >>> b = [77, 88, 99] >>> a + b [10, 20, 30, 77, 88, 99] >>> 'foo' + 'bar' 'foobar'
@EliahKagan
for is not doing anything special there. <> appears in list context, so it reads all available lines. But while is doing something special. <> is evaluated in scalar context, and that's not special -- the condition for while or if is always evaluated in scalar context -- but there is something special going on in that while loop. There are actually two special things going on.
The result of
<> (in scalar context) is checked to see if it is defined (in the sense of not being undef) rather than if it is true, and it is automatically assigned to $_. A for loop binds list elements to a loop variable automatically, but a while loop does not... and nonetheless while (<>) works to read lines of input one at a time and place each one in $_. This is special behavior.
You can undefine variables, in the sense of giving them their default value, which is
undef for scalars and is the zero-element array for arrays (there are other kinds of variables in Perl besides scalars and arrays but I've hardly talked about them thus far). To do this, use the undef operator. Note that, when you're using use strict; (which should be virtually always), applying undef to something to "undefine" it does not have the same effect as removing it as a variable. 0> my $s = 'some text'; $res[0] = 'some text' 1> undef $s; $res[1] = undef 2> $s; $res[2] = undef 3> $t; Global symbol "$t" requires explicit package name (did you forget to declare "my $t"?) at reply input line 1. BEGIN not safe after errors--compilation aborted at reply input line 7.
(
reply has use strict; enabled automatically, as does the Perl rename command. When you write actual script files, you have to write use strict; to get it--which you should. Constructs like perl -e do not automatically turn on use strict;.)
But applying
undef to a variable does have the effect of causing it to be as though the variable had been declared without ever having been given a value.
The
undef operator also returns the undef value. And you can omit the argument to undef. This is useful because it gives you the undef value. (When you use undef by itself as an expression, that isn't an undef literal: there is no undef literal. But it returns undef, so calling it without an operand gives you the same power as if there were an undef literal.) 0> my @values = (1..5); $res[0] = [ 1, 2, 3, 4, 5 ] 1> undef @values; $res[1] = undef 2> @values; 3>
Note that it returns
undef, which is a scalar, but it does not set its array operand to undef. An array cannot have the value undef, but it can have a single element that is undef. Such an array is quite defined, evaluates to true when used as a boolean, etc. It is rarely if ever what you would want. undef does the right thing, though, when given an array operand. It empties it out.
To check if a scalar is defined, you can use the
defined operator. This checks if it is not undef. It still gives an error if you use it on an undeclared variable with use strict;. It also should not be used on arrays. 3> my $x = 'foobar'; $res[2] = 'foobar' 4> defined $x; $res[3] = 1 5> undef $x; $res[4] = undef 6> defined $x; $res[5] = '' 7> my $y; $res[6] = undef 8> defined $y; $res[7] = ''
0> my @values; 1> defined @values; Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at reply input line 1. 2> defined $z; Global symbol "$z" requires explicit package name (did you forget to declare "my $z"?) at reply input line 1. BEGIN not safe after errors--compilation aborted at reply input line 7.
In Perl, numbers are true, except 0, which is false, and strings are true, except the empty string and the string '0', which are false. Note that even a string consisting of a single null character is true.
4> 1 or 'it was false'; $res[1] = 1 5> 0 or 'it was false'; $res[2] = 'it was false' 6> 'x' or 'it was false'; $res[3] = 'x' 7> '' or 'it was false'; $res[4] = 'it was false' 8> '0' or 'it was false'; $res[5] = 'it was false' 9> '0.' or 'it was false'; $res[6] = '0.' 10> '0.0' or 'it was false'; $res[7] = '0.0' 11> '-1' or 'it was false'; $res[8] = '-1' 12> ' ' or 'it was false'; $res[9] = ' ' 13> "\0" or 'it was false'; $res[10] = ''
One way to demonstrate that these rules would not work properly for
while (<>) if it were really testing <> for truth is to set the input record separator $/ to the numeral 0 (not to be confused with the null character):
(I pressed Enter after
1200345, then I pressed Ctrl+D at the beginning of the line after it showed me [0], since it was waiting for another 0 to end the record. Pressing Ctrl+D flushed the buffer and gave it one more iteration where there was text to read before end-of-input.)
One of the values
$_ takes on in that loop is 0, as shown by [0]. There, [ ] have nothing to do with arrays; I just had to use some characters in print to show where the strings ended.
Even though the string consisting of a single
0 character is taken to be false, that while loop nonetheless kept reading. That is one way to demonstrate that while treats readline expressions specially. When you return, if I'm still here, I'll show an example of what while is usually doing and how this is different. « first day (120 days earlier) ← previous day next day → last day (2447 days later) »
