« first day (119 days earlier)      last day (2448 days later) » 

09:53
@EliahKagan well, the impression I got was that there was no reason for Perl, because anything you could do in Perl, you could do in Python, and it would be better. But I think that at the very least that "better" is more than a little subjective, or perhaps means something like "easier to read at the cost of being three times as long"
but people seem to use Perl and Python to do quite different things. I have a general belief that approaches match tasks (rather than approaches being good or bad in themselves or that people are inflexibly inclined towards approaches). So I guess I think while it might be possible to do everything in language x that you can do in language y, there will usually be something about the task that suggests language x or language y (or z) as the most fun|easy|elegant|readable way to do it
10:33
@Zanna I find that Python is often more compact than Perl.
oh hmm
Not always. Some things in Perl are very compact, like file tests (run perldoc -f -X). Sometimes things can more reasonably be compacted into a single line in Perl than in Python. But, for example, suppose you wanted to introduce a local array variable called tokens, containing words read from a single line of input, which must be read from standard input, split on whitespace, with the splitting whitespace discarded. In Python 3:
tokens = input().split()
In Perl:
my @tokens = split ' ', <STDIN>;
Perl makes the case of reading from files instead of standard input very easy. <> will read from files whose names were passed as command-line arguments, or from standard input if there are no such files. If that behavior is acceptable, the Perl way is a little shorter, though it still doesn't beat the Python 3 way.
Perl bizarrely has nothing built in, or installed with any of the default modules, to trim/strip all leading and trailing whitespace from a string. You can install Text::Trim and use that, assuming your code doesn't have to run anywhere that might not have that module installed. Otherwise you can use s/// for it. To read a string from standard input into a variable called s in Python 3, with leading and trailing whitespace removed:
s = input().strip()
10:54
I was going to ask what you meant by "with the splitting whitespace discarded", but I realised what you meant, and also that this is probably what you nearly always want in such a case...
In Perl:
my $s = <STDIN>;
$s =~ s/^\s+//;
$s =~ s/\s+$//;
It can be done more compactly, but not without significantly decreased readability or (IIRC) performance.
wow
that seems really awkward
Of course, you can just write a subroutine to do it.
But that is also true in Python -- you can write functions with short names that contain multiple lines of code in Python, too. :)
In Perl, newline characters are not automatically removed from the end of lines when you read them. This is actually useful, for example because you can output the line that was read without appending a newline, and because information is occasionally lost in the more typical behavior of chomping the newline at the end automatically (it's possible to try to read a whole line but not have input end first). However, this means it is often necessary to use the chomp function on what was read.
So, for this code in Python 3:
s = input()
A typical way to achieve the same thing in Perl is:
chomp(my $s = <STDIN>);
That has the same effect as:
my $s = <STDIN>;
chomp $s;
chomp does not return the chomped string. Instead, it modifies it, and returns the number of characters that were chomped from it (0 or 1).
(I'm giving a 1 hour Skype lesson so while I'm not technically AFK, I'm mostly not here)
That's okay. I'll hold off on continuing what we had been talking about yesterday, which might require that you try stuff out. These are just some thoughts about code size. :)
However, I can stop typing stuff here if you think it may be distracting you or if you otherwise don't want to see it. (As far as I know, we're the only ones actively in this room at the moment.) You can let me know.
Culturally, it is encouraged when programming in Perl to use <> (or <<>> which is more secure but less widely supported), instead of <STDIN> whenever it is reasonable to do so, and to use a prompting module like IO::Prompt or IO::Prompt::Tiny in cases where it is not reasonable to do so because the goal is to obtain interactive input from the user. This makes things more compact.
But there are examples of this sort of thing that go in the other direction, too. In Python, the recommended way to create a standalone symbolic constant (i.e., one that is not part of an enumeration) is to just make it a regular variable but use all capital letters and avoid changing it. Static analyzers like pylint can notice if it is modified. So, in Python, one might write:
NUMBER_OF_REPS = 10
In Perl the old way to make a symbolic constant was:
use constant NUMBER_OF_REPS => 10;
That still works fine, but it is looked down on for a few reasons. Mainly the issue is that the constant is accessed through the bareword NUMBER_OF_REPS, which breaks the usual expectation to use a sigil, and also prevents it from being used in situations that require a sigil, like string interpolation. A newer method that is generally considered better is to install the Readonly module (it's not necessarily installed by default) and use:
use Readonly;
Then one can write things like:
Readonly my $NUMBER_OF_REPS => 10;
Even better is to install and use the Const::Fast module.
use Const::Fast;
Then one can write things like:
const my $NUMBER_OF_REPS => 10;
One thing about Perl that often saves space is that you do not have to convert strings to numbers before using them as numbers. For each of the dynamic types a scalar value can be, the Perl interpreter automatically keeps track of whether or not there was a value computed for it of that type and what it was, as well as whether or not it is current. So if you have strings that you then use as numbers, they shouldn't have to be converted to numbers again each time. Thus the Perl code
my @values = split ' ', <STDIN>;
works in place of the Python 3 code
values = input().split()
but it also works in place of the Python 3 code:
values = list(map(int, input().split()))
However, when a value is used as the key for a hash (a hash in Perl is a hash table, like a dict in Python), it is used as a string. Thus, care becomes required when there might be multiple string representations for the same number. Even aside from the situation of extra leading zeros, this can happen with integers, because leading and trailing whitespace are allowed.
The Perl interpreter won't generate string representations of numbers with whitespace added (e.g., you don't have to worry about spaces being printed from print 3;), but if it is not sanitized out of user input then it will still be there, and if the value is used a the key in a hash, then extra explicit processing is needed. Consider this Perl script, which reads numbers, one per line, stores them and their squares in a hash, then prints each "unique" pair in numerically increasing order:
#!/usr/bin/env perl

use strict;
use warnings;

my %squares;

while (<>) { # read one line at a time
    chomp;
    $squares{$_} = $_ * $_;
}

print "$_ => $squares{$_}\n" for (sort { $a <=> $b } keys %squares);
When the input is:
2
2
The output is:
2 => 4
But when the input is:
2
 2
The output is:
2 => 4
 2 => 4
To deal with all the possible situations where this can happen, people sometimes use tricks like $squares{$_ + 0} = $_ * $_, or writing $_ += 0 first.
The other thing that may stand out is, in Perl, one often writes things like:
my @sorted_numbers = sort { $a <=> $b } @numbers;
In Python this is just:
sorted_numbers = sorted(numbers)
Or if numbers is a list and it's sufficient to just make it be sorted, then:
numbers.sort()
You can use this in Perl, but it does not do the right thing for numbers:
my @sorted_numbers = sort @numbers;
The reason is that, if you don't pass it a block of subroutine to tell it what kind of comparison to use, then it treats the values as strings and uses lexicographic string comparison on them, regardless of whether or not they were ever strings. (Hash keys are strings in Perl, but this happens even when the list being sorted was never originally strings.)
In Python, each object has a specific type, and it knows its type. This forces you to convert objects between types explicitly in most situations where it's not already the type you need. But it means that the objects, rather than the operators your choose to use on them, know what needs to be done, which means that operators like < are numerical for numbers and lexicographic for strings. In Perl, scalars can act like any of several different dynamic types, but they use different operators.
This is not bad. It's a totally reasonable design choice, as is Python's. But it means you have to distinguish between operators like == and < (and several others) that perform numerical comparison and eq and le (and several others) that perform string comparison. It's sort of like the comparison tests with the [/test command in shell scripting and Bash's [[ keyword--except that the meanings are reversed.
For the same reason, you can use + in Python for both addition and string concatenation, and * for both multiplication and string repetition. In Perl, + and * are always arithmetic, and you must use . and x for concatenation and repetition.
12:14
The sigils themselves take up space--though they do have some advantages--and I find they often make something that's one line in Python of reasonable length take two in Perl. And due to Perl's distinction between scalars and arrays, when the Python code a[i] becomes $a[$i] in Perl (common, though that's not always how it would translate), the Python expression a[i][j] becomes $a[$i]->[$j]. Anyway, this is a huge wall of text; I'll hold off on sharing further thoughts on this topic. :)
@EliahKagan oops, sorry... no I just didn't look at the tab, no distraction :)
 
3 hours later…
15:04
@EliahKagan it matches all 1 byte characters so as well \nor better to say \0
I slept like a stone now for almost 20 hours lol
15:30
@EliahKagan oh please feel free! I only just finished reading it. I was enlisted to perform various urgent duties by my mum
@Videonauth excellent :)
yeah much needed sleep, had not slept since monday :)
@EliahKagan hmm I can see how lots of people would probably find that less accessible or harder to remember than how each object behaves in Python
@Videonauth ouch!
@EliahKagan hmm I think I remember doing something nice and clear like that when playing with Ruby...
@Zanna thats a normal time for me that i am awake right now, this might change in janurary when i get different medication
 
4 hours later…
19:14
@Videonauth I don't think I know what you mean. At least based on my own testing of GNU sed, a . matches any one character even if that character is represented by a sequence of multiple bytes. ...Let's see if SE chat will render this properly:
ek@Kip:~$ sed 's/./<&>/g' <<<א
<א>
ek@Kip:~$ sed 's/./<&>/g' <<<אאא
<א><א><א>
ek@Kip:~$ LANG=C sed 's/./<&>/g' <<<אאא
<▒><▒><▒><▒><▒><▒>
Ah, excellent.
So, each א takes multiple bytes to represent in UTF-8. In any locale that uses UTF-8, GNU sed has . match each of these byte sequences (not the individual bytes). So the alephs each are matched by . in he_IL.utf8, but also in en_US.utf8 even though English doesn't itself have א. In contrast, with LANG=C, sed treats each byte as a distinct character. Then two "characters"--produced by interpreting part of the sequence that represents the א--get matched for each of them.
nice!
This is all exactly what I expect, or at least what I hoped would happen. Are there sed implementations that otherwise support Unicode locales -- so that, for example, \w matches multi-byte charaacters that are considered word characters according to LC_CTYPE for the locale -- but where . still only matches single-byte characters? If not, then I think the rule for . in sed should simply be said to be that it matches any single character.
I assumed this was the case. I don't know that there are any circumstances when it isn't
19:32
In a UTF-8 locale, does . in sed match a trailing byte that fails to represent any character?
hm how can I add such a byte?
You can treat a single two-byte character as two characters by operating it with LANG=C, pull one character out, and then operate on that in a UTF-8 locale.
The answer seems to be no -- . is sensitive to whether or not a byte is considered a character in your locale. But, oddly, whether or not what I'd think should never be considered a character in a UTF-8 locale actually is considered one seems to vary across different UTF-8 locales:
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=C sed -n 's/./<&>/gp'
<▒>
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=en_US.utf8 sed -n 's/./<&>/gp'
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=he_IL.utf8 sed -n 's/./<&>/gp'
<▒>
Oh, I think I have figured out why.
Whether or not a byte that does not make a full UTF-8 character is matched by . does not vary across UTF-8 locales (or, at least, nothing that I have observed actually supports the idea that it would). The issue is simply that my machine Kip does not have the locale he_IL.utf8 installed. I deliberately tested en_US.utf8 on Kip to ensure that it would work to match the multi-byte character א even without any Hebrew locale also being installed.
But obviously I cannot really test he_IL.utf8 when that locale is itself not installed. On Io, which has he_IL.utf8 installed as well as en_US.utf8, they both work as expected. Since that's not the only difference between Kip and Io -- Kip is 14.04 while Io is 16.04 -- I'll install the language-pack-hepackage on Kip and test again to be sure. This is now, before installing the language-pack-he package:
ek@Kip:~$ LANG=C sed 's/./<&>/g' <<<א
<▒><▒>
ek@Kip:~$ LANG=en_US.utf8 sed 's/./<&>/g' <<<א
<א>
ek@Kip:~$ LANG=he_IL.utf8 sed 's/./<&>/g' <<<א
<▒><▒>
This is after running sudo apt install language-pack-he:
ek@Kip:~$ LANG=C sed 's/./<&>/g' <<<א
<▒><▒>
ek@Kip:~$ LANG=en_US.utf8 sed 's/./<&>/g' <<<א
<א>
ek@Kip:~$ LANG=he_IL.utf8 sed 's/./<&>/g' <<<א
<א>
19:52
aha you were right
And then, when I run those commands again (showing the final prompt so you can see he_IL.utf8 also refuses to match . to a byte that is not a character):
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=C sed -n 's/./<&>/gp'        <▒>
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=en_US.utf8 sed -n 's/./<&>/gp'
ek@Kip:~$ LANG=C sed -r 's/^(.).*/\1/' <<<א | LANG=he_IL.utf8 sed -n 's/./<&>/gp'
ek@Kip:~$
Interestingly, that didn't render properly... probably since there is no way to render it properly since SE chat uses Unicode. But you can see that something is matched when the half of a UTF-8 א is given to a sed instance with LANG=C, and nothing is matched when it is given to sed instances with LANG=en_US.utf8 and LANG=en_IL.utf8. The glitch itself may just be due to my terminal emulator, or my browser, or some combination of the two. In the terminal, there is a newline shown.
zanna@toaster:~/playground$ LANG=C sed -r 's/^(.).*/\1/' <<<ழ் | LANG=C sed -n 's/./<&>/gp'
<�>
zanna@toaster:~/playground$ LANG=C sed -r 's/^(.).*/\1/' <<<ழ் | LANG=en_UK.utf8 sed -n 's/./<&>/gp'
<�>
zanna@toaster:~/playground$ LANG=C sed -r 's/^(.).*/\1/' <<<ழ் | LANG=ta_IN.utf8 sed -n 's/./<&>/gp'
zanna@toaster:~/playground$
That's odd!
yeah... this character ழ் is actually 7 bytes...
zanna@toaster:~/playground$ wc -c <<< ழ்
7
not that I expect that to make any difference
I would definitely not assume that to be unrelated. Why is that happening?
I mean, I think wc -c always tells you bytes. But why is ழ் six bytes long?
ek@Kip:~$ printf '%s' a | wc -c
1
ek@Kip:~$ printf '%s' א | wc -c
2
ek@Kip:~$ printf '%s' ழ் | wc -c
6
Is ழ் a single character?
20:05
The dot is a modifier. Here's the base letter ழ
but that's still 3 bytes?
What do you get when you try it with that character but those simpler commands?
ek@Kip:~$ LANG=C sed 's/./<&>/g' <<<ழ
<▒><▒><▒>
ek@Kip:~$ LANG=en_US.utf8 sed 's/./<&>/g' <<<ழ
<ழ>
ek@Kip:~$ LANG=C sed 's/./<&>/g' <<<ழ்
<▒><▒><▒><▒><▒><▒>
ek@Kip:~$ LANG=en_US.utf8 sed 's/./<&>/g' <<<ழ்
<ழ><்>
zanna@toaster:~/playground$ LANG=C sed 's/./<&>/g' <<<ழ்
<�><�><�><�><�><�>
zanna@toaster:~/playground$ LANG=en_UK.utf8 sed 's/./<&>/g' <<<ழ்
<�><�><�><�><�><�>
zanna@toaster:~/playground$ LANG=ta_IN.utf8 sed 's/./<&>/g' <<<ழ்
<ழ><்>
What locales do you have installed? (locale -a)
20:12
oh en_UK is not a real locale... it's en_GB facepalm
zanna@toaster:~/playground$ LANG=en_GB.utf8 sed 's/./<&>/g' <<<ழ்
<ழ><்>
sorry!
FTR:
zanna@toaster:~/playground$ locale -a
C
C.UTF-8
en_AG
en_AG.utf8
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IL
en_IL.utf8
en_IN
en_IN.utf8
en_NG
en_NG.utf8
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZM
en_ZM.utf8
en_ZW.utf8
POSIX
ta_IN
ta_IN.utf8
ta_LK
ta_LK.utf8
That seems to explain it, then.
sorry, that's a really boring explanation and applies to many things - Zanna is clueless
Really your mistake was the same as mine: we both tried with a locale that was not installed, did not realize it wasn't installed, and were confused about why the effect was the same as using the C locale.
hahaha
I was thinking wow this must be the fault of our stupid Tory government! They've even messed up UTF-8!
Damned austerity! Even text formatting is going to hell in a handbasket
Doubtless they've handed control of it over to some unaccountable corporation and it will never get fixed
etc
So you're saying you were unsure, but that the current government May have been the cause?
20:21
precisely
I figured that maybe it was GB instead of UK because Northern Ireland had a separate locale or something... but supposedly that is not why.
@Zanna you mean encoding
well I wouldn't blame them for wanting a separate locale
+1 to that answer haha very useful
Now I'm tempted to start on the supposed Great-ness of Britain. But that won't lead anywhere good.
so . matches any character
It is far from obvious to me that "great" is any more misleading than "united."
> We were united in the face of the great crater that opened up in the middle of our city.
hahaha
I think "united" is far more likely to carry positive connotations than "great." One can easily use "great" in an negative way without irony (or sardonicism, or whatever we are calling it these days). Even to say something like, "They were united in their support of the foolish plan," carries a tinge of silliness that I don't think attaches to negative uses of "great."
20:37
The Acts of Union are looking shaky these days too
I mean, you can use "great" negatively in a silly way, but I think it is usually in longer phrases that themselves have positive connotations, like "one of the greats."
Anyway, as for why is three bytes in UTF-8, I think that even when a character is on the BMP and thus takes only one 2-byte element to represent in UTF-16, it can still take more than two 1-byte elements to represent in UTF-8.
I thought perhaps the reason is that it takes 3 times as much effort to pronounce as anything I can write with ASCII chars :)
Even control characters?
oh hmm which are you thinking of?
@EliahKagan excellent point...
@Zanna Informally, and somewhat more restrictively than the formal definition, I am thinking of any of the characters that are not whitespace but nonetheless are treated specially by most terminals and are typically entered by holding down Ctrl and pressing a key. For example, the character Ctrl+C enters is ASCII 003, and the character Ctrl+D enters is ASCII 004.
More formally, and actually less restrictively, the first 32 ASCII codes represent control characters.
20:46
oh! that's very interesting!
You can make your terminal ignore their special meaning (to it) and pass them on literally to programs by prefixing them with Ctrl+V, which (to employ an analogy) is to control characters in terminals as `` is to metacharacters in regular expressions.
do you mean backslashes, or really backticks?
I mean backslashes. Backticks are not special in a regular expression.
Oh, I see. I cannot easily write a \ as code in SE chat.
Yes, you are correct. I mean backslashes.
\
yeah, I figured it got eaten. I still haven't figured out how to type a single backslash in code in chat. But I'm guessing you used indenting to get that one :D
So typing Ctrl+V Ctrl+C writes a literal Ctrl+C ASCII 3 and sends no signals to anything, rather than writing nothing but sending SIGINT to (processes in) the terminal's foreground process group.
I did use indenting, yes.
I never remember how to do it. Is it \?
Yeah, so you can type something that is itself hard to represent literally as code in chat.
20:55
ah, another use for the much maligned triple backticks
can I type one? \
Well double backticks are sufficient.
oh yeah :)
In some dialects of Markdown, three backticks start and end code blocks. But SE doesn't support that anywhere -- or, at least, not for any user-contributed content.
it works in Telegram :) (to create a code block)
but Telegram wraps too hard. It destroys code and love poems
@EliahKagan TIL!
21:10
You can inspect character codes with a Perl one-liner (though I suspect there might be easier ways I haven't thought of):
ek@Io:~$ perl -wpe 's/./sprintf " %d", ord $&/ge'
foobar
 102 111 111 98 97 114
^A^B^C^D
 1 2 3 4
woah... that is awesome and it makes what you said here a whole lot more fun
zanna@toaster:~$ perl -wpe 's/./sprintf " %d", ord $&/ge'
ழ்
 224 174 180 224 175 141
^V
 22
hmm. I am easily amused
21:52
I have just realized I don't know how to make perl behave as sed does withh multi-byte characters. Even with the use locale; pragma beforehand, and the u modifier on the regex (as well as the g and e modifiers as shown), it shows the same thing for me. But . is documented to match any single character, even if it is more than one byte long. Similarly, length is supposed to give the length of a string in characters, but even after use locale;, length 'ழ' evaluates to 3.
There is something I'm not understanding about how this is supposed to work.
\X should match a single grapheme, which may be more than one Unicode code point, but even . should match any character, since that is a single code point even when it takes more than one 1-byte element in UTF-8 to represent. That is, I would expect length 'ழ்' to evaluate to 2 but instead it evaluates to 6, and I would expect length 'ழ' to evaluate to 1 but instead it evaluates to 3.
To clarify, . does not match every character in Perl unless the s modifier is used; it does not match a terminal newline. But I do not understand why it is matching parts of characters. Unless this is due to a bug, which I don't think is the case, my understanding of how one is supposed to make Perl treat strings as Unicode is apparently insufficient.
Time to read the some Unicode-related perldoc pages straight through from beginning to end!
I'll start with perlunitut.
22:12
It works fine out of the box in Python 3, as expected (and even without telling it to use the current locale, since Python 3 always treats strings as Unicode -- it has a separate byte string data type for byte strings):
>>> len('ழ')
1
>>> len('ழ்')
2
>>> ord('ழ')
2996
>>> ord('ழ்'[0])
2996
>>> ord('ழ்'[1])
3021
22:28
I believe I found the answer in perlunifaq. use locale; does not cause any filehandles to be opened with UTF-8 encoding -- not even special filehandles like STDIN that perl opens for me, which I had thought would be affected. Consequently, the characters are interpreted as latin-1. So I can use binmode on STDIN (just as I'd use it on a filehandle I opened myself for whatever reason hadn't give the needed encoding upon opening) or I can use the open pragma to affect everything.
ek@Io:~$ perl -wle 'print ord <STDIN>' <<<ழ
224
ek@Io:~$ perl -wle 'binmode STDIN, ":encoding(UTF-8)"; print ord <STDIN>' <<<ழ
2996
ek@Io:~$ perl -wle 'use open qw(:std :utf8); print ord <>' <<<ழ
2996
53
A: How do I read UTF-8 with diamond operator (<>)?

potylTry to use the pragma open instead: use strict; use warnings; use open qw(:std :utf8); while(<>){ my @chars = split //, $_; print "$_" foreach(@chars); } You need to do this because the <> operator is magical. As you know it will read from STDIN or from the files in @ARGV. Reading fro...

("each new STDIN" is not a correct description of what <> is doing -- <> doesn't change STDIN, which is precisely why binmodeing STDIN is insufficient for using <> or <<>>, to read from files whose names are passed as command-line arguments. But besides that, the answer is totally correct.)
22:43
Hmm... I'll read this in the UTC morning
So this is the version that of the character code printing one-liner that works right for UTF-8:
ek@Io:~$ perl -wpe 'use open qw(:std :utf8); s/./sprintf " %d", ord $&/ge'
foobar
 102 111 111 98 97 114
ழ்
 2996 3021

« first day (119 days earlier)      last day (2448 days later) »