Conversation started Aug 25, 2020 at 14:30.
Aug 25, 2020 2:30 PM
Welcome to APL Cultivation!
Do we have a topic?
Having treated new 18.0 features in the previous lessons, I was thinking to describe some of the language enhancements next. Unless we have other requests.
Sounds like a plan...
I really should install 18 on this box...
Yeah, but at least you can now use the bot for access to 18.0: ⋄ #⎕WG'APLVersion'
@Adám Illegal code
Aug 25, 2020 2:32 PM
@DyalogAPL Oh, of course it is. My bad.
ngn
ngn
^^^this code, officer!
So, I think one of the most exciting extensions is that of partitioned enclose X⊂Y which now allows the generation of empty segments.
⋄ 1 0 0 2 2 0 0 0 0 0 0 ⊂ 'KenEIverson'
@Adám Ken E Iverson
That's kind of hard to interpret, how about ⋄ {↑(⍳≢⍵)⍵}1 0 0 2 2 0 0 0 0 0 0 ⊂ 'KenEIverson'
@Adám
   1  2  3  4        5
 Ken     E     Iverson
Aug 25, 2020 2:36 PM
Let's say we have some sorted data, and we'll like to group it by interval.
values  ← 3 14 15 35 65 89 92
cutoffs ← 0 20 40 60 80 100
We want to reach (3 14 15) (,35) ⍬ (,65) (89 92)
That is, all the numbers in the interval [0,20) and in [20,40) etc.
To get the index of each value's interval, we apply Interval Index: ⋄ values ← 3 14 15 35 65 89 92 ⋄ cutoffs ← 0 20 40 60 80 100 ⋄ cutoffs ⍸ values
@Adám 1 1 1 2 4 5 5
RGS
RGS
@Adám is this a question? should we finish what you started?
If you want.
Now, you might think that Key could do the trick, but then we'd have to insert all the possible intervals.
⋄ values ← 3 14 15 35 65 89 92 ⋄ cutoffs ← 0 20 40 60 80 100 ⋄ (cutoffs ⍸ values) {⊂1↓⍵} ⌸⍥{⍵,⍨⍳≢cutoffs} values
@Adám 3 14 15 35 65 89 92
Can we do this in a more elegant fashion?
RGS
RGS
Aug 25, 2020 2:45 PM
This doesn't have the ⍬ in the middle:
⋄ values ← 3 14 15 35 65 89 92 ⋄ cutoffs ← 0 20 40 60 80 100 ⋄ where ← cutoffs ⍸ values ⋄ (1,(1↓where)>¯1↓where) ⊂ values
@RGS 3 14 15 35 65 89 92
Right, that is close.
@RGS If you change > to -, I think you'll get the correct result
@RGS The problem you have is that you are detecting if a value belongs to a new segment relative to its neighbour on the left, rather than how many segments ahead it is.
⋄ values←3 14 15 35 65 89 92 ⋄ cutoffs←0 20 40 60 80 100 ⋄ (1,2-⍨/cutoffs⍸values)⊂values
Aug 25, 2020 2:47 PM
@Bubbler 3 14 15 35 65 89 92
@Bubbler Exactly.
RGS
RGS
@Bubbler gosh, how could I forget about 2 f/... xD
This did not work in 17.1 because partitioned enclose only allowed a Boolean left argument.
(I hope the bot gets dfns autoloading and/or ]boxing option)
(Haven't seen moonchild here for a while ― downside of relying on 3rd parties.)
RGS
RGS
Aug 25, 2020 2:49 PM
@Adám "how many segments"?
ngn
ngn
(⎕io←0 would be nice too)
RGS
RGS
@RGS ah I think I got it! when we had a boolean, a 1 indicated a new partition would start there, or rather, we would have to jump 1 partition to the right; an integer n means we jump n partitions to the right, meaning we create n-1 empty partitions if n≥1
@RGS Yes, you were comparing interval indices, to see if they were growing or not. Instead, - computes the difference in interval index numbers, i.e. how many segments ahead of its neighbour on the left, any given value belongs in. 0 means "in the same segment". 1 means "in the next one". 2 means "skip one".
@ngn It can already do that: ⋄ ⎕IO←0 ⋄ ⍳3
@Adám 0 1 2
RGS
RGS
⋄ values←3 14 15 35 65 89 92 ⋄ cutoffs←0 20 40 60 80 100 ⋄ ⎕SE.Dyalog.Utils.repObj (1,2-⍨/cutoffs⍸values)⊂values
Aug 25, 2020 2:51 PM
@RGS Illegal code
ngn
ngn
@Adám the point is to not have to type that
RGS
RGS
oh ok :/ it would be annoying to type, but at least one could be sure of what one was getting
@ngn It originally used ⎕IO←0 by default, but having the bot use a different default than the main system (however wrong you may thing that default is) would be quite confusing.
⋄ values←3 14 15 35 65 89 92 ⋄ cutoffs←0 20 40 60 80 100 ⋄ ⎕JSON (1,2-⍨/cutoffs⍸values)⊂values
@Adám [[3,14,15],[35],[],[65],[89,92]]
ngn
ngn
@Adám same argument goes for ]box
Aug 25, 2020 2:53 PM
:-D
@ngn True, but there you don't get different values, just different display form, and ]box is on by default on TryAPL and in Jupyter Notebook.
Anyone up for making that expression a neat train?
RGS
RGS
@Adám a couple of minutes to give it a try would be nice, yeah
⋄ values←3 14 15 35 65 89 92 ⋄ cutoffs←0 20 40 60 80 100 ⋄ cutoffs (⊢⊂⍨1,2-⍨/⍸) values
@RGS 3 14 15 35 65 89 92
Beautiful.
OK, so what more can we do with the new improved ? How about splitting at a given set of indices? E.g. ⋄ 'KenEIverson'∊⎕A
@Adám 1 0 0 1 1 0 0 0 0 0 0
⋄ ⎕JSON 1 0 0 1 1 0 0 0 0 0 0 ⊂ 'KenEIverson'
Aug 25, 2020 2:59 PM
@Adám ["Ken","E","Iverson"]
Well, it's boolean left arg
But if instead of the mask, we started with the indices of the 1s: 1 4 5?
⋄ (⍸⍣¯1⊢1 4 5)⊂'KenEIverson'
@Bubbler DOMAIN ERROR
Huh.
I thought the bot was updated. Well well, that should indeed work:
     (⍸⍣¯1⊢1 4 5)⊂'KenEIverson'
┌───┬─┬───────┐
│Ken│E│Iverson│
└───┴─┴───────┘
Aug 25, 2020 3:03 PM
So the bot's not running the Issue 2?
That worked from initial release.
That's weird.
RGS
RGS
@Adám but the bot accepts the new left argument to ⊂ so smth weird is going on
⋄ 1 ,⍥⊂ 2 3
@RGS 1 2 3
⋄ 1 0 0 1 1⊂'KenEIverson'
Aug 25, 2020 3:04 PM
@Bubbler Ken E Iverson
The bot is using a cover for and that wasn't updated to handle this.
Anyway, this shows another extension in 18.0, namely that ⍸⍣¯1 conveniently works, but what you might not notice is that it also shows a further extension of . Observe:
     (⎕←⍸⍣¯1⊢1 4 5)⊂'KenEIverson'
1 0 0 1 1
┌───┬─┬───────┐
│Ken│E│Iverson│
└───┴─┴───────┘
Notice that the length of ⍸⍣¯1⊢1 4 5 doesn't match the length of the string!
RGS
RGS
@Adám yup, that's nifty
Until 17.1, the arguments of had to have exactly the same length.
Now, the left argument can have any length up until 1+the length of the right argument. Anyone care a guess why the 1+ ?
RGS
RGS
Aug 25, 2020 3:07 PM
@Adám in the "bin these values" prob above, it would be useful if any values where larger than the larger cutoff
is that it?
To allow some empty partitions at the end.
Yes, indeed. That'd be the way to say how many trailing empties you want.
⋄ ⎕JSON 1 0 0 1 1 2⊂'Hello'
@Bubbler ["Hel","l","o","",""]
So will assume that any "missing" elements in its left argument are 0.
Sometimes, you want to split into a head and a tail, like 1(↑{⍺⍵}↓)vector. What is now the shortest way to do this?
Aug 25, 2020 3:11 PM
⋄ ⎕JSON 1 1⊂'Hello'
@Bubbler ["H","ello"]
Spot on.
RGS
RGS
@Bubbler very clean
Another challenge: {(0↑⍵)⍵}vector
RGS
RGS
Just gonna share that ⋄ 3 (⊢⊂⍨1,⍨↑∘1⍤⊣) 'Hello' splits a larger head
Aug 25, 2020 3:15 PM
@RGS Hel lo
@Adám ⋄ ⎕JSON 2∘⊂ 'Hello'
@Bubbler ["","H","","e","","l","","l","","o"]
@RGS Sure, but at that point, ⋄ 3 (↑,⍥⊂↓) 'Hello' might be cleaner.
Wat.
@Adám Hel lo
RGS
RGS
Aug 25, 2020 3:16 PM
@Bubbler I had the same idea
@Adám indeed; just wanted to generalise the construct
@Bubbler You've discovered scalar extension. If the left argument is a scalar, will use that for all elements.
⋄ ⎕JSON 2 0∘⊂ 'Hello'
@Bubbler ["","Hello"]
Yup, you got it.
OK, how about ⋄ ⎕JSON ⊃,/{(0↑⍵)(1↑⍵)}¨'Hello' ?
@Adám ["","H","","e","","l","","l","","o"]
Aug 25, 2020 3:19 PM
That'd be Bubbler's 2∘⊂.
Not sure if this last one is very useful, but now you at least know about it.
Any more questions about X⊂Y?
Great.
Now that we mentioned ⍸⍣¯1 I should also throw in here that ⍸Y now works with duplicates in Y.
⋄ ⍸ 1 2 3 4
@Bubbler 1 2 2 3 3 3 4 4 4 4
(There must be a code golf challenge for this...)
It is now (almost) equivalent to {(,⍵)/,⍳⍵} where it only handled Booleans before.
@Bubbler If there isn't, we must make one. APL and K will be tied on this.
You can think of this as representations of sets.
Given a total set, e.g. 'abcdef' you can think of 1 3 5 and 1 0 1 0 1 0 as representation of a subset thereof.
Then 1 4 4 would be a representation of a multi-set.
The corresponding "Boolean mask" representation of that would be 1 0 0 2 0 0.
A better name might be a "counts" representation, as it gives you the count of each possible member of the full set, in the subset.
then converts the counts representation to an indices representation.
⍸⍣¯1 does the inverse transformation (save for leaving out any trailing 0s).
Any questions on the extension of ?
Great.
I won't go over the extensions to ↑[ax] because it really requires visuals. I'll show that in my next webinar.
Instead, let's go over the news in system functions for the remainder of this Cultivation.
We used ⎕JSON to visualise arrays here, but what can we do about matrices like ⋄ ⎕JSON 2 3⍴⍳6 and other high-rank arrays?
Aug 25, 2020 3:35 PM
@Adám DOMAIN ERROR
Exactly. JSON can't really represent those.
However, when speaking with the world outside, we probably want our matrices to be converted to lists of lists: ⋄ ⎕JSON⍠'HighRank' 'Split' ⊢ 2 3⍴⍳6
@Adám [[1,2,3],[4,5,6]]
⋄ ⎕JSON⍠'HighRank' 'Split' ⍳2 2 2
@Bubbler [[[[1,1,1],[1,1,2]],[[1,2,1],[1,2,2]]],[[[2,1,1],[2,1,2]],[[2,2,1],[2,2,2]]]]
Cool.
Aug 25, 2020 3:38 PM
Of course, this works universally, recursing even into namespaces: ⋄ mat←⍳2 3 ⋄ cube←2 2 2⍴2 ⋄ ⎕JSON⍠'HighRank' 'Split'⎕NS'mat' 'cube'
@Adám Illegal code
Oh, yes, we don't allow namespaces that simply, do we now?
      mat←⍳2 3 ⋄ cube←2 2 2⍴2 ⋄ ⎕JSON⍠'HighRank' 'Split'⎕NS'mat' 'cube'
{"cube":[[[2,2],[2,2]],[[2,2],[2,2]]],"mat":[[[1,1],[1,2],[1,3]],[[2,1],[2,2],[2,3]]]}
Oh, here's a challenge for APL hackers: Make the bot give you access to a namespace!
Anyway…
⋄ (dfns←⎕JSON '{}').⎕CY'dfns' ⋄ dfns.disp (1 2)3(4 5)
@Bubbler
Illegal code
VALUE ERROR
⋄ ⎕JSON '{}'
Aug 25, 2020 3:43 PM
@Bubbler ⎕CY is not allowed either (for quite obvious reasons).
@Bubbler #.[JSON object]
Nice.
⋄ ⎕SE
@Bubbler Illegal code
Aug 25, 2020 3:45 PM
There are a couple of other ways too.
Another thing that ⎕JSON can now do is to understand and create JSON5: ⋄ (ns←⎕JSON⍠'Dialect' 'JSON5'⊢'{noQuotes: [0xdecaf,0xC0FFEE] /* comment */}').noQuotes ⋄ ⎕JSON⍠'Dialect' 'JSON5'⊢ns
@Adám
912559 12648430
{noQuotes:[912559,12648430]}
Maybe most importantly, JSON5 allows trailing commas in lists and objects: ⋄ ⎕JSON⍠'Dialect' 'JSON5'⍠'Compact'0⍳3
@Adám
[
  1,
  2,
  3,
]
Compare with ⋄ ⎕JSON⍠'Dialect' 'JSON'⍠'Compact'0⍳3
@Adám
[
  1,
  2,
  3
]
RGS
RGS
Aug 25, 2020 3:50 PM
That's interesting
How so?
RGS
RGS
@Adám I am saying that the trailing commas now being allowed is interesting; I'm used to them, that is all :)
You're used to them from JavaScript. JSON doesn't allow them.
RGS
RGS
@Adám *Python; I know JSON didn't allow them
Ah, fair enough.
So, if there are no more questions, I'll close for today.
RGS
RGS
Aug 25, 2020 3:55 PM
@Adám Thank you very much for your time ○/
Of course. Let's start thinking about what else we might want to see in lessons like these, or if we should bring the season to a close.
Thank you for participating!
 
Conversation ended Aug 25, 2020 at 15:55.