Conversation started Apr 11, 2018 at 17:30.
Apr 11, 2018 5:30 PM
Welcome to the APL Cultivation!
2 and a half m...inutes
This week, we'll speak about function application, and how to get a better grip of that.
@EriktheOutgolfer What?
@Adám like, that many minutes earlier than 18:30+0, and also looks like you didn't get the pun
Next week, we can conveniently speak about multithreading using APL threads, as there is a webinar the day after on multithreading using OS threads.
@EriktheOutgolfer It sure wasn't early according to my computer clock.
@Adám mine is earlier than many other clocks tbf...
Apr 11, 2018 5:33 PM
So, some operators apply (to) their operands in intricate ways. How do you get a clearer picture of what they actually do?
Let's take outer product ∘.f as an example.
⎕←10 20 30∘.×1 2 3 4
@Adám
10 20 30  40
20 40 60  80
30 60 90 120
Sure, ok, but what actually happened?
It may seem simple, bu what about:
⎕←(3 2⍴10×⍳6)∘.×(2 4⍴⍳8)
@Adám
 10  20  30  40
 50  60  70  80

 20  40  60  80
100 120 140 160


 30  60  90 120
150 180 210 240

 40  80 120 160
200 240 280 320


 50 100 150 200
250 300 350 400

 60 120 180 240
300 360 420 480
What exactly got paired up with what?
Here's a trick you can use to analyse derived functions, that is both functions modified by operators and all tacit functions in general.
Let's replace the function (the operand) with a function which doesn't actually do the computation, but rather tells us what the computation would be:
⍞←10 20 30{'(',⍺,'×',⍵,')'}1 2 3
@Adám ( 10 20 30 × 1 2 3 )
Apr 11, 2018 5:41 PM
× is scalar. We can model that too:
⍞←10 20 30{⍺{'(',⍺,'×',⍵,')'}¨⍵}1 2 3
@Adám  ( 10 × 1 )  ( 20 × 2 )  ( 30 × 3 )
⎕←(3 2⍴10×⍳6)∘.{⍺{'(',⍺,'×',⍵,')'}¨⍵}(2 4⍴⍳8)
@Adám
┌────────────┬────────────┬────────────┬────────────┐
│┌──────────┐│┌──────────┐│┌──────────┐│┌──────────┐│
││( 10 × 1 )│││( 10 × 2 )│││( 10 × 3 )│││( 10 × 4 )││
│└──────────┘│└──────────┘│└──────────┘│└──────────┘│
├────────────┼────────────┼────────────┼────────────┤
│┌──────────┐│┌──────────┐│┌──────────┐│┌──────────┐│
││( 10 × 5 )│││( 10 × 6 )│││( 10 × 7 )│││( 10 × 8 )││
│└──────────┘│└──────────┘│└──────────┘│└──────────┘│
└────────────┴────────────┴────────────┴────────────┘
┌────────────┬────────────┬────────────┬────────────┐
Now we can see what's going on!
Even better if we use indices as arguments:
@Adám yup, I had that kinda figured out from seeing it before
that's a cool trick to show what's happening, btw
Apr 11, 2018 5:47 PM
⎕←({⊂'⍺[',(⍕1⊃⍵),';',(⍕2⊃⍵),']'}¨⍳2 3)∘.{⍺{'(',⍺,'×',⍵,')'}¨⍵}({⊂'⍺[',(⍕1⊃⍵),';',(⍕2⊃⍵),']'}¨⍳2 4)
@Adám
┌─────────────────┬─────────────────┬─────────────────┬─────────────────┐
│┌───────────────┐│┌───────────────┐│┌───────────────┐│┌───────────────┐│
││(⍺[1;1]×⍺[1;1])│││(⍺[1;1]×⍺[1;2])│││(⍺[1;1]×⍺[1;3])│││(⍺[1;1]×⍺[1;4])││
│└───────────────┘│└───────────────┘│└───────────────┘│└───────────────┘│
├─────────────────┼─────────────────┼─────────────────┼─────────────────┤
│┌───────────────┐│┌───────────────┐│┌───────────────┐│┌───────────────┐│
││(⍺[1;1]×⍺[2;1])│││(⍺[1;1]×⍺[2;2])│││(⍺[1;1]×⍺[2;3])│││(⍺[1;1]×⍺[2;4])││
Oops, that right-side should have been an , but you get the idea.
We can make an "eXplanation" operator: X←{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'}
How does it work?
First it captures its operand ⍺⍺ as f, then it makes into identity which is a common trick to make ambivalent functions. Finally, it strings together the left arg, the function character representation, and the right arg.
⎕←X←{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'} ⋄ ⎕←'abc'∘.(×X)'DEF'
@Adám
{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'}
┌─────┬─────┬─────┐
│(a×D)│(a×E)│(a×F)│
├─────┼─────┼─────┤
│(b×D)│(b×E)│(b×F)│
├─────┼─────┼─────┤
│(c×D)│(c×E)│(c×F)│
└─────┴─────┴─────┘
OK, now that we have a grip on ∘.f, let's look at f.g.
⎕←X←{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'} ⋄ ⎕←'abc'(+X).(×X)'DEF'
@Adám
{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'}
┌─────────────────────┐
│((a×D)+((b×E)+(c×F)))│
└─────────────────────┘
Apr 11, 2018 5:53 PM
The result is enclosed which shows us that if the arguments are vectors (as in this case) then the result is a scalar.
What happens with higher-rank arguments?
Something cool, I assume :p
⎕←X←{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'} ⋄ ⎕←'abc'(+X).(×X)(3 2⍴'DEFGHI')
@Adám
{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'}
┌─────────────────────┬─────────────────────┐
│((a×D)+((b×F)+(c×H)))│((a×E)+((b×G)+(c×I)))│
└─────────────────────┴─────────────────────┘
The left argument was a 3-element vector and the right argument a 3-by-2 matrix.
We can see how the left argument cells were distributed to the right argument cells.
⎕←X←{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'} ⋄ ⎕←(2 3⍴'abcdef')(+X).(×X)3 2⍴'DEFGHI'
@Adám
{f←⍺⍺ ⋄ ⍺←⊢ ⋄ '(',⍺,(⎕CR'f'),⍵,')'}
┌─────────────────────┬─────────────────────┐
│((a×D)+((b×F)+(c×H)))│((a×E)+((b×G)+(c×I)))│
├─────────────────────┼─────────────────────┤
│((d×D)+((e×F)+(f×H)))│((d×E)+((e×G)+(f×I)))│
└─────────────────────┴─────────────────────┘
Apr 11, 2018 5:56 PM
OK, now it is getting more interesting. The left arg was 2 3⍴ and the right was 3 2⍴.
The result became 2 2⍴.
In fact, the rule is that f.g removes the last axis of the left argument and the first axis of the right argument, so the result has the shape (¯1↓⍴⍺),(1↓⍴⍵).
well it makes sense when you look at it, but I couldn't tell it just by seeing the function work
@J.Sallé So does this illustration help clarify?
@DyalogAPL this one certainly does.
@DyalogAPL :O matrix multiplication
So if the left arg is shape 2 4 3 and the right arg is 3 5 1 the result should be shape 2 4 5 1:
Apr 11, 2018 6:00 PM
@Cowsquack no, dot product
⍞←⍴(2 4 3⍴0)+.×(3 5 1⍴0)
@Adám 2 4 5 1
(+.× is pretty standard)
@EriktheOutgolfer No, both.
Now let's go back to ∘.f for a moment. What is the rule about the shape of the result of that?
@Adám ah, never did I picture them as related like that before
Apr 11, 2018 6:01 PM
@EriktheOutgolfer APL opens the mind.
yeah on vectors it's dot product and on matrices it's matrix multiplication
The people who designed APL were very clever, and saw parallels nobody else saw.
⍞←⍴(2 4 3⍴0)∘.×(3 5 1⍴0)
@Adám 2 4 3 3 5 1
So the shape of ∘.f is (⍴⍺),(⍴⍵). ∘.f and f.g are definitely related!
The plot thickens
Apr 11, 2018 6:06 PM
In fact, Iverson suggested that the slightly anomalous in ∘.f be replaced with a number that indicates how many axes to combine. This way 0.f would be ∘.f. However, there is a more general alternative.
.
The powerful operator that few seem to get a proper grasp of. Let's explore it!
@Adám because it's a syntax error, no?
@EriktheOutgolfer No, it happens to be a syntax error on its own, but I meant that few know how to use it properly.
@Adám I certainly don't. I don't think I've ever even used that without raising errors >.>
I'll use a slightly modified X now.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ (⊂X)2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
Apr 11, 2018 6:14 PM
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  ABCD )
(      EFGH )
(      IJKL )
(           )
(      MNOP )
(      QRST )
(      UVWX )
OK, this just shows enclosing the rank-3 alphabet.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)⍤¯1⊢2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  ABCD )
(      EFGH )
(      IJKL )

(   ⊂  MNOP )
(      QRST )
(      UVWX )
Let's begin with negative rank, which is often what you really want.
f⍤¯N ⊢ B applies the function to cells of rank (≢⍴B)-N.
So in this case the array had rank 3, and the function was applied to sub-arrays of rank 3-1, that is 2, i.e. matrices.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)⍤¯2⊢2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  ABCD )
(   ⊂  EFGH )
(   ⊂  IJKL )

(   ⊂  MNOP )
(   ⊂  QRST )
(   ⊂  UVWX )
Apr 11, 2018 6:18 PM
And here, the function was applied to sub-arrays of rank 3-2, that is 1, i.e. vectors.
Still following? Now lets try positive rank.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)⍤1⊢2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  ABCD )
(   ⊂  EFGH )
(   ⊂  IJKL )

(   ⊂  MNOP )
(   ⊂  QRST )
(   ⊂  UVWX )
f⍤N apples the function to sub-arrays of rank N. So f⍤1 digs in until it finds vectors.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)⍤2⊢2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  ABCD )
(      EFGH )
(      IJKL )

(   ⊂  MNOP )
(      QRST )
(      UVWX )
So too does ⍤2 apply the function to matrices.
What about ⍤0?
@Adám apply to every rank?
Apr 11, 2018 6:25 PM
It applies the function to sub-arrays of rank 0, i.e. scalars. obviously isn't a useful function on scalars, but some functions are, e.g.
Consider e.g. the nested array:
⎕←2 2⍴(2 3⍴⎕A)(3 2⍴⎕A)(2 2⍴⎕A)(3 3⍴⎕A)
@Adám
┌───┬───┐
│ABC│AB │
│DEF│CD │
│   │EF │
├───┼───┤
│AB │ABC│
│CD │DEF│
│   │GHI│
└───┴───┘
It has four scalars. We can apply on each scalar:
⎕←∊⍤0⊢2 2⍴(2 3⍴⎕A)(3 2⍴⎕A)(2 2⍴⎕A)(3 3⍴⎕A)
@Adám
ABCDEF
ABCDEF

ABCD
ABCDEFGHI
Ah, but notice the description: on each. In general, ⍤0 is the same as ¨:
⎕←∊¨2 2⍴(2 3⍴⎕A)(3 2⍴⎕A)(2 2⍴⎕A)(3 3⍴⎕A)
@Adám
┌──────┬─────────┐
│ABCDEF│ABCDEF   │
├──────┼─────────┤
│ABCD  │ABCDEFGHI│
└──────┴─────────┘
Apr 11, 2018 6:28 PM
Except "mixes" the results while ¨ encloses them.
⎕←↑∊¨2 2⍴(2 3⍴⎕A)(3 2⍴⎕A)(2 2⍴⎕A)(3 3⍴⎕A)
@Adám
ABCDEF
ABCDEF

ABCD
ABCDEFGHI
⎕←⊂∘∊⍤0⊢2 2⍴(2 3⍴⎕A)(3 2⍴⎕A)(2 2⍴⎕A)(3 3⍴⎕A)
@Adám
┌──────┬─────────┐
│ABCDEF│ABCDEF   │
├──────┼─────────┤
│ABCD  │ABCDEFGHI│
└──────┴─────────┘
Actually, rank can do more than just that, in a powerful way that ¨ cannot compare to.
The derived function can be applied dyadically.
@Adám heh, the absolute humiliation of ¨
Apr 11, 2018 6:31 PM
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(819⌶2 3 4⍴⎕A)(,X)⍤1⊢2 3 4⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
( abcd  ,  ABCD )
( efgh  ,  EFGH )
( ijkl  ,  IJKL )

( mnop  ,  MNOP )
( qrst  ,  QRST )
( uvwx  ,  UVWX )
I'm concatenating the rank-1 sub-arrays of the arguments.
Let's use different ranks for the left and right arguments!
@Adám oooh shiz that's nice
Left arg will be:
⎕←819⌶2 2⍴⎕A
@Adám
ab
cd
Apr 11, 2018 6:34 PM
And right arg:
⎕←2 2 2⍴⎕A
@Adám
AB
CD

EF
GH
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(819⌶2 2⍴⎕A)(,X)⍤1 2⊢2 2 2⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
( ab  ,  AB )
(        CD )

( cd  ,  EF )
(        GH )
So here, we are concatenating rank-1 sub-arrays of the left arg with rank-2 sub-arrays of the right arg:
⎕←(819⌶2 2⍴⎕A),⍤1 2⊢2 2 2⍴⎕A
@Adám
aAB
bCD

cEF
dGH
Apr 11, 2018 6:37 PM
Quiz: Now can anyone figure out what Y is so that f⍤Y is ∘.f?
Remember:
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(819⌶2 2⍴⎕A)∘.(,X)3 2⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
┌───────┬───────┐
│(a , A)│(a , B)│
├───────┼───────┤
│(a , C)│(a , D)│
├───────┼───────┤
│(a , E)│(a , F)│
└───────┴───────┘
┌───────┬───────┐
│(b , A)│(b , B)│
├───────┼───────┤
│(b , C)│(b , D)│
├───────┼───────┤
│(b , E)│(b , F)│
└───────┴───────┘

┌───────┬───────┐
│(c , A)│(c , B)│
├───────┼───────┤
│(c , C)│(c , D)│
├───────┼───────┤
│(c , E)│(c , F)│
└───────┴───────┘
┌───────┬───────┐
│(d , A)│(d , B)│
├───────┼───────┤
│(d , C)│(d , D)│
├───────┼───────┤
So notice that each scalar in got paired up with the entire .
@Adám 0 15?
@H.PWiz Yes, very nice!
In other words, we need the left rank to be 0 and the right rank to be infinite. But since Dyalog APL only allows arrays of up to rank 15, that is enough. (15 = ∞ for very small values of ∞.)
⍤N can also take a three-element N. That's only useful for ambivalent functions. It then means that if the derived function is applied monadically, it gets applied to sub-arrays of rank N[1] and if it is applied dyadically, it is applied to sub-arrays of rank N[2] of and of N[3] of .
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(⊂X)⍤1 2 0⊢2 2⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
(   ⊂  AB )
(   ⊂  CD )
Apr 11, 2018 6:45 PM
I.e. applies to rank-1 sub-arrays.
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'} ⋄ ⎕←(819⌶2 2⍴⎕A)(⊂X)⍤1 2 0⊢2 2⍴⎕A
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ '(',(⍕⍺(⎕CR'f')⍵),')'}
( ab  ⊂ A)
( cd     )

( ab  ⊂ B)
( cd     )


( ab  ⊂ C)
( cd     )

( ab  ⊂ D)
( cd     )
I.e. applies to rank-2s of (which happens to be the entire array here) and rank-0s of .
Finally, let's explore how f∘g works. Any ideas for illustrating that?
Well, f∘g applies differently depending on ⍺⍵ so I don't know how to illustrate it properly
How about (again with a slightly modified X):
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'} ⋄ ⎕←(,X)∘(⊂X)'⍵' ⋄ ⎕←'⍺'(,X)∘(⊂X)'⍵'
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'}
(,(⊂⍵))
(⍺,(⊂⍵))
Apr 11, 2018 6:55 PM
Here is an example of how we can use this to analyse more complex trains, like this CamelCase splitter:
⎕←(⊢⊂⍨∊∘⎕A)'CamelCaseRocks'
@Adám
┌─────┬────┬─────┐
│Camel│Case│Rocks│
└─────┴────┴─────┘
(I know the isn't necessary, but it is in there for illustration purposes.)
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'} ⋄ ⎕←(⊢X⊂X⍨∊∘⎕A X)'⍵'
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'}
((∊∘ABCDEFGHIJKLMNOPQRSTUVWXYZ⍵)⊂(⊢⍵))
So now we can see how works and how is distributed to the outer functions.
And for an even more complex train, which splits on any number of delimiters:
⎕←' ,;'(⊢⊆⍨∘~∊⍨)'some delimiters;in,use'
@Adám
┌────┬──────────┬──┬───┐
│some│delimiters│in│use│
└────┴──────────┴──┴───┘
Apr 11, 2018 7:00 PM
⎕←X←{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'} ⋄ ⎕←'⍺'((⊢X)(⊆X)⍨∘(~X)(∊X)⍨)'⍵'
@Adám
{f←⍺⍺ ⋄ ⍺←'' ⋄ ∊'('⍺(⎕CR'f')⍵')'}
((~(⍵∊⍺))⊆(⍺⊢⍵))
Now we just have to note the obvious that ⍺⊢⍵ is .
This should also explain why and can get you the arguments when in a train.
I hope this lesson was useful. See you next week for multi-APL threading!
 
Conversation ended Apr 11, 2018 at 19:01.