Man, writing depth guards a million times makes me really want the Depth operator :)

@B.Wilson Do you have some good example use cases?

@Adám Basically all the Phase 2 competition problems have obvious domain generalizations which can be expressed concisely with Rank and Depth. Generally, though, the one I have been bumping into most often lately is making functions agnostic to a particular presentation of text data: character vector with newlines, vs vector of line strings, vs character matrix.

@B.Wilson Well, then all functions, really. Even primitives. E.g ? would benefit from from ?⍥0 and ⌹ would benefit from ⌹⍤2⍥1.

@Adám Exactly!

@dzaima Fixed in 19.0

> Given a non-empty character vector of single-letter grades ABCDF, produce a 3-column, 5-row, alphabetically-sorted matrix of each grade, the number of occurrences of that grade, and the percentage (rounded to 1 decimal position) of the total number of occurrences of that grade. The table should have a row for each grade even if there are no occurrences of a grade.

I failed to solve this one in APL style. I did not succeed to take into account the grades which were not present (as in example 2 or 3). Also sorting was difficult. Or can you use an axis of the matrix to sort on?

⍋[1]?

You mean a column?

yes

I managed to solve it but not with ⌸ as suggested: 'ABCDF'∘(⊣,∘(⊢,∘⍪10÷⍨∘⌊0.5+1000×⊢÷+/)(+/∘.=))

@Richard You can't, but don't need to, as the first column has unique values.

@rabbitgrowth Yeah, this is another one of those cases where it'd be useful to supply a "vocabulary" to Key.

What do you mean by vocabulary?

Tell it which keys to expect, and in which order.

I hoped the left argument of ⌸ would be the arguments/elements that are going to be counted.

Like this: {(⊣,(100÷≢⍵)(⊢,1⍎⍤⍕×)¯1+≢⍤⊢)⌸'ABCDF',⍵}

@Richard But it isn't. That, if given, has the keys, with the right argument supplying data to be grouped.

But the right argument doesn't have all the data, or you should add it manualy

I'm not following.

(your_function) ,'B'

Oh, the vocabulary, you mean. Right.

In this case ⌸ does not know it also has to count ACDEF

sorry

I hoped the left argument could be used for that

dyadic use of ⌸ I mean

Ah, prepending ABCDF to ⍵ is clever. I've seen this pattern multiple times here but can never recall it when I need it

I have a proposal to use the operand as vocabulary (and then use {⊂⍵} as actual operand to Key). With that, we'd have just 'ABCDF'∘{⍺,c,⍪10÷⍨⌊0.5+1000×(c←≢¨⍺⌸⍵)÷≢⍵}

Which is usefull if you don't know if all the instances are present

@Adám I think I tried to say that.

and i should say operand and not argument, my fault

If I could start over, I'd make ⌸ a dyadic operator, taking a vocabulary as right operand, but allowing a function to compute the vocabulary too. Then current f⌸ would be f⌸∪.

I'm worried if I can use ⎕FMT for col 3

You can, but ⎕FMT gives a character matrix, so you'd have to convert to numbers.

Why not using the left operand as vocabulary? The use of the left operand sounds a littek useless now

Btw, my solutions were 'ABCDF'∘{s←+/⍺∘.=⍵ ⋄ ⍺,s,⍪⍎1⍕100×s÷≢⍵} and 'ABCDF'∘{s←+/⍺∘.=⍵ ⋄ ⍺,s,⍪10÷⍨⌊0.5+1000×s÷≢⍵}

@Richard That's exactly my proposal. But I wouldn't call the left operand useless. Yes, you can always use f¨{⊂⍵}⌸ but some common operands can be optimised and avoid the intermediary (potentially expensive to compute and represent) nested value.

@Adám that;s nice. Did this also show up during the contest?

Or were there also solutions with ⌸

Probably I was to much fixed on using ⌸

I don't recall. There probably were solutions with ⌸.

'ABCDF'∘{s←+/⍺∘.=⍵ ⋄ ⍺,s,⍪⍎1⍕100×s÷≢⍵} is nice, so readable

mine looks like tacit for tacit's sake now

Note that inserting the vocabulary can be expensive too, if the data is large, because the entire thing will need re-writing in memory.

So you should use it wisely, as many other operators

especially the 'boxed' ones

There's a different approach, where you instead look up each required value. Wanna try?

something with ⍸?

No, ⍳

yes exactly. always mix them up

@Adám I was a little surprised how much faster this was over using Key, but with such a small vocabulary I guess it is expected. Did some testing (my laptop, 1e7≥≢⍵) and it seems like +/⍺∘.=⍵ is faster than ¯1+{≢⍵}⌸⍺,⍵ for vocabularies up to length 8, after that Key is faster.

Even for massive data?

Something with ⍳ and ⍋?

I don't know what you call massive, but with 1e9 characters from 10↑⎕A the Key variant is a bit faster

This is probably the fastest: {t←{⍺,≢⍵}⌸⍵ ⋄ ↑(≢⍵){⍵,(⊢,∘⍎1⍕100×÷∘⍺)(⍵⍳⍨⊣/t)⊃0,⍨⊢/t}¨'ABCDF'}

It can be sped up even further by avoiding the inner loop, though.

Oh, and the arithmetic way to round is probably much faster than the nasty executute trick too. Code provided for illustration purposes.

The idea is that we let ⌸ do its thing with the highly optimised {⍺,≢⍵} operand, then pick out the data we need.

0,⍨ is provided as a fall-back count for elements not found in the result of Key.

Does this make sense to the assembled?

{v,[1.5]{{(1-⍨≢⍵)}⌸⍵}v⍳v←'ABCDF',⍵} 3 8 4 7/'ABCF'

and another column for he average

Right.

'ABCDF'∘{s←¯1+¯2-/(1++/≢¨⍺⍵),⍨⍺⍳⍨{⍵[⍋⍵]}⍺,⍵ ⋄ ⍺,s,⍪⍎1⍕100×s÷≢⍵}

lol

Yikes.

Yeah, totally went in the wrong direction lol

Selfie!

Shall we round this up before anyone gets worse ideas? :-)

:)

See you next week for 2019-4: Knight Moves.

One question. How to sort on the first column of a matrix?

or any axis of an arrya

why not ⍋[x]

axis column.

Did you search APLcart for it?

no ... :(

I know I should order your T-shirt

Remember that sorting is just indexing using the grade. If you want to grade a specific column, select that first: M[⍋M[;x];]

But than still. Is ⍋[x] strange?

Yes. It'd be the axis operator, but not actually having anything to do with axes.

@Adám o yes

Also, how would it generalise to higher ranks?

not... ok:)

And what if you want to grade a normalised form of a specific column? E.g. sort by absolute value of the second column: M[⍋|M[;2];]

{⍵[⍋⍵]}⍢↓M ? :)

@rabbitgrowth my hero! :)

These things will be a bit nicer with Select in 20.0: M⊇⍨⍋x⊇⍉M

@rabbitgrowth That sorts each row, equivalent to {⍵[⍋⍵]}⍤1

Does it?

      ⎕←M←?5 5⍴10
5 1 2  5 10
1 7 9  6  4
7 6 5  5  8
2 9 9  2 10
9 4 2 10  9
      ↑{⍵[⍋⍵]}↓M
1 7 9  6  4
2 9 9  2 10
5 1 2  5 10
7 6 5  5  8
9 4 2 10  9
      {⍵[⍋⍵]}⍤1⊢M
1 2 5 5 10
1 4 6 7  9
5 5 6 7  8
2 2 9 9 10
2 4 9 9 10

Oh, my bad. {⍵[⍋⍵]}⍢↓ is the same as {⍵[⍋⍵;]}

If I'm not mistaken n ⍋⍤⊇∘⍉⍛⊇ M will sort M by the nth column in 20.0

Is that the same ⊇ as in github.com/abrudz/dyalog_vision/blob/main/… ?