Ah, another good example of your key with vocabulary!

+/'ACGT'∘.=,

@Silas Yes, I was just thinking that.

{¯1+⊢∘≢⌸'ACGT',⍵}?

Yes, indeed.

@rabbitgrowth yes tthat was mine also. Only took some time to figure out the ,

@rabbitgrowth You can make this one tacit.

@rabbitgrowth that was my initial solution for ∘.= :) Adding another row of 'ACGT' and later substracting 1

@Adám ¯1+(⊢∘≢⌸'ACGT'∘,)?

Sure, but you could avoid the paren.

I hope the recording of my Dyalog '23 talk on this subject will soon be published. Meanwhile, my slides are online.

(Without looking at my slides), anyone up for doing it without the potentially expensive 'ACGT',?

@Adám Hm, not seeing how at the moment

Go back to ¯1+⊢∘≢⌸'ACGT',⍵ and notice how ⊢∘≢⌸ is applied to the result of…

@Adám while still using ⌸?

Yes!

think here that brackets give nicer expression

@Adám trying

@Adám Sorry, I still don't see it. Does the solution look like ⊢∘≢⌸ (array) (function) 'ACGT'∘,?

No. Another hint: Replace 'ACGT'∘, with a fork

@Richard The idea is to remember which count is for which letter, then look for the required counts.

@rabbitgrowth oh, thought just removing brackets from here

No, that won't work because ⊢∘≢⌸ is monadic

ah, I'd got the ugly ¯1∘+⍤⊢∘≢⌸'ACGT'∘, working

Wait, what‽

???

Oh, that's clever!

get the counts including additions and then subtract 1

@Adám 'ACGT',⊢? Not seeing how that would help...

¯1+'ACGT'⊢∘≢⌸⍤,⊢

ahhh

I see

OK, now we're just waiting for the efficient solution.

I still would like you can choose your indexed items with the left argument of ⌸

That's my proposal. With it, the solution would be ≢¨'ACGT'⌸

exactly

How about +/1 2 3 4∘.='ACGT'∘⍸⍤, ?

@rabbitgrowth Building on the prior solutions, I worked out 'ACGT'+.=¨⊂

Actually, to handle a scalar, we'd need ⍤, but whatever.

@Silas That's just as expensive, and ⍸ should just be ⍳ here.

Ah, thought ⍸ tended to be better than ⍳

using sort?

no

That would allow you to append rather than prepend, which is indeed faster if there are no other references to the input, but that often wouldn't be the case.

No ideas left

@mitchelljohnstone you can do 'ACGT'+.=⍤0 1⊢ not sure if better though

Feels like a brute force implementation of key, but what about 'ACGT'≢⍤∩⍨¨⊂?

Oof.

I wrote down 'ACGT'≢⍤∩¨⍨⊂ as well

Never sure whether to use ¨⍨ or ⍨¨

Is ¨⍨ technically faster because you only swap once?

OK, so the way we can do this is to make ⌸ return pairs of (base count), then collect the found bases (which can be in any order, and might be missing bases) and their counts separately. Now we can look up 'ACGT' in the found bases, and use those positions to index into the counts, appending 0 to the counts for when a base isn't found.

@rabbitgrowth It was originally so, but I believe we've optimised that away.

{{(0,⍨⊢/⍵)['ACGT'⍳⍨⊣/⍵]},∘≢⌸,⍵}

@Silas In that, what do the 0 1 terms do to the rank operation?

@rabbitgrowth Yup. I had {(v,0)['ACGT'⍳⍨⊃(k v)←↓⍉,∘≢⌸⍵]}

I like yours better, though. Wonder if we can make the inner dfn tacit.

@mitchelljohnstone compare the scalars(0) of ⍺ with the vectors(1) of ⍵

@Adám That seems a lot slower than {¯1+⊢∘≢⌸'ACGT',⍵}, though?

@Adám yep, f⍨¨ → f¨⍨

@rabbitgrowth It does. Hm.

@Adám ((0,⍨⊢/)⌷⍨∘⊂'ACGT'⍳⍨⊣/),∘≢⌸⍤,

Right.

I can't really explain the performance. I suspect some optimisation has been removed.

What're you comparing?

,∘≢⌸⍵ with ⊢∘≢⌸'ACGT',⍵

{⍺,≢⍵}⌸dna is a bit faster than ,∘≢⌸⍵ but still very slow.

Considering that the only difference is including the unique elements in the result, this makes no sense.

Btw, is there a reason to prefer ⊢∘≢ over ≢⍤⊢? ≢⍤⊢ seems much simpler to me, as it maps directly to {≢⍵}, but APLcart for example uses ⊢∘≢, and Aaron Hsu also uses ⊢∘≢ in his PhD thesis

Looking now, but I imagine it's C code vs APL magic function inside key

@rabbitgrowth Historical reasons. ⊢∘≢ has worked since ⌸ (and ≢) was introduced in 14.0, but ≢⍤⊢ which indeed makes more sense, only came in 18.0

@Silas I thought we optimised {⍺,≢⍵} and {⍺(≢⍵)} for ⌸

Ah, yes, 19.0:

  {⍺(≢⍵)}⌸dna    → 1.4E¯2 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
* ⊢∘≢⌸'ACGT',dna → 8.4E¯3 | -41% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕

18.0:

  {⍺(≢⍵)}⌸dna    → 6.8E¯3 |  0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
* ⊢∘≢⌸'ACGT',dna → 6.3E¯3 | -8% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕

  ⊢∘≢⌸dna → 1.2E¯3 |    0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  ≢⍤⊢⌸dna → 3.5E¯3 | +188% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕

Is ⊢∘≢ a recognized idiom or something? I can't find it in the idiom list

Not an idiom, but ⌸ recognises the specific operand.

What version is that?

18.2.45505

We should fix this. @Silas Taking notes?

Yep, I can mantis them.

OK, on this (s)low note, let's stop for here. See you next week for 2020-2: Attack of the Mutations!

@Silas Thank you!

Nice weekend everybody!