Welcome to APL Cultivation!

Thanks! \o/

My first!

Now, first task for today is to decide on a subject.

Candidates are: a closer look at some of the more advanced operators ⌺⌸⍠, looking at the dfns workspace, anything else you might want.

I vote ⌺⌸⍠

⌺⌸⍠

OK, well I guess we can begin with ⍠ since we've never explicitly covered it before.

⍠ is a dyadic operator, but it is quite unlike all other operators in APL.

Syntactically, it is normal though. It always takes a function (monadic or dyadic) on its left, and always takes an array on its right.

this is variant not domino, right?

@TomCockram Correct. Domino, ⌹, is a normal arithmetic function.

And as Tom said, it is usually called Variant, although you can also call it Option. In fact, it has a system operator synonym, ⎕OPT.

Ok. What does it do?

Variant is special in that it sets options in an invisible set of options. You can't access this set directly, only observe modified behaviour in the operand function, because the operand function will check this set to know what to do.

This also means that, uniquely, the operand function will "know" that it is being called as an operand of ⍠. Usually, functions can't really detect (easily) who called them.

The left operand (the function) must be one of a fixed set of system functions (or functions derived from system operators).

The scalar operand is only allowed if a default key exists, in which case it is equivalent to 'DefaultKey' value.

Enough talk. Let's take an example. You might know about the system function to convert to and from JSON:

⎕←⎕JSON⍳3

@Adám
[1,2,3]

We can use ⍠ with the key 'Compact' to change the white-space behaviour of ⎕JSON. In essence, ⍠ set's the Compact setting to the corresponding value (0 or 1 in this case):

⎕←⎕JSON⍠'Compact' 0 ⍳3

@Adám
[
  1,
  2,
  3
]

neat

Ah, I came across this with regexes a few days ago

@xpqz Right. I'll get to those.

There are other options too. E.g. normally ⎕JSON will convert a JavaScript null to an APL enclosed string ⊂'null':

⍞←(⊂'null') ≡ ⎕JSON'null'

@Adám 1

However, if you instead want it to convert it to an object-type null, ⎕NULL you can tell it so:

⍞←⎕NULL ≡ ⎕JSON⍠'Null' ⎕NULL ⊢ 'null'

@Adám 1

Notice the ⊢. Does anyone not know why I put it there?

Whenever a dyadic operator has an array right operand, it will strand together with any literal right argument. There must be a function (or parentheses, or naming) to split them apart.

Another option for ⎕JSON is to convert JSON into an APL matrix that describes the JSON, rather than attempting to actually convert to an equivalent APL structure:

⎕←⎕JSON⍠'Format' 'M' ⊢ '[1,null,"hello"]'

@Adám
┌─┬┬──────┬─┐
│0││      │2│
├─┼┼──────┼─┤
│1││1     │3│
├─┼┼──────┼─┤
│1││┌────┐│5│
│ │││null││ │
│ ││└────┘│ │
├─┼┼──────┼─┤
│1││hello │4│
└─┴┴──────┴─┘

The exact details of this Matrix Format isn't important here, though. We can deal with those another time, or you can check out the docs.

Now that we know about a couple of options, we can look at how to specify multiple options. We can create a "dictionary" of key-value pairs:

⎕←⎕JSON⍠('Format' 'M')('Null' ⎕NULL) ⊢ '[1,null,"hello"]'

@Adám
┌─┬┬──────┬─┐
│0││      │2│
├─┼┼──────┼─┤
│1││1     │3│
├─┼┼──────┼─┤
│1││[Null]│5│
├─┼┼──────┼─┤
│1││hello │4│
└─┴┴──────┴─┘

Notice how we both got a matrix, and the null became [Null] (the text representation of ⎕NULL) rather than an enclosed 'null'.

We can also use ⍠ twice:

⎕←⎕JSON⍠'Format' 'M'⍠'Null' ⎕NULL ⊢ '[1,null,"hello"]'

@Adám
┌─┬┬──────┬─┐
│0││      │2│
├─┼┼──────┼─┤
│1││1     │3│
├─┼┼──────┼─┤
│1││[Null]│5│
├─┼┼──────┼─┤
│1││hello │4│
└─┴┴──────┴─┘

And if we check the docs for ⎕JSON we'll see that 'Format' is the principal option, which means we can specify it as a scalar:

⎕←⎕JSON⍠'M'⍠'Null' ⎕NULL ⊢ '[1,null,"hello"]'

@Adám
┌─┬┬──────┬─┐
│0││      │2│
├─┼┼──────┼─┤
│1││1     │3│
├─┼┼──────┼─┤
│1││[Null]│5│
├─┼┼──────┼─┤
│1││hello │4│
└─┴┴──────┴─┘

What happens if we set the same option twice?

@all Anyone venture a guess?

Same as passing it once I'd hope.

Oh, I meant setting it to two different values.

According to the docs, the rightmost one takes precedence.

Indeed. There are two ways you can think of it, both leading to that same conclusion.

1) ⍠ (like any operator) modifies its operand function. For simplicity, lets say we have two monadic operators applied acting on a function, f op1 op2, op2 gets to modify the derived function f op1. That is, the rightmost has the final say.

2) When we evaluate, we first have to process the inner derived function's operator (as in the previous point), which sets the hidden option. Then we proceed to the outer operator, which in turn overwrites the state. Only then is the function allowed to run, picking up the setting set by the rightmost (outer) operator.

@all Is this all clear?

yes

/me nods

Let's use the regex example.

If you're not familiar with ⎕R: Briefly, it is a dyadic operator, Replacing occurrences of its left operand with its right operand, in the right argument:

So this is APL's version of passing optional config parameters to a function, basically?

Pretty much, but it only applies to system functions. You can't use it in general for your own APL functions.

There are other (better, I'd say) methods for that.

⍞←'s'⎕R'S' ⊢ mississippi'

@Adám SYNTAX ERROR

'quote

Oops, missing quote.

⍞←'s'⎕R'S' ⊢ 'mississippi'

@Adám miSSiSSippi

This replaces all lowercase s with uppercase S.

Let's say we only want to replace the first 2. We can set the Match Limit to 2.

@all Hey, let's make this an exercise! Try it: The option key is 'ML'.

No?

I'm trying lol

I have unsuccessful attempts

What have you got? No need to be embarrassed.

's'⎕R⍠('ML' 2)'S'⊢'mississippi'

syntax error

⍞←'s'⎕R'S'⍠'ML' 2 ⊢ 'mississippi'

@JPeroutek miSSissippi

Right, so the trick here is to realise what syntactic classes were dealing with.

Aha. That's where ⍵ goes

I was literally just sticking ⍠ 'ML' 2 onto the end:

⍞←'s'⎕R'S' ⊢ 'mississippi' ⍠'ML' 2

@xpqz ⍵⍵, if you want :-)

@TomCockram  s ⎕R S ⊢  mississippi ⍠  ML  2

OK, this is important to understand.

Remember, ⍠ is a "normal" operator.

This means that it needs to modify a function, before the function is applied.

Did I get it right?

@JPeroutek You did.

@JPeroutek - looks like :)

so bracketing (⍠ 'ML' 2)?

This is really no different than ¨ in +/¨(1 2 3)(4 5 6)

@TomCockram You can, but never have to.

⎕R is also an operator. It takes two operands, in our case 's' and 'S', and derives a new function.

It is this derived function that ⍠ needs to act upon by taking it as its left operand.

So the order is FunctionToBeModified ⍠ options ⊢ argument

Ah, hah! Which explains why the 'S' had to come before the ⍠

Exactly.

Alternatively, we can parenthesise: (FunctionToBeModified ⍠ options) argument

⍞←'s'⎕R'S'⊢'mississippi'(⍠'ML'⊢2)

@TomCockram SYNTAX ERROR

@TomCockram That won't work. You have MonadicOperator function array on the right.

Remember that a dyadic operator can bind with its right operand to form a monadic operator.

We would have to parenthesize if FunctionToBeModified was dyadic, correct?

@JeffZeitlin Only if the function to be modified itself is derived from an operator with an array left argument, as is the case here.

Oh I see, looking at the correct solution above. I'd need to set the ML and then have my argument 'mississippi'?

@TomCockram Yes, the modification of the function derives a new function, which is then applied.

You could also do something like this right

⍞←('s'⎕R'S'⍠ 'ML' 2) 'mississippi'

@JPeroutek miSSissippi

To eliminate the right tack

Yes.

Naming a derived monadic operator:

⋄ ReplaceWithS←⎕R'S' ⋄ 's'ReplaceWithS 'mississippi'

@Adám
miSSiSSippi

This also means we can name the combination of ⍠ with one or more options.

⋄ OnlyTwo←⍠'ML'2 ⋄ 's'⎕R'S'OnlyTwo 'mississippi'

Only2←⍠'ML'2 ?

@Adám
miSSissippi

GMTA :)

@JeffZeitlin Ninja'd.

We can even do both:

⋄ ReplaceWithS←⎕R'S' ⋄ OnlyTwo←⍠'ML'2 ⋄'s'ReplaceWithS OnlyTwo 'mississippi'

@Adám
miSSissippi

A really common thing with regexes is wanting case insensitivity. That is 'IC'1 (Ignore Case), but it is also the principal option:

⍞←'ss'⎕R'__'⍠1⊢'MISSissippi'

@Adám MI__i__ippi

So if the setting is the principal option, it does not have to be specified? Only the value?

"Principal option" meaning that if you specify only a value, it will be applied to that option?

Yes and yes.

But it only works if that is the only setting you're changing. Though, you can always use ⍠ twice:

⍞←'s'⎕R'_'⍠'ML'3⍠1⊢'MISSissippi'

@Adám MI__i_sippi

Is it generally considered 'better' to repeatedly use ⍠, or to provide a vector of parenthesized options to a single ⍠?

Ok, so quick question. What if I'm trying to set the principal option to a value that is equivalent to the name of another option? Like in the case of ⎕R, what if I tried to set 'IC' to the value of 'ML'

Obviously a non-sensical value

@JeffZeitlin Personal style. I like using one ⍠ for each options, because I dislike parentheses.

@JPeroutek You'll get an error and a message to that effect. The bot doesn't give detailed messages though. Try it online!

I assume that the documentation for the various system functions includes descriptions of applicable options for ⍠?

I see.

@JeffZeitlin They do.

Here is another example where we use ⍠ on ⎕R to do something entirely unrelated to regular expressions…

Sometimes, your input can be of various forms and you need to normalise it. Say you get some text, but it could be a character scalar, a character vector, a vector of character vectors, an enclosed character vector, or even a character vector with literal newlines.

So we want to normalise all of these to become a vector of character vectors.

⋄ VecOfVecs←''⎕R''⍠'ResultText' 'Nested'
⋄ VecOfVecs 'a'
⋄ VecOfVecs 'abc'
⋄ VecOfVecs 'abc' 'def'
⋄ VecOfVecs ⊂'abc'
⋄ VecOfVecs 'abc',(⎕UCS 10),'def'

@Adám
┌─┐
│a│
└─┘
┌───┐
│abc│
└───┘
┌───┬───┐
│abc│def│
└───┴───┘
┌───┐
│abc│
└───┘
┌───┬───┐
│abc│def│
└───┴───┘

See? :-)

Also note that we often add additional options to existing system functions based on customer demand.

E.g. in version 18.0 (due this summer) we're adding options to ⎕JSON to automatically split high-rank arrays so they can be represented as JSON, and an option to process and generate JSON5. And for ⎕R/⎕S we're adding an option to turn regexes off so you can do literal replacements without worrying about having to escape characters that have special meaning in PCRE.

If you have other requests for options to existing system functions/operators, feel free to email [email protected]

@all Any questions?

About this topic or something else? Because I have a question about TODO comments

@TomCockram About this. Can that wait 10 minutes until the lesson is over?

@Adám of course, no problem

I also have questions regarding other topics. I believe I understand Variant, at least in the contexts presented so far.

OK, so before we finish, I'll just briefly mention one more usage of ⍠ that isn't really related to this, and we can't try it online either.

When using external .NET methods, APL will coerce its arrays into an appropriate type for the called method.

However, .NET methods can be overloaded (different code depending on the type of the argument), and then APL can't know which one you want.

You can use ⍠ with the method and the option 'OverloadTypes' to choose. The value has to be a .NET data type, e.g. Double or Int32.

This option is the principal option too, so the calling can be done simply with MyDotNetMethod⍠Double ⊢ argument

If the method takes multiple arguments, you can specify a vector of types: MyDotNetMethod⍠(⊂Double Int32) ⊢ argument

If the .NET method is overloaded, does OverloadTypes have to be specified, or will it detect the argument types?

e.g., if there is an overload that takes an Int32, and one that takes a string, will DotNet 'foo' correctly invoke the string overload, and DotNet 42 invoke the int32?

@JeffZeitlin You don't have to. By default, APL will use the type matching the current type of the argument. E.g. if the method can do both Double and Int32, calling it with 42 will choose the integer option unless you specify otherwise.

In such cases, it can be dangerous to rely on the default, as APL will pack values into the smallest type it can, so 0.3+0.7 gives an integer!

⍞←⎕DR 0.3+0.7

@Adám 83

The digit 8 means 8 bits, and the digit 3 means integer.

OK, this concludes the lesson on ⍠