Welcome to APL Cultivation

Last week, we used a user command ]RunTime to compare the speed of two otherwise equivalent expressions.

Today, let's talk about user commands then.

You may remember system commands like )save and )clear and )off.

The system commands are an integral part of the interpreter (and have been so for a very long time). That is, for Dyalog APL, they are written in C.

System commands are not APL functions, but rather a way to directly interact with the system. Thus, they do not follow APL syntax. Instead, they act more like commands on a command line. That's why they're called commands.

Sometimes, this non-syntactic way is really useful tor day-to-day stuff, and you'd want that for your APL code as well.

In comes user commands. They have exactly the same syntax model as system commands, they just begin with a ] instead of a ).

The only thing built into the interpreter is that whenever it sees a line in the session beginning with ] it takes the rest of that line and calls ⎕SE.UCMD with the line as a character vector argument.

@TechnicalMachine I'll get you write access in a moment.

Now, Dyalog APL comes pre-installed with a "user command processor", i.e. a function ⎕SE.UCMD which takes care of the rest. The default user command system is tightly integrated with SALT, but you could write your own drop-in, should you with to do so.

@TechnicalMachine You should have write access now.

Dyalog APL also comes loaded with more than 80 pre-defined user commands, some are simple and complex. All are written in APL, and you can change them as you see fit.

]?

@Adám

74 commands:

 ARRAY         Compare  Edit
 CALC          Factors  FromHex  PivotTable  ToHex
 EXPERIMENTAL  DBuild  DTest
 FILE          CD  Collect  Compare  Edit  Find  Replace  Split  ToLarge  ToQuadTS  Touch
 FN            Align  Calls  Compare  Defs  DInput  Latest  ReorderLocals
 NS            ScriptUpdate  Summary  Xref
 OUTPUT        Box  Boxing  Disp  Display  Find  Format  Layout  Rows
 PERFORMANCE   Profile  RunTime  SpaceNeeded
 SALT          Clean  Compare  List  Load  Refresh  RemoveVersions  Save  Settings  Snap

Let's have a look at some of them.

There are simple things like ]CD:

]CD

@Adám
/home/runner

Oh, you can cd from the REPL? Cool

And a diff tool called ]file.compare. Btw, user commands are case-insensitive just like system commands.

One really cool command is ]DInput. If you've ever wanted to enter or paste a multi-line statement into the session, you'll like this one.

What is a multi-line statement? Remember that you don't have to assign dfns before you use them; you can insert them inline. And dfns may have multiple lines. Effectively, you then have a single multi-line statement.

Now, as soon as you press Enter in the session, you code will be executed, and if it has any un-closed braces, e.g. 2+{a←⍳10 it will fail.

@Adám hm, imagining multi-line dfns among other stuff just smells like rotten bad practice from 500 km away

However, if you enter ]dinput you will get a new prompt indicated by a dot · and then you can begin entering (or pasting) multi-line statements. ]DInput will keep track of your brace-nesting level and indicate it with more dots.

@EriktheOutgolfer Why? What's wrong with:

That's also really nice

2+{
    a←⍳10
    ⍵,¨a(⌽a)
    }3 4

@EriktheOutgolfer ?

@Adám uh, yuck, at least in my perspective, that's godawfully misorganized

just imagine a dfn with 8 lines and 30 tokens outside the dfn, both left and right of it

@EriktheOutgolfer Well, you don't have to use it. You can also just type ]dinput f← and then paste a multi-line dfn there, beginning on that line. That'll define it in the workspace.

@Adám didn't say I have to (btw APL would really benefit from a line continuation character)

@EriktheOutgolfer Well, we are considering that, but 1) do you really want people to write overly long APL lines, and 2) that means executing from the bottom and up!

@EriktheOutgolfer However, my proposed array notation will allow you to split long arrays over multiple lines.

There are also various code analysis tools, like ]Calls. It will produce a calling tree:

]Calls  getEnvir  ⎕se.SALTUtils

@Adám
Level 1:  →getEnvir
  F:rlb               F:splitOn           F:splitOn1st        F:GetUnicodeFile    F:SALTsetFile       F:UnixFileExists

Level 2: getEnvir→UnixFileExists

Level 2: getEnvir→SALTsetFile

Level 2: getEnvir→GetUnicodeFile
⍝ Read a Unicode (UTF-8 or even UCS-2) file
⍝ This version allows excluding specific 1-byte characters before the translation
⍝ This prevents TRANSLATION errors in classic interpreters
  F:condEncl     F:numReplace   F:Special      F:Uxxxx

Level 3: GetUnicodeFile→condEncl

@Adám yikes no, in my humble opinion that would be a good practice only for assigning literals to variables

@EriktheOutgolfer Well, then my array notation should do, right?

This says that the getEnvir function in ⎕SE.SALTUtils calls these six functions, which in turn call the other listed functions, each at its level. This is really useful if you're trying to extract some utility function and need to know its dependencies.

A workspace stores information about each function; who was it last modified by, and when. This information can also be saved in script files with ]Save if you turn on "atinfo tracking". You can turn turn that on with ]Settings track atinfo.

Then you can list which functions were recently modified: ]Latest 20180501 -by=Fred

track isn't the only setting:

]Settings

@Adám
┌─────────────┬──────────────────────────────────────────────────────┐
│compare      │APL                                                   │
├─────────────┼──────────────────────────────────────────────────────┤
│cmddir       │/opt/mdyalog/16.0/64/unicode/SALT/spice:[HOME]/MyUCMDs│
├─────────────┼──────────────────────────────────────────────────────┤
│debug        │0                                                     │
├─────────────┼──────────────────────────────────────────────────────┤

oh boy

These are basically like OS environment variables, but used just by SALT.

E.g. edprompt determines if the editor should ask you before writing changes to scripted items back to their source file.

And varfmt determines how ]Save should save variables; as XML or as APL statements that produce the value.

cmddir tells SALT where to look for user commands. As you can gather, you can just drop your own or downloaded user commands into the mentioned /MyUCMDs dir and you're in business. Watch my webinar next week about how to write your own user commands!

If you've ever written anything moderately complex as a tradfn, you may have been annoyed that, as you edit along, your list of local variables on the header line is not neatly ordered. ]ReorderLocals allows you to sort the header row of all (or some of) the functions currently in the workspace: ]reorderlocals MyFn or ]reorderlocals F* or just ]reorderlocals

@Adám what do you mean by "neatly ordered"?

@EriktheOutgolfer Alphabetically.

Sorted alphabetically, or by first occurence? (or some other order)

@H.PWiz Sorting by first occurrence is difficult, e.g. with ⍎⍕(,¨⎕A)'←⍳26'

If you're on Windows, you have a few goodies special for you.

It is soon time to upgrade from 16.0 to 17.0, but now you've spent a whole year customising 16.0 to your liking. One of the first projects I did for Dyalog was writing a tool that allowed you to easily migrate your settings between versions:

]CopyReg 16u64 -to=17u64 does the job (you may need admin privileges, though).

At this point, I should mention that all this user commands have a whole host of options which you can specify with various arguments or modifiers. It would be too much to go into details about it all, but you can always get documentation about any user command with ]?cmdname:

]?calls

@Adám
Command "FN.Calls". Syntax: 1-2 arguments (last arguments merged).
Accepts modifiers -details -file -full -isolate[=] -treeview

Arg: pgmname namespace ; Produces the calling tree of a program (arg 1) in a class (arg 2, default current namespace)

]??Calls ⍝ for more info and examples

Script location: "/opt/mdyalog/16.0/64/unicode/SALT/spice/Summary.dyalog"

Actually, from 17.0 we prefer ]cmdname -? but the old syntax will continue to work. The benefit of using -? instead of a leading ? is that auto-completion will work.

@Adám and, uh, isn't ]?cmdname actually a separate command than ]cmdname? or are leading ?s special syntax?

@EriktheOutgolfer User commands must have valid identifier names, and the ]? syntax is handled by the user command processor. Yet another reason to move to -?…

Now that we are talking about the special syntax of user commands, the command processor has another few tricks up its sleeve.

@Adám and will that -? be a "default" for the user commands built into Dyalog?

because, well, what if a command uses -help instead?

@EriktheOutgolfer The -? is also handled by the processor, so no change is required in any existing command to handle that.

@EriktheOutgolfer Watch the webinar to learn how the user command help system works.

If for some reason you want to capture the result of a user command, you can do so with ]varname←cmdname

And if you want to silence a user command, you can do that with ]←cmdname.

Remember that I said everything after the ] is passed as argument to ⎕SE.UCMD? That means that you can even call user command under program control: ⎕SE.UCMD 'cmdname'.

And of course, anything else you'd write on the line just goes inside that character vector.

In fact, that's how I made the bot handle user commands.

There are also commands that let you get an overview of things:

]Summary ⎕se.Parser

@Adám
 Name        Scope   Size  Syntax
 Parse       P      17112   r1f
 Propagate           2728   r2f
 Quotes              2240   r1f
 Switch              2600   r2f
 deQuote             1496   r1f
 fixCase              120   r2f
 if                    48   r2f
 init        PC     14024   n1f
 splitParms          3384   r1f
 sqz                13160   r2f
 upperCase          13248   r2f
 xCut               12936   r2f

This analyses the ⎕SE.Parser class and tells you a little bit about each function. P means public, C constructor, and the syntax is whether they have a result, number of arguments, and type (function/monadic operator/dyadic operator).

]Xref will produce a cross reference of all items in a namespace, which ones call or reference which, how they do so (global/local) and what type they all are.

You may already know about ]Box. It is responsible for that nice boxed output you see from the bot and on TryAPL. You can turn that on and off, and decide exactly how you want it to display things with the user command. For now, let's just see what the current settings are by the bot:

]box ?

@Adám
]boxing ON -style=min -trains=tree -fns=on

There is a lesser known, but very useful, companion to ]box called ]rows. I'm sure by now you've entered a statement that caused way too much output, so your session would just scroll and scroll. Right?

@Adám aaaaargh!!!

Well, the ]Rows user command can protect you against that but limiting output to the current height and width of your window.

]Rows ?

@Adám
]rows OFF -style=long -fns=off -dots=· -fold=off

So if you do ]rows on -fold=3 it will cut any output four lines before the bottom of your screen, insert a row of dots (or whichever character you choose, e.g. ]rows on -fold=3 -dots=~) and then display the last three lines of the output.

It will then also (by default) not wrap lines that are too long, but rather will cause them to continue beyond the right edge of the screen (scroll horizontally to see it).

Again, see ]?box and ]?rows for the full details.

If you prefer boxing off during normal work, but want to display some results boxed here and there, you can use ]disp and ]display for that. ]disp is much like ]box -style=mid and ]display is like ]box -style=max.

As you saw above, the bot uses -style=min, but that doesn't always give you enough information:

⎕←2 3⍴'' (⍳3) (0 0⍴0) 'a'

@Adám
┌─┬─────┬─────┐
│ │1 2 3│     │
├─┼─────┼─────┤
│a│     │1 2 3│
└─┴─────┴─────┘

OK, we've go three empty (or are they filled with spaces?) elements. But what are they really?

]display 2 3⍴'' (⍳3) (0 0⍴0) 'a'

@Adám
VALUE ERROR

Oh, of course, limitation in the bot's handling of user commands.

]display 2 3⍴'''' (⍳3) (0 0⍴0) ''a''

@Adám
┌→────────────────────┐
↓ ┌⊖┐ ┌→────┐ ┌⊖┐     │
│ │ │ │1 2 3│ ⌽0│     │
│ └─┘ └~────┘ └~┘     │
│     ┌⊖┐     ┌→────┐ │
│ a   │ │     │1 2 3│ │
│ -   └─┘     └~────┘ │
└∊────────────────────┘

(Don't actually double your quotes in real life!)

Now we can see what exactly each thing is; we've got two empty character vectors and one 0-by-0 numeric matrix. We can also see that the a is a scalar, and the 1 2 3s are vectors (not e.g. one-row matrices).

Then there are of course all the user commands which are just covers for SALT functions, like ]Save and ]Load.

You probably want to keep you code in text files so you can keep it on GitHub.

Let's say you've written a nice function myfn. Save it to a text file with ]save myfn /path/ (it will be called myfn.dyalog).

Load it back in with ]Load /path/myfn

After you save or load your function like this, if you edit the function using the editor in APL, SALT will offer to save your changes to the associated file.

@Adám aw, how kind

The same goes of course for namespaces and classes. Even character matrices, vectors and vectors of vectors.

haven't ever been in any other IDE which has ever offered that

@EriktheOutgolfer Well, as mentioned, you can ask it to shut up forever and always save: ]settings edprompt 0 -permanent

Oh, I should mention that if you change your ]boxing or ]rows settings, you can keep your settings by saving the session or just add the settings you want to a function called Setup which must be stored (e.g. using ]save Satup /path/setup) in MyUCMDs/setup.dyalog in your documents directory or home folder (depending on OS).

OK, let's stop here for now. Tune in next week for more useful user commands, and then the webinar on how to make your own on Thursday next week.