The APL Orchard

chrispsn

12:39

@Adám is this a new idea? what inspired the symbol choice?

Razetime

@chrispsn not that new, it's been in extended for a while

chrispsn

@Razetime ah ty

dzaima

interestingly enough, ⍛ first appeared as atop. Then, (ignoring this,) this, adding it to Extended (and two days later, dzaima/APL)

chrispsn

13:00

@dzaima thanks! love seeing the seed of an idea and how it changed over time

dzaima

@dzaima (fwiw, ⍤ was mentioned as atop first here)

(thing i use to search APL chars for those wondering)

Elias Mårtenson

13:57

Was ⍛ ever implemented in Dyalog?

dzaima

no

from my first message, its first mention was in may 2019, which is "pretty recent"

Elias Mårtenson

@dzaima but one could easily implement it as a custom operator, no?

Is there a library of these that people use?

dzaima

@EliasMårtenson well, of course

Elias Mårtenson

How do you make it work? Dyalog gives me invalid token if I try: ⍛ ← {⍵ ⍵⍵⍨⍺⍺ ⍺}

Is there a way to mark a symbol as a valid one-character name?

dzaima

oh, you of course can't have dyalog just recognize ⍛

Elias Mårtenson

14:01

Why not? All you have to do is to flag ⍛ as a valid one-character name?

dzaima

and then Dyalog can't add an actual ⍛ without breaking backwards-compatibility

Elias Mårtenson

@dzaima Well, it could. If you flagged it as a custom name, the built-in functionality would no longer be available. But that would be fine, since any file containing such symbols wouldn't need the "new" Dyalog functionality.

dzaima

that'd come with some runtime penalty on every ⍛ the dynamic parser encounters, i assume

Elias Mårtenson

@dzaima yeah, but not too bad I guess? The parser is already quite slow in Dyalog.

dzaima

@EliasMårtenson it's decently fast for what it is, really. You also wouldn't be able to add custom syntax (e.g. multiline comments/strings, or like anything that's not builtins. iirc i said this in a previous discussion)

Elias Mårtenson

14:09

Well... :-)

In the context of Dyalog, I'm sure you're right.

It's not complicated in principle though.

In dyalog, are there any one-character symbols that are available for use by users?

dzaima

@EliasMårtenson no

Elias Mårtenson

I guess Dyalog could have a special range of symbols that were guaranteed to never be used, so they were available to uses. Perhaps all the 𝕒𝕓𝕔𝕕𝕖𝕗 letters?

dzaima

@EliasMårtenson well, in the context of any language that wants backwards compatibility and ability to extend the language in the future

Elias Mårtenson

That would be kinda neat.

@dzaima Well, I'm obviously thinking about this in terms of Kap, and it works quite well there. However, every symbol, includig single-character ones, have a namespace similar to how it works in Lisp.

So you can define your own custom ⍳ that is in a different namespace, which will not interfere with the existing implementation.

But Dyalog doesn't work that way, so I was trying to think of ways it could be implemented there, because I like custom symbols :-)

dzaima

(also IDEs would be very unhappy if you define and use ", and then the language adds it as a string delimiter)

Elias Mårtenson

14:15

Well yeah, if you define a custom symbol that is later being used as a new kind of token on the tokeniser level, then you're kinda screwed even in Kap :-)

dzaima

so you agree that no language can have all of extensibility, custom chars, and backwards-compatibility

Elias Mårtenson

@dzaima No. I still firmly believe it's possible. But neither Dyalog, nor any other array-based languages that I know of could easily do it.

Arguably, Common Lisp can do it.

dzaima

is common lisp extendable though?

Elias Mårtenson

Very much so.

Or wait, what do you mean by extendable?

dzaima

@EliasMårtenson e.g. it could decide to add [] as an alternative to () for improving readability (i assume lisp doesn't use [])

Elias Mårtenson

14:21

Yes, you can.

CL has a concept called the "readtable" that maps a character to a function that is responsible for parsing anything that follows that symbol. There are existing reader extensions that parse [...] as quoted arrays for example.

dzaima

but could common lisp have an update that just introduces that (by default, everywhere), and doesn't break anything?

Elias Mårtenson

That wouldn't really be a problem. You can replace the entire parser if you really wanted.

dzaima

my point is about whether the base language can, not a user

Elias Mårtenson

All you do is switch out the readtable and you can do anything you want. If you want a symbole being interpreted in the "standard" way, you just switch back.

Well, in CL the base language is kinda fixed in stone, and all development that has been done on CL since the 90's has been in the form of user-level stuff, so there is really no difference between what CL can do vs what users can do. Everybody are users.

dzaima

@EliasMårtenson so that's a "no" to "extendable"

Elias Mårtenson

14:25

So there is for example a reader extension that allows you to use C syntax. It literally converts the C code to lisp as it's being read, so from the point of view of the caller they are just reading a file, and out pops Lisp code.

@dzaima Is it extendable. If the industry really wanted to release a "new version" of CL, then that would be fine.

It's just that there is no way that'll ever happen since there is no need to. Since the language can be updated using its own mechanisms, there is no need to define a new standard, really.

dzaima

@EliasMårtenson well, this isn't a question about if it'll happen, but about if any language can do this

Elias Mårtenson

Well, I argue it can.

I really can't think of a program that uses custom syntax that would stop working even if there was a new CL. Assuming that the new CL would still be backwards compatible to the old one.

dzaima

i guess if your way of defining a new entry in the readtable is doable somehow before anything further on is tokenized/parsed, it's possible

Elias Mårtenson

@dzaima Yeah. That's exactly how it works. You can inject code that runs as the reader is tokenising the symbols, so the first line in a file runs at read-time (i.e. just after the first form has been parsed), and it switches out the readtable which is then subsequently used for the rest of the file.

The key is that you can run code at different phases in CL, you can have code run at read-time, at compile-time, at load-time or execution-time.

Now that I think about it, the same could be done in Dyalog.

I guess I have to change my mind, I think it is possible to support this stuff in APL.

dzaima

that means you must have a full read-time parser in your syntax highlighter (aka, my paste would need to implement a part of lisp)

Elias Mårtenson

14:32

Not sure if it's worth the effort though.

@dzaima For sure. And that's the reason most true syntax extensions are quite limited in scope, and doesn't do stuff like introducing new comment syntax or whatnot. Because they screw up the editors.

dzaima

and any existing extension would retroactively look ugly if the base language defined it as new comment/string syntax

Elias Mårtenson

There is one that introduces the #I syntax for infix expressions for example. You can do #I(2 * (a + 1)) which is then read as (* 2 (+ a 1))

@dzaima Well, that's always going to be a problem with any language that supports custom syntax. But then again, languages that do tend to have very regular basic syntax and doesn't really get changed that much (since there is less demand for it as people can do what they want). This includes Lisp and its cousins, as well as Forth.

dzaima

also

Elias Mårtenson

Woah

OK, that's both confusing and very neat

dzaima

not quite sure why this doesn't work ¯\_(ツ)_/¯

Elias Mårtenson

14:41

I'm confused. Why doesn't this work? mlochbaum.github.io/BQN/…

dzaima

i assume the tokenizer just isn't designed to prefer custom chars to the default ones

which makes sense, since this extended thing was added as an afterthought, not as a core goal

the first line of the tokenizer directly hard-codes the character '#', so you just wouldn't be able to replace it

Elias Mårtenson

Is it because x is considered part of a word?

dzaima

yeah

Marshall

15:05

@EliasMårtenson It's only designed to add characters not in the character set already. So it won't complain, but you'll get undefined behavior. I think since it does lookups with sorting and binary search (dyadic ⍋) it does end up favoring existing characters.

@dzaima SHARP APL used ⍤ for Over (on) and ⍥ for Atop (upon). I think Roger had mentioned these before and Adam just swapped them.

They're in Rationalized APL (1983). I'm guessing it was Iverson who came up with the symbols.

Elias Mårtenson

15:35

@Marshall Funnily enough, it did work with ↕

code_report

18:55

Say I want a snippet of code that gives me a 26 element array for letter frequency, i.e.:
{+⌿⍵∘.=⎕C⎕A} 'bcd'
0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
However, if you pass this a one letter string, you get:
{+⌿⍵∘.=⎕C⎕A} 'a'
1
How do I avoid this?

rak1507

@code_report ,'a' instead of 'a'

code_report

19:09

ty!

Adám

22:15

@code_report Better solution: {¯1+{≢⍵}⌸⍵,⍨⎕C⎕A}

@rak1507 Enter reexecutes the block, Shift+Enter edits the dfn in the Editor, and <IL> (which I've assigned to Alt+Enter) inserts a line inside the block (in the session transcript) so you can add a line to the dfn.

Adám

22:48

@chrispsn In APL, underscored glyphs are generally conceptually related to their plain counterparts, e.g. </≤,=/≡, ∊/⍷ (member/find), ⍳/⍸ (index of/interval index of), ⊂/⊆` (enclose/enclose-if-simple). I needed a glyph for something closely related to ∘…

rak1507

23:42

@Adám Alright, cool

thanks for forwarding on xpqz's book as well I will have a look tomorrow :)

Transcript for