The APL Orchard

RGS

00:43

Was trying to implement "pretty printing" of APL arrays, mimicking the ]box on style. I have to say, I just kept wishing Python was array oriented because manipulating all those strings was annoying as hell

I got the main things working, at least. May still have some hidden bugs, but some things already print decently:

 >> 2 2⍴⊂2 2⍴⍳4
┌──────┬──────┐
│ 0 1  │  0 1 │
│ 2 3  │  2 3 │
├──────┼──────┤
│ 0 1  │  0 1 │
│ 2 3  │  2 3 │
└──────┴──────┘

Bubbler

00:57

Cool. Are the extra spaces around the center intended?

RGS

@Bubbler They were "intended" in that I put them in, but then didn't do the side margins with extra spaces, so I removed them meanwhile

Was doing some debugging, now I also got this:
        2 3⍴(1 2 3)(2 2⍴0)1(3 1⍴0J1 0.001 0)
┌───────┬───────┬─────┐
│ 1 2 3 │ 0 0   │1    │
│       │ 0 0   │     │
│       │       │     │
├───────┼───────┼─────┤
│   0J1 │ 1 2 3 │ 0 0 │
│ 0.001 │       │ 0 0 │
│     0 │       │     │
└───────┴───────┴─────┘

Bubbler

Oh wow... but that scalar

RGS

You mean the lonely 1? Dyalog APL prints this more or less like I have it there.... Except that Dyalog APL aligns decimal points and I think J's for complex numbers as well

Well, gotta go sleep. Ttyl

Bubbler

Good night.

(Just checked dfns.disp, and it gives all contents left-justified with no margins at all)

Adám

04:28

@Bubbler @RGS it actually has an option to centre:

      0 1 disp 2 3⍴(1 2 3)(2 2⍴0)1(3 1⍴0J1 0.001 0)
┌───────┬─────┬───┐
│ 1 2 3 │ 0 0 │ 1 │
│       │ 0 0 │   │
├───────┼─────┼───┤
│0.000J1│     │0 0│
│0.001  │1 2 3│0 0│
│0      │     │   │
└───────┴─────┴───┘

Razetime

05:30

@Adám for {P5.G.ln↓⍉↑⍵×⍺∘-@3⍳¨⍺1 1⍺}:

for {P5.G.ln↓⍉↑⍵×(⍳⍺)⍬⍬(⍺-⍳⍺)} :

Adám

@Razetime What is ⎕IO?

@Razetime No idea. @dzaima what is going on here?

Razetime

05:46

@Adám ok , works with ⎕IO←0

Adám

@Razetime Then simply use 4 instead of 3 for ⎕IO←1 :-)

Razetime

@Adám he did say it was sort of unfinished

with ⎕IO←1

Adám

@Razetime Is tis right, or is that a stray line?

Razetime

stray line

it doesn't do this with ⎕IO←0

Adám

Oh, of course, ⎕IO affects the ⍳s too.

Razetime

06:11

so that causes the stray

RGS

08:59

@Adám I'm not gonna go down that rabbit hole

Elias Mårtenson

@Adám And that's why I never implemented ⎕IO :-)

Adám

@EliasMårtenson Good choice.

Elias Mårtenson

Last night I had this idea. In APL, you can modify arrays using the ← operation. Like foo[2;3] ← whatever. Now, when your language has immutable objects, this use of ← is not useful. However, the general idea of deriving an object based on some other with a subset of it modified is still useful. So a general syntax for this may be the following:

Let's choose the symbol ⇺ for immutable modification. Now, the expression foo[2;3] ⇺ 5 will return a new array, similar to foo but with one cell changed. So you could do something like foo ← foo[2;3] ⇺ 5 to create this new array, and reassign it to foo. You could even imagine a new operation, like ⇷ that would do the two things at the same time foo[2;3] ⇷ 5 which then would behave remarkably similar to the APL way of doing it, without having to suffer from modified arrays.

Please critique this idea.

It feels like this is something that someone must have done already. It's not particularly bleeding-edge language research.

Adám

09:15

@EliasMårtenson Why require the array to be named? You could write something like 5@(⊂2 3)

Elias Mårtenson

It doesn't have to be named. I just used it as an example.

You can definitely do something like (⍳10)[4] ⇺ 100 this would evaluate to a n array: 0 1 2 100 4 5 6 7 8 9

Adám

So this is just a different syntax for the @ operator.

Elias Mårtenson

I figured someone thought about it already.

Adám

      100@4⍳10
1 2 3 100 5 6 7 8 9 10

Elias Mårtenson

This puts me in a dilemma. I was to be compatible with Dyalog where it makes sense, and this is a case where it makes sense. But I use @ for character literals already so I need to find a different character for that :-)

Adám

09:20

@EliasMårtenson I told you back then that @ already was used, no?

Elias Mårtenson

Yes, but I didn't pay attention to what it was used for :-)

I didn't realise it may actually be a useful feature.

Adám

It is used for changing values at positions ;-)

However, implementing a "structural under" operator gives much more power.

Elias Mårtenson

My operation would be as generic as possible. Every operation/function would have a "modification method" that, if it exists, means that the ⇺ operation can be applied to it. So you could, for example, do something like: (↑x)⇺100 to modify the first element of an array.

Well, not modify. Return a new array with the first element replaced.

Adám

That is exactly what structural under does.

Elias Mårtenson

So is @ the structural under operation?

Adám

09:24

No, but it should have been imo. We propose ⍢ for it, which is what Extended Dyalog APL and dzaima/APL use, and is probably what Dyalog will use if we ever get around to add it.

Elias Mårtenson

OK, sounds good, and I don't use that symbol yet :-)

Adám

Current Dyalog @ is but a subset of structural under.

@EliasMårtenson Just don't give it special syntax, please. Make it a normal operator!

Elias Mårtenson

Well, yes. The special part happens in the backend

Adám

So it'd be 100⍢↑x

Elias Mårtenson

OK. But what about structural change of a dyadic function?

Oh wait, You are applying an operator on a number?

Or did you mean to write 100↑⍢x

Adám

09:29

@EliasMårtenson Yes.

@EliasMårtenson You'd write e.g. 100⍢(2∘↑)x

The idea is that an array is treated as the corresponding constant function.

Elias Mårtenson

I haven't implemented Dyalog style ∘ at this point, and I never considered doing it. Is there some good documentation somwhere that explains precisely how it works?

RGS

@EliasMårtenson You mean this?

Elias Mårtenson

09:45

@RGS I mean exactly that. Thank you.

RGS

np, I'm gonna be here all morning :P

      ]create # C:/users/rodri/documents/test -source=both
Linked: # ←→ C:\users\rodri\documents\test
      ]link.list
┌─────────┬─────────────────────────────┬─────┐
│Namespace│Directory                    │Items│
├─────────┼─────────────────────────────┼─────┤
│#        │C:/users/rodri/documents/test│0    │
└─────────┴─────────────────────────────┴─────┘
      ]link.break #
Unlinked 0 items: #
      ]link.list
No active links

The message after ]link.break is very unclear

Adám

@RGS Can you make a GitHub issue about that, and explain what you'd want instead?

RGS

@Adám Yeah. Now I understand it says it unlinked 0 items because the link was syncing 0 things, and I thought it meant it broke 0 links. Maybe it is not that unclear, I am just a dummy

Elias Mårtenson

10:15

Hmm, the ∘ operator makes sense. However, from a parsing perspective it collides with its use as the Null function (left argument of .). So I'd need to special-case it when building the AST.

Perhaps I should go back to my original idea of having outer join be a separate symbol, and not a special case of .

Adám

@EliasMårtenson Absolutely. That ∘. is a real wart. I suggest dyadic f\ and/or f⍀

RGS

Does anyone have the magic TIO apl incantation at hand?

Adám

@EliasMårtenson Also, should you ever want object.member syntax then I'd recommend even separating f.g outer product a different symbol.

Elias Mårtenson

KAP already supports ⌺ for outer join. I could easily use \ or ⍀ of course instead. What is the reasoning behind that?

Adám

@EliasMårtenson But ⌺ is used by Dyalog for stencil code.

@RGS tio.run/##SyzI0U2pTMzJT////3/ao7YJj3r7HnU1P@pd86h3y6H1xo/…

Elias Mårtenson

10:19

@Adám Good point. Although I'll deal with that when the time comes :-)

@Adám Oh is it? I don't know what that is. And I thought that symbol was so appropriate :-)

Adám

@EliasMårtenson You can always look up symbols from the IDE, or even use APLcart.

@EliasMårtenson You could use ˙

Elias Mårtenson

@Adám I don't have Dyalog. I should probably install it.

@Adám sorry for a stupid question, but the IDE is not included in the free linux release?

Adám

@EliasMårtenson The GUI IDE (called RIDE) isn't. But it is easy to get from GitHub. Not a bad question at all, btw.

RGS

@Adám thanks...

Adám

@EliasMårtenson I'm trying to conserve glyphs, and dyadic f\ and f⍀ are not defined. J uses dyadic f/ for outer product.

RGS

10:32

(@Adám turns out I was right after all: codegolf.meta.stackexchange.com/users/56656/wheat-wizard)

dzaima

@EliasMårtenson for food for thought, dyadic \ or ⍀ could be used for providing the first item to start with. (unfortunately dyadic / & ⌿ are already taken :/)

Adám

@RGS no

RGS

Hm ok, what was I defending, then? I thought our discussion was regarding WW being a mod; did we disagree on caird, then?

Adám

Yes.

RGS

woops ⍨

Adám

10:37

@EliasMårtenson I think f⍁ or f⍂ would be better for a "table" operator (i.e. ∘.f). ⌺ is such a good fit for "stencil".

Elias Mårtenson

I found a bug :-)

Adám

In?

Elias Mårtenson

photos.app.goo.gl/eVVACZaHpzvuxamRA

I took a screenshot

Adám

Oh, is that a combining ring?

Elias Mårtenson

The formatting of the boxing isn't counting grapheme clusters correctly.

Adám

10:40

Right.

And if you throw in RtL scripts, it gets worse.

Elias Mårtenson

It was the first thing I tested because I struggled so much getting it right myself :-)

dzaima

if it did, the output couldn't be a character matrix

Adám

It could, but it'd have spaces on the right on some lines.

Elias Mårtenson

@dzaima But it isn't a character matrix in this case. It's a matrix containing what is essentially strings.

dzaima

@Adám at which point you're literally at the place for what vectors of vectors is meant for

Adám

10:42

Well, in this case, disp is simply using a matrix to cause the system to print a specific text.

The Windows IDE instead puts every character into a cell in a grid, so RtL and combining characters are ignored.

It makes reasoning about the data easier, but makes human language unreadable. I think that's fine, though, as you don't need proper rendering until at the GUI frontend, and that'll just work.

dzaima

(still better than dzaima/BQN :p (and when/if i fix that, i doubt i'll support grapheme clusters, it's probably already impossible enough to get what they are, let alone handle them properly))

Elias Mårtenson

@Adám So the windows UI is different from ride? Because I did my test in ride.

Adám

@EliasMårtenson Yes, the Windows IDE is very different. (Though the main parts; session, editor, tracer are similar.)

Elias Mårtenson

@dzaima It is possible. You just let your Unicode library deal with it. That said, that still only covers languages where each grapheme cluster fits in a box. Arabic will of course not work, but even things like Devanagari will be problematic.

dzaima

@EliasMårtenson "Unicode library" if it's not in the default java language, i'm deeefinitely not bothering

Elias Mårtenson

10:49

@dzaima It is. What you want is BreakIterator.getCharacterInstance(). It returns an iterator that splits a string into a sequence of grapheme clusters. For the purposes of terminal-style rendering, you can consider each substring something that fits in a single box.

Adám

@EliasMårtenson But there are additional issues. Some characters have double width. Should boxing take that into consideration? How would you display a 2-by-3 matrix where one row is Chinese and one is Greek?

dzaima

better yet, what about a 2-by-2 matrix where the first row is a single grapheme cluster and the second - 'ab'?

Elias Mårtenson

@Adám I currently don't. This is mainly because the wide characters are considered legacy in Unicode. However, any language with Unicode support has functions that allows you to check if a character is a wide character. It's not really that important though.

@dzaima Good point. What I do is that I only render rank-1 arrays as strings. Multi-dimensional arrays render each character using its @-representation

Adám

@EliasMårtenson What do you mean by "the wide characters are considered legacy in Unicode"?

dzaima

@EliasMårtenson dyalog definitely can't just not print a character matrix as 2D text

Elias Mårtenson

10:54

So no, 2 2 ⍴ "abcd" does not render as a pretty box of ab(newline)cd

dzaima

@EliasMårtenson that's afaik the primary way to output multiline text in Dyalog

of course text with newlines works, but Dyalog really doesn't like newlines

Elias Mårtenson

@dzaima That's not what I was suggesting. In dyalog's case, my opinion is that grapheme clusters should be handled for rank-1 arrays (i.e. "plain" strings). For higher rank arrays, the combining characters should be replaced with replacement characters, so that it's clear where things go.

dzaima

@EliasMårtenson that'd be directly messing with the output, removing information

Elias Mårtenson

Because right now, trying to render that in ride at least gives me a complete mess.

Try this: 2 2 ⍴ 'aåbc'

dzaima

@EliasMårtenson that just cannot be presented in any way other than 2 2⍴'aåb'

Elias Mårtenson

10:57

@dzaima It wouldn't. It would add information. It's definitely more clear than this mess: photos.app.goo.gl/jhT5JD4a2NSAikoj7

dzaima

@EliasMårtenson to your human eyes at least

Elias Mårtenson

The issue here is that what Dyalog is trying to write is invalid Unicode. Note how the conbining character ends up on the next line, so the text renderer gets confused since you're not allowed to start a line with a combining character.

dzaima

@EliasMårtenson it has no other option though, besides literally outputting 2 2⍴'aåb'

Elias Mårtenson

But you can't do that, since the å is two characters: The simple character 'a' followed by a combining character. What happens is that the 2 2 reformatting breaks that unit and drops the combining character at the beginning of the next string. This is invalid.

Adám

@EliasMårtenson Here's the Windows IDE version:

dzaima

11:02

@EliasMårtenson right, it can't. So it does nothing and pretends combining characters don't exist

Elias Mårtenson

@Adám I see. So the windows ide does the same thing. It simply doesn't pay attention to special unicode characters.

Adám

@EliasMårtenson I isn't invalid. How can you allow the ring to combine with the a if they are not adjacent?

dzaima

@EliasMårtenson what output would you expect?

Elias Mårtenson

@dzaima The point is that it can. It can detect combining character and display them individually instead. Some terminals do this.

@Adám I wouldn't say it should combine with anything. The output routine could replace the combinding character with this thing:

Check the wikipedia page on combining characters, it represents them with a dotted circle with the combining character on top of it:

Combining character

In digital typography, combining characters are characters that are intended to modify other characters. The most common combining characters in the Latin script are the combining diacritical marks (including combining accents). Unicode also contains many precomposed characters, so that in many cases it is possible to use both combining diacritics and precomposed characters, at the user's or application's choice. This leads to a requirement to perform Unicode normalization before comparing two Unicode strings and to carefully design encoding converters to correctly map all of the valid ways to...

Adám

One possibility would be to redefine how character arrays work. Instead of being defined in terms of code points, they could be defined in terms of graphemes. I think Raku does that.

Elias Mårtenson

11:04

So it could look like this: a◌́ instead of á

That will make it clear that there are combining characters there, no information is lost, and everything is drawn in the right place.

Adám

Right, that could work, but only using RIDE, not in the Windows IDE (which effectively uses space instead).

Elias Mårtenson

@Adám Of course. This would be a solution for text-based output.

dzaima

@EliasMårtenson so you are proposing loosing information - now you can't distinguish 2 2⍴'a◌́cd' and 2 2⍴'ácd'

@dzaima (which is probably acceptable, if you can guarantee only humans will look at that as debug output, which i doubt Dyalog can)

Adám

@EliasMårtenson How would you display 2 2⍴'ab',⎕UCS 9676 768? (the last two chars are the dotted ring and a combining grave accent)

dzaima

@EliasMårtenson what about those funky 4-character emojis? i doubt those have special non-combibing combination markers

Adám

11:09

Not to mention the style selector characters…

Elias Mårtenson

OK, that didn't render well in my firefox. I porbably copy-pasted incorrectly.

dzaima

heh

Elias Mårtenson

@Adám You mean the variant selectors?

Those can also be displayed as their own things (there is a set of symbols used to represent the VS characters)

dzaima

@dzaima ah, it does use a zero width joiner between those all

Elias Mårtenson

@dzaima Oh for emoji styling? Yes, you are right. And again, what I suggest is that those a ZWJ should be displayed with a replacement character too.

Adám

11:12

@EliasMårtenson I don't think you can come up with a consistent scheme. How would you represent 2 3⍴⎕UCS 9 13 65 10 0 65?

Elias Mårtenson

Because absolutely none of these special characters makes any sense in a multi-dimensional character array.

dzaima

@EliasMårtenson right.

Adám

@EliasMårtenson What we could do, is to (upon output) replace all not-normal-single-width characters with placeholder characters.

Elias Mårtenson

@Adám My scheme is very simple: If it's not renderable as a single grapheme, it gets a replacement character.

Adám

@EliasMårtenson That's is totally possible. Although you'd probably only want to do so for higher-rank arrays, not for vectors, since ⎕UCS 65 66 13 67 68 makes total sense.

Elias Mårtenson

11:14

@Adám That is exactly what I'm suggesting. With the added suggestion that certain special characers (like the combining characters) are replaced with something more nice than just a U+FFFD. For combining chars, that would be the dotted circle with the combing char added to it.

@Adám Exactly. That's why I said that all of this is only applicable for rank-2 and higher arrays.

Adám

@EliasMårtenson Dyalog's fancy output code is written in APL, so I can easily experiment (and make real changes ― I've done so before). Give me the range of characters to replace and I'll give it a try.

Elias Mårtenson

For rank-1 character arrays, all that needs to be done is that the number of grapheme clusters is used for measuring the width of the string instead of the char count.

@Adám I have started rewriting my "fancy" (not as fancy as Dyalogs) display code in KAP itself instead of Kotlin. A 100 line function for laying out a rank-2 array become 2 lines of APL.

Adám

@EliasMårtenson Even that is likely to fail. How would you render ⍪(⎕ucs 65 13 66)(⎕ucs 65 66 67)?

@EliasMårtenson Can you not port the Dyalog code?

Elias Mårtenson

@Adám Maybe. It depends on the license I guess?

Adám

@EliasMårtenson All our public-facing APL code is MIT, unless otherwise noted.

Elias Mårtenson

11:18

@Adám A newline? For me, I simply escape special character in my string rendering routine, so that would be "A\rB"

Adám

@EliasMårtenson That makes it awkward to print "documents".

@EliasMårtenson Heh, but that uses two characters!

Elias Mårtenson

True, but it will be measured as such, so when placed inside a larger rank-2 array it doesn't screw up the indentation.

I always render strings with double-quotes, which is different to how Dyalog does it so my solutions are not applicable.

Adám

So if you print the two vectors a\r↓ and bc↑ the arrows will not point at each other?

dzaima

this all would be much nicer if there were separate output commands for human output and machine output. (i'm planning to do something like that for BQN - •← for pretty human-readable, •Out for consistent, well-defined output (including things like no nested arrays))

Elias Mårtenson

Also, currently I don't even have a "plain" output. Everyting is always boxed, so if you want to display a plain string you have to use the print function.

Adám

11:20

@dzaima That's close to ⎕← and ⎕FMT, no?

dzaima

@Adám more precisely, ⎕← and sendToSTDOUT ⎕FMT

Adám

Yeah.

@EliasMårtenson source

Elias Mårtenson

@dzaima That is a very astute observation. In my case, everything is for human reading. You have to use print for machie output. It has formatting arguments that specifies if you want to have PLAIN, FANCY or READABLE output (the last one being output that you can feed back into the parser to get the same data)

@Adám Thank you. I need to study this thing :-)

dzaima

@EliasMårtenson but it does store the output as a character matrix

@EliasMårtenson ah, in which case i wholeheartedly agree with whatever pretty-printing you may want to do on ⎕←

@Adám (importantly, •←2‿2⥊"abcd" is free to box things, print in roundtrip, etc whereas •Out 2‿2⥊"abcd" will always literally output the lines of the matrix (even if they include newlines), separated by newlines)

Adám

@EliasMårtenson Are you planning a difference between 3⍴⍳3 and 1 3⍴⍳3?

@EliasMårtenson Btw, it is only so complicated because tries to be so compact. The traditional display is more informative and much simpler code.

ngn

12:05

@EliasMårtenson what do you mean by "without having to suffer from modified arrays"?

apl makes a copy (if necessary) when modifying an array (namespaces are the only exception)

dzaima

@ngn you just have to say namespaces are "mutable" every opportunity you get don't you

ngn

@dzaima i wish i didn't have to

dzaima

it's annoying that simple scalars are arrays, otherwise i could say that namespaces aren't mutable arrays

regardless, even if you find them pointless, namespaces definitely allow doing things you just could not at all do nearly as nicely otherwise, and pretty much never harm you otherwise

ngn

@dzaima i don't find them pointless. i find them poorly designed - inconsistent with the rest of apl.

dzaima

@ngn "inconsistent" - is that just that they're mutable or do you have other problems with them?

@dzaima (because to me they'd lose 99% of their usefulness if they were immutable)

ngn

12:13

@dzaima mainly that they are mutable, but also that they don't allow arbitrary keys, and maybe a few other minor issues not worth mentioning.

@dzaima how so?

dzaima

@ngn right, no arbitrary keys is also kind of bad. But with arbitrary keys, dot syntax becomes strange (and it's definitely something i'd want)

@ngn imo they're for giving access to mutable things. A dialog UI is mutable. A drawing canvas is mutable. A file is mutable. So I want a mutable interface to them.

ngn

@dzaima "file" issues apart, the rest can be mutable only when refcount=1, and still be useful

it's just easier to think in terms of values instead of pointers to values

dzaima

@ngn if I have an immutable namespace of a window, and create a copy of it, should I now have two windows?

ngn

@dzaima what is a window in programming language terms? a dictionary?

dzaima

@ngn a special object that somehow connects APL and a window. What specifically that means is the question here - I'm saying the best way is to make the APL object mutable, you want to somehow make it work with it being immutable.

ngn

12:22

ok, so if you copy a window, you should get two equivalent representations of the same graphical object. this doesn't mean it has to be drawn twice.

the window-drawing part that belongs presumably to some graphics toolkit is not part of the language we're considering (apl)

dzaima

@ngn So the APL-side object can be nothing more than a pointer to the OS-based structure, since any other data it would go out of sync if it's changed in another copy. So what you're proposing is equivalent to a file descriptor, but for windows.

ngn

yeah, that's how windows work anyway. they have some sort of id.

dzaima

so you're just pushing the mutability problem back to the OS/whatever provides you with the window structure, while at the same time making syntax for using it just awful and defining other pseudo-mutable things (think, a graph element on the window) nigh impossible.

ngn

yes to the first part

dzaima

@dzaima here's a proposal: Wrap the window ID in an immutable object that knows the ID is a window. Then make syntax such that thatObject.size calls the OS-specific function to get size of the window from the ID. Is that new completely immutable object still immutable?

ngn

12:31

i don't understand why we're discussing windows. a window is a resource outside of the control of our language. it needs to have some id acting as pointer. that doesn't mean the rest of the language shouldn't be designed properly and consistently.

there is this double-think some apl users (dyalog in particular) seem to have: when you ask "should arrays be values?", they say yes. then you ask them: "namespaces are arrays, should they be values too?" and they say no. but the conditions are the same. what's so different about namespaces that makes people alter their reasoning?

dzaima

a window is just one of the simplest examples to think about. You're right that the only places where mutability is just outright necessary is for outside interactions (another example being a canvas - you don't want to keep it in RAM so all interactions to it must be through some (mutable) interface, mutating the GPU contents) but the system of mutability is just so nice for many other things

@ngn what's a "value"?

ngn

@dzaima something immutable

dzaima

@ngn Namespaces are "immutable" from the perspective of the array (the reference never changes), but not from the perspective of the user of it (what precisely they reference might change)

ngn

@dzaima "value" means that what you reference doesn't change either

dzaima

@ngn but that's not the way it's used in the first case

ngn

12:50

let's look at it like this: both arrays and namespaces are like mappings. arrays associate integer indices with the corresponding elements. namespaces associate keys with corresponding elements. so, namespaces are like a generalization of arrays. it's a small generalization - just let the keys be anything (or maybe just identifier strings, like in dyalog's case) that doesn't have anything to do with the mutability of the container.

it's a simple argument. i hope i'm not being too annoying by repeating it so many times.

dzaima

@ngn Mutable namespaces are more than a just mapping of items though - it's information about a unique thing's mappings. The case when I just want a hashmap is the 1% here (though if they allowed arbitrary keys, the 1% would drastically increase, but still would leave quite a bit of need of some mutable structure too)

ngn

@dzaima "unique things" - so, you're attributing identity to namespaces. it's like you assumed they have reference semantics before we even started arguing. that's ok - so, why not give arrays identity too, then?

dzaima

@ngn One structure with identity is enough. And structures without identity are useful too. So we want both

(kind of hard to make a case of GUIs to you because i'm pretty sure your answer would be to have no GUIs)

ngn

checkmate :)

dzaima

@ngn what?

Razetime

13:00

I heard a chess term

ngn

i feel checkmated :)

dzaima

@ngn ah (i assumed i was the one getting checkmated :p)

ngn

it's like: sentence A says something, we proved that something, now under the same circumstances statement B says the same thing, so let's consider it false because both truths and falses can be useful sometimes :)

dzaima

@ngn where did i do that?

ngn

it's good that containers A (arrays) be values. is it good that containers B (namespaces) be values too? no, you say, because "we want both" values and references.

dzaima

13:08

@ngn where did i say B?

ngn

i'm just labelling them A and B for convenience, to illustrate the logical fallacy

you said the "we want both" part

dzaima

ah, i horribly misread that message (not about the label part, but like everything else ._.)

@ngn I don't consider an array of mutable namespaces (which includes a simple scalar namespace) to have identity

Just as much as a vector of file descriptors doesn't have an identity

(the only difference between the two is how the "pointer" is viewed, so i think it's a fine comparison)

Marshall

@ngn I think the problem here is that you're starting with the idea that namespaces are intended as a kind of container like an array. Maybe that was the case in Dyalog; I wasn't there. For BQN I was trying to solve some problems of code encapsulation and found that namespaces do that and are also very similar to closures, which already exist. If I wanted to make arrays with non-integer keys then I would have dictionaries and not namespaces.

As the conversation with Elias shows, K-style dictionaries fit poorly with APL arrays because they only have one key while arrays have many. My preferred approach would be to extend arrays so that each axis has a domain which might not be a prefix of the natural numbers. But I think this would be too complicated in BQN, which doesn't have a lot of facilities for working with axes.

ngn

"namespace" is dyalog's word for "dictionary", no?

Marshall

@ngn Dyalog users sometimes try to use namespaces as dictionaries, and it usually doesn't go well.

ngn

13:17

exactly, cuz they are mutable (+other problems) :)

dzaima

@ngn "namespace" to me implies mutability, whereas "dictionary" need not be mutable

xpqz

@Marshall I literally just this minute wrote:

Hash←{keys←(1500⌶),⍺⋄vals←,⍵⋄{⍵⊣⍵.(Vals Keys Default)←vals keys ⍬}⎕NS''}

Should I be ashamed of myself? Or prepare for bad things to happen?

dzaima

@ngn no, because they're not meant to be used as arbitrary dictionaries

ngn

i think you're mixing up cause and effect

Marshall

@xpqz That's implementing a dictionary using a namespace. You're not actually using the namespace as a dictionary, since it only has three fixed keys.

ngn

13:19

@Marshall this conversation was about mutability. i think multidimensionality is a separate independent question.

dzaima

@ngn how so?

Marshall

@ngn I'm explaining why I would choose to add a mutable thing with keys and values before an immutable one.

dzaima

@dzaima (i'm fairly certain the original namespaces were just that - a space for names (i.e. variables))

ngn

@dzaima "they are such-and-such" vs "people don't like to use them because". i think the former is the cause, the latter is the effect.

Marshall

Dyalog namespaces were developed to work with Windows GUI objects. Originally they weren't first class; namespace references were added later.

dzaima

13:22

@ngn but them not being designed for being dictionaries caused the decisions about mutability & etc

ngn

and i thought the value of values would be so obvious..

Marshall

My paragraph on namespace history on the APL Wiki.

ngn

yeah, i know the history

Marshall

@ngn Even the APL2 bit?

ngn

not those details, but they are not essential. what matters is what we actually have in the interpreter here and now. i don't care about N years ago - that's gone.

what we have now is a design in which some objects are mutable and some are not.

Marshall

13:29

@ngn But surely there is a reason APL2, Dyalog, and BQN all ended up adding fairly similar mutable name-value associations? I still think Dyalog's version of namespaces is a mess, but I ended up designing something that has the same central ideas.

dzaima

@ngn if you want to understand why a feature is useful, though, you need to know what it solves (and what it solves includes objects with identity, and also allows users to create their own objects with identity; You may never want identity, but i certainly like it. What it doesn't solve is a need for a dictionary/hashmap structure)

ngn

@Marshall bqn because you come from dyalog and don't know better (yet), dyalog because of backwards compatibility, and apl2 - i don't know :)

dzaima

@ngn BQN could very well get immutable dictionaries in the future. But namespaces still have a place after that

Marshall

@ngn I come from J and have never considered myself a Dyalog programmer. I'm pretty sure I've never added namespaces to a Dyalog program except to debug them.

Razetime

12

Q: Topple the sandpile

(There are related questions about infinite sandpiles, and finding identity elements of sandpiles.) Given a matrix of non-negative integers, return a matrix of the same dimensions, but toppled: If the matrix doesn't contain any values larger than 4, return it. Every "cell" that is larger than ...

I think @ might work here

but not sure how

ngn

13:37

@Marshall but are you convinced that programming with value semantics is better than with ref semantics? @dzaima doesn't believe this, and there seems to be no way to convince him otherwise

dzaima

@ngn "better" - often, but not always. There are times when both are "better"

Marshall

BQN already had mutable data before namespaces because it has function and modifier closures. Even in Dyalog function closures can be put into an array with ⎕OR, although the interpreter makes scopes vanish when the function finishes so this is hazardous.

dzaima

e.g. in this code, g is a namespace referencing the window canvas. I can just do g.BG '1' to set its background to #111111. In an immutable world, that'd either look like g ↩ g BG '1', after which I'd need to make some "write to screen" call (never mind needing to refcount the GPU buffer), or there'd be a global list of all canvases and the function mutates that. Both are significantly more annoying than just g.BG '1'

ngn

@Marshall ok, ignoring closures.. (they are a can of worms, mutability in diguise) do you agree that values are better?

Marshall

@ngn I don't really think any kind of programming is always better (there are cases where one design is strictly worse than another, but it's very much a partial order).

ngn

13:42

@dzaima you keep giving examples that involve things outside the control of the language

Marshall

I think sticking to immutable data is the right thing to do most of the time, but I also think most functions should be pure and yet I've allowed side effects in BQN.

dzaima

@ngn how is a canvas outside the control of the language? (besides my remark about GUI buffers)

ngn

@dzaima it draws on a screen, presumably?

a screen is inherently mutable

dzaima

@ngn it doesn't need to necessarily. It could also be written to an image file, processed/analyzed, etc

Marshall

@ngn A pseudorandom number generator is an example that's definitely inside the language. Using a function that takes initial state and creates function closures provides a very clean interface and lets state changes be localized to the parts of code that have access to a particular copy of the generator.

@ngn I do agree that immutable values are always the right choice in cases where you don't use mutable values.

ngn

13:46

@Marshall so is there anything stopping you from designing namespaces as values?

Marshall

@ngn No, but they wouldn't accomplish the functionality I want in that case.

ngn

@Marshall they would eliminate the need to implement mark-and-sweep gc

Marshall

@ngn No, function closures require exactly the same implementation support (they can pretty easily be used to implement namespace-like things with awful syntax).

dzaima

@dzaima all I'm saying there is I have a type of thing, and I may want to call BG on the thing. Doing that in an immutable world is hard because I necessarily need to write the changes back to wherever I got the object from, and as such you need the structure to be at a consistent location so it's easy to update, heavily decreasing the amount of freedom in structures of things

cannadayr

what functionality do ppl want from namespaces? the only thing that comes to mind for me is code encapsulation but i found lexical scoping plus functions as values was enough for me. granted i havent written a large bqn codebase or interacted w a large codebase so grain of salt etc

ngn

13:51

@dzaima i don't understand your argument. would it be different if you want to set just one pixel, and the canvas is modelled as an ordinary matrix?

Marshall

@cannadayr Yes, code encapsulation is the only major reason closures aren't enough.

ngn

@cannadayr tables?

dzaima

@cannadayr for me it's mostly GUIs & things outside the language. Grouping code is also nice, but closures are often good enough

cannadayr

@ngn whats a table? like a rank 2 array?

ngn

@cannadayr a transposed namespace

imagine a namespace with keys 'id' 'name' and values (3 5 8)('alice' 'bob' 'charlie'). when we transpose it, we get a vector of namespaces. when we index that with, let's say, 1, we get a namespace with id:5, name:'bob'

cannadayr

13:56

i think i lack some of the context because i havent used a namespaced apl-family language. the most i interact w namespaces are import/export semantics and i dont see why those have to be mutable at runtime (i dont do gui programming tho)

dzaima

@ngn No, that's not the purpose of namespaces. That's what immutable dictionaries are for

ngn

to me they are the same thing. dyalog "namespaces" are just handicapped dicts.

dzaima

@ngn So that's your issue.

ngn

no, that's their issue

dzaima

@ngn that's your issue with them

ngn

13:58

that's my main issue with their namespace design

dzaima

So you're calling Dyalog namespaces bad because they don't do a thing they aren't meant to do.

@ngn the discussion isn't about the operations on the canvas, it's about the storage of the canvas and how you operate on it (so no, my argument would be no different). In a mutable world I could just have nested containers for a GUI and have the pixel matrix as a leaf element somewhere far off and just call BG on it, whereas I'd need to backtrack-rewrite everything in an immutable situation, or have a drastically different setup for storing them

Marshall

@cannadayr There's a restricted version of namespaces that have to be immediately destructured. That's the version in the self-hosted BQN (still only mostly implemented). These aren't mutable at runtime like you say. However, it's natural to extend it to general namespaces, which is what dzaima/BQN does.

dzaima

@ngn "To me, k and C are the same thing. k doesn't have pointers, just some handicapped integer indexes in arrays, so from that I conclude that k is bad"

ngn

@dzaima the distance from apl/k to c is far greater than between apl and k

dzaima

@dzaima "To me, A and B are the same thing. A doesn't work well if used as B, so A is bad"

ngn

14:07

@dzaima yeah, maybe it is a problem of not setting the right goals

rak1507

Maybe namespaces don't work well as dictionaries, but what other options are there?

dzaima

@ngn "To me, arrays and dictionaries are the same thing. Arrays cannot store a key 'abc' in them, so arrays are bad."

ngn

@rak1507 um, is doing it right an option? :) of course i mean any-keys-allowed immutable namespace/dicts (for me there's no difference)

rak1507

Yeah it would be great if there was a better solution

dzaima

@ngn at which point is the threshold at which you can consider two things "equal enough" to be able to blame one for not doing things the other does?

@ngn (also that message should be "the distance from apl/k to c is far greater than between mutable namespaces and immutable dictionaries")

Marshall

14:11

@ngn You are in favor of both function closures and the eval function, right? Because if you put them together (with local scoping for eval) you literally just get namespaces, except without any control over what variables can be accessed.

rak1507

Sorry to interrupt but I was wondering if someone could explain what ⍤0 1 does, I used it in a solution and it works somehow but I don't really understand entirely how it works

dzaima

31 mins ago, by ngn

@Marshall ok, ignoring closures.. (they are a can of worms, mutability in diguise) do you agree that values are better?

Marshall

@rak1507 What are the argument ranks?

rak1507

The way I ended up using it was (⊃⍤0 1)∘(2d array)¨stuff where stuff is something like (1 1 1)(1 1 2)(1 2 2)...

dzaima

@ngn also what about "To me (immutable) dictionaries and (mutable) namespaces are the same thing. Dictionaries aren't mutable and so are handicapped, so namespaces are better"?

@dzaima (i'll stop with this spam of comparisons now)

Adám

14:16

@rak1507 It applies a function over and over again (if necessary) such that the function only ever sees a scalar as left argument and a vector (or scalar) as right argument.

Marshall

@rak1507 In general ⍤0 1 matches 0-cells of the left argument with 1-cells of the right argument. In this case, the left argument is always a vector (rank 1) while the right argument has rank 2, so the left argument can be seen as a list of scalars and the right argument as a list of rank-1 rows. So ⊃⍤0 1 matches each scalar to the corresponding row.

ngn

@dzaima it's funny when you're copying my style of expression :) i guess your last few messages do make sense if we don't consider immutability a virtue, but i strongly believe we should

rak1507

Is there a better way of doing it? The 2d array was actually ↑something, and I feel like there might be a more simple way to solve the problem

Adám

@rak1507 Yeah, try explaining in English what you want.

For every triplet in stuff, you want to select that element from the corresponding row of your 2D array?

rak1507

I am bad at explaining things haha sorry, maybe a simple example would be (1 1)(1 2)???(1 2 3)(4 5 6) returning (1 4)(1 5) where ??? is the function

Marshall

14:19

@rak1507 Walk through what happens and be sure to say "each" a lot.

dzaima

@ngn immutability is good, but as with most things, it isn't always 100% no-matter-what the absolute best thing. If we assume that immutability is always 100% no-matter-what the absolute best thing, then of course we shouldn't have mutable namespaces

ngn

@Marshall closures - not sure. i'm a half-hearted supporter of their inclusion in k, though i never implemented them in my own dialect.. eval - it's a necessary evil. i don't think i've ever actively argued for it. in ngn/apl ⍎ was less powerful than in dyalog (no access to locals). i understand that closures and eval are mutability in disguise.

Adám

@rak1507 (1 1)(1 2)⊃¨¨⊂(1 2 3)(4 5 6)

rak1507

(a b)(c d)???(x)(y) -> (x[a], y[b])(x[c], y[d])

Adám

@rak1507 Well, if your data is orthogonal, you'll get better performance if you can use flat arrays, so with two matrices, you can do (2 2⍴1 1 1 2) (⌷⍤0 1⍤1 2) (2 3⍴⍳6)

rak1507

14:24

@Adám Ah yeah, that works, I think I tried something like that but didn't realise my data had something like (1)(2 3) when it should be (,1)(2 3)

Adám

Ah yes, that'll mess you up.

rak1507

Yeah, that's much simpler (the first one)

Going to look at the second one now but the multiple rank operators scare me..

Adám

@rak1507 Nothing to be scared of. It simply means ⍤1 2 for each vector (row) on the left and the entire matrix on the right, ⍤0 1 for each scalar on the left and each row on the right…

So first, we pair up rows on the left with the entire matrix (each row gets the entire matrix to choose from) and then when we have a vector paired with the matrix, we pair a scalar from the vector with a row from the matrix.

rak1507

Alright... I think this is one of those things I'll need to play around with myself to fully understand

Marshall

@ngn Well, I guess I'd encourage you to think that closures demonstrate that mutability is a useful tool for code organization rather than that allowing mutability is a downside of closures. I started out wanting everything to be pure: in I, the only parts of the language are functions, and whitespace and parentheses to specify precedence, and the only impure functions are get, set, and print.

I've gradually come to realize that language purity can lead to messy and fragile programs, and a few well-designed impure features can make programs easier to write and reason about overall.

Adám

14:34

@rak1507 Watch RikedyP's series on ⍤, especially the second episode.

rak1507

Alright I will thanks

ngn

@Marshall "i tried and couldn't do it" is not a good argument - i know that from crypto :) you should look at what the best in the trade have already done.

rak1507

Btw the problem I was trying was projecteuler.net/problem=18

ngn

@ngn (to clarify, slightly off-topic: in cryptography they often get these questions: "i designed a cipher, and no matter how hard i tried, i couldn't break it. could you please review it?" - 100% of the time it's something stupid. everyone can design a cipher they cannot break.)

@Marshall "purity can lead to messy and fragile programs" - i would argue the opposite. it's harder to reason when objects can change without you touching them. you never know if a function you call has a ref to the same ns that you have and changes it when you're not looking.

"a few well-designed impure features" - agree. but let's keep them to a minimum.

dzaima

14:57

@ngn keep either the usage to a minimum, or features to a minimum? I'd argue that the former should be done, and the latter is pointless

ngn

@dzaima the amount of impurity to a minimum :)

dzaima

@ngn still, in the language syntax, or in usage?

ngn

ideally usage, but it's hard to measure

if we could know how much each feature is used, we could remove unpopular features easily

dzaima

I'll agree that 99% of code should work with immutable data, but for that 1% with mutability I'd rather have a well-designed, nice system, rather than something squeezed to a minimum

Transcript for