Showcase: ⌹ strikes again

@Bubbler that's a really nice solution.

@Bubbler Nice. I really enjoy your "showcase" series.

Today there's the informal meeting? at 16h30 BST?

As a v18 data point, RIDE is crashing a lot for me -- several times a day, with no discernible pattern. v17 crashed occasionally, but it seems to have increased since my upgrade. The error I see is ECONNRESET - I've enabled core dumps, but none are produced. I also see no evidence of problems using the interpreter in jupyter context, so it seems to be an issue with RIDE.

The crash is pretty severe in that the only way to exit the unresponsive RIDE application window is to use a system force-quit or kill -9.

@Adám Thanks. Btw, do you have any updates on this?

@RGS Yes :-)

@xpqz Hm, but you don't have a consistent way to repro the hang?

Sadly not, so as a bug report it's pretty useless, sorry.

@Bubbler Well, we're still busy with the current competition, so we have not really started serious work on next year's ― only collecting ideas for now. But if you want, I can take it up with @Brian who is leading that work. Alternatively, you can email him. His email address should be guessable…

@xpqz ECONNRESET I think means that the interpreter has crashed, and then RIDE just hangs. Do you know how to start the interpreter without RIDE?

@Bubbler Btw, I mentioned what you're work here (during a meeting about RGS's internship!) and on APL Wiki. Our management was very keen on me praising you for it. So there you have it; we really appreciate your efforts!

@Adám I assume I run the interpreter in the terminal with the weird package path I discovered the other day?

@xpqz Yes, exactly. What you can do then is have the terminal interpreter listen for an incoming RIDE connection. Then it becomes easier to detect who is at fault. If RIDE fails, you can kill it and restart and reconnect to the same interpreter.

So RIDE checks first if there is a running interpreter?

Can I start the interpreter in daemon mode somehow, rather than serving up the repl?

(I could rtfm...)

@xpqz Ah right, on macOS it skips the connection screen. I'll have to investigate how to get to the connection screen… But no, it doesn't. Rather, it normally (on other OSs) lets you choose to spawn, connect to an existing, or wait for an incoming interpreter.

I can get to that screen as File > Connect

@xpqz No, it always creates an IDE.

@Adám Wow, that's an amazing news. For the challenges, I guess I'll write an email after I collect some ideas.

@xpqz Ah, ok, then you can start an interpreter with RIDE_INIT="SERVE:*:4502" and then you can Connect to localhost on port 4502.

@Bubbler :-D Trickling single challenges is fine too. That's what we do internally.

@Adám I think this falls foul of Dyalog on Mac not really being a fully-fledged *nix install:

(base) Stefans-MacBook-Pro:AoCDyalog stefan$ ln -s /Applications/Dyalog-18.0.app/Contents/Resources/Dyalog/dyalog /usr/local/bin/dyalog
(base) Stefans-MacBook-Pro:AoCDyalog stefan$ which dyalog
/usr/local/bin/dyalog
(base) Stefans-MacBook-Pro:AoCDyalog stefan$ RIDE_INIT=connect::4502 dyalog
Stream 0: Can't access /usr/local/bin/aplkeys/default
Stream 0: /usr/local/bin/aplkeys/default line : 1 can't open. "/"

@xpqz Ugh. I'd complain (again) about that.

I'd be known as 'that guy' at this rate.

No, this is good. Dyalog on macOS is a relatively new thing, and we need your feedback. Nobody will be upset or annoyed.

@Adám except for xpqz

;-)

@Adám Say I have an array m where the scalars are integer vectors

Right, e.g. ⍳2 3?

Is there another way of writing (⊂v)≡¨ m without ¨?

@Adám yes

@Adám is there some docs somewhere on how to run the interpreter locally in a Docker container, and have RIDE talk to it?

At first I was thinking about scalar expansion and I just wrote (⊂v)≡m thinking it would "auto-map", but of course it doesn't because it is comparing the array ⊂v with the array m

@xpqz I didn't take a close look, but let me know if you find something useful here or here.

@RGS m∊⊂v?

@Adám oh wow, that is interesting

@RGS nah :) not my first rodeo. I know how the sausages are made.

@xpqz xD that is an interesting expression, "I know how the sausages are made"

@Adám what is more idiomatic? m ≡¨ ⊂v or m ∊ ⊂v?

Ha. Yeah, isn't the full quote something like “If you like laws and sausages, you should never watch either one being made.”

@RGS I used to write ≡¨ until I saw Morten use ∊…

@Adám I guess that answers my question :D

@xpqz I do have a Docker image with cutting-edge Dyalog APL and RIDE installed, though it isn't configured for local usage. You might have a chance if you clone it, add EXPOSE 4502 \n CMD RIDE_INIT=SERVE:*:4502 dyalog (is it right?) at the end of Dockerfile, and build it at Docker Hub.

Or just write a fresh Dockerfile like this:

FROM bubbler9903/dyalog-apl-gitpod:latest
EXPOSE 4502
CMD RIDE_INIT=SERVE:*:4502 dyalog

and docker build then docker run.

Does that give v18?

The vanilla docker pull dyalog/dyalog is still 17

Yes, mine is 18

Ok, let me try that. The docker pull dyalog/dyalog works as advertised, but 18 would be nice.

@xpqz @Bubbler That's due to the rather severe issues with 18.0. We're just about to issue "18.0 SE"… Once we do that, we'll update the Docker images too.

what does SE stand for?

@RGS Second Edition

@Adám ah ok. why not 18.1?

@Adám I'm aware of it, so I added a workaround to the image to make ZF RIDE work properly, but I have no guarantee that SERVE will work...

@RGS Because we're not adding any features, even minor ones, only fixing bad bugs.

@Adám ah I see

Btw, in a sense, 18.0 contains a huge change: It (hopefully) implements lexical scoping for dfns ― correctly.

@Adám what does that mean?

@RGS Since the introduction of dfns, there was a bug where the lexical scoping didn't behave as expected. We're now, finally and after much sweating, fixing this.

No more auto-shadowing?

Congratulations!

But my question was actually "what was wrong?"

@xpqz What? That doesn't have anything to do with it.

oh

I've learnt to live with this, but it still feels scope-odd:

      v←7
      f←{v←8⋄⍬}
      f⍬
┌⊖┐
│0│
└~┘
      v

7

      f←{v+←1⋄⍬}
      v

7

      f⍬
┌⊖┐
│0│
└~┘
      v

8

@xpqz It just overshadows the global v with a local v, exactly as if you had ;v in a tradfn header.

@xpqz What is it that feels odd? The use of the same symbol for local and global assignment?

I assumed a variable would only become local if one of that name wasn't already visible -- like for the modification case. The two cases seem at odds -- if we accept the first case, then the second should be an error, imho.

@RGS Hang on. I'm just putting together a minimal repro.

@xpqz No, that'd defeat the whole purpose of the auto-localisation; being able to use any name without worrying about outside names.

Then you can never have closures.

@xpqz I don't understand that conclusion.

@Adám +← 1

@xpqz ngn/apl works like you assumed. it always surprises old aplers, they take it for a bug.

in dyalog you can use v⊢←8 instead of v←8 when v is in the outer scope

Also ⎕THIS.v←8 if you don't want to depend on a global v already existing.

 M17←{
     _Op←{⍺⍺ ⍵}
     VAR←'outer'
     M←{
         VAR←'inner'
         F1←{F2 ⍵}
         F2←{VAR}
         F1 _Op ⍵
     }
     M ⍵
 }

@RGS @xpqz ^ should give 'inner' because F2 should be able to see the members of the scope in which it was defined, i.e. that of M.

Hmm, I just tried the below, and it does what I'd expect actually:

      x←1
      add←{x+⍵}
      add 5

6

      add2←{x←2⋄add ⍵}
      add2 2

3

@Adám for me it gave 'inner'...?

@RGS In version 18.0, right?

@Adám yes

so add is properly closed over x.

@RGS Try 17.1 online!

ah ok, but then wasn't 18 SE supposed to fix this..? What are you fixing if 18 already fixes it?

@RGS 18.0 attempted to fix this failed to do so universally, and broke other things in the process. 18.0 SE should have a proper fix.

alright, good luck!

@xpqz Right, that's lexical scoping. If you redefine the two functions as tradfns, it'll give 4. Try dynamic scoping online!

CMC: Golf this code. Bonus if you can remove some peppers.

^ Related challenge

@Bubbler ((⍴⍵)⍴¨⍺↑¨1),⊂⍵ → (⊂,⍨⍴⍴¨⍺↑¨1⍨)⍵

@ngn It's like in C -- you have to declare a variable in a block if you want the shadow. If you just refer to it, it's a variable defined in any surrounding lexical contexts - or an error. Dyalog treats x←5 as a declaration, whereas ngn/apl would only treat it as a declaration if no x was found in any outer scope, including globally?

@xpqz yes. i took this scoping rule from coffeescript, which i used initially instead of js. the problem is, you don't have locality this way - you have to look at the whole file (or compilation unit) before you can know for sure whether the variable is local or not.

Yes. Perl solved this with my $x.... for introducing a block-scoped lexical variable.

Not an option for apl of course.

@ngn I had no idea about v⊢←8

Obvious (but only after someone points it out)

@xpqz APLcart.

@Adám o yeah. I was so convinced it was deliberately impossible I never thought to look...

@ngn philosophically, this is the same problem as with ⎕io, ⎕ct, etc, and also tradfns - you can't reason about the code in isolation. only dyalog got it right this time.

tradfns have the mechanism to indicate which variables are shadowed. That's a nice feature, I think.

@xpqz there's no lexical scoping though

which means, if v is not local, it's the v in your caller's scope, not in your surrounding lexical scope

Does k behave that way, too?

@xpqz of course not

k doesn't have closures but locals are locals

scope in k is limited to local and global, there's no intermediate

Can a k-fun refer to globals though?

@xpqz yes

So it behaves like ngn/apl here?

@xpqz well, k has no closures, ngn/apl has full lexical closures

if you're asking about how scoping is determined - k is more like dyalog

  a:0;{a:1}[];a
0

the a in { } is local, but similar to apl you can use modified assingment to change the global a

  a:0;{a::1}[];a
1

:: is ⊢←, +: is +←, etc

I see.

@xpqz you can't do that with intermediate scopes though: {a:0;{a::1}[];a}[], and that can be very annoying

Announcement: Dyalog APL 18.0 Issue 2 is now available including for Docker.

\o/

@KrzysztofSzewczyk Welcome. So, an intro to APL, right?

yeah

I've installed APL-64 already on my PC

let me switch devices

Now, I know you're a low-level programmer, which means APL will be very different for you, but can I ask how you are with mathematics in general?

I'm feeling good with maths

Linear algebra?

I'm fine with it, I believe

if I won't understand something, I'll ask

Great on both accounts. That'll make things much easier. Since your foray into J wasn't so successful, let's just start over.

Try to think of APL less as a programming language (despite the name) and more as an alternative mathematical notation.

APL was originally envisioned as a better replacement for Traditional Mathematical Notation (I call that TMN) that gets rid of inconsistencies and ambiguities, while generalising various fundamental concepts.

So, and as opposed to J, APL uses TMN symbols for basic things, even if this means non-ASCII code.

@KrzysztofSzewczyk Are you able to start it so you get a REPL with a 6-space prompt?

yeah

also; the mathematical aspect of APL is mainly the reason I wanted to learn APL

Cool. You should then be able to use APL as a fancy desktop calculator:

      2+3
5

currently my brain thinks mostly in assembly and I find it horrible, APL just seems so much better

yeah, works for me

Many people prefer developing APL in a very interactive fashion. They'll experiment with formulas in the REPL (we tend to call it "the session") until they have stuff that works, then they copy that into their functions for keeps.

that's how I used to write some J programs

to make sure I'm not making a mistake

OK, you did get that far.

hmm, but I still don't understand some topics like gerunds and adverbs and such so I wouldn't say that I know J

I like the fact that you can hover over some operators in APL

and the application will tell you how they work

Don't worry about gerunds. They are plain weird, and we don't have them in APL. We also prefer different (non-grammar) terminology.

@KrzysztofSzewczyk OK, let's quickly move towards explaining APL's equivalent of adverbs. First some terminology.

We call things like + and × functions, and use the same name for user-defined ones. Just like in J, user-defined functions adhere to the exact same syntactic rules as the built-in ("primitive") ones.

alright

You know about TMN's ∑ and ∏ right?

yeah

1st one denotes sums, the 2nd one I've seen used to denote a factorial

Hm, the second is usually product.

there are more uses of these, but that's just something from the top of my head

yeah, product, forgot the name

But really, these symbols just a fancy way of denoting multiple applications of + and × respectively.

yeah

So APL generalises this concept with something called operators which are higher-order functions that take operands (as opposed to the arguments of functions) and determine a derived function to be applied instead ― based on the operand or operands.

E.g. / is an operator that takes a single operand (so we call it a monadic operator) on its immediate left, and derives a new function with the same relationship to the operand as ∑ has to +

Try it:

      +/3 1 4
8

that's interesting

I always explained this operator to myself as "reduce"

I've heard that APL has implicit reduce, though. Doesn't it?

@KrzysztofSzewczyk Right, that's what it is, but it is useful to have the mathematical background. It is called reduce because it reduces the number of dimensions of its argument by 1.

@KrzysztofSzewczyk No, not implicit reduce, but implicit map.

ah, yeah, obviously

that's a very clean explanation

Another operator (this one dyadic) is the power operator ⍣ which is like the superscript 2 in f²(x) that you might have seen in TMN.

So rather than raising a value to a power, it applies a function that many times.

Yes, nice!

I remembered i. from J and tried to find something like that on the bar above

@Adám how does that work

     1(+⍣2)10
12

The parenthesis is there to separate 2 from 10 since they'd otherwise be joined to a list before the ⍣ operator could bind the 2.

I see

the power operator

yeah

so, wait

how does this exactly work, I don't understand this code

Dyadic operators take on operand one each side, + and 2 in our case, and derive a related function based on those operands, the "add twice function" in our case.

We can even name the function:

     AddTwice←+⍣2
     1 AddTwice 10
12

Now we don't need parentheses because 2 and 10 aren't adjacent.

ah, okay, now I understand it

I think

Maybe I should mention that ⍣ treats the left argument of the derived function as "constant" in the sense that the same argument is used over and over, with the right argument being updated upon each iteration.

So 1(+⍣2)10 is 1+1+10 and not e.g. 1+10+10

yeah, okay

I get it now

Another simple monadic operator is ⍨ which, when given a dyadic function swaps the arguments:

      10-1
9
      10-⍨1
¯9

ah, nice

now I'm wondering what can I do with this knowledge lol

also, I've seen you use this arrow operator

does it mean functions in APL are tacit by default?

or you can explicitly refer to parameters?

You can do both.

is there anything more that I'd need to know?

what do I do now

Sorry, I just got a phone call. I'm back now.

@KrzysztofSzewczyk The arrow is assignment.

yeah I see

I can't find it on the bar //ah, no, it's the first symbol, I'm blind

@KrzysztofSzewczyk All the way left?

@KrzysztofSzewczyk As opposed to J, APL has a very neat explicit notation for functions too.

in J it's totally horrible

verb =. monad define
NB. code
)

@Bubbler Under the covers the RIDE front end to Dyalog calls mapl, not the dyalog binary. The mapl script, which is called on all non-Windows platforms sets various environment variables, most important of which is DYALOG. If you copy the mapl script to /usr/local/bin you'll need to alter it so that DYALOG still points to the installation directory. If you want to use a standalone RIDE then you'll need to install it (github.com/dyalog/RIDE has the most recent release).

@KrzysztofSzewczyk APL uses curly braces and refers to the right argument by the right-most letter of the Greek alphabet (⍵) and the (optional) left argument as the left-most letter (⍺):

If you call mapl from a terminal window then you'll need to set the environment variable ENABLE_CEF=0, but even so you won't get APL characters. Your best bet might be to use either PuTTY on Windows, or a terminal window of some sort on Linux to ssh to the mac.

ahh, okay, makes sense

      MyFun←{⍺+2×⍵}
      10 MyFun 3
16

You can also use such functions anonymously, inline:

      10 {⍺+2×⍵} 3
16

how can I get alpha and omega on the bar above // ah, no, it's there, sorry

@xpqz Sorry .. mouse troubles. Are you getting any aplcores in your home directory ?

I see, APL like J evaluates expressions from right to left

no operator precendence

@KrzysztofSzewczyk It'd be beneficial to get used to the glyph shortcuts as you go along. You're using the RIDE interface which allows backtick as a prefix key (backtick isn't used in APL). Based on visual resemblance, you can get ⍺ with `a and of ⍵ with `w

@Bubbler The latest docker images also contain zero footprint RIDE ..

ah, cool, okay

@KrzysztofSzewczyk Yes, that part is exactly the same. It is important to know, though, that operators like / and ⍣ have higher binding strength than functions like + and ×:

      2+⍣1-10
¯8

Even though we move from right to left, - doesn't have a chance to grab 1 as its left argument before ⍣ snatches it as its right operand.

ahhh

okay

- therefore becomes negate. (Note that negative numbers use a high minus ¯.)

@KrzysztofSzewczyk Do you want some exercises?

yeah

You wrote a factorial formula above. How would you define a factorial function?

@AndyS No.

      fac←×/⍳
      fac 10
3628800

Nice. And if you wanted an explicit function?

      fac←{×/⍳⍵}
      fac 10
3628800

Great. Be careful with tacit functions consisting of more than 2 functions. Are you familiar with how J works in that regard?

I just use 13:'something' or 3:'something' (forgot which one was it) to suggest me a tacit version of a complex verb

I have zero clue how does it work in reality

OK, then don't worry about it for now. Use explicit if in doubt, or doing something complex.

Can you think of a way to add the first N even numbers, i.e. 2+4+6+…+(2×N)?

@AndyS So reading this back, there is basically no practical way on a Mac to run dyalog on the command line, or to attach a stand-alone RIDE?

hmmm

give me a sec

@xpqz Well, you'll be in trouble because you have no APL characters (nor a lot of the special commands). But you could try

      sum←+/2×⍳
      sum 3
12

@AndyS Why is the situation different on MacOS than on Linux here?

@AndyS But if the goal is only to get an interpreter running so one can RIDE into it, that should be enough. One can always enter )off.

RIDE_INIT=SERVE:*:4502 /Applications/Dyalog-18.0.app/Contents/Resources/Dyalog/mapl

@KrzysztofSzewczyk Nice. The important thing here (which seemed to handle just fine) is to think in terms of whole arrays. ⍳ generates an array, and you're multiplying with the array, not mentioning the individual elements.

try that in a terminal window, and then attach a standalone RIDE to it and see what happens. The other thing to try is to start as normal and then hit Ctrl-F12. That should open up the log window .. perhaps we'll see

something of interest when the problem next happens

alright

so, what now?

How about adding the first N odd numbers? 1+3+5+…+(2×N)-1 ?

I can see it working

You may well end up in trouble with tacit definition for this one. Don't be dismayed.

yeah, I found it out

APL may play a large role in my attempt to stop using the numpad, lol

What do you mean?

numpad input doesn't work in the REPL

but only the minus operator, everything else works

how bizzare is that

Odd. Minus works by me.

@AndyS well, I'll be damned. I had to change the port, as my jupyter kernel is using 4502, but otherwise it seems to work.

@KrzysztofSzewczyk Any success?

yeah I already did it

but I'm trying to find a tacit form

odd←{⍵+((+/2×⍳)⍵-1)}

Do you want me to go into the exact rules of "trains"?

@KrzysztofSzewczyk odd 10 gives 82 but should give 100

I'll eventually figure it out, give me a second

oh, this is bad

(btw good work on everything else so far, haven't been following closely but I noticed you are handling APL well :D )

@KrzysztofSzewczyk SORRY my result was wrong because I usually have ⎕IO←0

ah, I see the mistake

lmao

no I don't see the mistake

:P

does APL have someting like a cap in J?

1 -~ [ => tacit form of x - 1

@KrzysztofSzewczyk You don't need it. 2-trains are atops.

no idea how to say it in APL

uhh

okay

@KrzysztofSzewczyk That's not cap, just an identity function. APL uses ⊣ and ⊢ for left and right.

I'll happily learn about trains then

Btw, you don't need to use the swapping trick to write x - 1. The ⍨ can also take an array as operand, and derives a constant function. This allows you to write {⍵ - 1} as ⊢ - 1⍨

hmm

well then, I'll try to write it tacit by myself

So, APL's rules for 3-trains are exactly like J's: (f g h) Y is (f Y) g (h Y) and X (f g h) Y is (X f Y) g (X h Y)

wait

how do I get control shortcuts back

ctrl+stuff just gives me ↓⍳'⍺∇⌈∆⌊'∘⊥_I

Control shortcuts?

Wait, are you on Windows?

yup

Oh, I just assumed Linux based on your homepage. My bad.

I don't like Windows, but I'm often working with malware and packing sooo

tools available for linux about this topic are quite mediocre

On Windows, Dyalog hijacks your control key by default. If you run the installer, you can uninstall the Dyalog IME (not IDE!) which is what provides you with both the backtick shortcuts and hijacks your control key.

can I disable or enable it?

it may come handy later but

not now

Windows should allow you to switch layout using an indicator next to your system tray on the task bar.

OH! I didn't notice it switched it

sorry

You can also install an alternative layout. If you prefer backtick, then this is available (for a US layout) without touching Ctrl.

ah, no, I can live with ctrl keys

@KrzysztofSzewczyk Which ones in particular do you want? Options>Configure>Unicode Input should allow you to keep Ctrl+X/C/V.

@KrzysztofSzewczyk I personally prefer using right-side Alt (AltGr) as APL shifting key.

@KrzysztofSzewczyk Back to trains?

yeah

OK, so, APL's rules for 2-trains are very simple: (f g) Y is f g Y and X (f g) Y is f X g Y

alright

You can get a J-style hook (Y f g Y or X f g Y) with (⊣ f g)

I have no idea what a J-style hook is

Then don't worry about it.

The important thing to know about trains is that every other function will be called on the argument(s) and the intervening (even-numbered, when counting from the right) are used in between those results.

alright

E.g. , is concatenation, so +,× is the concatenation of the sum and the product:

I'll make myself something to drink and try the tacit whatever in a second

      2(+,×)3
5 6

@KrzysztofSzewczyk Ping me when ready. I'll grab one too.

@Adám is it good practice to try and write ⎕IO-less code? e.g. I had a 2 elem vector from which I wanted the 2nd element. Is it good practice to write ⊃⌽ instead of 1⊃/2⊃?

@RGS ⊃¯1↑

@xpqz are you suggesting an alternative or saying that is the best alternative?

@RGS vtc as duplicate ;-)

@RGS Not sure what is best, but that's how I'd think of "take the last element of a vector"

@Adám Actually, this question may be slightly different. It asks if one should write ⎕IO-independent code when it is easy to do so without setting or referring to ⎕IO.

@Adám yup, was about to write something like that in my flag to reopen

@xpqz thanks :)

I'm here all day.

@RGS Dunno. A matter of style, really. I've lately begun writing ⊃twoElementVector⌽⍨condition instead of twoElementVector⊃⍨1+condition or twoElementVector⊃⍨⎕IO+condition

@Adám this is an interesting one ⊃twoElementVector⌽⍨condition

@Adám sorry for the delay, but that's just how it goes. This one is a bit smaller: odd←{⍵++/2×⍳⍵-1}

no need for the parens

Right. Sometimes it just makes more sense to use explicit forms. Especially if you need multiple monadic (one-argument) function applications, like +/ and ⍳ here.

Next up: The sum of the first N squares: 1+4+9+16+…+N²

alright

I'm thinking about a tacit form

for so long now

What have you got?

fn←{+/(⍳⍵)(×⍣2)1} I can use the operand swapping operator

to get iota omega at the end

and then it probably will work

but I have no idea how to use this operator here

You shouldn't need the power operator to do repeated multiplication. You can either do self-multiplication, or simply raise to the power of 2 with ⍵*2

* is power

I didn't know

@xpqz Excellent ! Next time it doesn't work, just ask me to go out for a cycle ride ! I have had this sort of effect on software all my working life :-) Developer hate me for it

fn←+/2*⍳

there we go