The current chapter in the ongoing function-definition-syntax-saga: In my current implementation, full-form function defintions: ∇ (x) foo (y) { x+y } are global, while short form: ∇ foo {⍺+⍵} are local.

It works, and it's what one wants most of the time.

But man, it's not easily to explain.

Someone on Mastodon suggested ∀ as a prefix for global definitions. It's not too bad, since one could use it for variables too: ∀foo ← 1.

And I still haven't decided on a syntax for operators. I'm looking at making them quite verbose (since defined operators aren't overly common). I want to use defoperator or defop.

@EliasMårtenson "what one wants most of the time" I would definitely never ever want that

@EliasMårtenson imo the syntax for custom functions and custom operators should at least be roughly the same. ∇ left (leftOperand name optionalRightOperand) right { … } isn't that bad

@dzaima (if that's intentional behavior, there should definitely be some actual syntactic difference)

@dzaima But that would mean that declaring a monadic function would look like this: ∇ (foo) a { a+1 }

That would declare foo to be a function that adds 1 to its argument.

@EliasMårtenson there's no reason ∇ foo a {a+1} couldn't still work

@dzaima that's what tradfns do, and they work just fine

@dzaima But how would you declare a monadic function that accepts two arguments? Currently the syntax for this is: ∇ foo (a;b) { a+b }

@EliasMårtenson that, again, still works

You mean this? ∇ foo a;b { ... }

@EliasMårtenson what's that currently?

@dzaima Currently that is not valid synax. Arguments (left and right) have to go inside parens.

@EliasMårtenson so that, too, stays

the only behavior that's changed is that, if the "object" 2nd from right is (a b) or (a b c), then it defines an operator instead of a function

afaict that should allow all currently existing functionality to stay

Hmm... Let me write down all alternatives in a list. One sec.

'∇' arg? (name | '(' arg name arg? ')') arg? {…} where arg is a name or (a;b;c) in some mix of regex and grammar syntax

@dzaima If I understand you correctly, you're proposing this: gist.github.com/lokedhs/af01fd756e08969dae3d04eb41c6d9ea

Am I right?

@EliasMårtenson yeah

Hmm, I guess that's possible.

@dzaima (tradfns for some unknown reason don't allow destructuring the left arg, but ¯\_(ツ)_/¯)

I remember now why I shied away from that approach before: If I parse this: (foo) (bar), then foo could be either a function name or an argument name, depending on what comes next. With the current implementation, seeing an opening paren immediately tells the parser what it is.

I mean, I could just parse the thing as a (possibly-semicolon-separated-list of symbols) and then resolve it once I see the { character.

letting implementation details dictate big language usability decisions

@dzaima Good point.

I'm at work now, but once I get home I will implement this and see how it goes. Thanks for taking the time to assist on this.

that's i guess a thing good about my mess of a tokenizer - with how bad it was (its mess of tokenization intertwined with grouping parentheses and statements), I didn't feel too bad squeezing parsing strands in the tokenizer too. :p

(fwiw BQN headers have the advantage that things know their type, so a dyadic operator's header is leftArg LeftOperand _name_ RightOperand rightArg, no parentheses needed)

@dzaima Are you saying that 1‿2‿3 is seen by the parser as a single token?

@EliasMårtenson it's a token containing other tokens. :p

(so my tokenizer is really a tokenizer and mini parser intertwined, ¯\_(ツ)_/¯)

@dzaima I see. So what is 1+2 then? Also a token with tokens?

@EliasMårtenson it's a line token of 3 tokens

Sounds like the tokens and AST nodes are mixed up

@Bubbler exactly

this pre-grouping of things makes actual parsing (or compilation in the case on BQN) much more simple, and if I didn't do it in the tokenizer, I'd probably have an entire pre-parsing parsing step (but that'd be more inefficient as I'd be duplicating objects often, and 1 object is better than 2 objects)

With heart in mouth I have penned a very terse introduction to dfns for the MENACE book. (Gentle) feedback please: gist.github.com/romilly/c1eeb90fb691ed573d56c341fdc27208

There are examples that follow in the text but I have not included them in the gist.

@RomillyCocking Looks good to me. If you want, you can add "For more information about dfns, see APL Wiki."

I will. TY.

@Adam You were correct - the book has turned into a tutorial :)

@Adám Hello. Yes, after a long while! Currently trying to understand and write a parser for K's grammar for a hobby project!

@dzaima I implemented your suggestion. It can now handle regular function definitions, but with the new more flexible format. This means I'll be able to add support for operator definitions. I'm not proud of the messy code though.

github.com/lokedhs/array/commit/…

@EliasMårtenson fwiw, my BQN header mess (and more of it) before I rewrote it

@dzaima What's the purpose of canBeImmediate?

@EliasMårtenson BQN operators (called modifiers) can be "immediate" if they don't contain 𝕨 or 𝕩, which means they evaluate upon getting operands instead of when getting arguments. So Fn ← -{•←"hello" ⋄ 𝕗} (like APL fn ← 5{⎕←'hello' ⋄ ⍺⍺}) prints "hello" before calls to Fn

@dzaima (i guess only this is the actual parsing of it, but it's outsourcing matching parentheses and getting types of things)

@dzaima (canBeImmediate is the boolean telling whether the code contains 𝕨 or 𝕩, but that can be overridden by the header explicitly asking for arguments)

@dzaima (also functions can be immediate too, so •←"a" ⋄ {•←"b" ⋄ •←"c"} ⋄ •←"d" prints all of those. This gets extremely annoying when making side-effect-only functions)

@dzaima actual compilation time breakdown

@dzaima (for c.bqn;) "collect" is the part that takes care of logic of what sequences of "tokens" are what (monadic/dyadic call, derivation, train, etc); "tokenize" obviously tokenizes; "compO" converts "tokens" to bytecode through whatever necessary means; "isE" is a part of "collect"; "builtin" converts a character to an instance of the appropriate builtin; "funType" gets function type + immediateness

@dzaima Ah, I think I understand. :-)

@dzaima got rid of isS (at 1.3%) and isE down to 1.4% (them being the pattern parsers) by writing code just manually checking the types

@dzaima c.bqn compiling c.bqn

bitarr stuff appears to be on the top

@dzaima If you know you're going to add all the bits before doing anything else, a better implementation of add() (which, for anyone else following along, places a new bit into a bit vector) would be to keep a 64-bit buffer v, which you append to with v=v*2+(b?1L:0L) (could also shift, but the multiplication is more likely to be fused into an lea instruction). After 64 appends, write it. You don't even have to clear it because the next 64 appends will flush those bits out.

Oh wait, that's for writes with big-endian ordering. Given that I don't think little-endian writes are as fast, you might actually want to walk backwards using that method to write the output.

@Marshall that's a good idea (i was kind of going for as branchless as possible, but that really doesn't make sense when you consider that it's 64-periodic); will play around after ~20 mins

But you could also right-shift v and add the new bit at the left. Getting Java to branchlessly convert a boolean to a top bit might be tricky but is probably possible.

@Marshall The JVM optimiser will also rewrite code in a way that may surprise you.

Trying to get rid of a branch in the bytecode is often not worth it.

@dzaima I think branch predictors can usually work with length-64 loops, but you might also benchmark length-32 loops as well. I used 32-bit accumulators in Dyalog, but I think that was just for convenience when supporting 32-bit as well as 64-bit architectures. I also used unrolled loops of 8 sometimes, which can be a little faster but not a big difference.

@EliasMårtenson Not worth it because Java will figure it out and make it branchless anyway, or because you can't get Java to consistently make it branchless? In this case, the bit to write can be arbitrary, so branching on each write bit is a huge cost, like a factor of 5 or something if the rest of the code is good.

@Marshall A bit of both. The JVM optimiser has special knowledge of "typical" bytecode generated from the the Java compiler, and has special-cased opimisations for a lot of those cases.

So unless you're ready to benchmark, trying to predict that the optimiser will or will not handle is note really feasible.

@Dogbert Cool. Feel free to hang out here, although the focus is mostly on languages that stay closer to original APL. Are you aware of the k tree?

One case for example is that trying to get rid of method calls is often unnecessary, since the JVM very heavily inlines small functions.

@EliasMårtenson I think it's safe to say we're ready to benchmark.

@EliasMårtenson The inlining stuff is good to know though.

@Marshall inlining is the reason I made the r() function for getting array rank - short to type, but can eliminate array bounds checks when the .shape.length is inlined

@Marshall precise branch predictor behavior is probably at the edge of my knowledge, but IIRC (and that's a big if) (much?) longer periods can get 100% prediction too

@dzaima oh, turns out I didn't actually go full-branchless ¯\_(ツ)_/¯

The JVM is capable or re-optimising if it detects that the predictoin was wrong.

@EliasMårtenson this is about the CPU not JVM

Now, under what circumstances it does that these days I don't know. It's been a very long time since I worked with the JVM internals.

@Adám Ping?

@Adám yep, joined k tree, doesn't seem as active as this one. I'm interested in APL as well. Will try asking K related stuff there first!

@Marshall even with Long.reverse, I get an improvement of 0.165 to 0.13 for a‿b←<˘•Rand 2‿100000⥊3 ⋄ 10000•Time"a=b"

huh. java is fun

@dzaima Are you warming up the JIT before testing?

@EliasMårtenson yeah. I'm running the expression 4 times, and do that a couple times in fresh REPLs

There are certain optimisations that only kick in after a rather significant number of invocations.

And at least as of 10 years ago, the JVM was not able to optimise a running function, so you needed to return from it and call it again for the optimised version to be called. This may have changed though.

@EliasMårtenson well, that expression is called 30000 times beforehand, and 300000 or 3000000 don't seem to make a difference

@dzaima Yeah, that should be enough :-)

@EliasMårtenson there's some on-stack-replacement optimization stuff, but >one exit is definitely needed for proper optimizations

@dzaima What Java version are you using by the way?

@EliasMårtenson java 8 oracle currently because that appears to be needed for visualvm (maybe), but as i'm not using it currently I can switch out

@dzaima For benchmarking I strongly recommend Jprofiler. It's really good.

@EliasMårtenson thanks. i tried looking at options, and i happened to have visualvm in an installed state so i just used it

Jprofiler is commercial, but you have a time-bombed demo license you can use if you want to try it.

ah, well that's disappointing

@dzaima java 14 looks about the same - 0.17ms for old, 0.13/0.14 for new

@dzaima I agree. It's sad that there really is nothing open source that comes close.

for my current type of testing, visualvm seems to be fine (beside slowing everything down like 100x)

though Jprofiler does say that tokenization takes 22% of time instead of visualvm's 10%

Currently trying to implement the Bailey–Borwein–Plouffe formula : Try it online!

not sure what's going wrong

Is there a clever tactit thing for picking the 4 corners of a 2-d array?

@Razetime The summation looks fine.

And what are you trying to accomplish with the {2 2 2 2⊤⍵}¨ bit?

@FredrikNiemelä Pong. Hur mår du?

@Adám Bara bra tack. :) Are you available for a call?

> The structure of this formula allows a simple manipulation to generate any desired hexadecimal digit without calculating the previous digits, and each hexadecimal digit is of course just four binary digits.

@xpqz ⋄ (⊃,⊃⍤⌽,⊃⍤⊖,⊃⍤⌽⍤⊖) 3 4⍴⎕A

@Adám
┌→───┐
│ADIL│
└────┘

@Razetime But you did not implement such a manipulation. You just implemented the summation.

I did not understand this right then

Also, beware that converting decimal digits to binary is different from converting units. So 0.5 in binary is 0.1, which is different from the 110, which is 5 in binary.

@xpqz ⊃¨(⌽∘⍉⍤⊢\4⍴⊂)

i.e. to convert decimal digits to binary you cannot simply treat the decimal part as an integer and convert to binary.

Not tacit, but: {↑(⊣/⍵)(⊢/⍵)}⍣2

@RGS can ⊥⊤ be used?

@xpqz nice. (⊣/,[.5]⊢/)⍣2

@xpqz ⋄ (⊣⌿,⍥(⊣⌿,⊢⌿)⊢⌿) 3 4⍴⎕A

 @Adám
┌→───┐
│ADIL│
└────┘

@Razetime I don't think you can use them directly, no

@dzaima Ha, that's what I wanted to come up with ... ⌽⍉ as rot90. Nice!

Got lost somewhere around power operator instead of doing things with scan :D

@Adám That's ... beautiful and hideous in equal measure.

@xpqz ⋄ (⊣⌿,⍥⍪⊢⌿)⍣2⊢ 3 4⍴⎕A

@Adám
┌→─┐
↓AD│
│IL│
└──┘

dzaima/APL, ⊃¨⌽∘⍉⍡4

@Adámis there a way to use ⊤⊥ with floating point values

@dzaima reading about ⍡, I love itttt

@Razetime Not to convert to non-int bases.

@MartinJaniczek This: ⋄ {⌽r⊣{r,∘⊃←⌽⍉⍵}⍣4⊢⍵⊣r←⍬}3 4⍴⎕A

@Adám
┌→───┐
│ADLI│
└────┘

@MartinJaniczek "reading about" outside of this chatroom i believe there are a total of 11 words talking about ⍡ :p

@dzaima well I had to google what it does somewhere... of course the first result was aplwiki

@MartinJaniczek the 2 words left are here. (i doubt this counts :D)

@Razetime I think you want to have a look at something like this

It's basically iterate - "apply fn N times but give me also the intermediate results" hackage.haskell.org/package/base-4.14.1.0/docs/src/…

@RGS Went through a rabbit hole

which is what my mind went to when reading the problem

@MartinJaniczek yeah. It's a pretty fundamental operator to not have

@dzaima (though it's a good question what it should do with a function right operand. Any option leads to something being awkward, and I've changed it before and would probably change it again if there was any reason to)

@FredrikNiemelä Should be available in a few minutes. Or we can schedule.

@Adám In a few minutes works perfectly for me.

@Razetime does that help you?

@xpqz With negative indices you would just have to select by 0 ¯1 along each axis, so this is fairly easy in J and BQN: BQN example.

@RGS nah but it was a cool link

You might want to ask for help somewhere like math.stackexchange.com

@dzaima what weird case does the ⍣ behaviour result in? (stop if x g f(x))

To understand how to work with the formula to compute a specific set of digits.

@MartinJaniczek the question is whether or not to include the initial argument in the result array. If you do, then the result array length is one above the number of calls made to f. If you don't, you lose the information of the original argument. (plus the general problem of ⍣ of it calling f at least once guaranteed)

currently I include the initial argument as that seems to be often what I wanted, but it has the strange result that the result will always be at least 2 items

Even with {⍵}⍣≡ ?

(sorry, with the T thing)

@MartinJaniczek Yeah

I'd expect 1 item result in that case if you're keeping the original item. That is, don't include the final item at which you'd stop

@Marshall negative indices is one thing I occasionally miss from other languages.

@MartinJaniczek that'd discard the result of precious computation. (really the calculation and checking should be merged, but that's, like, even harder to do)

@dzaima My feeling about it is: meh :D So, you keep all computation done, and let the user ¯1↓ if they need the other behaviour

I guess that's more generic and works for more usages

@MartinJaniczek and that's what I do

@RGS It works now!

it's definitely shorter than computing the individual digit formula

@Razetime What do you mean "it works"? :P What is supposed to be the argument..?

@RGS This question

Ah I get it. Nice!

problem now is to find out how to make it work theoretically for any possible input

@Razetime because of rounding issues?

no that's what the question asks for

the pattern can appear anywhere in the binary digits

@Razetime Yes, but why wouldn't your code already be working theoretically for any possible input?

because it only generates pi till ≢⍵ precision

@MartinJaniczek I've discussed what I know of your application with a couple people here at Dyalog. If you'd like to talk with us, maybe make a presentation to give us a bit more context, then I think we can help determine what might work for your application. Please feel free to contact me via email to set something up!

@Brian Thanks Brian, that's very kind! I'll definitely ping you on email if I manage to convince the relevant people about the APL approach.

@MartinJaniczek Even if you don't convince them, it would be useful for us to understand a use case in the real world. If you can't share details, that's fine, just getting a broad understanding of the challenges you face helps us develop frameworks to address them.

@FredrikNiemelä Available now?

@Adám Yes

@FredrikNiemelä Join on Zoom: +\758 ¯435 8447

@Brian Ah, understood! I think I can give you some broad context about the problem itself and my approach and the difficulties, eg. on a call if you'd like.

@Adám I'm there...

@xpqz if order doesn't matter, not so clever works too: 2 2↑¯1⌽¯1⊖⊢

also: (2↑¯1⊖⍉)⍣2

@ngn ಠ___ಠ so obvious

@ngn what magic is this

Oh.

@MartinJaniczek a mix of this idea and this

@MartinJaniczek remove middle rows, then remove middle columns

Right, you rotate one edge next to the other one and then take just those two... transpose, repeat

yep. (transpose goes first here but it doesn't matter)

Announcement: aplwiki.com/wiki/ can now be abbreviated to apl.wiki/

Nice url

Hm, now I want apl.chat to link to here… It is a tad expensive, though.

How much?

Aww, some polish shop has already got a.pl

33 GBP/yr

trya.pl is available though

@rak1507 I know, we'll take that if we ever decide on a server-side URL shortener.

Oh cool

apl.wiki was on sale, really cheap, so I could help myself.

@dzaima Thanks, fixed.