@EliasMårtenson Well, finally fixed this one. That took an embarassingly small amount of code. Internal compiler errors are worth reporting now!

@Marshall Nice. I'm curious as to the internal workings of your compiler. Do you have the changelist?

it's open source, right?

@Razetime Yes, I'looked at the code, but the compiler is written in BQN and it's hard to follow the code.

(well, for me, since I'm not good at it)

@EliasMårtenson wait, BQN is written in BQN?!

@Razetime Well, the compiler is.

This is the entry point to the parser, if I'm not wrong: github.com/mlochbaum/BQN/blob/master/src/c.bqn#L102

It's impressive work. My parser is much more, hmm, traditional.

@Razetime and also most of the built-ins for the online version are written in BQN too (all the prerequisites for that are listed on the 2nd line)

I remember he said the website was made in BQN

but this is a bit beyond me

@Razetime the HTML is generated from ±markdown by BQN is what i think you're thinking of

I'm fixing support for complex numbers for the trig functions. While doing that I came across this page: planetmath.org/complexsineandcosine

It contains the following quote that I don't understand: "Because these series converge for all real values of z, their radii of convergence are ∞, and therefore they converge for all complex values of z (by a known of Abel; cf. the entry power seriesMathworldPlanetmath), too."

It can't be true that all series that converge in the reals also converge for complex numbers?

@EliasMårtenson it could for power series

(and there's also a prerequisite about "radius of convergence")

What does it mean that the radii of convergence is ∞?

@EliasMårtenson planetmath.org/radiusofconvergence

Oh I see

@EliasMårtenson you know there are better (easier to implement) formulas than the series, right?

@dzaima yeah "made" meant generated

@ngn Yes, of course. But as part of implementing it I started reading about it. :-)

I'm using the (e^(i*x) - e^(-i*x)) / (2*i) formula in practice.

@EliasMårtenson can your apl implement primitives in apl?

@ngn Yes. Symbols like + or - or whatever are just plain symbols. The only thing that's special about them are that the parser treats them as single character symbols rather than constituents in a multi-character symbol.

this formula is a perfect candidate for that :)

The tokeniser has a list of symbols that are "single character", and there is nothing that would prevent you from modifying this list at runtime (although I don't provide an interface to do so at the moment)

@ngn I could, but then any calls to sin/cos with real arguments would go through this formula, making it very slow as opposed to the case now where it directly calls into the platform implementation of sin/cos.

@EliasMårtenson it could eval the formula only in the complex case

i didn't do this in ngn/apl. i should have..

@ngn True, but the selection between the different versions (int, float, complex) is done at the native level. At that point it's easier just to write the expression in Kotlin. Not saying it's the ideal way to do it, it's just what happens due to the implementation: github.com/lokedhs/array/blob/master/array/src/commonMain/…

@ngn since it'd be always called on scalars, it'd result in a lot of useless temporary objects, and the runtime overhead would probably be huge too

@dzaima why is it always called on scalars? can't the user do things like 2○1j2 3 4j5?

@dzaima Yes. If I had an optimising compiler things would be different of course.

@ngn how else could you have native pervasion for non-complex but executed pervasion for complex?

@dzaima i guess it could by deciding on depth 1, but KAP doesn't have specialized arrays anyways

@dzaima if there's at least one complex, go down the complex path. otherwise do the usual optimized thing.

@ngn that requires going through the entire input twice, which could possibly mean evaluating things twice with lazy evaluation

@ngn also x←1⊃2○1j2 3 4j5 will only evaluate 2○3 and not even touch the complex numbers

@dzaima i was assuming the array knows its type. doesn't it?

(for some definition of "type" that distinguishes between real and complex)

@dzaima that's all true. In BQN things are different because it's BQN all the way down.

@ngn the type of 1+2 3 4 is ArraySum2Args if i understand KAP correctly

@ngn No. An array contains "things" (instances of type APLValue in practice). There is no such thing as an "integer array".

@EliasMårtenson i see

@dzaima Yes, that's correct. You've actually read the code. I'm humbled. :-)

and you're worried about the performance of the formula :)

If you call .collapse() on that array, you end up with an APLArrayImpl which is just a Kotlin array of type Array<APLValue>.

@ngn well, it could be a big factor multiplier to an already very slow thing, no?

Well that, and a shape.

@ngn I'm not actually. The performance is just a justification why I don't go out of my way to implement it in apl itself.

@dzaima (i.e. already being slow isn't an excuse to be fine with making things much much slower)

@dzaima maybe. i'd expect iterating through the elements one by one to cause the greatest slowdown.

you have to test the type of each "thing" individually before deciding what to do with it

@ngn Yes. That is correct. However, the dispatch turns out to be quite fast in practice. The JVM does a pretty good job at that.

fast is relative

@ngn JVM is pretty fast for what it does. Of course it won't magically make KAP competitive with dyalog

@dzaima For numeric operations, of course not. That's not the goal.

KAP does have one advantage shared with BQN: No need for backwards compatibility.

I'm guessing that's why defined functions execute faster. Dyalog has to defer parsing because definitions may change at runtime.

@ngn checking of type probably should be the fast thing - it's ±a switch. But there is a lot of management of function calls otherwise that probably results in a lot more overhead

@dzaima dyadic is more complicated, but it's still quite simple as it'd be ±100% predictable branching

@EliasMårtenson Here's the line for that test. It just says two subjects that aren't operands can't appear next to each other.

At that point r and ro are rearranged so each subject, function, or modifier is one token long: for example parenthesized subexpressions are represented by the closed parenthesis only. And expressions are reversed, to match evaluation order. ro is a mask of modifiers and their right operands, meaning »⊸∨ro finds all operands (see shifts).

So the test ∧⟜«ro»⊸∨⊸<r=0 gets non-operand subjects ro»⊸∨⊸<r=0 and marks each 1 that's followed by another 1.

_err_ is an error reporting utility that takes a mask of errors for its argument; g⊸⊏ says how to modify the source indices to correspond to that mask.

@Marshall typo: grammer

@ngn Got it.

How on earth people write their own languages, and then implement more features in that language, is beyond me. It seems like complete wizardry

@rak1507 In any language where you can define functions, you can extend the language from within it :P

(fwiw, that fancy r.bqn file isn't actually valid BQN code, as you can't assign to primitives. It's transpiled by replacing each builtin there with a variable (with some extra logic as there's also reassigning, but we need to use the old versions in the old places))

@rak1507 i often wonder about "why", not so much about "how". so many people think they can implement or design a language properly and almost always the results are tragic.

Scathing!

Is it tragic if they get something out of it?

@ngn "people think they can implement or design a language properly" citation needed. I'd guess the goal much more often is to design something better, not everything.

@rak1507 well, the learning experience they (we?) get could be valuable

Yeah, language designers that actually say their languages are good are pretty rare (but it's not closely correlated to the quality of the language unfortunately). But how are you supposed to design a good language without making a few bad ones first?

@dzaima citation provided: "people think they can implement or design a language properly" --ngn :)

@Marshall also: one needs a purpose, a good definition of "good" in order to judge how well one's done

@ngn The purpose of dzaima/APL was to be an APL for dzaima. I think i've achieved that purpose quite well :)

@ngn unreliable source :p

in my experience, every time i challenge someone about the performance of their pet apl-like language x implemented in technology y which they learned yesteryear, the answer is "but it's more expressive" - they escape into some vague and hardly useful criterion

golfing languages are probably an exception. at least they optimize for something - the length of a solution to a typical codegolf.se challenge

@ngn I'd assume that contrived and useless (to you) criterion is probably derived from the criterion of "appeals to me"

@ngn I consider optimizing for one thing at the cost of anything else a very bad approach to language design.

@dzaima "appeals to me" - good, but should anyone else care about your language?

@Marshall i think a language must be the best at something, or else why should anyone use it?

Being all round decent at lots of things beats being great at one thing and terrible at the others, in my opinion

the "okayest" language :)

Hey, it worked for python

@ngn If it appeals to you, you should care about it. If it doesn't, don't. (in the case of dzaima/APL, I'm perfectly happy with precisely 1 user)

btw, i think i suck at language design. that's why i never designed one of my own.

@ngn Languages are used to accomplish programming tasks. All or almost all programming tasks require adequate performance across a few metrics to accomplish. A language that does well on exactly one metric generally won't be good for any purpose. So yeah, you want to use a language that's good at more than one thing.

And I believe that optimizing only for one metric does usually preclude doing adequately in others: there are always terrible tradeoffs to make in service of that one metric.

@dzaima why do you keep telling this chat about it then :) mostly in parenthesized messages with internal details no-one cares about :) (i hope this doesn't sound insulting, i'm just trying to give you a frank reality check)

@ngn what's that "best at something" of k? It's not the best at performance, not the best at brevity, what else is there?

@dzaima $/byte ratio ;)

Harsh, I think it's interesting to see development of a language

@Marshall a combination of metrics is also a metric

@ngn in that case, it'd be impossible to make a language that isn't the best at something

Counter argument: PHP

@dzaima true, but some metrics are more important than others

@rak1507 The negation of a metric is also a metric.

@rak1507 lol :D

@ngn but those other metrics might be important to others (even if you consider them stupid)

@ngn because there are other people who are interested. Me not wanting a bigger audience doesn't imply I want everyone to not use it

What do you reckon are the main mistakes people make when designing their own programming languages?

@ngn From the original comment, is your complaint that expressiveness isn't well defined? I agree it's not, but I don't agree that well-defined computable metrics are better; just that they may be easier to optimize.

common sense leads me to believe that the number of distinct important metrics should be much less than the people who create languages. yet, everyone here has one or two.. there's almost no collaboration (bqn obviously being an exception)

Seems like I need to make my own language to finally join the cool kids

@ngn The original message was more accurate imo :p

@ngn True about the collaboration, but I think APL Orchard is doing a pretty good job of improving that situation!

@dzaima ok maybe there aren't as many "important metrics" as there are "useful metrics for at least one thing", but there's a lot of the latter and they're not "useless" by definition

@rak1507 unnecessary complexity, ignoring performance ("we can make it fast later if needed"), wanton remapping of primitive operations to symbols even though established symbols exist, not caring about the usership..

@ngn most of the common important metrics have already been reasonably optimized for. What's left are many smaller metrics, and finding people to collaborate with on those is much much harder (especially in the APL community where there are already not that many people)

@rak1507 I think one problem a lot of designers have is seeing a language as a collection of features and choosing ones they like rather than trying to look at how features interact, what each one's role is, what else could accomplish that, and so on.

Interesting

@Marshall not really. this chat is partially to blame for the proliferation of apl dialects :)

@ngn I can account this chat for 2 APL impls (dzaima/APL and RAD; idk about ngn/apl), plus BQN

Each one brings something new to the table though, so overall, they will all influence each other and get better

@dzaima "Reasonably optimized for": a lot of wiggle room in that phrase, but I think there's a big gap between what we have and what we could have on pretty much every front. Possibly humans have done what humans could reasonably be expected to do, but I am sure there are lots of simple human-understandable ideas that could greatly improve existing programming languages.

@dzaima I would have made BQN with or without the chat, although of course I've gotten some good advice from you and others here.

@Marshall collection of features vs how they interact - very true

Personally I think it's great that so many people are designing their own APLs. Where I think we have a problem is that there's not enough communication, so everyone takes most of their decisions from Dyalog, GNU/APL2, or K (plus non-array languages) and ideas developed away from those languages have no chance to spread.

@Marshall in my experience you should adapt to what people already know or else they'll ignore your little language. sad but that's the way it works.

no-one will learn bqn for bqn's sake. especially with bqn not being the champion at any single given criterion. chances are, people looking at bqn already know at least one of the big three languages and they are radar scanning for improvements on them.

@ngn A good benefit of smaller languages is to influence other languages. Even if the language doesn't get known, some other language using ideas from it could be

@Marshall At least for me, I like a lot of features of GNU APL, but there are some annoying things. Thus, I end up with a language that has most of the nice features of GNU APL, but add/change all the things I personally dislike. I don't really build this thing for anyone else at this point. If, at some point in the future, there are actual users of this thing, that may change.

@dzaima That's a great point. Small languages with a small or nonexistent userbase can afford to experiment, while an established implementation can't.

@ngn Eh, I've been surprised: a few people do seem to want to do that (it's not a lot, just more than I expected). See this thread for example. That's why I've been spending a lot of effort on tools and documentation to learn the language.

Personally I think any lazily evaluated array language is completely different, afaik there are no others

@dzaima right, i think that's true. i can think of a few such little-known but very influential languages. but if another language adopts your features then you get neither fame nor $ :/

@ngn right, that's a useless point if a requirement of making a language is fame/$

@dzaima Either you go for fame, or you go for something good.

the great injustice in the array programming world is proprietary interpreters

we can't beat a good proprietary interperter with 100 mediocre free interpreters

(i use "we" here with my free software hat on; when i work for money, i do work for money)

well, there may be a chance if array programming becomes a lot, lot more widespread

@rak1507 Doesn't R qualify as an array programming language? It's reasonably popular.

I guess it could, don't have enough experience to really know

@rak1507 It has a lot of the same functions. I've taken some ideas from R with the label support.

@EliasMårtenson passes my requirement of implicit mapping of primitives

You mean the same behaviour of the maths functions like + on arrays?

Ok cool that gets my vote

maps arith primitives, check. uses <- instead of = for assignment, check. single-letter lang name, check.

@EliasMårtenson the fact that i don't need a loop to increment each element. That's a good enough sign of a lang being an array-y for me

R is definitely one of the better GNU projects

@ngn still needs == for comparison though

@dzaima erases half a check :)

R is really good. Its syntax is a bit all over the place though.

But I believe that comes from its legacy, as being based on S?

what's the difference between <- and =?

yes pretty much

@dzaima = is a top level operator

@Razetime unfortunately that tells me nothing

@dzaima <- is the reassignment operator. = works in simple cases, but I honestly don't know when it fails. The documentation warns about relying on =.

@dzaima = assigns in a normal context, but in function, it acts as a custom parameter specifier

<- always assigns to variable

This is a good article on that

@Razetime ah, so kind of like = vs := in python, except both are valid as a complete statement (not that i know python much better than R)

yep sorta like walrus

How come ⎕AVU exists when it's just ⎕UCS ⎕AV?

@rak1507 heh, it's not mentioned here

Weird

@EliasMårtenson dyadic ~ not a thing in KAP?

@rak1507 Not yet, no. I haven't implemented it. Seems like that's something I'll do tomorrow.

{⍺/⍨~⍺∊⍵} it is

You'll see several other functions that haven't been implemented yet as well.

Ok, no worries

Its implementation is trivial. I could do it right now if it wasn't for the fact that it's almost 2 here and I'm going out biking tomorrow morning.

Its implementation is trivial, so it's no big deal that it's not in the language yet

I guess it could be useful if it would make a difference to lazy evaluation

Speaking of features that needs an implementation. Do you know of an efficient implementation of the Gamma function? I need it for the ! function.

Not a clue, I would use a builtin function if one exists

@rak1507 There isn't. Java doesn't have support for Complex numbers, which is why I have to reimplement everything.

@EliasMårtenson i tried to find one once, but gave up ¯\_(ツ)_/¯

Oh, that is annoying

There are Java libraries for it of course, but remember that KAP is a multiplatform project, so I'd need Kotlin implementation.

I will likely look at Maxima and see how it's implemented there. Mainly because I'm a contributor to that project so I know it quite well.

mathworld.wolfram.com/GammaFunction.html there's probably something you can use here

@EliasMårtenson gitlab.com/n9n/apl/-/blob/master/apl.js#L591-614

@rak1507 Yeah, that's a tricky issue. For now it'll have to force a collapse because a lazy-evalauted result must have a shape. There is no way to defer the computation of the shape.

Actually maybe not

I have some ideas how to do it. We'll see if I implement it.

Is there a concept of infinity in KAP?

@rak1507 Well, kinda. Floating points numbers can have infinity as per IEEE. But it's not formally supported. I've decided to hold back on that until I implement support for bigdecimal and bignum.

@EliasMårtenson is it intentional that .1 is invalid number syntax?

@dzaima Yes.

I don't think I've written a decimal number like that in my life :-)

Cool, btw +.×⍨ (and same with outer product stuff) is invalid, but {⍺+.×⍵}⍨ works

I've seen it, of course, but always dismissed it as a US thing. We never did in in Sweden when I went to school (and we use , as decimal separator too)

@EliasMårtenson it's useful for writing common things shorter - .5*⍨⍵, .5×⍵, etc

@rak1507 That's because the ⍨ binds to the ×, not the +.×.

@EliasMårtenson Sleef has tgamma and lgamma implementations (reference here), which I expect would be state of the art in performance at high accuracy (1ULP). Code is open source but might be hard to port since it's pretty macro-heavy.

Ahh

I just changed the way that stuff is parsed to make it better, but I haven't updated the web version yet.

@EliasMårtenson that's incorrect parsing of operators

@dzaima Please test again tomorrow :-)

@EliasMårtenson i wouldn't write it on paper that way either, but for constants intentionally less than 1 i often write it as such in code.

@dzaima I mean, adding support for it in the tokeniser is trivial. I just never thought about it because it looks so weird to me :-)

@dzaima BQN also specifies that .1 is illegal.

sue him :)

@dzaima in a java project more about floating point numbers that i have, there are 24 occurrences of [^0-9]\.[0-9], and 26 of [^0-9]0\.[0-9]

@dzaima (most of the no-0 ones have only 1 digit after ., whereas many of the ones with a starting 0 continue with more zeroes)

Anyone know if sample solutions to phase 1 of the problem solving competition will be published?

@dzaima You're right. I didn't even realise I do that until I started thinking about it just now. I do write .5 sometimes when I mean one-half. I think it's special because I read it in my mind as "one half" and not "point five"

@JamesHeslip I don't think it can hurt. Email me their email addresses and I'll add them to our internal beta tester list.

@Adám Done.

Anyone know how I can get ≠ (1 1)(1 2)(1 3)(2 3) in extended to work as it does in dyalog 18?

@rak1507 isn't that particular example just 1 1 1 1? Either way, monadic ≠ is ⍧ in Extended

Thank you! and yeah it is lol, maybe a bad example

Changing the way primitives works is quite annoying...

@rak1507 AFAIK yes, a "cream of the crop" solution set will be published eventually

Cool