Aw geez, I had tried variations of that, I knew it must have needed ≡¨ and ⊂ but I guess I just hadn't gotten it right! That gives me the bool vector. Hmm I need to study that and why I didn't get it right. Thanks!
@Bubbler wow that's even better, all in one step. More studying to do, thanks
Ah, I'd tried applying ⊃ to v, but your solution is from the other side, making the search str into an array -- ok that makes some sense
OK, I apologise for the soapbox, but I now think I have another reason why dfn's are not optimised in Dyalog: It's hard, and I'm not sure it's really that important.
I specifically wanted to make {⍺+⍵}/⍳N roughly the same speed as +/⍳N. After I implemented support for specialised arrays, the latter is significantly faster in both KAP and Dyalog.
Turns out that in order to show that {⍺+⍵} is equvalent to + in the general case (not specifically just a single function) you need to do flow analysis to determine where the arguments are used.
<moon-child> not sure how the implementation works, would have to check the source
<moon-child> the problem isn't flow analysis, but name resolution / late binding. E.G. imagine f←{x⍵}. Depending on if you follow that with x←+ or x←5, the 'fast' form looks very different
<moon-child> if not, then either you don't allow corecursive functions (bad), or you have the same issue as dyalog. Dyalog doesn't allow changing name class either fwiw; x←5⋄x←+ will fail
Oh wow... The JVM just gave me the weirdest problem. I mean, dynamic compilation is great and all, but this issue is bizarre. The following code runs in 0.5 seconds in KAP: +/⍳1000000000. I can run that code repeatedly and it consistently takes roughly 0.5 seconds. However, if I change the code around a bit, trying various variations such as +/1+⍳1000000000, then after a while things seems to slow down.
And when that happens, +/⍳1000000000 (the same as the original snippet) will now consistently run for 2 seconds instead of 0.5. No matter what I do, it's now running 4 times slower.
The only explanation I have is that the dynamic compiler recompiled the code in a more inefficient way.
(it's not GC by the way, I forced GC before the test function)
I do allow corecursive functions. You can refdefine a function. You just can't make a function into a value.
I was reading this, and I don't understand this quote: "The following J example does not use reduction, and is therefore not readily expressed using APL scan:" with the example J code being: <\ 'abcdef'
@Wezl nice fonts! an interactive demo would be nice. here is my take: ktye.github.io/ui.html#ui.k (there should be a black screen you can type on). this is the source that includes the font: ktye.github.io/ui.k
@EliasMårtenson I think it's the compiler is inlining & assuming things while evaluating that because it sees only one thing ever happening, but on the other expressions it's forced to deoptimize
(and a native build is forced to consider all paths from the start, so has that slowdown always)
@dzaima Yeah, I figure it's something like that. The problem in this case is that the initial version runs 4 times faster than whatever the JVM comes up with later. It doesn't realise that its new version is much slower.
I'm hoping this is only a problem with microbenchmarks, or I will have to start digging into how one can tune the JVM bytecode compiler.
And I can think of about 69420 other things that are more entertaining than that.
@EliasMårtenson microbenchmarks will stay artificially inflated, yeah. There's not much to tune here - inlining everything is faster than inlining nothing
@EliasMårtenson I think it's out of necessity. When compiling just +/⍳10000 it can assume that + will have both arguments as scalars. But after +/1+⍳10000 it realizes that's not true always, and now has to check on every invocation of + (or something like that idk what your impl actually does)
you'd ideally want the jvm to constantly reoptimize every builtin & function on every new line of KAP code executed, but that's quite a bit of work
except that the JVM doesn't need to generate unlikely code until it's actually needed
while compiling + it can just inline the invocation in whatever pervasion implementation (or even straight into the impl of /), assuming other pervasive builtins don't exist, for example. And when another one's class file is loaded in, it's forced to deoptimize that
@dzaima (and it kind of has to do that in the case of Java - the overwhelming majority of inline-worthy functions aren't marked final and thus can technically be overridden dynamically by loading in another class file (kotlin is better in this aspect with open and final by default))
It would be nice if there was a tool that allowed me to see the optimisations that the JVM performed. This is the first time since I started working with Java in 1996 that I have seen such a weird optimisation edge case.
@EliasMårtenson there definitely are flags to the JVM to make it spam you with a mess of optimization information. Problem is, it spams you with a mess of optimization information
@dzaima But Common Lisp allows you to assert things like "variable x is guaranteed to always fit in a register, so don't worry about checking if it needs bignum-extending"
One solution I'm considering is if I have two integers A and B which both fit in a Long, so when adding them the optimised path is called (fun add(a: Long, b: Long): Long), but if the result overflows I throw an exception that contains the result as a bignum.
Because in the JVM, a try/catch is very lightweight if an exception is not thrown.
I have only thought of two other solutions: One being a special value that is updated in s different place (thread-local perhaps?) and the other being a special integer value (Long.MAX_VALUE perhaps) which can act as a sentinel value indicating that the value is not valid.
@dzaima But that limits your integer size to 56 bits, no?
@dzaima Fair enough. Probably better to add that on a higher level, building it all on a simple bigint library. That allows me to leveral GMP for the native backend.
The thing that scares me though it making sure that all numerical operations support the type tower.
KAP still doesn't do the right thing for complex numbers for certain trignometric functions (I think I only support complex numbers for sin/cos/tan. And I don't even have the hyperbolic functions yet.
The thought of computing the correct result for arctan for a massive rational number is somewhat terrifying
@Adám Hello, APLers! I made my "unofficial" announcement of the Dyalog's student competition at CodeForces, a large competitive programming platform: codeforces.com/blog/entry/88048 – This post already gained some attention. So let me know if there are any critical mistakes in this post. @MortenKromberg, @Adám
@rak1507 , yeah, compared to SPOJ.com, AtCoder or even Open Kattis, CodeForces is much more boring in their language choices. But posting some blogs at CodeForces related to APL could be a good first step. For example, if someone posted elegant and concise APL solutions to a few problems from their archive, it could help promote the language, I believe.
@AndriyMakukha I'll keep that in mind! I've done a few in APL before but none were particularly interesting, maybe if I spot something that would be nice in it I'll post it
@Razetime These platforms are all good for studying algorithms and practicing coding. However, my favorites are CodeForces and Open Kattis.
@Razetime CodeForces publishes all solutions by all participants as open-source. So if you are stuck on a problem, you can learn by reading someone else's code. It also has lots of educational editorials, blogs, useful discussions and even live-streaming.
@Razetime Unlike feature-rich CodeForces, Open Kattis, on the other hand, is extremely spartan and simple. And there is a charm to it. Additionally, this problem archive is covered in a book on competitive programming by a Singaporean professor. (I don't have that book, but it would be handy to get "unstuck" if you can't get some problem right.)
Returning to what is probably a lost cause (arguing in this forum that Dynamic Scope is somehow "simpler" than Lexical), I think you provided a clue with your comment that "all dynamic scope does is lose information". I think that is exactly it: there is less information required to understand "what a function does" in Dynamic scope.
I am trying to chose my words carefully here: I am not claiming that dynamic scope makes it easier to predict what the entire system does, only what the individual function is trying to express.
I also disagree with 'there is less information required to understand "what a function does" in Dynamic scope. ', as you need to know exactly what every called function does. Can you tell me what {a←⍵ ⋄ _←foo ⋄ a} returns?
Nope, that was just a 'fun' answer, the point is that there's no clear way to analyse exactly what a function returns without knowing exactly how everything it calls works
Dynamic scope requires more information to fully know what something does
@rak1507 It depends on the definition of "does". The function calls a function called foo and then returns the value of a. That is a 100% COMPLETE description of what the function does. It depends not only on the "value" of a, but also the "value" of foo - but you can see that in the source.
@rak1507 The same way that you can reason about what an operator does without knowing what the operands are.
I guess my main issue is that with dynamic scope, there's no way of knowing just from the function calls if there are any side effects and what they do
There's no way to call a black box function and be confident it won't change anything you don't want it to without checking beforehand
@MortenKromberg Dynamic scope doesn't at all change how much information is required to understand what a function does. It just makes it harder to find, which I guess is what you see as a benefit
@rak1507 Anyway, I really just wanted to thank @dzaima for helping me realise what it is that makes Dynamic scope feel simpler TO ME. Not claiming it is better for "programming in the large". I think I might claim that it is better for writing simple models of the style that APL was originally invented for. Like the fact that one of the attractive things about spreadsheets is that you can always refer to a cell, you don't have to pass it to every function that will need access to it.
Yes, and the nice thing about Dynamic scope is you can replace the globals behind the back of some code and still have it work, without having to refactor it (I can hear the screaming echoing down the wires, so I'd better leave now).
@MortenKromberg why is ability exclusive to only dynamic scope?
you can change anything you want that given code doesn't touch it with lexical scope too, but at least it's much more clear what could be affected by the change
@dzaima OK, I may be ignorant about what is possible with lexical scope systems, but it was my impression that you could not refer to and change values in its caller?
@dzaima I'm not sure we're really disagreeing about anything(?). Lexical scope allows you to group things together and predict how the whole construct will work. Dynamic scope allows you mix stuff up in ways that are ultimately less predicable but require less up-front planning (which you may regret later).
@dzaima @rak1507 Perhaps we do disagree about whether dynamic scope ever has any merit whatsoever :D
"require less up-front planning" do you actually change which scope you want a function to read from or mutate often enough for dynamic scope to be beneficial?
@dzaima Probably not, but then I also don't modify semi-global values often enough for dynamic scope to be a problem. But believe it or not, lexical scope still feels so hard to accept when writing APL, I think because the fundamental unit of thought for me is the function. Nested functions make my head hurt. Why would you write a function that can only be used in one place, etc...
@MortenKromberg right, if you only access local variables 99% of the time, lexical and dynamic scope are equally good 99% of the time. We're discussing the 1% here though
@ngn I am comfortable with small dfns, which I think of as explicit (as opposed to tacit) compositions. Multi-line embedded dfns is where I start to feel uncomfortable.
Please note that I'm not trying to convince any of you that I'm right and you are wrong, that dynamic is better than lexical, I'm trying to explore this feeling that I still have that dynamic scope is simpler, in order to understand it.
APL is a tool of thought, and if it doesn't help me think, it ain't working
@dzaima Yes: one of the things that I think we SHOULD do is come up with a way to write a tradfn in which all names which are defined using assignment in the body of the function are automatically localised.
IMO, using ∇ to close the line editor was among the biggest ever design mistakes of APL. An it isn't even "core language" or even "language".
If it had been ∆ then lexical scope would have become an APL thing much earlier, since nesting functions would have been obvious. Without nestable functions, lexical scope can't work.
That said, it bothers me that InnerProcess has to be redefined *for every element of largeData:
OuterProcess←{
acc←⍬
InnerProcess←{
⍝ long complicated
⍝ function that
⍝ eventually
⍝ computes a
condition←?2
⍝ then does
condition:acc,←⍵
}
acc⊣Process¨⍵
}
OuterProcess¨largeData
I cannot define InnerProcess once and for all, outside OuterProcess because of lexical scoping.
If I write one of the two processing functions as a tradfn, then lexical scoping applies, and I can get away with defining InnerProcess only once.
I know it sounds bizarre: The reason for this is that many existing apps are starting to become littered with little dfns, because of the attraction of writing short inline expressions with locals that do not escape. This works fine until you get a mixture of dynamically and lexically scoped things on the stack, at which point the programmers who are used to dynamic scope can get very confused about the situation they have created.
@Wezl the glyphs are bitmaps encoded in char vectors. 0x... are char literals in that version of k. the font has been created with the great Windows paint. here are more fonts and converters: github.com/ktye/i/tree/master/_/i2/kui/f
@ngn and, as I said, by looking at just modification (and ignoring modified assignment which you want to for some reason), you can't modify any parent variable, so it's also no-scoped
what will happen if one dfn "sees" a local variable from another, but somewhere in the call stack there is a tradfn with a local variable of the same name?
