5:47 AM
6:26 AM
Wrong it is better the normal loop because it is possible to write it without create intermediate array of binary for calculation in pseudo code if aa is the filter and w is the array {j=1;r=@;k=len(w); goto B; A: if aa(w[j]) then r=r,w[j];++j;B: if j<=k then goto A; return r} no intermediate binary array only the array result,
2 hours later…
8:47 AM
2 hours later…
10:47 AM
Still, @arcfide, I think it'd absolutely be worth drawing those points out in a blog post - how removing the each and being less abstracted changes performance
In particular: the trade-off (or false trade-off?) between generalised iterators (filter etc) and the inline, 'whole array at once' solutions. This is pretty much what I was getting at with stackoverflow.com/questions/51916990/…; also, there was a good exchange about it 8 (!) years ago here: reddit.com/r/programming/comments/ce7cx/comment/c0s93d8/…
1 hour later…
12:01 PM
Just under 2 weeks until Dyalog '18 - you can see the full schedule at https://www.dyalog.com/user-meetings/dyalog18/programme.htm
Tweeted by dyalogapl on October 16, 2018 at 9:48 AM
Make APL Great Again. https://twitter.com/dyalogapl/status/1052134125421940736
Tweeted by philipthrift on October 16, 2018 at 9:57 AM
2 hours later…
2:02 PM
@dzaima
so one can consider one filter primitive F for array for to see the one is more fast
r←(q F) w;k;i;c
k←↑⍴w⋄→E×⍳1≠⍴⍴w
r←''⋄→A×⍳''≡0↑w⋄r←⍬
A: i←0⋄→C
E: r←⊂,¯1⋄→0
B: c←q w[i]⋄→E×⍳(c≠0)∧c≠1⋄→C×⍳∼c⋄r←r,w[i]
C: i+←1⋄→B×⍳i≤k
the argument q would be a function that return only 1(true) or 0(false)
the argument w should be one array or list [should be ⍴⍴w=1?]
something as
{0=3∣⍵}F ⍳20
┌6──────────────┐
│ 3 6 9 12 15 18│
└~──────────────┘
and make a time competition with {(⍺⍺¨⍵)/⍵}
so one can consider one filter primitive F for array for to see the one is more fast
r←(q F) w;k;i;c
k←↑⍴w⋄→E×⍳1≠⍴⍴w
r←''⋄→A×⍳''≡0↑w⋄r←⍬
A: i←0⋄→C
E: r←⊂,¯1⋄→0
B: c←q w[i]⋄→E×⍳(c≠0)∧c≠1⋄→C×⍳∼c⋄r←r,w[i]
C: i+←1⋄→B×⍳i≤k
the argument q would be a function that return only 1(true) or 0(false)
the argument w should be one array or list [should be ⍴⍴w=1?]
something as
{0=3∣⍵}F ⍳20
┌6──────────────┐
│ 3 6 9 12 15 18│
└~──────────────┘
and make a time competition with {(⍺⍺¨⍵)/⍵}
2:32 PM
I just don't understand you wanting that function if it's gonna be slower always than writing proper APL
4 hours later…
6:22 PM
@Adám is this a bug? The funcion seems to work fine with odd numbers but not with even numbers ⍨. Also, for some reason the error looks weird
1 hour later…
7:44 PM
@RosLuP Here's another core "mental shift". In addition to what I wrote the other day, one is best served to stop "thinking with loops" and stop any notion that APL is "simulating" anything. Both of these enforce a mental notion of abstractive indirection that is antithetical to high-performance, direct, readable APL code.
In the short term, the use of an explicit looping construct, or the use of Each, enforces element at a time execution of complex interpreted functions over small amounts of data with significant structural overheads in order to develop the correct solution.
In the case of the current and foreseeable interpreter technology, this is obviously a problem across the board.
Now, the cases where this might be useful is when you are working on chunks of large amounts of data that need to be processed, but that data is too large to fit into memory. In that case, if MMAPing files isn't going to work for you, then chunking the data up and loading in slices at a time can be useful, and there the overheads of Each are overwhelmed by the amount of data you are working with.
However, that really only comes into play with data in the 32GB+ range of working memory requirements.
Anytime where you begin to think in terms of low-level, inner code iteration using complex (non-primitive) user-defined or derived functions on small sized arrays, you're destroying your code on multiple leves.
The value of a compiler that allows you to get performance from very poorly written, less readable, less clear code is questionable at best.
These algorithms are insanely expensive, break down, and often can't handle programs of any meaningful size.
If you wrote APL like that, it would be very easy to result in quad+ nested loops on a single line, and then have a series of 20+ lines of that code in a single function.
APL is a tool of thought, and if you can't think fast enough, no amount of tech is going to be worthwhile.
And if it takes 40 minutes to get an answer or 5 hours to compiler the answer to improve the performance to 1 minute or one could simply rewrite the code to be clearer, shorter, more direct, easier to read, and faster at the same time without the need for a compiler, which one are you going to take?
Because not only is that easier to read code easier for the human to read, it's also vastly easier to compile efficiently.
There are issues with the current interpreter technology related to excessive intermediate arrays (the interpreter actually does a better job than you might think at this), and those can be solved, but the good news is that not only can those things be solved, but if you write canonically good APL code, the issues can be solved using very dumb/simple algorithms for doing this fusion, and across a wider array of possible code forms, than is possible with the vastly more complex solutions.
In the same way that lexical scoping helped functional programming languages handle closure creation in a sane way that was performant, the directness of expression in APL permits most widely desired fusion operations to be performed with a high degree of success using dirt simple algorithms that don't require any explosion in the asymptotics of your compiler.
8:01 PM
Anyways, the moment that you introduce a goto, a loop, an Each, or some other form of such statements into the high-performance internal parts of your code (the "inner loop" sections of code written in other languages), you destroy that symbiosis and make it significantly harder to get good performance.
And what's worse, is you don't gain anything in terms of readability, ease of manipulation, efficiency of coding, or the like.
The 5% good is that it allows someone who doesn't understand the canonical APL form to write something down, even if it works more like their traditional FP languages or Python types code, since that may be all that they know. And from there, one can begin to see patterns that allow for the manipulation of that code into something more canonical.
So, it can be a good sketching tool, and a good learning tool, and it's helpful at the out edges of code to orchestrate chunks of data munching, but there are very clear lines about where that is good and bad.
When I give workshops, I rant on this a bit each time, to ensure that the functional programmers in the crowd understand that they are almost always wrong at the beginning when they start trying to use Each, Power Operator, or the like to solve their problems.
Tacit functions don't really count, but yes, I would say that if a beginner has managed to start doing tacit programming and the nesting level begins to exceed 2 then that newb should take a step back and re-assess.
This is a full compiler front-end, and you can go back and see previous versions of it that do some of the optimization passes, too.
You'll notice that there are very few, and the ones that are there, follow a very particular, peculiar pattern.
1 hour later…
9:16 PM
@Quintec I don't think I answered your question directly. In short, rather than apply your own user-defined function to an array using either each or writing the function so as to handle array at a time thinking, just inline the expression directly where it is used rather than creating a function abstraction to contain it. That is usually the right answer.
10:14 PM
@Quintec In most cases you need an expression that is a composition of primitives that will result in a Boolean array. This is, in the strict sense, the application of a series of Boolean functions to an array. However, I was just trying to highlight the difference between a composition of primitive Boolean functions composed together to form a direct logical expression as the left argument to Replicate, and the use of a separate, user-defined Boolean function being applied.
A Boolean function which is lifted to the higher-ranked array domains is more reasonable than a Boolean function that works only on scalars, but the main point I was making was about avoiding, if possible, the indirection of having a separate Boolean user-defined function rather than just having that function inlined.
« first day (629 days earlier) ← previous day next day → last day (2015 days later) »