The APL Orchard

12:01 AM

@Ada Sort of. So, × is also overloaded; monadic × is different from dyadic ×. The main trouble with / is that it's actually specified inconsistently. Which in principle could mean there are disputes over how to interpret it, but in practice it's not as big of a problem. (See here)

@Ada Though note also that in e.g. dzaima/apl, / is always reduce and ⌿ is always replicate

Ada

Yeah, I've seen dzaima's apl a bit before.

Marshall

12:12 AM

Self-hosted BQN up to about 6 characters per second now. Exciting stuff.

Moonchild

12:24 AM

@Marshall probably still faster than the earliest versions of perl6, and they're now faster than the other mainstream scripting languages

Bubbler

1:05 AM

Started writing a review on the competition problems (best viewed on nbviewer with font override, or Binder)

Jo King

1:31 AM

@rak1507 My phase 1 and 2 answers

3

Jo King

1:42 AM

@Adám I'm pretty sure I submitted a file for Phase 2, but when I checked the website to get my Phase 1 answers, it says No entries submitted for Phase II. Can you check if it actually went through?

Bubbler

3:30 AM

Digging into (mostly working but slightly buggy) APL syntax highlighting support in Jupyter notebooks, it turns out it uses CodeMirror's APL highlighting which lacks a few Dyalog built-ins.

Function names look like APLX's (by monadic ⊣ being "Stop")

Bubbler

4:12 AM

Erm, no, it looks like Pygments

Bubbler

4:25 AM

Actually both are right, I guess. CodeMirror is used for live notebooks (and it is almost perfect because Dyalog kernel comes with custom CodeMirror highlighter), and Pygments is used for static viewers which use nbconvert (including GitHub preview and nbviewer)

Adám

4:36 AM

Phase I (we haven't finished judging phase II):
73 participants with at least one correct solution, of which 38 self-claimed students
61 with more than half right, of which 29 self-claimed students
47 had a correct solution to all problems, of which 24 self-claimed students

Bubbler

@Adám So it's around 1:1 on all levels, interesting

Adám

@rak1507 About 33 years, but I didn't really get into it until 22 years ago.

Adám

4:50 AM

@rak1507 My father taught me a lot, but I also read An interactive approach and APL2 at a glance and many APL Press books including Algebra, but most important was not being afraid of experimenting.

room topic changed to The APL Orchard: Learn, teach, ask, code, golf, & discuss. See aplwiki.com/wiki/APL_Orchard for access and info. [apl] [array-manipulation] [j] [k] [tips]

@JoKing I don't see it.

Jo King

oh dear...

@Adám can i submit it now?

Adám

5:16 AM

@JoKing I think the system will refuse, but I suggest emailing it to contest@ will an explanation. I'm right now asking my colleague in charge of the database to look into it.

Jo King

will do! i really should have double-checked it earlier. i bet i just uploaded it without actually clicking "Upload"

TessellatingHeckler

5:33 AM

@Adám inside my cmdlet, I ⍎ whatever string comes in, and pass the result to the output function. If the string contains something like "⍵[1]←5" then there is no output, and I get a Value Error. I have found the dfns.dyalog.com/n_do.htm page about a way to apply a function which is known in advance to have no output, but what could I do if I don't know whether the function will have an output or not?

Passing a PowerShell scriptblock in, executing it inside Dyalog APL, having the powershell cmdlet print to the console. And setting up an array in PowerShell, passing in a Powershell scriptblock which will modify it, and it gets modified in the outer session.

and works without the powershell scriptblock part, which is slightly more impressive

PS C:\> $hash = @{"abc"=1; "efg"=2}
PS C:\> ⍎ "hash←⍵ ⋄ hash[⊂'abc']←17" $hash

that assignment inside APL affects the powershell $hash variable

ab5tract

6:11 AM

Ihave been thinking about it and I think the reason that I view this "FP is correct enough for general applications" as some degree of
"dangerous" is that it just handwaves the entire error domain as inconsequential without any regard for actually codifying the impact. Yet when we deal with the only "real" numbers in the sense that getting them wrong affects the real world in an immediately tangible way, we know for a fact that FP is not good enough and do not use it at all.

There is no way we would justify using FP if there were not an overwhelming cost to arbitrary precision rationals. You won't be able to convince me that the approximate has any value next to the "actual" without the performance implications. Therefore it is vital to ask every once in a while: "Does it still cost too much to do this correctly?"

Or to slightly mangle the idiom that refactoring is how much money you are willing to bet that your refactoring will work as expected in production, I think it is highly suspicious that the one time programmers have to put real money down on their technical decisions, they don't trust FP anywhere near the operations.

Adám

6:49 AM

@TessellatingHeckler Append ⋄0 0⍴0 to the string before ⍎ing it.

@TessellatingHeckler Are you sure you need hash←⍵? I think ⍵[⊂'abc']←17 works.

@ab5tract Maybe an alternative is ball arithmetic where we continuously track how much error we may have accumulated?

ab5tract

@Adám Interesting, I hadn't heard of that technique before! I should note that

Adám

@Moonchild I'm ready.

@ab5tract NARS2000 implements it.

ab5tract

Oops, lost my edit due to missing the timeout. Meant to say "I should note that I am not advocating abandoning FP (that's simply not possible) -- only pushing back at the widespread reasoning related to why FP is 'mostly ok'".

I think the sweet spot is having some way to toggle arbitrary precision without changing any code other than the config line (a ⎕ in the APL case). Unless you are doing gradual typing at which point the Rat as the "highest order and therefore default" representation makes the most sense.

@Adám cert issues :/

RGS

7:10 AM

@Adám why is that helpful? The error is going to be an estimate by excess of how wrong the result might be, but it won't tell us by exactly how much we are wrong nor the direction (if we are too high or too low) so what is the point?

Unless I'm misunderstanding something, in which case please correct me.

Bubbler

@RGS You can at least be sure that the error doesn't grow too much over some threshold that your application would allow.

RGS

@Adám "X had a correct solution to all problems" means "had 10 golden trophies"? Or do you have further tests that could mean I can get less correct solutions than golden trophies?

ab5tract

@RGS You could save a more expensive round of arbitrary precision code that runs only for members that exceeded the threshold

RGS

@ab5tract @Bubbler I see, makes sense.

ab5tract

It would likely result in quite some timing variations for the same code against different data depending on how often that threshold was reached. I wonder how large those effects would be in real world usage.

At least with Ball you are no longer in the dark about just how wrong your numbers are, so it takes care of my complaint about the impact of FP imprecision not being measured before being dismissed

RGS

7:19 AM

But how are these error estimates computed..? With FP? Then those can go very wrong as well, so I'm lost again... Especially when I think about the fact that those estimates have us potentially combine large and small numbers :O which doesn't sound good

Adám

@RGS It means any trophy, even silver. The only problem I have more tests for is P10, namely the argument 'ab'

RGS

@Adám +← 1

@Adám I did notice there was some discussion here about it; what should the result be? For 'ab'?

Adám

a
b

@RGS Theses are not estimates, they are indications of the maximum error. If FP can't represent the maximum error exactly, simply take the next higher FP value, since the error surely is less than that.

ab5tract

@RGS Now you are stumbling into the exact dark corners of FP that lead me to seek arbitrary precision rationals so I don't have to worry about any of those concerns while trying to Just Do Some Accurate Maths

RGS

@Adám oh bummer

@Adám but surely the error an operation incurs in is dependant on the operands; how can you cap beforehand how much error there is involved?

Adám

7:25 AM

@RGS We very clearly stated that the result should look like {⎕←⍵}¨ :-) Don't worry, I counted failure on that point as -0.2 (silver=1, gold=2).

@RGS I don't understand. It is known how much the error will be for any given (basic) FP operation on any given FP number(s).

RGS

@Adám hm, do we agree that said error depends on the values involved in the basic FP operation?

@ab5tract yup, and why are you so concerned with JD'ingSAM?

Adám

7:44 AM

@RGS For sure.

RGS

@Adám yes you did, my bad I didn't realize character vectors weren't being handled correctly

Bubbler

@ab5tract But if you want really accurate, you can't represent irrational numbers in Rat (trying to do so will give you at least some amount of error), and then what you need is a CAS.

RGS

@Adám ok so what are you going to do..? Are you going to tabulate the errors a priori and then look them up?

Adám

@RGS While unintentional (I didn't realise that 'ab' would be substantially different from 'a'0, which was a tested-for edge case), it turned out to be a way to grade submissions on a finer scale.

@RGS No, there are formulas for each operation.

Moonchild

@ab5tract real-world applications especially are never about 'completely accurate'; always about 'to within an acceptable margin'. To be sure, you have to know what that margin is, and whether your calculation method is appropriate, but fp is absolutely justified sometimes. I think you're right that a lot of people use it without thinking, for applications to which it may not be suited, but that doesn't mean it's useless

@Bubbler (and I maintain that an APL CAS would be great)

RGS

7:52 AM

@Adám yup I understand :)

@Adám exactly, formulas; are you going to compute those formulas using floating-point arithmetics? Then the results of the formulas might be rigged as well!

Moonchild

@RGS you can implement arbitrary-precision arithmetic with integers

Adám

@RGS The formulas can well be designed to overestimate correctly. It could even be done with intervals, if a lookup table is absolutely necessary. I am sure it is possible to always return a ball with a guarantee that the real value is inside.

Moonchild

@RGS (or, more likely, you implement rats; since fp is basically fixed-radix with a limited number of digits, error of arithmetic operations on representable numbers is rational)

ab5tract

@Moonchild I think that this is just tautological reasoning -- it doesn't matter because it doesn't matter. I haven't heard any argument for using FP besides the performance. Are you really saying that given 0 or minimal performance overhead you would still use FP? If so, under what line of reasoning?

RGS

@Moonchild if the error estimates are computed with arbitrary precision, might as well just have calculated the original operation in arbitrary precision, no?

@Adám hm ok, I guess that is feasible... Even though it sounds annoying :p

Moonchild

8:01 AM

@ab5tract no, I'm saying that given the (inescapable) performance characteristics of fp, its accuracy characteristics are an acceptable compromise for some applications

ab5tract

@Moonchild That's not the argument I'm making. And considering how few languages even allow for arbitrary precision natively, I think the amount of calculations that are shunted into "well it's only an approximation, I mean, who is even going to use this software to be the only source of information for their decision?"

without any evidence that the data mangling actually has the minimal impact that they presume

is a far larger problem than programmers are willing to admit. If you make it too costly to do it correctly, it will generally be done wrong

RGS

@ab5tract "If you make it too costly to do it correctly, it will generally be done wrong" is a sentence with arbitrary precision

Adám

@TessellatingHeckler OK, John now understands the issue and has updated the internal issue tracker accordingly. Can't promise anything, but at least there's an awareness. Maybe we can include it when we get around to review our OO. We're also missing abstract methods, and protected fields, and it'd be nice to have public operators and dfns.

ab5tract

@RGS I always prefer my turtles to go all the way down :)

And when you develop a culture that any questioning of FP's legitimacy to the crown, so to speak, is considered to be some kind of kooky side issue that "all good programmers" already know how to deal with properly.

Then people are not likely to even know that there are issues with FP representation in the first place because the culture at large has already declared it "not an issue"

Adám

@ab5tract There are plenty of articles, tutorials, and videos out there, and I sure hope they teach FP issues as part of CS.

ab5tract

8:14 AM

CS is only a subset of programmers, and a pretty small one at that.

And if the education says both "there are terrible dragons here who will devour dollars, so be careful" and then "for the vast majority of applications FP is just fine"

How do they know when to choose the difference? Is it in some kind of government qualification for safe software?

And how can they even quantify when to switch between the two?

Adám

@ab5tract True. I personally know of a large and respected financial services company that uses FP for money. They say they can't switch to integers (of the smallest used unit) because the accountants would not accept that all amounts suddenly were changed. So instead, they keep the "lie" alive. I wonder how much money is created out of thin (FP) air instead of being issued by a government…

RGS

@Adám (this has been mentioned in this chatroom before, hasn't it?)

Adám

Possibly, but it sure is relevant here.

ab5tract

In the words of critical media theory, "the workings of a system become visible only once it no longer works". So those articles are likely to be encountered after someone encounters whatever the potential failure mode is in the first place.

@Adám That feels like a decent anecdote for my argument :O

@Adám It's exactly this "known unknown" of an unquantified/unquantifiable degree of rounding errors that surprises me so much when FP concerns are so easily brushed off. No one has done any real accounting on it but people who usually require extensive data to trust in something seem strangely fine with just trusting their gut on this one.

(Please note that I'm not addressing my criticisms against anyone here in the chat! I'm speaking about how these discussions unfold on a large scale)

Adám

@ab5tract Humans have a curious tendency to trust their gut on unknown facts. Obviously, heavier objects fall faster, diseases can't be spread by something invisible, information cannot be transferred through solid objects, and maggots are formed of old food.

ab5tract

8:28 AM

@Adám And those beliefs always work just fine -- until they don't :/

Adám

At least FP errors rarely kill people. In fact, I know of a case where they saved people.

ab5tract

@Adám It's absolutely unclear to me how accurate that statement is precisely because of how unknown the effects of the problem are. Who knows what kind of business decisions have been made due to butterfly flaps in FP&A calculations?

That said, I do hope you will now share this example!

RGS

@ab5tract slightly related but on the joking side of things

ab5tract

@RGS :D bats do look pretty tough when they are hanging like that. pretty cool that they get their own permanent black leather jacket xD

dzaima

@ab5tract ok, that is a separate, probably reasonably real problem (that also happens to be applicable to very many other things in programming). It's not some magical thing that always works, but it's still more than acceptable when the error of the input/output is several magnitudes above the error of fp (and in the case of analog i/o rationals are mostly precisely pointless)

Adám

8:36 AM

@ab5tract A rabbinical school room-mate of mine went on to work for one of the worlds leading electronic medical equipment companies, refining control systems for radiation therapy machines. When they improved the algorithms (thus lessening the errors in computations) a the patient survival ratio dropped statistically significantly. Turns out they had gotten rid of a radiation spread which critically killed of metastases just outside the tumour.

ab5tract

@dzaima Indeed I think we are on the exact same page, just reading different paragraphs or summarizing the content differently. I'll sum up what I'm arguing here: for my personal definition of "21st century programming," it should be trivial to switch between fractional number representations.

@Adám Wow, that's a very special case indeed :O However, considering that this was an unintended radiation spread, if it had been doing anything besides targeting cancer the chances of a positive impact seem remote.

dzaima

@ab5tract though i fail to see how rationals would help much. They limit you to a very few "acceptable" operations, which mean they're unusable for wide classes of things, and in the cases they can be used, i'd expect using just integers would probably be much better anyways

ab5tract

@dzaima What operations are they limited to? In Raku they only degrade to floats when you put floats into the mix

Bubbler

@ab5tract you lose pow, exp, log, trig to name a few

ab5tract

8:53 AM

```

`> (my $f = 4/5).WHAT.say; ($f ** 10).WHAT.say
(Rat)
(Rat)`

dzaima

@ab5tract ^; pretty much the only operation benefitting from you using rationals is division, and the amount of cases i divide by an arbitrary number is rare enough for it to not be worth a data type

ab5tract

My experience does not agree with these limits you are describing

At least not all of them

dzaima

@ab5tract is 10 a rational?

ab5tract

10 is an integer

** is exponentiation

Bubbler

@ab5tract If you give a fraction as a power, you most likely get an irrational, e.g. 2**(1/2)

ab5tract

8:57 AM

Sure, but you didn't say that rationals can't handle irrationals. You said you lose exponentiation

dzaima

(as a side-note, a bad thing about arbitrary precision rationals is that if you're not careful, they can explode in size very easily, and you'll never really care about the precise last digits of a 1000-digit rational. fp at least won't allow a single calculation to DOS your program)

ab5tract

```
($f ** 1/2).WHAT.say
(Rat)
```

Moonchild

@ab5tract that is ($f ** 1)/2. Try $f ** (1/2)

Bubbler

@ab5tract What would you say the result of 2**(1/2) in rational, then?

Adám

@ab5tract Ctrl+K for a multi-line monospace block.

ab5tract

9:00 AM

Yeah, it falls back to Num. But I don't get the argument here.

Bubbler

Representing an irrational number in rational surely gives you some amount of error.

ab5tract

"Rational numbers can't handle irrational values" seems like a small categorical issue with well defined boundaries

dzaima

@ab5tract you do lose (the less used) half of exponentiation

ab5tract

unlike "I have personally decided that this math only requires approximations"

Bubbler

Then let's list what can be done: add, sub, mul, div, sign, abs, min, max, integer power.

ab5tract

9:03 AM

and equality

Bubbler

ok, comparison.

dzaima

@ab5tract have you ever needed to check 2 non-int numbers for exact equality?

ab5tract

not exactly an uncommon operation

@dzaima have you ever used a language where you actually can use equality on fractional numbers?

Adám

@ab5tract NARS2000 allows that.

Bubbler

@ab5tract And J.

dzaima

9:08 AM

@ab5tract i haven't, but i've never felt the need for it (possibly due to the domain being often being screen-related or angles; i was more curious as you do have the operation available and i just fail to see any case it could possibly ever be the correct thing to want)

ab5tract

What I'm saying is that the question asked about exact equality of fractionals is precisely oriented towards what doesn't work about FP. You can't do exact equality so it is assumed to be useless. Again, just confuses me.

@dzaima so your position is that arbirtrary precision rationals cannot be used for money calculations?

or statistics, which governments, militaries, and businesses all rely on to decide how they are going to run their corner of the world?

Bubbler

My argument is that you'll eventually need FP when you need to evaluate certain math functions.

dzaima

@ab5tract why would i need exact equality comparison for money?

ab5tract

That's so irrelevant to my point -- it's never something I argued as untrie.

Bubbler

@ab5tract I do know some money stuff needs logarithm to solve, which definitely can't be solved in rat.

dzaima

9:11 AM

@ab5tract ah, that wasn't a reply to my message

ab5tract

@dzaima I honestly don't think I can convince you of any utillty of arbitrary precision if you don't find the idea of being able to accurately compare fractionals is of no practical use to anythingto be of any practical use

user435118

Hi. I’d like to explore Code Golf and this site a bit. What is the best language to learn and where can I get started?

Bubbler

@ab5tract And multiple functions in statistics require square root.

ab5tract

uff, that was a mangled sentence that I didn't manage to fix in the edit

@Bubbler All the more reason to have as accurate a representation as possible prior to engaging in a lossy operation

Adám

@Daniil This room is for APL, which we'll be more than happy to teach you here. APL is interesting in a code golf context for being both fairly competitive relative to dedicated golfing languages, while at the same time being a "real world" general purpose programming language. If APL isn't for you, then The Nineteenths Byte is a better place to go, it being the general chat room for the site.

dzaima

9:19 AM

@ab5tract If you only ever divide once, by a constant, you're much better off by just pre-multiplying everything and doing integer math. If you are possibly doing division by an arbitrary number (doing division by a constant many times being equivalent to an arbitrary division), the numerator is going to be mostly a random number, and equality checking it is not very useful as you'd need to have calculated it precisely by the same expression

user435118

@Adám Ok thanks, I’ll go to the main room

dzaima

@dzaima (if checking if the thing's been calculated by the same expression is the goal, then you also have the chance of hitting the specific rational by pure luck, which might or might not be desired)

@ab5tract if dealing with roots and not needing symbolic calculations, fp is definitely good enough. The error is tiny in general (iterated sum of a large array being about the worst thing, and it's still good enough for most purposes, the dozen or so calculations pre-root would have almost no effect to anything), and you will have to do some rounding at some point

ab5tract

@dzaima You keep arguing about domains that FP maps to fine. My argument is completely orthogonal: your code should not look any different when you notate a mathematical thought that requires arbitrary precision to one that doesn't

dzaima

@dzaima (i'm thinking about the general case here obviously; in the off-chance that the division is controlled, there might be some utility for equality, but that's a very specific case that i don't think would be hit often enough)

ab5tract

We have difference in opinion: to me math should be accurate as it's first priority in all cases until I ask it not to

Or need it to be due to limits in rational representation.

To pretend that your case negates my case is just weird to me especially since I'm not arguing that my case negates yours.

dzaima

9:31 AM

@ab5tract then i guess my argument is that it's not worth making root/log/exp/trig harder to use (i.e. in a typed language where an argument to a function has to be either a rational or a float, or in a dynamically typed language where knowing the type might be very important but its not described anywhere) just for the off-chance that the user wants precise calculations (which they probably won't)

ab5tract

@dzaima I will bet a million invisible internet bucks that a majority of programmers don't even know their calculations are inaccurate

:)

dzaima

@ab5tract again, probably true (also depends on at what point does one become a "programmer"), but of no fault of fp

ab5tract

" in a dynamically typed language where knowing the type might be very important but its not described anywhere)" -- not sure what you are describing here. dynamically typed languages that include rationals in the native library is a population of one that I'm personally familiar with

@dzaima That's again orthogonal to my point. This all started because I dared ask whether doing fractional math with actual precision was performant yet.

It is you who took up the position that not only is it not performant, it is never useful.

dzaima

@ab5tract ah, it appears raku allows specifying types of function arguments/result, which is an improvement over most dynamically typed languages

ab5tract

yeah it was designed specifically to rectify the worst sins of Perl 5's attempts to make life as simple as possible WRT numbers

like interpreting "123abc" + 2 as 125

now I would argue in the case of gradual typing you always have to start with the most accurate representation and degrade gracefully from there

dzaima

9:42 AM

(my thought originally being it'd be quite hard to tell whether a function gives the result as a float or rational, which, as we now know, can have consequences)

ab5tract

what I'm discussing here is more that the utility of arbitrary precision is that the code gets to remain the same

@dzaima true. I should probably reiterate my point that my comments are really only about what it takes to match my expectations of "21st century programming"

It's so much easier to have a style guide that enforce ⎕ARBITRARY ← 'Rational' in statistical analysis code than to worry about the knowledge (and impact) of fp imprecision being evenly distributed

and I would also say that I agree with you that FP is probably 100% fine for most cases

so you could first do all your work with ⎕ARBITRARY ← 'Bell' to see if there is even anything to be concerned about

but this is only actually usable/useful if you don't have to change the actual expressions in any way during these tests

dzaima

@ab5tract it's not never useful, what i'm trying to get at is that it's almost never better than the alternatives (either scaled integers, or floating-point; rationals are definitely prettier than scaled integers but i wouldn't say that immediately makes them better)

ab5tract

@dzaima It's the homomorphism of the "notated idea" that provides the value here

And if I recall correctly, there are quite some dragons to beware of in integer math as well that COBOL programmers know better than anyone.

dzaima

@ab5tract about the only problem i know of is overflow, and if that's a possibility, using arbitrary precision is an option (some langs allow erroring in case of integer overflow)

ab5tract

10:00 AM

Record-keeping errors during integer math could each have orders of magnitude of impact on the result

dzaima

@ab5tract i'm not sure what "errors" during "integer math" means?

ab5tract

I don't think it's too much to expect that a programming language can easily switch to a mode of operation that uses your computer to compute correct values. Even raising the question is informational to users. Providing drop in support provides a lot of value (to the people who use math to model the world) at the cost of implementor time. What is the issue we are arguing about?

@dzaima 100 * 0.75 become 10000 * 75 with a counter somewhere keeping track of your zeros. If you are saying everyone always writes perfectly working code then I have to wonder who you have been working with and are they hiring!

dzaima

@ab5tract sure, having rationals in a language is good. Having it be the-default-until-(log|trig|exp) is the weird thing though, especially if the conversion is transparent to the user

ab5tract

Considering the amount of garbage statistics in scientific papers that are only caught during experimental replication, the accuracy of computational output is maybe a bigger issue than you realize.

@dzaima this kind of conversion is so automatic as to be invisible already

7 + 3.5

you can force explicit type conversion in the expression or allow some form of overloading/multiple dispatch

dzaima

@ab5tract and that's the bad thing. If my calculations randomly switch from rational to float without my knowledge (i.e. calling a function written by someone else (or even past self) that could work just fine with rationals but for some reason doesn't), now precise equality checking might randomly fail

ab5tract

10:11 AM

if you are doing the latter, it is not surprising to the user whatsoever that a math operation ends up with a type that is a consequence of the arguments

dzaima

@ab5tract are those garbage statistics really caused by the 10^-50 error of floats?

ab5tract

@dzaima But I think we are safely on the same page now.

@dzaima That error rate is cumulative across calculations. Again my point is: we have no idea of the impact and I am not personally interested in accepting the shrug-and-move-on approach.

@dzaima When the type of the thing truly matters, the approach that relies on user type checking at the call site of where the typing is important (if you put a return signature on your function that needs a Rat then you won't be able to output a Num) does look like the version that requires explicit conversion.

dzaima

@dzaima (2^-50 actually)

ab5tract

It's in all the expressions where that isn't an issue that you save all the extra typing

But that is really about language design and so I suggest we just let that stage of the discussion sit, at least for the moment.

dzaima

@ab5tract for basic +-×÷ the error should be ~1ulp (2^-52 for double precision), and even in the worst case bias of all going in the same direction, even with a million calculations you'd have only lost ~20 bits of precision and still have 32

ab5tract

10:21 AM

I don't see how you can argue that considering that multiply and divide have a much larger impact on the output for variations than add or subtract

You don't just lose data at the rounding errors, the data that arrives at the rounding errors is imprecise

And the impact of that will be entirely dependent on what you are doing to that data

dzaima

@ab5tract addition/subtraction is actually the worse offender in terms of error - it's much easier to lose precision due to very different magnitudes, whereas mul/div only change the error of the last bit or two (and keep the relative error of the arguments)

ab5tract

' it's much easier to lose precision due to very different magnitudes' <-- this is only true for addition/subtraction?

dzaima

@ab5tract mostly. you also lose some precision with log/exp due to the exponent needing place in the mantissa/vice versa, but that's already outside of rationals

ab5tract

Look, until the inconsistency between FP and arbitrary precision doesn't affect accounting ledgers and whether or not your medical device is accidentally leaking more radiation than expected, I don't buy the argument that the error rate is acceptable, let alone intangible

* acceptable in the sense of "this is the best our brand new programming language can do without a third party library"

dzaima

10:41 AM

@ab5tract if you actually really need to keep track of 0s and can't use fp then going to rationals is definitely the sane thing, but in most cases you can get by with storing all the numbers multiplied with a constant

@ab5tract you can easily see that with e.g. 1e100=(1e100÷○1)×○1 - there's no error even with the 10^100 magnitude difference. It's not gonna be equal always (e.g. 1e96), but even then the difference is just 1 bit (dzaima/APL)

ab5tract

@dzaima You keep reiterating that the user can just do it an entirely different way. I think we can leave this behind for now and let the chat move on.

dzaima

@dzaima (in fact, for all of 10*⍳300 the max difference is only 1ulp)

ab5tract

But to be clear about the impact of multiplication and division when dealing with imprecise input still holds. If you do a bunch of addition that introduces various levels of imprecision to a value and then use that value in a multiplication, the impact will be amplified.

So I think I will trust the fact that any time math intersects with the real world in a way that impacts money, you suddenly can't trust FP at all.

dzaima

@ab5tract sure, the absolute error increases, but that's still the case even in just (a+b)*c where the error of a+b could get extended into an absolute error of 1e300 if c is large enough

ab5tract

And since there are very few things that are calculated in this world that have the value of money (life not being one of them in many spreadsheets), I think the idea that all the other values we are getting out are "good enough" is laughable.

They are only "good enough" because no one computed the accurate values in order to verify whether they are indeed good enough or not.

dzaima

10:53 AM

@dzaima absolute error in the case of numbers with extremely different magnitudes is gonna be bad, but that's just how fp works and you do just need arbitrary precision to get around that

ab5tract

And considering the only time this is guaranteed to be double checked (money) FP is consistently incapable, I am not willing to accept the general consensus that it is not even a question that can be raised without the kind of push back I am receiving now.

Anyway! I've got to run for now. I've definitely gained some food for thought from our discussion @dzaima and I hope I have provided at least a bit of the same for you.

dzaima

@ab5tract that's acceptable i guess. But when i'm drawing a circle on the screen i won't care if it's off by 1/1e10th of a pixel

HighlyRadioactive

11:30 AM

CMC: Find a simple translation from 1+ to APL.

Adám

12:08 PM

@Marshall Are you continuing the APL Seeds, or shall I cancel the event going forward?

@HighlyRadioactive Do you have spec for 1+?

Marshall

@Adám Cancel it.

Feeds

Adám has removed an event from this room's schedule.

Marshall

@ab5tract Well, at least as I understand your arguments you are raising a very strong point: that floating point is unacceptable in any language. If I said the same about linked lists, dynamic typing, or big-endian numbers, I would also expect a lot of pushback.

Note that the reason computations have to be exact in finance is not that people deem the loss of a few hundredths of a cent here or there to be unacceptable but that because people control what transactions are made they could manipulate those computations to make money. That's a rare circumstance.

ngn

12:34 PM

@Adám that comment - wow. aren't you religious anymore?

TessellatingHeckler

@Adám ⍵[⊂'abc']←17 throws a SYNTAX ERROR. Appending ⋄0 0⍴0 is another magic answer which does seem to help, thanks :)

dzaima

@Adám 1+

Adám

> unknown [verifiable] facts

@dzaima Right. That does seem rather trivial to implement. Probably not too hard to translate either.

ngn

@Adám e.g. the approximate age of the universe?

rak1507

Still not quite sure what ⍺⍺ and ⍵⍵ do, can someone give me an example please?

Adám

12:44 PM

@rak1507 They are the operands in a dop ("dfn" style operator).

So, ⌽¨Y is equivalent to ⌽{⍺⍺¨⍵}Y.

Bubbler

@ab5tract My last two cents: We agreed that we have certain applications where ordinary FP inaccuracies are unacceptable, and we also agreed that certain functions are simply incapable of exact calculation. You suggested to push such functions to the very end of the calculation (maybe you could even set up a CAS to find the evaluation strategy that minimizes the error), but it's not always applicable (e.g. arccos(sqrt(1-sin(x)))).

I think one possible compromise is to use FP with configurable precision, so you can increase the number of significant digits/bits during calculation until you can convince yourself (or maybe prove) that the resulting error is small enough for the application. (Such thing can be done with rationals too, by truncating the continued fraction. And this "approximate rational" can support exp/log/trig functions)

dzaima

@rak1507 note that just the occurrence of them in {} is what converts the function to an operator - only ⍺⍺ but no ⍵⍵ makes it a monadic operator, but ⍵⍵ makes it a dyadic one

rak1507

Alright, thank you both @Adám @dzaima

Adám

@rak1507 An example (notes here):

perv←{⍺←⊢               ⍝ Scalar pervasion
    1=≡⍺ ⍵ ⍵:⍺ ⍺⍺ ⍵     ⍝ (⍺ and) ⍵ depth 0: operand fn application
             ⍺ ∇¨⍵      ⍝ (⍺ or) ⍵ deeper: recursive traversal.
}

TessellatingHeckler

I was rather hoping it would be possible to use a powershell scriptblock inline with APL code in it, and as long as it wasn't executed by powershell it would be OK. But PS needs the code inside to be syntactically valid powershell, so it's going to have to be a braced block in double quotes ⍎ "{}" $things or ⍎ "{}" 1 2 3 if I can get a long-right argument working.

@Adám ⍎ does not seem to like multiline string input; when you commented about operators and multiline input yesterday, what kind of usage were you imagining? I had only been thinking of a single one-shot line of APL at a time.

Adám

12:55 PM

@TessellatingHeckler Maybe I want to use an intermediate value or a helper function before computing the result. ⍎ doesn't do multi-line, only multi-statement.

TessellatingHeckler

@Adám Right now every invocation seems to be a clear workspace (with an uninitialized random seed), state isn't persistent. Ideas of defining a function which comes back to powershell, and stays as "an APL function" ready for next use, might need some work

Adám

@TessellatingHeckler No, I mean like {sum←+/ ⋄ (sum ⍺)>(sum ⍵)} or {lc←⎕C ⍺ ⍵ ⋄ ≡/lc}

TessellatingHeckler

  PS C:\> ⍎ (1,5,10) "sum←+/ ⋄ (sum ⍺)>(sum ⍵)" (2,3,4,6)
  1

I think making it so ⍎ ⍳5 works, but with arguments it needs ⍎ "{}" $Y is where I will go.

is it possible to define the type of a :Field? I have :Field Public Instance A and it comes out in the DLL as type object

There is :Signature for marking the return types of a method

ab5tract

@Marshall It's unfortunate that even with all those words I wasn't being clear :(

HighlyRadioactive

@Adám The point of this challenge is #

ab5tract

1:07 PM

I am in no way saying that it is unacceptable to have floating points in a programming language. I am saying that according to my own personal standards I expect moving forward in computing, it is unacceptable for a language to not provide a way to perform those calculations more precisely.

And I would further argue that the responsibility of knowing when it's acceptable to use an imprecise representation should fall onto the user of the imprecise representations -- not onto everyone else.

It is completely ok to expect a language to handle the casting in an expression like 8 + 1.5 to produce a float instead of a type error and I'm honestly confused why it's such a heretical leap to include (or allow seamless inclusion) of a type capable of arbitrary precision into the automatic casting layers common in many programming languages.

Other than performance, which is apparently still slow and which is a question I feel a bit regretful in even asking at this point.

dzaima

@ab5tract i guess that could make sense (as 3/4 of the int→float conversions are not lossless), but that'd be adding a layer to an already somewhat confusing part of the language

ab5tract

@Bubbler Thanks for the summary and additional technical details. I appreciate it.

Bubbler

1:22 PM

@ab5tract You said Raku gives a Rat whenever applicable, no? Do you mean you want that in other languages too?

TessellatingHeckler

@ab5tract What would I, an ordinary math pleb, expect the arbitrary precision result to be if did 2×Pi ? Right now I expect some long scribble of floating point approximation, but at least predictably bad.

dzaima

@TessellatingHeckler the idea is to include trying rationals before resorting to float, and as pi would already be a float, that'd be float multiplication

ab5tract

@Bubbler To whatever degree that math is possible in a language, it should be homomorphic (minus the type specified in the type conversion for those requiring Integer.toFloat(integerVal) + 7.5 as that is inevitable/hopefully generic-able) to do the same thing with arbitrary precision

It doesn't need to be on by default and I can definitely understand the arguments against ever even turning the feature on in one

*one's own code

My preference would be to have it on by default but that's what configuration settings are for :)

dzaima

@ab5tract "on by default" so you mean introducing another complete-code-behavior-changing thing to ⎕IO ⎕ML ⎕CT?

ab5tract

not on by default.

Marshall

1:29 PM

@ab5tract "Every language must support X" is also a very strong statement: as before, replace with OOP, exceptions, or monads and see how you feel about it. The "heretical" thing (but no one is calling you a heretic, just wrong) is not to advocate for the feature but to advocate for the feature as a hard requirement.

Bubbler

For the record, Dyalog has ⎕FR to increase the precision of FP numbers, which is a weak but valid way to support my alternative.

ab5tract

@Marshall Ok I am really over re-explaining myself. I never said it was a hard requirement for a language to exist in the world. I said it was a personal specification over whether a new language being designed today has it's screws on straight.

And saying that I am "wrong" without evidence is exactly how heretics are treated.

dzaima

@ab5tract (to be clear, i'm completely fine with rationals if they're clearly separated from floats - just like there's the f and d suffixes for 32/64-bit floats, such a suffix for rationals would be good; them being quietly inserted at places until there are floats is the thing i don't particularly like)

Marshall

In the case of array languages supporting multiple numeric types leads to some pretty tricky issues. If an array consists of floating and rational numbers, is it acceptable to convert the rationals to floats? If so, you're giving up your accuracy guarantees. If not, then adding a rational to a float array means it won't be optimized by normal array typing methods. This is especially problematic if integers are exact by default.

@ab5tract Uh no, heretics get excommunicated.

Without evidence?

ab5tract

You don't need to lecture me on heresy @Marshall. You need to have some belief system that a heretic is challenging for there to be an excommunication.

There is no quantification of the issue of imprecision in floating point math across all systems in use around the world. My issue is with everyone pretending it is otherwise when the only time we double check we find the technique to be mostly unusable.

TessellatingHeckler

1:37 PM

@Adám (I missed this message) That's cool! A couple of things - first if you go that way so people can make arbitrary PowerShell cmdlets in APL it would be necessary to have multiple Attributes on a Field, and beneficial to specify the field type. Second, I have no other cmdlets to make, and now I that can put attributes in after the compile, that'll do me for this use; changing the language just for that is a bit overkill.

ab5tract

And I don't even understand how I can be "wrong" when all I'm saying is that the responsible move going forward is not to just continue pretending the issue doesn't exist but rather to make affordances at the implementation level.

dzaima

@Bubbler also NARS2000 has arbitrary precision floats with ⎕FPC

Adám

@TessellatingHeckler You actually want the attributes on a property, not a field, I think.

TessellatingHeckler

@Adám I don't understand the distinction; they are .Net object properties, instance variables, but in Dyalog they appear to be called :Fields

docs.microsoft.com/en-us/dotnet/csharp/programming-guide/…

variables defined directly in a class appear to be "Fields" in C# as well(?)

Properties are fields wrapped in get/set methods!

Bubbler

@ab5tract You're not wrong, I get your point, but the only problem is that high precision computing is apparently not considered a "general" enough application area for the current language implementors.

Marshall

1:46 PM

@ab5tract I honestly don't have any animosity towards you, but I think you are either being overdramatic in how you're portraying the people arguing against you (please stop) or you are confusing our attempts to convey our own opinions with attempts to shut yours down. You believe I should add arbitrary-precision rationals to BQN, no? I consider this decision incorrect--BQN is subjectively better with only one numeric type, and in that case it should be double-precision float.

dzaima

@ab5tract but the floating point error is something you can mostly not worry about. Most calculations happen in roughly the same magnitude, and then you need to get into the millions of operations for anything to be visible (at which point there's a possibility that after doing those millions of operations on rationals you could end up with the number taking up a megabyte if the numerators happened to be coprime enough)

Marshall

@ab5tract The discussion here has been very technical and raises specific issues with your characterization of floats and with rationals. I think that it is far more wrong for you to act as though people are dismissing you without thought than to push rationals in the way you are.

dzaima

(a simple example of summing the reciprocals of the first 10000 naturals in NARS2000:)
https://dzaima.github.io/paste/#0RZrLgRw3DETvikJ3Hwz@QDIt5eFofHAAzsSR@D3MdGtlybszPSQIFAoFcH/@8Wf71f7957@//m7B149192gjxoyz9l7nzrv2bDdy3IjdY44Ve90bfbdsK/ix5czcp/PRM/LuczPOGLxeD54xzx2Xxfph4csP67DM7RF86B5WZZPLVn2eHz/r68y121mtzZjR@JN@pmcfuTYGZM92c9/Wg5997p6@zsrTdo621lj77FuWZ2fjPmay2Wqbc7TT2J7zLd6Pfte4PLTP2HPfrwWcOudcva95ZmvtpOcIXuu9sWOfnRdjnzZcmGXz7NbbbHOw8W6TT2PWOFg2cerGR3cmL@HkOO2c5OtwjoEBHQPHHRq8vhbcmyzA@utiMSZz/MVDGRsnn9lX8lIs1o/D0rEGVnnaO/YmGGx2xgn2YW2MyIbpSxv3wL42tH0TFPzVxuBYgwgECzwWsF27GorzPfduuLTfzgvEg…

Bubbler

Each language has its own design choices, and often, one of the top priority is performance. The designers naturally choose the most performant single number type (or one for integer and one for non-integer), and then float64 is the most sensible single choice because it is supported by mainstream CPUs.

ab5tract

@Marshall Obviously if you use a design constraint of having only a single numeric type, you can only have one.

@Marshall Why do you feel so comfortable being dismissive of someone who is remarking on a feeling of being dismissed?

You misinterpreted me a number of times

Bubbler

1:56 PM

If you want a language that supports precisely what you want, you might need to implement one yourself.

ab5tract

@Bubbler I know this. This is where I started from when I asked if any of the much-more-knowkedge-able-in-Computer-Science crowd here had heard of any tricks to make rationals perform well enough to lose that issue.

ngn

@ab5tract no chance of that. fp is cast in hardware. unfortunately.

ab5tract

I think it's best if I jump all the way out of the chat for now because I honestly only asked because if there is any croqd which would know how to do rationals quickly I imagined them being in this chatroom.

TessellatingHeckler

/ offloads the rationals to the GPU. GPUs magically make everything faster.

ab5tract

@ngn I've seen so much magic in regards to performance since encountering APL/K/etc that I guess my sense of what is and isn't possible is all confused at the moment

ngn

1:59 PM

@TessellatingHeckler gpus make computations faster but transfering the data to/from the gpu is very expensive

TessellatingHeckler

if it's not too much of a hot button question right now, why are rationals much slower? Are they not two integers, and integer calculations are fast?

Bubbler

@ab5tract You can still try out the configurable precision to make some tradeoffs between speed and accuracy.

@TessellatingHeckler Machine sized integers are fast. Higher than that, it gets slow.

dzaima

@ab5tract (as for a more complete answer to that, the time arbitrary precision rationals will be of comparable performance to floats would need to be about the same time array languages lose their edge on performance - arbitrary precision requires dynamic (possibly of megabytes) memory access so it can't really be simd/hardware, and so is very much non-array-language-y)

TessellatingHeckler

or is the issue that the integers become so huge they go beyond int64 and have to become bignums, as in dzaima's paste?

Bubbler

And the magical speed thing in array languages and GPU is just massive parallel processing. I don't think it's feasible to utilize it for arbitrary precision arithmetic.

dzaima

2:04 PM

@TessellatingHeckler keeping a separate below-64-bit type and an arbitrary precision type would probably be a good implementation choice. But either way, to add 2 numbers you'd have to simplify the fraction at some point which involves division, and division is still about one of the slowest operation you can do (not sure how it compares to trig/log/exp & similar)

Bubbler

It can help reduce the performance gap a bit, but not much

ngn

@dzaima gcd can be computed with shifts and and-s (no divmod), but it's still heavy compared to a flop

dzaima

@ngn right, i'm stupid :)

TessellatingHeckler

@dzaima ok, I see

dzaima

@dzaima but if you have a gcd you still have to divide the numbers by it, no?

ngn

2:08 PM

@dzaima right, i'm stupid :)

dzaima

:p

(unrelated, i have no idea why ":p" is often my goto "emoji"; the name i call it by myself it literally "colon p" and any time i use it and think about it, it seems inappropriate. nevertheless, i continue to use it, so if any of y'all think my use of it is weird, know that i think so too)

Bubbler

To make a higher-precision arithmetic even faster, just design a hardware that does the job.

Like Google invented TPU to accelerate their neural networks...

ngn

@Bubbler not enough soldering experience, unfortunately :)

dzaima

(the difference between ^^ and arbitrary-precision rationals is the "arbitrary" part. Maybe i'm being too paranoid though and it's actually possible to optimize for 64-bit ints well enough (keeping fall-back to arbitrary only for when strictly needed) that the only problem is the constant need for division)

ngn

@Bubbler but there are these "fpga"-s now, there's still hope

Bubbler

2:17 PM

@ngn My company does have some of them. They work, but they're damn slow :(

Marshall

2:28 PM

@ab5tract The distinction I'd like to make is that when a group calls someone a heretic, they are not really telling that person to reconsider their beliefs but to stop sharing them publically, or in some cases to recant, that is, publically state those beliefs are wrong. I don't want you to do this unless you have changed your mind, so I am pretty annoyed that you seem to be accusing me of this pattern. If you don't mean to, then there is surely a better word than "heretic" for what you mean.

I am very much at a loss as to how the group of three or four people that disagreed with you could have expressed this without making you feel dismissed. For what it's worth I do think problems with computable representations of real numbers are very important and consider the discussion you raised (but not the meta-discussion) to be welcome. I just think IEEE-754 is usually better than other options in the tradeoffs it makes.

@ab5tract If you think I am comfortable at this point you are quite wrong.

ngn

2:47 PM

@Marshall for the record, i share @ab5tract's dislike of floating point, and yes, rats could be the answer in certain cases

i just don't see much benefit in arguing over "should" questions. it's a matter of taste. or, it depends on the applications you envision for your languages.

AviFS

@Adám @Marshall Wasn't today APL Seeds?

Adám

@AviFS It has been cancelled.

AviFS

@Adám ... Forever?

Adám

@AviFS Nobody showed up, so Marshall will start writing articles instead.

AviFS

@Adám Uh oh. When was this? I thought there were a few people two weeks ago, and I came in and out, but didn't say anything

Or do you mean no one showed up today?

Adám

2:54 PM

@AviFS I think 2 times ago, then he just skipped last time, and then I asked him today.

AviFS

@Marshall Sorry for the delay, was searching through.

@Adám I see, that's a bummer. At least we'll have the articles written up though. Which on the positive side, in reality, will be much more helpful for future users. Much easier to peruse through an edited article (monologue) than a scrambled, in-the-moment dialogue. Only downside, aside from the communal/ritual aspect, would be the FAQ the other format provides.

Marshall

4:29 PM

@AviFS I think you've got it right. In the interest of not looking like a ragequitter, I'll put it this way: I received a pretty clear signal that for array implementation people are more interested in well-composed material that they can read on their own time than interactive discussion.

Personally I also found that typing stuff live isn't so easy; I think markdown-hopefully-with-nice-illustrations (which is coming eventually!) will probably have a better return on the effort.

ngn

6:05 PM

@Marshall tbh, i'm interested in one thing: any good ideas worth stealing :)

Marshall

@ngn I'll do my best. The compiler is extremely well suited to a K style because it never has to use arrays with rank >1.

ngn

6:26 PM

@Marshall i'm looking at the compiler now. it will take me a while to decode.

Marshall

@ngn I was planning to add some little comments explaining sections and variables soon--for example cb gives indices of character literal boundaries. I can do that now if you're interested.

ngn

@Marshall i've just tried the vim plugin. looks great, except that some of the characters are a little too wide. i know there's a custom font, but wouldn't it be simpler to avoid wide characters?

@Marshall thanks, but i prefer to study the code :)

Marshall

Also note that src/c.bqn is the compiler I'm currently working with. The differences with c.bqn are fairly small except that it cuts out all the Wasm stuff. I will probably build Wasm support into the bytecode compiler rather than keep working with the old Parse.

@ngn I tried to do that for a while but just couldn't find good characters to use. Font support also tends to be too inconsistent to identify characters that will have good support across many fonts.

ngn

6:46 PM

@Marshall does the shape of the glyph for "reshape" have any significance?

⥊ could be ϟ or ⇋

also, obviously there are many ascii chars you haven't used yet

Marshall

@ngn It suggests ravel order a little. I wanted more of an s with square corners, but I don't think there is one.

@ngn They tend to look out of place because they're not as geometrical. * and ~ are the simple ones that are left and they are much too easily confused with ⋆ and ˜.

@ngn ϟ could work. I'll have to think about that.

Oh, didn't realize it's a Greek letter. I want to avoid conflicts with alphabets.

ngn

7:05 PM

@Marshall modern greek doesn't use it :)

but you're right, letters better be left for identifiers

i desperately need something in vim to tell me what the character under cursor is supposed to mean in bqn

Marshall

@ngn Beyond my Vim abilities at the moment. Any tips?

ngn

@Marshall generally, the status line is good for such things. people usually customize theirs (i don't use it, but i know the "airline" plugin is very popular), so i think vim-bqn shouldn't take it over by default.

another option is to use K (a keybinding in normal mode) or <F1> for that. by default K shows the man page for the word under cursor.

@Marshall i can write the vimscript, but the real problem is the content

Marshall

7:21 PM

@ngn The primitive names are all here; were you thinking about something longer?

ngn

@Marshall what if you update that table but forget to update the vimscript?

Marshall

@ngn Fair enough. Given that the highlighted version of the README is generated I could probably pull the material there out into a symbol/name/URL table.

ngn

@Marshall that sounds good

Ada

> Oh, didn't realize it's a Greek letter. I want to avoid conflicts with alphabets.

...

RGS

7:47 PM

@all is there an easy way for me to find out when I wrote my first message in this chat room?

AviFS

@Marshall You mean half a swastika?

Marshall

@AviFS No, it should curl around one more step.

dzaima

@RGS i haven't seen a way. had to dig mine up by trying to recall what were the first things i did here :p

AviFS

@Marshall Totally know what you mean, but that was the immediate association and I couldn't resist. Sorry for the poor self-control :p

Jeff Zeitlin

@Marshall - so, like a 5 in the classic LED/LCD style?

dzaima

7:50 PM

@dzaima ..appears a reasonably good start would be searching your name and going to the oldest message actually

Marshall

@JeffZeitlin Let's just do this:

┌─
└────┐
    ─┘

RGS

@dzaima that is clever; another thing I thought of was searching for things you'd reply when Adám asks you for your background haha

Marshall

Not that I'm fixated on this particular shape. I haven't come up with any symbol that really conveys the idea of Reshape well.

Jeff Zeitlin

@Marshall - Hmmm.

U+1105 might work, unless you want to avoid Hangeul

RGS

@RGS That led me to this chat message which seems to be on the day I first participated here, so apparently I only started learning APL "for real" by the end of March

Marshall

7:55 PM

@JeffZeitlin I want to avoid scripts both because they could be wanted for identifiers as ngn notes and because they tend to be rendered in a script-y way (subtly curved or variable-width strokes) that doesn't look like other mathematical operators.

Jeff Zeitlin

U+2440 OCR HOOK? Kinda like what you want, but rotated 90 degrees

Transcript for