@cairdcoinheringaahing edit the post and you can see some code blocks start with <!-- language-all: lang-apl --> and some don't; I imagine the new syntax highlighter changes the default behaviour if the language is unspecified
seems this is now approximately the slowest single thing in dzaima/BQN in compiling b←3⋄⊢d←(b×b)-4×2×1⋄((-b)+√d)÷2×2 with Marshall's compiler, taking ≈2% of total time (approx. 0.1% worse than monadic ⍋)
@dzaima The bitarray copy is a lot easier to improve though... Boundaries are the tricky part, but in the interior the idea should just be to construct each destination word from the previous and current source words, since you really don't want to do partial writes.
Timings for primitives in the self-hosted BQN self-compile in ms, out of 6 seconds total. But / is missing because adding a profiler breaks /⁼.
The ¨ is all coming from ⊐¨, so it's not that interesting. Somewhat surprising that ⊐ isn't a big cost in dzaima/BQN, although maybe it's more important in a larger source file. But obviously a hash map is going to be a lot faster than sorting.
for reference, my latest timings (for 1e4 iterations): https://dzaima.github.io/paste/#0xZm9jhRHEMdzP8U4tmi6qrr6Q8Ikji37FU6s5eRWtgAhAgJEcFqWOxnEIizLATqDQPKFlgj4CI43mSehZ26neqzpYKOu1QZzo9HeT1XV//pXTb877XcPF9@Lg28e@MDTb/rdo1/u/3676/LFraPj4zvDxd3f7h4dd@s7492je79Ol/npv2s/8/Hgmwc@8Hrg6rd/rLpu/MeJbP7kiw48GQxoff7DmnwTArcnO3uy3pNB2pOBiyalyClMZCG65mTXp4gxTFzgDGNKliYuxPZc/ebzRObslMvExmYwBCGj0Jzsy8sJzAtYYBNShFiKDH1zsGuSShYuNJw8saQSCJpzfSfxkkPpgyGMnlG4rNc4lBNZJ2QccvFbZLcngwTtS@z7iYuEq8tqwcRQMunac/WPXwtZmMhclgsHEAecbl/8zcluChcKF5qYovex1H57Gbv8MOk@SiopmcjOeygdidqn8uH5Dz/9@PN6bJFC5o0DtMR7MvIOtMhW/ydzhmxCS…
@dzaima i appear to have fixed that by adding an optimized copier for the case when the required shifting is 0 (because shifting by 64 is stupid) - down from 1261/100000 mistakes on random copying to 0/10000000
I am working on a lazy-evaluated APL dialect with proper first-class functions. It is starting to become more and more usable, but I still don't have a good syntax for hashtables. Does anyone here have experience in an APL dialect with hashtables? I'd like to know if the way it's implemented in languages like Dyalog is the ideal way to do it?
I'm looking at the Dyalog docs for this stuff, and to be honest I don't understand how it works. But that might be because the examples use ∘ in a way that I am unfamiliar with. I guess it's a Dyalog extension.
A hashtable is just an array where an index is an arbitrary value (or sometimes just a string or symbol) instead of a sequence of numbers, so I would say to use indexing.
Dyalog doesn't really have hashtables, only namespaces. There's also a mechanism to hash an array to make later lookups faster, but it doesn't have any support in the syntax.
Yes, and I was thinking about making a hashtable look and behave like a regular array where the leftmost column contains the keys. If I implement it using a linked hashtable in the backend I can preserve order as well. So I could do that, or I could simply invent a new syntax for a full-featured hashtable.
@EliasMårtenson If hash tables are an important part of the language, it's probably better not to make them act as arrays but to have ways to get keys and values as lists.
Yes, you have a point. My reference use of them are for the JSON parser. I need to have some example programs that do some REST stuff, parsing data from a webservice, and I want the syntax for accessing data in nested hashtables as clean as possible.
Some other potentially important operations are K's group (=), which takes a list and produces a dictionary where the keys are the unique elements of that list and the corresponding values are lists of indices where those values were found, and a function to turn any dictionary into one that maps the same keys to themselves, analogous to ⍳⍴A for an array A.
I have been active with GNU APL, but I felt that I wanted to explore what could be done if an APL implementation could completely ignore any compatibility concerns and just adopt whatever imperative style it wanted, along with traditional APL code. I also wanted to make it lazy evaluated so that parallel evaluation can be more efficient.
@EliasMårtenson I guess you could take a page from Javascript and make hash.key equivalent to hash['key']. But this does conflict with the inner product of hash and key functions, so you would want to either not use . or change the symbol for inner product (or get rid of inner product).
@EliasMårtenson What's imperative style here? Like programming paradigm generally, or something more specific?
Yeah, the period is out of the question. I considered using some completely new characters as well. So far I have only adopted a single new character outside the traditional APL character set: λ is used to create a functional closure
Yeah. I've implemented support for custom syntax (kind of like a Lisp-like macro system), so I have things like if/else and while implemented in userspace.
So you can do things like result ← if(foo) { something } else { somethingelse }
But it's implemented in userspace, and my standard library implements it like so:
No reason to dig into it yet. It's not ready for real usage :-)
I'm working in reimplementing the display function in KAP itself. That has exposed a lot of issues with the way the tooling works. I need to work on the editor and things like that.
You might enjoy my take on anonymous functions here. It has default argument names like dfns, but the header syntax also lets you declare names, so it can be pretty similar to a λ without requiring the extra character, just a : after the header.
It's not like Lisp macros. In Lisp, the macro returns the code that is the expansion of the macro invocation and the expansion is then passed to the compiler. In my case, you can't evaluate any code during expansion, and instead the syntax form describes how the parser should handle the tokens.
And during execution the handler form is run, accepting the values that was collected during parsing.
Yes, I see. I was thinking about something along those lines, but my problem was actually limitations in the parser (or at least what I believed would be limitations in the parser).
The issue for me is in a form such as the following: { ⍵+2 } 1. Is this the value 3, or is it an array with two elements: An anonymous function and the value 1?
@EliasMårtenson The natural way to handle everything is to unify functions, arrays, and hashes, so that a hash table is a function where each argument's domain is a finite set, and an array is one where each argument's domain is a prefix of the natural numbers.
But there are a lot of practical reasons to separate them.
@EliasMårtenson { 𝕩+2 } 1 to apply the function and { 𝕩+2 }‿1 to make a list. Having an explicit character for stranding saves a lot of small annoyances.
It would be possible to strand together adjacent values with a subject role, but the easy way to convert a function to a subject role is to put it in a list and pick the first element, so there would probably need to be more syntactic support.
Or something like F←{ 𝕩+2 }⋄f 1, which is currently an error (F has a function role and f has a subject role).
@EliasMårtenson interesting project! Something like that was my goal with dzaima/APL, but i stayed pretty close in syntax, focusing more on builtins
at one point i really wanted to have plain literal if/while/etc statements, but couldn't really think of a good way to make the syntax work. Something macro-like and user-definable is a pretty nice way about it
@EliasMårtenson Any comments on the import structure (namespace/export/use)? Imports are going to be a problem in BQN since it has no namespaces and I'm pondering if I should add some kind of module system that isn't first class or add dictionaries and namespaces.
The namespace system in KAP is takes directly from Common Lisp.
Even to the point of exposing a way to quote symbols (single quote).
Every symbol belongs to a namespace, and when looking up a symbol by name, the system first looks at the current namespace. If the symbol does not exist there, it searches for it among the exported symbols in any of the "used" namespaces.
Finally, if it doesn't exist there, it interns the symbol in the current namespace.
You can also explicitly specify a namespace using the colon notation: namespace:name
You can also add an explicit import of a symbol from a namespace into your own.
I'm not saying this is the best way to do it. I'm usually a Common Lisp programmer, so that's what I'm used to.
@EliasMårtenson Sounds a lot like what I'm looking for, although I hope I can find a good way to adapt it without adding reserved words. Does namespace(...) always go at the top of a file? Do exports have to go after the definition?
It doesn't have to go on the top of the file. The namespace directive tells the tokeniser to change the default namespace. When you include a file, the old namespace is saved so the directive only lasts until the end of the file.
And exports can go anywhere. It just sets a flag on the symbol.
It's actually implemented as a flag on the namespace mapping of the symbol, but that's just a technicality because symbols are immutable. From a design perspective it's just a flag.
And I haven't settled on the syntax for namespaces. I agree with you that having reserved words for this stuff is not great.
I implemented it that way because it was trivial to add it to the parser without spending much time on it. Because as well know, one of the two hardest problems in computer sciense is naming things.
@Marshall The biggest problem i see is being able to bind variables at compile-time. The two options i see is having statically known exported symbols (i.e. more syntax and forced compilation of imported files at compile time), or namespaces (and potentially namespace destructuring for spilling contents out)
I'm taking the Lisp route where compile-time is intermixed with parse time. Now, I don't natively compile, but I do build a syntax tree of the code that is eventually evaluated. However, these can be rebuilt by redefining functions. You may note that this means that the semantics of the parser may change depending on what was evaluated before, and that would be completely correct. This is why defsyntax works.
@Marshall yes, it is. Again, very similar to how it's done in Lisp. I think you may notice a trend here :-)
@dzaima The import syntax I had been thinking of was essentially Python-style import ⟨a,b,c⟩ as ⟨q,r,s⟩ from file.bqn, with no *. I still think that basically makes sense.
@Marshall that's not a bad way to do it. I dislike Python with a passion, but credit where credit is due. It may even be a better way to handle things and I'm open to changing my implementation.
The basic problem is that a block or file can define any number of values, but only return one. Sometimes you want to get multiple values out, with names to distinguish them. So the block should be able to mark variables for export—here Lisp export is perfect, semantically speaking—and the caller should be able to select some of these and possibly rename them.
@EliasMårtenson Python's definitely a mix of good and bad ideas.
@dzaima I'd like to use the same strategy for blocks and files, so that a file is really just a block without the braces. So it would be nice to have some way to distinguish a block that returns the last value with a block that returns a namespace. Maybe this could be put in the header somehow?
@dzaima Was just about to write the opposite: you should be able to use ⟨q,b:r,s⟩. When there are no renamings I'm not sure it's a problem to have ⟨q,r,s⟩ match a list or dictionary, but you could also put a leading : in front of one of the names or something like that to indicate it's a dict.
@dzaima Yes. It avoids accidental shadowing and helps the implementation optimize.
I don't know whether I'd even allow exporting all variables.
I'm thinking you'd just use an arrow for exports, so it could look like this:
@Marshall so you need to mark what you want to export. Putting it in the header would work (e.g. ⟨a,b,c⟩←𝕨𝕊𝕩:), but having some syntax to export at or near the place of definition would possibly be clearer
It would also be useful to have a generator-like syntax where you can return as many values as you like and they're collected in a list once the block ends. That would be different in that → acts on a value instead of a symbol.
@dzaima Or just the symbol with no assignment. It acts on the rest of the statement up to the next arrow, and that has to be a valid assignment target.
@Marshall your previous proposed syntax kind of collides with that. i would say that you'd need the assignable on the left for a namespace output, but a→a←2 is kind of bad
@dzaima hm, a→2 could itself include the assignment, but then the fact that a←2 and a→2 do partly the same thing is very strange
I had been thinking that the export and return syntax could be unified, but that doesn't work because inner blocks can declare the same variable multiple times.
And the generator thing does work with returns, so it shouldn't have the same syntax as exports.
@EliasMårtenson you are doing what I wanted to do but have so far been too lazy to start, I am excited to try it! lazy evaluation sounds like it'll be really powerful in APL
@EliasMårtenson The problem here is that adjacency is overloaded: a b, where a and b are objects can mean '2-vector of a and b', or 'apply a to b'. The semantic roles of a and b decide the syntactic meaning, which is problematic. (BQN resolves this with the stranding symbol, as marshall mentioned.)
@EliasMårtenson A secondary problem is that, with longer sequences, you have to distinguish between monadic and dyadic application (and, if you don't have true first-class functions, operator application). Marshall's I resolves this by only having dyadic application.
