The APL Orchard

2:36 PM

@user280555 Hi Heneli. I you want write access to this room, please email me: adam@ with the same domain as www.dyalog.com.

@ngn aplcart.info/dev is a quick demo using List.js instead of DataTables.js. Load time is about 100% faster, but it seems to me that searching is slower, and its "fuzzy search" finds strange results — I don't quite understand the settings for it.

@ngn Btw, do you still experience "skipping" letters on the main site? I've stopped updating the URL on keystrokes.

ngn

@Adám no, it doesn't skip them anymore. there's just a prominent delay when i type 3+ letters quickly.

Adám

@ngn It has to look through 13000 cells… Suggestions are welcome.

@ngn Do you understand the fuzzySearch options?

dzaima

2:52 PM

just alternating showing & hiding all rows every 100ms makes chrome use up all my cpu, and unless the fuzzy search is too complicated, it shouldn't really have much of an effect on the speed

Adám

@dzaima No, but somehow it matches things that aren't there. e.g. "product matrix" matches "(÷¯1+∘.+⍨∘⍳)Is Hilbert matrix of order Is Tacit Monadic Function"

dzaima

@Adám my message wasn't about the fuzzy searching, but it seems it has a threshold in which it allows omitting letters - conjgte → conjugate

Adám

@dzaima Ah, and it also allows adding letters that are not supposed to be there. I can't see how to turn that off. I just want the input split on spaces and filter to those rows that contain all input words.

ngn

why use any lib.js? if an exact search is ok, it should be simple enough to do directly through dom

Adám

@ngn That'd kind of require knowing some JS.

ngn

3:11 PM

@Adám not a lot. just how to loop through the values (for(...){...}), check if the description matches (.indexOf(searchTerm)>=0, you may also have to use .toLowerCase()), and tableRows[i].hidden=0 or 1.

@Adám btw, values is not accessible. it seems to be overshadowed by some library function.

Adám

@ngn well, oninput, but yeah.

ngn

huh, how did i manage to edit the old message instead of posting a new one... never mind

Adám

@ngn I'm a bit unsure about the looping. I'd do document.getElementsByTagName("tr") and then for each one look at all its children and their .text… ugh.

ngn

@Adám i assume the order of <tr>s is the same as in values?

Adám

@ngn values?

ngn

3:17 PM

@Adám the big array here

Adám

@ngn Oh, that's just for the mockup. I'd read in from the tsv like in the real site. But I don't understand. Should I really regenerate all the html all the time instead of making the whole giant table once, and then hiding rows?

ngn

@Adám you don't have to regenerate it on every keystroke, you can use .hidden= to show/hide specific rows

@Adám where is that .tsv and how do you instruct list.js to load it?

Adám

@ngn aplcart.info/aplcart.js

ngn

@Adám the main site loads very slowly

Adám

@ngn Yes, but I've timed it. It is entirely due to DataTables' rendering, not my fetching and splitting data.

dzaima

3:30 PM

@ngn yeah, the "Load time is about 100% faster" above wasn't a lie :p

ngn

throw everything away. start from scratch :)

and no libs :)

Adám

@ngn I just did today.

@ngn But it is a bit above my level. Maybe I can use this?

dzaima

could be a fun challenge to make that. i've already got a very similar thing for Canvas, though this is like 100x more data

ngn

@Adám that's very close to what i had in mind!

that answer mentions .style.display="none" to hide elements but .hidden=1 is newer, better, and faster

Adám

@ngn OK, I'll have a go at it.

ngn

3:35 PM

table.getElementsByTagName("tr") - that also looks suboptimal. table elements have a special property for quick access to the rows, .rows i think

dzaima

i wonder if fancy search could help - prioritize keywords over description, code over keywords or something similar

Adám

3:52 PM

@dzaima Then I really would have no idea how to do it, but if you want to contribute, I'd be delighted. The main work was putting together the giant table. Now we just need the best possible interface for it.

dzaima

hmm, i thought about allowing a single e.g. B showing functions only related to booleans, but the return type isn't a thing mentioned in the table (e.g. searching for all currently is hard)

Adám

@dzaima I've thought about allowing an input element to choose/exclude one or more categories. Really almost every column could have its own filter. But I try to keep it simple to use too.

dzaima

@Adám (as you may have inferred, I am currently playing around with the table, and i do know JS)

Adám

@dzaima hehe

@dzaima Btw,

html←⎕XML 0 'table' ''⍪,[1 2]1 'tr' ''⍪⍤1 2⊢2 'td',⍤1 0⊢1↓⎕CSV⍠'Separator'(⎕UCS 9)⍠'QuoteChar' ''⊢file

@ngn Even without that, it is instant. Now I just need it to search for the words separately…

ngn

@Adám you implemented it already?

Adám

4:02 PM

@ngn "implemented"=copy and paste :-)

ngn

@Adám git push-ed?

Adám

@ngn No, but I can. Hold on.

@ngn ok, pushed and live on /dev2

ngn

@Adám wow, that is instant :)

Adám

@ngn Now for modifying that pesky inner loop to consider each word separately. I suppose it should be clever and bail out early if a word isn't found.

ngn

@Adám what is a word? do glyphs and numbers count as words?

Adám

4:07 PM

@ngn very simple: space-separated. w.split(' ')?

ngn

@Adám will it be an OR or an AND between the words?

Adám

@ngn and

:51102280 well, I can check one word at a time, going through all 7 cells, if by the end of the 7th cell, that word wasn't found, we're done.

ngn

yeah ok

Adám

@ngn But there are too many nested loops; I can't hold it in my head.

ngn

@Adám make a function like searchMatches(searchString, rowDescription)

or m(s,d) as i would call it :)

Adám

4:15 PM

@ngn Making a JS function always takes me sooo many tries to get right :-(

ngn

@Adám huh, idk what you mean

it's just const f=x=>... where x is the arg

es6 is already ubiquitous, you dont have to write function f(x){return ...} anymore

Adám

@ngn IE can't =>

ngn

ie still exists!? :)

Adám

@ngn As long as Win7 does.

@ngn Ugh, JS doesn't allow infix functions!

ngn

@Adám just like k

Adám

4:21 PM

@ngn very sad. I can't name my functions if they are not infix.

ngn

the newest ie, ie11, was released in 2013, is unsupported, and has 2% usage, says here

@Adám you sould put a <!doctype html> at the top. i can see some encoding issues, eg search for "adjoint"

Adám

@ngn Sure, the other pages have that. I just made a quick push so you could see.

ngn

@Adám js vars are global by default, you should put a var before t=

(if you are going to reuse that part of the old code)

Adám

@ngn t=?

ngn

t=(esc(w)).split(/\r?\n/g).slice(1).map(function(w){return w.split('\t')}).filter(function(w){return w!=""});

in the old aplcart.js

Adám

4:37 PM

@ngn Oh, I'll have to rewrite that anyway, since I'll instead be loading tsv and creating html.

Yay, it's working!

@ngn @dzaima aplcart.info/dev2

ngn

see? js is easy

Adám

:-/

ngn

@Adám if (td[j].innerHTML.toLowerCase().indexOf(words[k]) > -1) { - it might be better to search in the .textContent instead of the .innerHTML

Adám

4:53 PM

@ngn Just in case I decide to put markup there?

ngn

@Adám they are the same, except that .innerHTML is escaped and .textContent is what you actually see rendered on screen

Adám

@ngn Oh, right, then it is important for < and > etc.

ngn

for instance now you'd have to type & to find info about the spawn primitive

here's a way to avoid inner loops like

found = false;
for (j = 0; j < td.length; j++) {
    if (td[j].innerHTML.toLowerCase().indexOf(words[k]) > -1) {
        found = true;
    };
};

found=td.some(x=>x.textContent.toLowerCase().indexOf(words[k])>-1) //es6

dzaima

@ngn or if you're using es6, just use includes

ngn

@dzaima oh, right, i forgot about that one

and maybe we can search the whole <tr> instead of each <td> separately

dzaima

5:06 PM

personally i'd make (and have made) an array with all the contents and not use the dom objects until necessary, but that's understandably more complicated

ngn

@dzaima yeah, it's probably better to generate the html in the browser and keep the values from the tsv for searching

they could be pre-lowercased too, instead of calling .toLowerCase() on every keystroke

Adám

@ngn But at this point, we're seeing instant reaction, so it can wait.

ngn

~30ms per keystroke on my laptop

gamers, traders, and cats will probably notice it

hm, actually that's just the cpu spike. the actual delay from key event to screen rendering is several times longer

Adám

5:22 PM

@ngn ok, let's see if I can get the functionality down — then we'll look at performance. For now I need to translate all my jQuery to JS.

ngn

@Adám github.com/nefe/You-Dont-Need-jQuery#readme

Adám

@ngn Thanks! That's great help.

@ngn I don't understand this: el.parentNode.removeChild(el);

document.querySelector('body').removeChild("#w")?

ngn

@Adám jquery has a el.remove(), i think. this is its equivalent in pure dom.

Adám

@ngn Yes, I understand, which is what I'm trying to use, but what goes into the final paren? a css selector?

ngn

@Adám the same element, the one you want to remove

Adám

5:30 PM

@ngn document.querySelector('body').removeChild(document.querySelector('#w'))?

ngn

@Adám is that your code you wish to translate?

Adám

$('#w').remove();

But I know that it's parent is body

ngn

@Adám var el=document.getElementById('w'); el.parentNode.removeChild(el);

Adám

@ngn But why go to to the element to find its parent when I already know the parent?

ngn

@Adám ok, sure, you could do document.body.removeChild(document.getElementById('w')) too

Adám

5:34 PM

@ngn Ah, document.body works, nice.

ngn

also, elements with ids are available as global variables: document.body.removeChild(w) should work too, though some developers might frown on this

i don't know how standard it is according to the w3c, but in practice it works in all (both?) major browsers

Adám

@ngn all four.

@ngn Wow, you've just made my JS experience so much calmer. Thank you muchly!

ngn

@Adám welcome

Adám

5:54 PM

Aaand it's broken.

ngn

@Adám does it need wd40 or tape? :)

Adám

@ngn Dunno.

ngn

what's wrong?

Adám

@ngn It is not filtering right. Looks like it is searching for each letter…

ngn

@Adám looks ok to me except for the problem with innerHTML i mentioned before

Adám

6:01 PM

@ngn I broke it locally :-/

function F() {
    var f, tr, td, i, j, k;
    w = q.value.toLowerCase().split(' ')
    tr = t.getElementsByTagName("tr");
    for (i = 0; i < tr.length; i++) {
        td = tr[i].getElementsByTagName("td");
        for (k = 0; k < w.length; k++) {
            f = false;
            for (j = 0; j < td.length; j++) {
                if (td[j].textContent.toLowerCase().indexOf(w[k]) > -1) {
                    f = true;
                };
            };
            if (f){break};
        };
        if (f) {

ngn

@Adám you changed if (!found){break} to if (f){break} without the !

Adám

@ngn :-/ Thanks.

dzaima

ಠ_ಠ doing too fancy stuff is hard

Adám

@dzaima My fault. The keywords' list wasn't carefully designed for that.

dzaima

6:18 PM

eh, got around it by adding extra score for the description starting with a search term. probably makes 5 other places worse, but ¯\_(ツ)_/¯

Adám

@dzaima OK, maybe I should add words from the description (and code) and sort all the keywords by descending order of importance. Only 1800…

dzaima

here's the mess of js that makes that happen
https://tio.run/##tVjrjtvGFf6/TzFhiiyJlcjd/Aq8ooq1vHGdOF5jd414K6vBcDgS6R2R7MxQlyQGAgQIgqL/@yZ9gPZN/CLuOTO8r5A0Kaof4nDmXL5zmXOOxGi2oerDh8lHT65mt3cvL0mi12J6NMEHETRbhQ7PnOkRIZOE0xgXsFxzTQlLqFRch86r28/Hnzkk6B5mdM1DZ5PybZFL7RCWZ5pnQLxNY52EMd@kjI/Ny4ikWapTKsaKUcHDs1aUSLN7IrkIHZWAGFZqkoIkhySSL0NnSTf46sPXQR69F1wlnOuagRaCUal9ppRT0@tUCz69ePkcT8iYfJ5mMdnnpSRbugdoBI4mgaWyHIrJtNBESdZKfKuc6SSwJ8ZbQe2uSZTHICcOHU0jwZ1KCDVb1CGUMa7UPd@bN6MndGo4eUaepvpPZURcSzcGwkfkwnNAhqZyhf7/JoJA3dc2JloX6lEQrFKdlJHP8nVAI1nG3wYVVmQlhj0CU/kudM6c6fu//fT@7z9PAtqDx3vweAPvcleASqpTwNcDdu…

Adám

@dzaima ew.

ngn

6:49 PM

match scores look complicated to me. tbh i'd rather have on/off buttons for the categories of entries

Adám

@ngn Is there a way for me to globally have B=document.body?

dzaima

@Adám just do B=document.body in the global context

ngn

@Adám var B; in the global scope and B=document.body after the ondocumentready event - inside the $(function(){...})

Adám

@ngn Hm, then it is simpler to have function B(){return document.body}

ngn

technically, var B is unnecessary as assignments are global by default, but one day you'll embrace "use strict" and it will complain about this

dzaima

6:55 PM

@ngn shouldn't document.body already exist at the point in time that any script can run?

ngn

@dzaima not guaranteed, as Adam put his <script> in the <head>

if he puts it before the end of the </body>, it should be guaranteed, i think...

Adám

And "mini-jQuery": function $(s){return D().querySelectorAll(s)};

ngn

@dzaima that's the whole point of jquery's $(function(){...})

dzaima

@ngn ah, so it isn't. TIL

ngn

@Adám well, you don't use it all that much, but if you prefer ...

Adám

7:00 PM

@ngn Hm, it doesn't auto-map :-/

ngn

@Adám jquery is an array-oriented language :) the dom isn't

Adám

@ngn Seems like a bug to me.

ngn

@Adám there's a spec and it doesn't say it should auto-map

dzaima

@Adám not every language is APL

Adám

@ngn Clearly a mistake ;-)

ngn

7:04 PM

such specs are made by big corps who try to defend their own business interests and the result is usually a mess, as everyone tries to make what they already have into a standard, and specify it complicated ways so the competition can't reimplement it

in practice nobody cares about good minimal api design or the users' needs

@Adám you could shorten this bit:

    if (found) {
        tr[i].style.display = "";
    } else {
        tr[i].style.display = "none";
    }

as tr[i].style.display=found?"":"none" or better tr[i].hidden=!found

Adám

@ngn done

ngn

td = tr[i].getElementsByTagName("td") -> td=tr[i].children

or better: you could search all words in tr.textContent.toLowerCase() and eliminate the innermost loop

Adám

@ngn I didn't quite understand what to replace with what. Ah, maybe I do now.

ngn

7:19 PM

@Adám is there a way i can test this code locally?

Adám

You just need to clone the repo and run a local server to serve the files. There are no callbacks to the server other than for getting the files.

I run MiServer.

ngn

function D(){return document}; - that's going a bit too far :P

dzaima

@Adám dev2 currently doesn't seem to even need a server - just opening the file in chrome is enough if there aren't dynamic file loads

Adám

@dzaima Oh, right, because that's the next step, to non-jQuery fetch the data from tsv and convert to html and insert into the document…

ngn

@ngn oops, i meant tr[i] there, not just tr

Adám

7:30 PM

@ngn Yeah, I know.

ngn

here's a shorter Q():

function Q() {
    var fw, ws=q.value.toLowerCase().split(' '), tr=t.getElementsByTagName("tr")
    for(var i=0;i<tr.length;i++){
        var s=tr[i].textContent.toLowerCase()
        for(var j=0;j<ws.length;j++){fw=s.indexOf(ws[j])>-1;if(!fw)break}
        tr[i].hidden=!fw
    }
    Z()
}

function W(){B().classList.toggle('w');q.focus()}

also, instead of .get / .setAttribute('class',...) you could use .className

history.replaceState(..) - today i learned

Adám

@ngn Does the ternary require parens? B().className=(null!=s.get("w")?"w":"");

ngn

@Adám no

Adám

@ngn 20 levels‽ ****

ngn

@Adám well it's a c-like lang

Adám

7:42 PM

@ngn OK, how do I translate

html←⎕XML 0 'table' ''⍪,[1 2]1 'tr' ''⍪⍤1 2⊢2 'td',⍤1 0⊢1↓⎕CSV⍠'Separator'(⎕UCS 9)⍠'QuoteChar' ''⊢file

to JS?

ngn

@Adám i need to consult the aplcart to understand what these ⍠-s mean

dzaima

@Adám again, not every language is APL

ngn

@ngn oh, they're not there yet

Adám

I guess I can split on line breaks, split on tabs, escape, wrap each in <td>, flatten, wrap each in <tr>, flatten, wrap in <tbody>.

@ngn I only added the principal options, both because they are often used, and because they don't always show their name.

@ngn It is pretty simple, TSV (separator=tab) and take " literally.

@dzaima Well, just lots of looping/mapping.

ngn

@Adám something like '<tbody><tr><td>'+esc(x).replace(/\n/g,'<tr><td>').replace(/\t/g,'<td>')+'</tbody>' (i haven't tested)

may be off by one if there's a \n at the eof

Adám

7:50 PM

@ngn I just do .filter(function(w){return w!=""}) at the end.

dzaima

or something like this for a more non-hardcoded version

ngn

@dzaima counter-intuitively, it's faster to do .innerHTML= than to manipulate the dom

Adám

@ngn And /\r?\n/g

ngn

@Adám i don't use windows

dzaima

@ngn eh, i kind of thought that'd be the case. damned non-array-orientatedness

Adám

7:53 PM

@dzaima No, but there's no guarantee on what someone might PR and I won't notice.

ngn

@Adám how did you generate the tsv?

Adám

@ngn Hand typed.

ngn

@Adám ah, so you know what the line separator is then, no need for /?/

Adám

@ngn Right, I know now, but it has happened that others PR'd it.

dzaima

@ngn i've had git mess with line endings sometimes (breaking everything, obviously), so where possible, i wouldn't rely on them

Adám

8:00 PM

@ngn OK I've pushed. gotta go for 10 mins. Care to look why I get TypeError: s is undefined in E(s)?

ngn

@Adám 0. no tsv in that dir, 1. function(d){d.text()} -> function(d){return d.text()}

Adám

@ngn D'oh, i'm in dev2

@ngn And about return, yeah, that's one of those that cause me to get it wrong all the time.

ngn

so much simpler in es6: d=>d.text()

Adám

@ngn I know.

ngn

there must be an es6 to old js compiler

dzaima

8:13 PM

@ngn babeljs

ngn

@dzaima thanks :) so @Adám, if you make this part of your build process and trick ie into loading that instead of aplcart.js, there would be no need to write function so often

Adám

@ngn I'd rather avoid a build process.

ngn

well, tradeoffs. it might be easier to find that person who still uses ie and spend 5 minutes installing ff on her computer than dealing with inconvenient syntax throughout aplcart's lifetime :)

Adám

@ngn I'd say once Win7 is out of life support, I'll go =>

@ngn Easy way to skip the first line of the tsv?

dzaima

@Adám lineArr.splice(1)

Adám

8:25 PM

lineArr?

dzaima

@Adám the array of lines you've gotten

oh right, you replace immediately & don't actually split

Adám