I wonder why \nulldelimiterspace by default is set to 1.2pt, independent of font size.

Also, is it this \nulldelimiterspace that is added on both sides of fractions(?) that explains the "lie" mentioned in this question: tex.stackexchange.com/q/144159/52406? With it, the fraction behaves somewhat like something fenced. @egreg?

@PauloCereda Hardware Store is epic :D

@Plergux ooh

Lots of TeX Live updates this morning: some infrastructure change?

@JosephWright oh wow, 122!

@JosephWright oh no

@mickep no generalized fractions do't use nulldelimiter space (even if using a null delimiter) compare:

\documentclass{article}

\begin{document}

\showoutput


$x\frac{1}{2}x$


$x{1 \over 2}x$


$x{1 \overwithdelims..c 2}x$

$x\left.{1 \over 2}\right.x$
\end{document}

( tex.stackexchange.com/questions/631200/… )

@samcarter ooh twitter.com/hyxpk/status/1484918791566151685

@PauloCereda Such a beautiful duck!

@samcarter it is!

@cis A board shows a position. There is no reason why it should expect that it relates to a previous position (or to know how exactly, typically chess books show only selected positions). Handling moves is the task of skak or xskak. But as you have illegal positions with two black kings you will need your own parsing commands.

@JosephWright There's something strange going on: if I try from several mirrors I get different listings.

@samcarter OOH

@cis yes there are fen and pgn, and chessboard can store fen. And you could parse the fen to build a pgn. But typically the pgn is given. All chess programs you use to record a game can output them, so why should chessboard bother with creating one? It looks quite odd to use as input complicated setpiece lists to get out a simple pgn.

I just exchanged email with Jeffrey Ratcliffe of gscan2pdf about scanning formats. He says he uses djvu. I've never used it, but apparently it is a format that is optimized for scanning. But it does not seem that TeX offers any djvu support. Presumably one would need to convert it to PDF first.

I wonder if it would make sense to archive as djvu and convert to PDF on the fly if one needs to insert it in a TeX document.

@FaheemMitha wow I remember looking at djvu decades ago, but not heard of it in recent years? unless you have millions of scans are any space savings really worth using a format that isn't supported in other tools you have?

@DavidCarlisle Possibly not. I've never used it, so don't know. But if it's easily converted to PDF, it might be at least worth looking at.

@FaheemMitha found this github.com/kcroker/dpsprep (I used to have djvu files ages ago)

...but really never used

@Rmano Thank you.

Unfortunately not in Debian, but I suppose not everything can be in Debian.

I'm not finding a djvu converter in Debian, but perhaps I'm not looking hard enough.

@Rmano So did this dsprep thing work for you?

@FaheemMitha djvulibre?

@DavidCarlisle I'll check it out. Thank you.

BTW, since we are on the topic, is there any consensus of a default resolution for B&W scanning to PDFs? Say in terms of ppi?

@FaheemMitha not used it was just the first hit looking for djvu debian

@DavidCarlisle OK.

@FaheemMitha no, never used it. I remember that translating djvu was not trivial (unless you want to drop all metadata)

@Rmano Hmm.

@FaheemMitha I do tend to agree with the point made at karl-voit.at/2015/04/05/digitizing-paper about target file format.

Btw, I'm in the process of doing some stuff of the sort scanning to PDF and using tesseact for OCR. I'm glad with results. If you'd like, I can share my notes / "script" on it.

@gusbrs Sure, would be interested in your thoughts.

I have to do quite a lot of scanning, but haven't quite standardized things yet.

@gusbrs That doesn't mention distributed version control, which is also a good idea. As long as the files are not too large. Hence in part my interest in keeping file sizes down.

Actually, I use OCR with gscan2pdf to produce PDFs, and it mostly works quite well with typed pages. With PDFs created by TeX it does particularly well, for some reason. Almost perfect results.

I wonder if OCR would work with dejavu. The author of gscan2pdf says he uses dejavu, so perhaps gscan2pdf supports it.

I have some version of the Fijitsu ScanSnap. It's worked pretty well for me in the past.

@FaheemMitha Large file sizes are not a problem. LFS exists, after all.

@TeXnician LFS?

@FaheemMitha git-lfs.github.com

Supported by all major platforms.

@TeXnician Oh that, yes. But in that case, the file isn't actually part of the repository.

I use Mercurial, but neither Mercurial nor Git do particularly well with large files.

I thought git-annex was the best. Created by Joey Hess.

@FaheemMitha Well, but it's controlled in history. Yes, by reference but that does suffice to track changes. Especially for binary files that don't really have proper diffing. The LFS store is under your control as well so backing it up is no problem either.

@TeXnician Yes, I see.

@FaheemMitha I have several repos with large files and Git with Git LFS does very well. At least for me who just wants version control of large files and does not care about representation (I just want to be able to back them up).

@TeXnician OK. Have you ever tried git-annex?

I don't use Git, so have no opinion about anything Git-related.

@TeXnician What are your thoughts on an optimal resolution for B&W?

@FaheemMitha No, I have not.

@TeXnician OK.

@FaheemMitha I have no real opinion on that. If I were you I would try several and look at which point OCR becomes unreliable and use something that OCR is still reliable.

@TeXnician OK. The OCR runs after the scan is complete, right?

@FaheemMitha Usually. Unless you have some interesting setup that does it on the fly. But all setups I know have scan, then OCR.

@FaheemMitha djvu may buy you some size difference over pdf, but I doubt if enough to make them amenable to version control. And, I don't get it, do you expect to be "editing" your scanned documents frequently? Why VC? You sure need a proper backup, but that is true for everything anyway.

@gusbrs I trust version control more than a regular backup.

Because regular backups can get corrupted, and you wouldn't know about it.

@FaheemMitha but for binary images especially scans that you are not planning to edit, it seems a strange choice

Unless you test the backups regularly, which I do, automatically. But I still feel safer with version control.

@DavidCarlisle An unorthodox choice, certainly.

@FaheemMitha You could simply use backups with integrity hashes as many software solutions do.

@FaheemMitha that's one way of describing it:-)

@TeXnician I use Borg Backup. Not sure whether it does that or not.

But I don't think it makes integrity guarantees. You can run a check on the backups, though.

@TeXnician What examples of this are you aware of?

@FaheemMitha borgbackup.readthedocs.io/en/stable/usage/check.html

@FaheemMitha Here you have them: gist.github.com/gusbrs/0473e50c8ce471e11cd3a63c1d779e59 . Those are self notes, so I've kept some of my attempts and reasons for the kind of scan, resolution etc. I'm using just basic libraries for image manipulation, tesseract for OCR, and qpdf for merging. So stuff I presume you'll find in Debian.

@gusbrs Thank you.

@TeXnician Yes, I just mentioned Borg. Anything else?

I see that my scripts run borg check and email me the results. I don't do a full verification check, probably because that would be too intensive. I forget.

@FaheemMitha There are proprietary solutions that do this. I can't really remember names but have used some when I was still on Windows.

@TeXnician Presumably you still need to run checks, though.

With version control, I think corruption would be more readily evident. Though I could be wrong.

@FaheemMitha About djvu and OCR, I did use sometime ago for this djvubind. But, in the end, though I could export it to pdf later with djvu2pdf, I could not do it with metadata (including the OCR got lost). So it became a deal breaker for me.

@FaheemMitha The nice thing about some of the proprietary stuff is that it hides stuff so that you don't need to do that manually. But yes, how would check something without running checks?

@FaheemMitha About resolution and scan type. One thing I think is worth mentioning. As far as I searched, I've mostly seen recommendations for TrueGray and at least 300dpi for best OCR results. And it does correspond with my (light) testing. Of course, this makes your images quite bulky. So one of the things I considered important was to be able to run the OCR on a "better" scan, and then be able to merge the OCR results with an image with is just enough for me to read it well. ...

... So I do scan at 300dpi, run OCR on it, then downscale it to 150dpi, then merge the latter with the OCR results. I could go further down, or use jpeg compression for smaller sizes. But you get the gist of it.

BIT ROT

/leaves

@gusbrs If the OCR got lost, then I can certainly see how that would be a deal breaker.

@gusbrs That's an interesting strategy. What kind of tools do you use to do that?

@FaheemMitha That, and I also considered long term support in choosing pdf. I'd be willing to bet I'll still have the tools available to open these pdfs in 20 years. Can you say the same for djvu?

@FaheemMitha The ones I just sent you. ;-)

@gusbrs That's a good point, certainly. I have no idea.

@gusbrs I see, you use scanimage.

@FaheemMitha Yes, scanimage, the tiff and pnm libraries utilities, tesseract, and qpdf.

@gusbrs As mentioned, I use gscan2pdf. What OS do you use?

@FaheemMitha I've used gscan2pdf a lot too in the past, but my reasoning to move from it is also in the notes. An inconvenience, it does occasionally hang on large scans, and you have to start over. Most importantly, the tesseract OCR overlay grants much better results than the method gscan2pdf uses to do the same task (which I think is called hOCR), and I could not find a way to tell gscan2pdf to use something else.

@gusbrs gscan2pdf uses tesseract. I know this, because it keeps complaining. And your OCR option is certainly configurable.

Ah, I use Linux. Mint to be more precise.

@gusbrs OK. Is that a Debian Derivative?

@FaheemMitha Well, I have tried. ;-) If you find out how to do it, you can tell me.

@FaheemMitha There are two branches IIRC. The main one is Ubuntu, but there is a Debian branch as well.

@FaheemMitha Indirectly. It is on top of Ubuntu.

@gusbrs I could tell you right now, but I need to go do yoga. Or try to. But I'll try to follow up with you later. Are you going to stay in Chat?

@FaheemMitha I'm around. Not always necessarily logged in, but it eventually gets to me. And feel free to follow up. :-)

@gusbrs Can I message you if you are not actually in Chat?

If your email is public, I could email you.

@DavidCarlisle Hm, perhaps I do not understand something then. Where do the spaces between the x and the fraction come from in your example in $x\frac{1}{2}x$? Or do you mean that it is set tight?

@FaheemMitha As far as I know, it will generate a ping as long as one has been recently to the chat, which is the case.

@FaheemMitha It is not, but I can send it to you in case yours is.

@DavidCarlisle In the $x\frac{1}{2}x$ example, I see the following:

....\mathon
....\OML/cmm/m/it/10 x
....\hbox(8.44843+3.44841)x6.38612
.....\hbox(8.44843+3.44841)x6.38612
......\hbox(0.0+0.0)x1.2, shifted -2.5
......\vbox(8.44843+3.44841)x3.98613
.......\hbox(4.51111+0.0)x3.98613
........\OT1/cmr/m/n/7 1
.......\kern1.23732
.......\rule(0.39998+0.0)x*
.......\kern1.2373
.......\hbox(4.51111+0.0)x3.98613
........\OT1/cmr/m/n/7 2
......\hbox(0.0+0.0)x1.2, shifted -2.5
....\OML/cmm/m/it/10 x
....\mathoff

Isn't this: ......\hbox(0.0+0.0)x1.2, shifted -2.5 the \nulldelimiterspace?

@mickep yes, sorry, I lied

@FaheemMitha if the name is offered as completion when you type @ the ping will work, if not, not.

@DavidCarlisle Oh, no problem.

@DavidCarlisle (It seems all TeX people are liars! (Me too))

@mickep ooh the liar paradox

@PauloCereda Ohh, indeed :P

@UlrikeFischer Bär causing trouble again

@DavidCarlisle I'm not surprised ;-)

@DavidCarlisle oh no

@UlrikeFischer Gert "Bär necessities" Fischer should intervene. :)

@PauloCereda I can blame @UlrikeFischer for bash commandline usage

@DavidCarlisle but she uses PowerShell

@PauloCereda He suggested a new shirt:

@DavidCarlisle I saw that ;-) But I thought it is better if I take the blame and you try to figure out how to pass the umlaut around mylatex.

@UlrikeFischer mylatex was written to speed up pre-emtex sbtex on 8+3 dos filesystems, accented filenames wasn't part of the design

@gusbrs The OCR option is under Scan Document -> OCR scanned pages -> dropdown menu (in my case containing tesseract and GOCR as options).

@FaheemMitha I know, and the underlayed OCR text does work with gscan2pdf. But, as I've said, the difference is in the method used to underlay the text. Try to scan and OCR some pages with gscan2pdf and then open the pdf and try to copy-paste from it. You'll see that each word gets its own "block". Whereas, if you do it straight from tesseract as I've been doing, you get a proper block of text, with (mostly) correct recognized lines, etc.

@UlrikeFischer Is this a bear that bathes a lot?

@gusbrs First, what's the problem with each word getting its own block? I'm not even sure what that means.

@gusbrs If it is a bug in gscan2pdf, I think the author would be happy to fix it. He's very helpful and cooperative.

@FaheemMitha If you don't mind the difference, then you're fine with the hOCR method. But the selected region gets all cranked up. It looks bad to me. And what is a line and what is not becomes a good question. To some extent, the quality of the pasted text will depend on that. If you just do search, it may not matter to you.

@FaheemMitha I don't think it is a bug, it is just the method he chose. I don't know why though. And I prefer the tesseract "built-in" one.

@gusbrs I not infrequently cut and paste the OCR text. But I don't recall having major difficulties doing so. Of course, the OCR quality itself was quite variable.

@FaheemMitha ;-)

@samcarter @UlrikeFischer @barbarabeeton ooh twitter.com/shouldhaveaduck/status/1485648087943356420

@FaheemMitha Try the methods out with some samples. If you don't care about the difference, you may well go with the convenience granted by gscan2pdf, which is indeed considerable (I do like it).

@gusbrs I'm still unclear about what it means to have each word treated as a separate block vs the other option.

Is this documented anywhere?

@PauloCereda ohh

@FaheemMitha As I've said, use both methods, and open each pdf in, say, evince, and try to copy paste from them.

@gusbrs You mean write tessaract directly on the PDF?

@FaheemMitha Not that I've seen. I'm telling what my experience is. If you'd like to see what the hOCR file looks like open the gscan2pdf tmp folder and look for the files there. If you examine some of the OCR files, you'll see each word gets its coordinate, and why it works as it works.

@FaheemMitha As per the notes I've sent you. ;-) Do take a look at them. And yes tesseract can receive an pdf "configfile" as option to directly produce a pdf, and it can also produce a pdf with only the OCR text with -c textonly_pdf=1, so that you can manipulate the image and merge both parts later.

@gusbrs Ooh, homework. Love it.

@PauloCereda -- I'd name the one on the right "Motley".

@samcarter -- How about a wood duck? (Not quite as gaudy as a Mandarin, but quite colorful.)

@barbarabeeton Maybe you have some knowledge about or could be interested in this: chat.stackexchange.com/transcript/message/60236055#60236055 and/or this: chat.stackexchange.com/transcript/message/60236108#60236108 ?

@barbarabeeton ooh

@mickep all the math parameters are lengths, they can be set whenever you chaange the fonts size with commands like \large (which is why font size commands are not supported in math mode)

@mickep -- I have absolutely no idea why \nulldelimiterspace is set to 1.2pt; it just is. It's stated as a fixed value in the TeXbook with no reason stated. I'll ask a couple of people who might know, but without much hope to get a knowledgeable answer. If I had to guess, I'd say it's maybe because DEK couldn't really see anything less making the output clear. (With the facilities he had in the early 1980s, what was the size of a pixel?)

@DavidCarlisle But wouldn't it have been more natural to set it to 1mu or something? (Apparently the value 1.2pt has worked well...)

@barbarabeeton Thanks! I'm curious to hear if you will get some answer.

@mickep mu lengths are a bit weird and not used in lots of places that you might expect, such as \delimitershortfall or \scriptspace they are all (normal) lengths

@gusbrs Wir haben keine Badewanne, sondern eine Dusche ;-)

@UlrikeFischer Auf jeden Fall, ein sauberer Bär. :-)

ooh Bärs

I mean Bären

ooh Bärendregt

ooh lämbda cälculus

@PauloCereda dïnner

@DavidCarlisle \delimitershortfall I can understand since it is vertical, but also \scriptspace amuses me a bit that it is fixed, independent of font size.

@barbarabeeton Here is a wood duck for you:

@samcarter -- Well, different kind of duck, but definitely one who works in wood.

@mickep In modern engines (aka. LuaTeX and relatives) \scriptspace can be set separately for every math style.

@MarcelKrüger As in \Umathspaceafterscript, iirc. But even in luatex, it is by default some fixed value independent of font size, right?

@DavidCarlisle oh no

Big cosmetic change in the home page

@egreg It's so "responsive", so "responsive"...

@gusbrs As usual, the look “improves”.

@egreg Lots of quotes. ;-)

@egreg Indeed