« first day (1047 days earlier)      last day (1352 days later) » 

1:46 AM
I thought of that, but I still didn't know how to do it, but I will try to figure it out
 
2:31 AM
Shall we continue in the Crystal documentation?
@Zanna Alternatively, do you want a hint about that?
 
3:02 AM
@EliahKagan yes :)
@EliahKagan I think I need a hint, as I suspect the thing I need it something I don't know about
Back shortly...
 
3:16 AM
@Zanna When not thinking of a solution, it can be useful to solve a simpler related problem instead.
In this case, the simpler related problems that come to mind are (a) printing a multiplication table without proper alignment, (b) printing a single row of a multiplication table (with or without proper alignment, whatever that means in a single row), and (c) printing a single column of a multiplication table (with or without proper alignment).
 
:)
ok, I'll try that later
 
Also, if you were thinking that the multiplication table should have labels showing the factors, then a simpler related problem is to do it without labels as the factors. If your factors range from 1 to n (where n is often 10 or 12) then those labels look repetitive anyway and you might not ultimately want them.
 
@EliahKagan sorry, I don't think I read this part yesterday. If it's not departing too much from the topic, could you say what "fully buffered" means? I don't think I have any idea what buffer means in this context
 
@Zanna You're certainly under no obligation to try any of them now (or even at all), but if the simpler problem you're thinking of solving is one that seems like it would take an extended time, then you might want to pick a different simpler problem to solve initially whose solution is immediately apparent.
(Probably not all problems have a usefully illuminating simpler problem that can be solved very quickly. But many do.)
 
I did think of trying to do some such things, but it was not immediately apparent to me how to do anything much
but, I'm used to that XD
 
3:25 AM
@Zanna Well, you can probably print the ones column (for 1*1, 1*2, ...), for instance.
 
probably
 
If the solution is not apparent, it may be worthwhile to think of what that output would be and of how to produce that output without thinking of it as the result of multiplication.
@Zanna Full buffering, also called block buffering, is actually simpler than line buffering, so I'll say something about that first.
 
3:51 AM
In unbuffered I/O, all read and writes perform a system call to tell the kernel to perform an operation on a particular file descriptor. In bufered I/O, data are read or written less often in larger amounts; to faciliate this, they are staged in a buffer. An input buffer is a region of memory into which usually multiple bytes of data are read. An output buffer is a region of member into which usually multiple bytes of data are written.
This allows operations that conceptually involve multiple small read or small writes to be coded that way, in terms of functions for reading and writing that are themselves implemented to use a buffer.
Reading or writing the same amount of data in fewer actual reads and writes (i.e., fewer system calls) offers a performance advantage because system calls are slow and sometimes for other reasons.
@EliahKagan By this I mean, it's good to be able to write code that reflects the problem being solved, rather than the low-level details of the solution. Many problems in which input and output are used are naturally solved in terms of writing or reading a small number of characters at a time, and doing that many times.
For example, suppose I want to read numbers separated by whitespace--where by numbers I mean integers represented by sequences of digits--and each time I get a number, I want to do something with the number. Then I should use the facilities provided by my language or library to perform that high-level operation!
But suppose I am writing such a facility. Or maybe what I'm doing requires specialize treatment for some reason. (Also, some languages really don't offer a good out-of-the-box way to do that, instead focusing on reading input line by line.)
To simplify the example, I'm imagining the input is endless, and also that no read errors will occur (that is, no I/O errors--the input can still be ill-formed), though obviously that's not usually the case.
To read just one number, I would read one character at a time until there is nothing to read or I get to something that is not whitespace. If it's not a digit, that's an error. If it is a digit, I would store it, then read and store one character at a time until I get something that is not a digit. If that character is not whitespace either, the input is ill-formed. Otherwise I'm done reading that number. I do whatever needs to be done with it, then start over again to read the number.
That description is a sense quite low-level. After all, the high-level goal is, "Read whitespace-separated numbers." And you can, inefficiently, even write low-level code that corresponds directly to it. But in another sense, it is a high-level description, because what "read one character" means is to consume a single character, such that the next character will still be possible to consume later.
 
4:12 AM
To flush a buffer is to cause pending operations to be performed. So, flushing an output buffer actually writes the data currently stored in it--and then the buffer is empty. Flushing an input buffer actually uses the data stored in it.
Flushing an output buffer is effectively always something that's done by the program writing output (usually as an automatic result of filling it all the way or of closing the file descriptor, but sometimes explicitly).
In contrast, flushing an input buffer is typically not done by the program reading input (and it is not universally clear what it would mean for a program to do that). The major example of forcing a flush on an input buffer that would not otherwise have occurred is when you press Ctrl+D in a terminal.
The asymmetry is because, to write buffered output, a program writes into a buffer until there is no room in the buffer (either it is full or another write needs to be done that would exceed its remaining space). Then it performs a write of the total number of bytes in the buffer. A library usually takes care of this, but it is still something that can be--and is--implemented in program logic.
In contrast, to read buffered input, a program attempts to read some number of bytes. If the read ought to complete before that number have been successfully read, that is not something the program can know or handle itself, because it does not know about that condition, because the read has not completed.
I've effectively described full buffering (block buffering). For example, the dd command performs buffered reads and writes. To specify the size of the buffer, you can use bs=, which stands for "block size." Otherwise a default of 512 bytes it used. It is not guaranteed that every read and write is of this size, because what if the size of the input file, or the output file, is not a multiple of that size? However, every write but the last (assuming all succeed) is of that size.
So, in full buffering (block buffering), an output buffer is flushed when it is full, when the file is closed, and when an explicit flush is performed. And an input buffer is flushed when it is full, and when reading succeeds but with fewer bytes than requested.
In line buffering, all that is the case. There is a buffer of some size, and it is flushed under those conditions. But it is also flushed when a newline is written or read. Thus, when a full line of output is ready (or the rest of a line, if it is longer than buffer size), the program flushed the buffer even if it's not full. And when a full line of input is ready, the read call completes even if there are fewer characters on the line than the number of bytes the program attempted to read.
@EliahKagan to read the next number
@EliahKagan There might be additional exceptions to this.
@EliahKagan Input in a terminal is line-buffered by default, which is why when you run cat with no arguments, you don't see what you've typed until you press Enter to send a newline or press Ctrl+D to force the buffer to be flushed.
 
5:33 AM
@Zanna Sorry, I got distracted by the other stuff. If you still want to continue in the Crystal documentation now/soon, I'm up for that. If not, there's always the future. :)
Regarding buffering:
In computer science, a data buffer (or just buffer) is a region of a physical memory storage used to temporarily store data while it is being moved from one place to another. Typically, the data is stored in a buffer as it is retrieved from an input device (such as a microphone) or just before it is sent to an output device (such as speakers). However, a buffer may be used when moving data between processes within a computer. This is comparable to buffers in telecommunication. Buffers can be implemented in a fixed memory location in hardware—or by using a virtual data buffer in software, pointing...
79
Q: Buffered vs unbuffered IO

sud03rI learned that by default I/O in programs is buffered, i.e they are served from a temporary storage to the requesting program. I understand that buffering improves IO performance (maybe by reducing system calls). I have seen examples of disabling buffering, like setvbuf in C. What is the differ...

4
A: Difference between buffered io and unbuffered io

William PursellThere are multiple layers of buffering. If you call write, no application layer buffering will occur. If you look at the file from another process you will see the data, but that does not mean they have been committed to disk, because there is a layer of buffering happening in the kernel. Sinc...

 
 
5 hours later…
10:59 AM
@EliahKagan I'm sorry, one small mishap and after that one distraction just led to another and another
@EliahKagan ok, I understand :)
I think I should have remembered this from reading about what a journalling filesystem does at some point. That text probably used the same language you are explaining here...
I think Vim calls its open files buffers
@EliahKagan I like the rollercoaster analogy
also amused that wikipedia has an article on "queue area"
sometimes I give myself the pointless task of explaining the significance and use of some object or practice to a being in another galaxy...
but, I guess there are really people using the internet somewhere who have no experience of queuing
 
11:22 AM
@Zanna I'd say this is another kind of buffer. But it is very conceptually similar in that it holds data that may not have been written to a file but likely will be written. Also, calling the container for text one edits in a text editor a "buffer" is better terminology than "open file," since there may or may not be any open file that corresponds to it.
Some other text editors, such as Emacs, call them buffers as well.
@Zanna That reminds me, though it is not the same issue, that I'd wanted to explain why I had said "because system calls are slow and sometimes for other reasons."
The conventional wisdom, which was once often true, is that buffering improves performance physical I/O operations carry overhead. That's actually still true, but more of lower levels of buffering that an application does not control and that happen even when a program uses unbuffered I/O. It contributes, though not nearly as much as in the past, to the performance benefit associated with a program being written to use buffered than unbuffered I/O.
Hard drives have on-boardcache in which buffering is performed, i.e., when a write has been performed and as far as the kernel has concerned has succeeded, the data may or may not yet have been written physically to the storage medium. Either way, a read of the same sectors on disk is guaranteed to work as though it has.
And kernels perform buffering as well, i.e., when a write system call completes, it's guaranteed that subsequent read system calls that access the data see the value, but the physical device may or may not yet have been accessed.
@EliahKagan because physical I/O operations carry overhead
@EliahKagan rather than unbuffered
@EliahKagan on-board cache
^ Sorry, I was distracted by something unrelated and didn't have an opportunity to read over those messages during the edit window.
 
12:29 PM
@EliahKagan as the answer you linked to last said, I think
 
12:46 PM
@Zanna Yes. That's why I included that one.
 
1:13 PM
@EliahKagan it doesn't contribute as much as it did in the past, because physical I/O operations aren't as slow as they used to be, or because hard drives do their own buffering and/or the kernel does too, so that it seems like things have been written to the disk although they actually haven't?
 
It doesn't contribute as much as it did in the past because of kernel-level and disk-cache level buffering.
 
because the first buffering seemed general, I reduced it to "buffering isn't as useful as it used to be because of this other buffering", so I asked that silly question
I guess the first buffering is the buffering we were originally talking about, that done by the application!
 
That's not a silly question. I also wish my answer were more specific, as to how much of the performance benefit of using buffered I/O (in the sense of an application buffering to make fewer system calls) is due to the overhead associated with system calls vs. other factors.
(But I don't think I know the answer more specifically.)
 
anyway I learned a lot from your explanation
thank you very much for that
 
Cool. :)
I was going to say that you might want to take a look at how buffered I/O is implemented in Crystal's standard library. But the various functions involved, and how they call each other, turns out to be somewhat convoluted.
 
1:22 PM
it might still be interesting though
 
When I called puts, one of the methods (a method is a function, basically) that got called was write_byte.
Here's the implementation of that method: github.com/crystal-lang/crystal/blob/5999ae29b/src/io/…
 
(I am going afk for about 1 hour, then may have a small bite of time)
 
If you're interested to do the multiplication table thing at that time, then you could consider how you might do it in the mean time.
If you wish.
 
yes :D
 
 
1 hour later…
2:45 PM
@Zanna I am available, if this works for you. (Totally okay if it does not!)
 
I was mopping the floor thinking about Bash scripts and wanting to throw down the mop to start working on it... I finally finish and decide to read the recipe for what I'm going to cook... I notice I haven't typed up the adaptations I made to the recipe on the previous page... the recipe has a soundtrack (Succotash by Herbie Hancock) and since then I've just been dancing XD
my attention span is regressing...
would you like to read some more of this documentation?
 
@Zanna That all sounds worthwhile! :)
@Zanna How so?
 
@EliahKagan haha totally :D
 
@Zanna If you'd rather do that now than the multiplication table thing, sure!
 
@EliahKagan I mean I can never finish anything hahaha
@EliahKagan I'll be doing that whenever I have ideas, like at 3am XD
 
2:55 PM
So, where were we? Interpolation?
I think we read and talked about the section on interpolation and were going to move on to the next section.
But did you want to try String.build, which interpolation uses behind the scenes?
 
that sounds like fun
@EliahKagan yes
 
3:11 PM
input = String.build do |io|
  while line = gets
    io.puts line
  end
end

STDERR.puts
STDERR.puts "Got #{input.size} characters."
STDERR.puts

print input
puts appends a newline, unlike <<.
icr(0.35.1) > String.build { |io| io << "hello" << "world" }
 => "helloworld"
 
hmm I'll try it...
 
<< can also be used with actual FileDescriptor objects.
 
@EliahKagan what is the STERR part for?
oh I got it
you have 3 lines to get a clear line above and below the message
cool :)
@EliahKagan did we already met this |io|?
there was a thing like that in the HTTP server example
there it was |context|
we are giving the name, right?
yes I can write anything there and it works the same way
 

« first day (1047 days earlier)      last day (1352 days later) »