@brama To reduce the cost of reading the last line in a large file, you could jump to an arbitrary position near the end of the file using SetStreamPosition before reading in the lines. e.g. something like this:
Module[{s}
, s = OpenRead[$file]
; First @ CheckAll[
SetStreamPosition[s, Max[0, FileByteCount[$file] - 1000]]
; ReadList[s, String] // Last
, Close[s]
]
]
Here, it is assumed that the maximum line length plus newline sequence is less than 1000 bytes.
If the maximum line length is unknown, or the file encoding contains variable length multibyte characters, then the same principle could be used but there would need to be more trial-and-error processing to find the last line start.
So, I was typing up a reply but it didn't fit in the comment field anyway. So I'm just going to paste it here.
But maybe this will help: whether associations are atomic or not is entirely up to us. It's not dictated by any implementation detail, just by whether we think it makes obeys the definition (hint: it doesn't).
But I think what you're really saying is that Associations are "NormalQ", i.e. they aren't equivalent to their FullForm in a literal sense. That means that they require special support in almost all kernel-level traversal (e.g. Part, Map) and mutation (e.g. Append) functions. One place they don't yet have this support is the pattern matcher, which is because the pattern matcher is very delicate code and it is quite a big job to add an entirely new class of expression to it.
So, in conclusion, there are two concepts that many people (including us), confuse: AtomQ and NormalQ.
AtomQ means: can I traverse this, or is it atomic?
clearly, Associations are not atomic
But historically, people have used AtomQ to mean something else: is this an ordinary expression, or does it have special traversal semantics?
e.g. SparseArrays
obviously, associations DO have special semantics
so they are not NormalQ
Currently, NormalQ does not actually exist. We've somewhat abused AtomQ to mean, in some cases, NormalQ.
anyway, here's my take, and I'm going to push hard on this because I think I'm right and everyone else is just a bit confused: Association isn't AtomQ, but neither is it NormalQ.
It clearly can be traversed, but it has special traversal semantics that are not 'ordinary FullForm'.
Okay. Thanks! That all makes a lot of sense to me, and yes, from that standpoint I was conflating "Atom" and "Normal" however since the latter a new term (though not a new concept) I only had atomic to choose from.
@TaliesinBeynon I understand. Hopefully soon we can chat. Hopefully I'm not driving you crazy asking. :^)
I have been curious about it for long. Now that Mathematica 10 arrives, I think it's time to ask the question: How the new Association data structure would be used to improve Mathematica programming?
There are a few related aspects of this question:
(1) How fast is the key-search in Association...
@Mr.Wizard I still think this is a bug in TeXForm, but need to ask at the latex forum first to verify if the Latex code generated from the above is valid or not. Currently Latex does not like it. Thanks.
@E.Doroskevic Write a comment to one of his answers and he will be notified about that comment.
Also you may want to adjust your hopes, the idea that anyone here would hand out his email is ridiculous and even when talking to somebody one on one here on StackExchange you are very unlikely to persuade someone to take continue the discussion of email...
@E.Doroskevic Sure you do. Since you have an account on Stack Overflow and Android Enthusiasts you will be awarded 100 reputation points upon creating an account for this site.
@Pickett I will look into that just now again and see if I can open an account on Mathematica to get those 100 reputation points you've mentioned. Thank you.
It's only a list of error/info messages but if you spot a mistake, it would be nice if you ping me in the IDEA chat. I'm about the prepare a new release and since I have now all messages at one place, it can be checked easily.
Photography by Tracy Howl and Paul Clarke Has our newfound massive availability of data improved decisions and lead to better democracy around the world? Most would say, “It’s highly questionable.” Conrad Wolfram’s TEDx UK Parliament talk poses this question and explains how computation can be key to the answer, bridging the divide between availability and [...]
Hi guys, I'm trying to understand how to control the error in the "Projection" NDSolve method. It seems that even if I set AccuracyGoal -> infinity and PrecisionGoal to a large number, it is only giving me ~10 digits of precision in the solution.
I've used a lot of SetPrecision calls to try and set the precision of all the inputs to the same as the WorkingPrecision, which I set to double the PrecisionGoal
how do I plot the following step function such that the value of function between 0-1 is 3, 1-2 is 5,...and 4-5 is 4. a = {3, 5, 7, 1, 4};Plot[a[[Round[n]]], {n, 1, 5}]