@DavidCarlisle In DocScape, I'm frequently running out of registers in large or complicated documents, so I'm developing more and more schemes for reusing them. Most of the time, it is enough to avoid blatant wasting (i.e. there is some simple remedy like "don't allocate a new box for every page but reuse these boxes when pages are destroyed and created again"), but in the long run I'll certainly have some sort of allocation table with use count, garbage collection and all ;-)
@DavidCarlisle Well for instance in DocScape you can have any number of pages "open" (i.e. created, but not shipped out) at the same time, so there is a basic need for having several boxes and other registers allocated for every page, existing in parallel. (ctd..)
The "naive" approach is to just use \box@<pageno> for every page, currently limiting the page count to at most about 16.000. As I said, this is one case of blatant wasting, but OTOH a reusing scheme would then limit the number of simultaneously open pages...
@DavidCarlisle Yea, that's the way to got, but it requires more thought ;-)
@StephanLehmke We reached the conclusion that allocating globally was fine provided you assign locally, although I guess trying to deal with >16k pages at the same time might be an issue :-)
@JosephWright This is not as absurd as it might sound. With DocScape we're talking data based publishing. Often the first couple of document generations basically show what's in the data base, and what design means need to be taken to "compress" it sufficiently for getting a catalogue down to 1000 pages or so. So the first documents can be quite large...
@JosephWright But mind, I'm not asking for any means to be taken especially to cater for such needs. That's my (paid) job after all ;-)
If at all, I'm moving away from LaTeX towards implementing everything with primitives (or, at some point in time, in Lua).
@JosephWright Yea that's right. But that was only one example where registers are (and could) be used. For instance, in DocScape all procedures and functions are implicitly grouped, you have local variables, parameters, return values, recursion and everything without having to care too much for side effects. This currently works because for instance arithmetic is done very primitive, storing everything in macros.
Trying to do register arithmetic will open a can of worms...
I was thinking about adding types to variables and basic operators, to allow for better runtime error reporting and maybe faster execution, but if that's involve registers things might get hairy..
@MartinScharrer When copied from the documentation of standalone, the code has spurious whitespace around the words including commands e.g. \ documentclass { beamer }. Just to let you know :)
@StephanLehmke We came to the conclusion that as a 'local' register in TeX is grouped for assignment as well as for allocation, doing global allocation but local assignment made the most sense. There did not seem to be a gain to using local allocation when we looked at this, as it's not the same as local allocation for a functional language.
@JosephWright I'm not criticizing the decisions made by the LaTeX3 team. OTOH, the scoping rules of TeX are probably one of the most difficult things to grok for programmers, so it might make sense to provide a programmer's interface where block structure agrees with group structure.
@percusse This should be caused by listings. A recent change in the settings should have fixed it. I just tried the code example in the current version and it works fine. Which version do you have?
@StephanLehmke Well, as I'm used to TeX and other languages where there is no scoping of variables based on functions, so to me it's quite straight-forward :-)
@StephanLehmke there are cases where one is missing them, but providing something that goes so much against the nature of TeX means you would need to keep state to an extend that you really don't want (and only for a few cases where it comes in really handy).
@FrankMittelbach @StephanLehmke I have to agree with Frank here. I'm really really happy with the fact that the only "active" white-space is <newline><newline>^+
@tohecz I think there's a misunderstanding here. In programming "block structure" means the nesting of procedure definitions. In particular, as a general rule if I define a variable outside of all procedures, it will be (defined and assigned) global while if I define a variable inside a procedure, it will be (defined and assigned) local. This is also called static scoping while TeX (like a lot of script languages) uses dynamic scoping.
... and introcude command like \static\def\x.... ? That would change the meaning of \x only inside the now-defined macro?
... on the other hand, it would be very fragile considering TeX's way to use macros, e.g. you often have \def\macroA#1#2{some_action_on_arguments_1,2\macroB}, then \def\macroB#1{some_action_on_argument_3} instead of \def\macro#1#2#3{...}. This makes even the definition of "locality" very problematic...
@tohecz I'm sure this wouldn't work for the usual way of programming TeX macros. But then again, most macro packages are write-only. I've observed a remarkable reluctance of even seasoned TeX programmers to read and build upon the code of others. But in particular with LaTeX, the way of writing "commands" is different. Look at the ifthen package, for instance. I doubt the majority of LaTeX users who can write a "command" would understand the macro scheme you're citing.
@tohecz Yes: \@ifundefined{relax} returns true. Every token whose meaning is equivalent to \relax is considered "undefined" for this test (also really undefined ones, because the token is built with \csname).
@tohecz That's the basic idea. But in addition to that it would be nice to have return values (which are transported out of the group) plus local variables. And that's the beginning of the discussion: Local macros are easy, local registers are hard.
@Stephan: \def\let@return{\let\return\@return} and then in the "procedure" something like \xdef\@return{#1.#2.#}\aftergroup\let@return could help I think
@egreg And one of the most annoying things is the way \aftergroup is used to manage colors. It sure does its job, but it makes it so easy to introduce subtle errors... :-(
Speaking about colors, tell me, is \begin{document}\color{black}Hello\end{document} equivalent to \begin{document}Hello\end{document} as far as the package xcolor is loaded?
@tohecz Nope, I believe text will have no color at all if you don't explicitly set it. So for instance, if you include a PDF generated this way later with \includegraphics, the text will come out in the current text color of the including document.
@tohecz Furthermore, this is typical for the way \color shouldn't be used. You'll have oodles of color switching stuff on the vertical list, with unpredictable behaviour.
@PauloCereda I use a name, sometimes, or something related to the problem. It rarely happens that the name leads to an old file where a similar problem was analyzed.
@StephanLehmke From memory, one of the big issues we had with 'local' registers (and macros) was that you still need to keep a track of what is defined in relation to TeX groups. If we imagine doing something of the form \def\foo{\begingroup\newlocaltoks\footoks, then if we use \foo again inside \foo itself the local assignments don't 'disappear'. That's not what happens in functional languages, and so it did not really seem to help programming TeX.
@JosephWright With etex.src approach the allocation is made in the same way as usual (starting from the end of the pool), so a new \localcount\foo will allocate a different register. Not sure if it's really useful.
@egreg Yes, but one of the early LaTeX3 decisions was that allocating 'new' items should check for an existing allocation: not sure what the expected behaviour would be. As I said, the big issue was that you still need to have the \begingroup, so why not just allocate globally and set locally.
@JosephWright As I said, I'm not criticizing the decisions made fot LaTeX3. I was just trying to argue that there could be uses for that kind of thing. Furthermore, I don't understand your example. Surely, \newlocaltoks would reset \footoks to a neutral value when called recursively? Otherwise, havoc would result if I call all my local variables \i ;-)
@StephanLehmke As I've just said to @egreg, it would raise an error due to the prior existence of the csname
@StephanLehmke No, I understand. Just trying to explain the reasoning. We did try it for quite a while, but it was all about the benefits versus the additional complexity.
As we have no namespacing in TeX, the best approach seems to be to insist that all names are globally-unique
One problem seemed to be that there was a danger of people coming from other areas trying something of the type \def\foo{\newlocaltoks\footoks ... without the grouping and wondering why it did not work
@JosephWright Yes, but still one could reuse registers by letting the global names to \undefined. The basic idea of keeping the meaning of a global name can easily fall down as soon as people (like me :-) try to define register arrays with \csname.
@StephanLehmke There are some open questions about how best to handle arrays :-) For example, property lists are currently done as a single csname 'packed' with data, which works well for small lists but less well as they grow. On the other hand, one csname per entry makes copying harder and impacts on performance generally as it fills up the hash table.
I added some code for 'data tables' (l3dt), which seems to be on the wrong side of the performance question, when compared with say datatool, which uses a hybrid approach of about half a dozen csnames per table.
@JosephWright I had imagined that LaTeX3 offers a higher level interface to defining macros. Then there could be one with implicit grouping. One could disable local registers everywhere else.
@StephanLehmke Yes, I can see that concept but then what happens if you assign a local variable which is not local to that particular macro?
One thing I've said before is that we could talk for twenty more years about the 'best' approach for LaTeX3 (concepts, syntax, ...) without delivering anything, and while I'm keen to retain 'flexibility' we also need to have some decisions made. The only way to really test these things out is to write code which is actually usable, and that requires some confidence that things will stay about in some way.
What I notice is that although etex.sty offers local register creation, LaTeX programmers have survived mainly with the assign-globally approach. It works reasonably well with TeX's flat namespace and macro expansion basis.
@JosephWright One may need those for, say, a big "one-time" computation: doing it with (local) registers and not with \int_eval:n can save much time. But it's difficult to think of an application that requires thousands of registers, connected to real typesetting and not of the "let's-do-it-all-with-TeX" kind.
@egreg As you say, it's a question of dealing with 'real' problems. I suspect @BrunoLeFloch would point out that the way to do that sort of thing is to use a group then access registers by number, not name, to avoid filling up the hash table :-)
@StephanLehmke's problem for 'big databases' are probably borderline for the aims of LaTeX3: they do seem to be best addressed with custom code built from the ground up
@JosephWright This is certainly true for systems like DocScape, FO or DocBook processors. But I believe that data based publishing, CMS, Web2print and such are on the rise. Why shouldn't LaTeX3 cater to be a backend for laypeople wanting to do this kind of thing with a free system?
@StephanLehmke Perhaps I should have phrased that slightly differently. I meant that the first target has to be to get something which can deal with 'small' documents (a book, a report, a thesis, ...). As you say, database driven work is important, but there are specific challenges for very large databases that may need specific tools that at this stage we are not in a position to tackle. Does that sound reasonable?
@StephanLehmke I'm not ruling stuff out, but want to have some intermediate goals. The LaTeX3 idea of separate formatting from processing, for example, should fit well with data-driven input.
Also, it's a team effort, and different people bring different ideas :-)
@JosephWright Of course. As I said, I'm not at all demanding anything. For sure the classic manually edited LaTeX document, which will make about 99% of the use base, has to be in focus. Furthermore, writing any kind of automatic system always means one can afford to put a lot of work into the basic programming (as a lot will then be generated automatically), so writing dedicated extensions should be part of that work.
@DavidCarlisle 2^n doesn't go up that quickly for small n. 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4092..ok, I forget 8k as I've only used it twice.
@JosephWright For very large databases it's probably much more convenient to have a way of querying with an external program. TeX shouldn't be misused for doing what is better done differently. Of course "small databases" are a different cup of tea. But if one is typesetting an English dictionary, with scores of examples for each term among thousands of them, ...
I guess that Lua can have a big part in integrating external calls. But it's a different job than LaTeX3
@egreg Well I at least wasn't talking about querying a database with TeX. Hereabouts, querying is done with SQL, then structuring, transforming and aggregating with XSL, leading to an XML representation of the document content. But still, depending on the size and complexity of the database, this XML can be quite large and deeply structured. Furthermore, if a lot of design decisions wrt. aggregation depend on the size of things, those have to be done with TeX.
@egreg I'm not completely sure what you mean by this, but if you're thinking about generating something like a pre-formatted "content" stream, that's the FO approach which is vastly inferior to what DocScape offers.
@egreg :-) Basically, XML as is is only syntax, no semantics. Of course, you can assign a semantics to a certain XML syntax specification. XML formats with predefined semantics are FO, XSL, RNG, SVG etc. The semantics of the DocScape data files are just the data itself :-)
@StephanLehmke Out of curiosity, how do you map data to page elements? I imagine it's a template-like method to assign chunks of data to well-defined layouts. :)
@PauloCereda Exactly. There's a markup language for laying down the design. It's a bit on the level of LaTeX markup, but instead of writing down the content immediately, you can access the XML data content in a structured way.
The semantics behind the "design markup" is a bit different from LaTeX however. Instead of producing content elements which are contributed to the vertical list, you "place" stuff on a design grid. Much more similar to textpos or flowfram, but with a lot of bells'n'whistles.
@JosephWright Ah the technology. :) I say the Duchess of Cambridge with her red dress - here our TV narrators don't do their homework about the United Kingdom, so they reduce their comments to general stuff about fashion. :)
@StephanLehmke Brilliant! It seems a very clever implementation. :)
@JosephWright I received a request for a demo from "the LaTeX3 team" (in the person of Frank), but for reasons unknown to me, our CEO is a bit hesitant to come to a decision. Mind you, it's all in german currently...
Well DocScape has the honour of having been treated to an open source clone (by another regular here), so you can always look at that :-)
Should we close this question as too localized. The op found a solution (see last comment). I think the linked bibliography style isn't a standard style and so doesn't related to TeX.SX
In LyX, I'm using the reference style apsrev.bst, which I downloaded from here:
http://www.maik.ru/pub/tex/revtex4/
This reference style is the one used in Physical Review.
But now when I add references, they show up as question marks in brackets (?).
Why does this happen? how can I fix it?
...
I am using a bit modified bibtex style but with quite common base. It is based on utphys 2.7 which is based on "IEEE Transactions bibliography style (29-Jan-88 version)"
I'd like to remove comma after titles ending with question marks. Some automated hack will do. (Not changing the bbl)
Bibtex ...
@StephanLehmke: I have a simple project of mine in which the dataset is mapped to page elements. But the heavy work is performed by the template engine itself, not TeX. :) Thankfully, I could incorporate several lambda functions to it. :P
@JosephWright: I use xkeyval and have a value which contains a comma; actually some key=values as well: \adjustbox{key={a=1,b=2,c=3},...}{...}. For some reason the commas in the values break things in a weird way.
The {a=1,...} part actually is taken as part of the adjustbox content! Looking at the log file xkeyval actually sanitizes the value and scans it for ,! No idea how it will handle it from here. Do you have any idea how to fix this? I find this features of xkeyval very strange.
@Canageek yes but n isn't that small, I wrote "something" as I was dashing out, but etex gives you registers up to 32767 which should be plenty if you are not @StephanLehmke
@egreg no we had various sports in the morning (tug of war, egg and spoon etc, then we closed the high street and the entire population of the village (~200 people) sat at a long table down the centre of the road and ate tea and cake and sandwiches, then we retired to the pub for the evening. Not a drop of rain:-)
@DavidCarlisle It reminds me of a very similar small village where I was for some days years ago, in Dorset. It's called Marston Magna: nobody can find Marston Parva, of course. :)
In LyX, I'm using the reference style apsrev.bst, which I downloaded from here:
http://www.maik.ru/pub/tex/revtex4/
This reference style is the one used in Physical Review.
But now when I add references, they show up as question marks in brackets (?).
Why does this happen? how can I fix it?
...
I am using a bit modified bibtex style but with quite common base. It is based on utphys 2.7 which is based on "IEEE Transactions bibliography style (29-Jan-88 version)"
I'd like to remove comma after titles ending with question marks. Some automated hack will do. (Not changing the bbl)
Bibtex ...
