To (partially) answer my own question from yesterday: in the data dumps, the users table includes a column for the network profile ID. So, one could download the data from multiple sites, load it into a database, and join on that (if my rudimentary DB clues are correctly applied), and from there one could do cross-site user-based queries. I think. I don't think SEDE does this though.

(This will remain academic for the nonce, as I don't have, nor yet know how to write, a parser for the XML format in the data dumps. And converting data including posts and comments to CSV seems like rather a bad idea.)

@MonicaCellio CSV is a singularly badly designed format.

@TRiG I assume that there's some way of escaping commas that are in the data, but that sounds inconvenient if you have a large amount of data (like, say, all the SE posts). And I can't think of a single character that it would be safe to use instead of a comma, so changing CSV to something-else-SV probably isn't the right way to go. But I don't know; I'm kind of new to this.

On this question (judaism.stackexchange.com/q/41240/5323), I was trying to figure out what tag to use for a drasha question, and settled on exegesis .....later research revealed that the only other question with that tag was Baby Seal's Pardes question.....

What tag can be used for a specific drasha question, instead of a "drasha theory" question?

@Shokhet I think the one you want is midrash

@MonicaCellio I guess a non-printable char would work

@MonicaCellio An escape character would be reasonable, wouldn't it? Instead, if there's a comma or a linebreak in the data, you enclose the field in quotation marks. If the field already has quotation marks, you enclose it in quotation marks again. (See Raymond Chen via Joel Spolsky.)

@DoubleAA OK .... maybe we need a separate drasha tag, cuz midrash includes a lot of stuff from Midrash, and a lot of drashos ....

@Shokhet Midrash and drashos are just different conjugations of the same thing, really. If you have two or more subcategories you think the material could be split into (which I think you do) perhaps post a proposal on Mi Yodeya Meta with proposed tag names and definitions for what precisely goes in which tag. (Writing sample tag wikis could be a good way to do that.)

@DoubleAA Done.

@MonicaCellio You can probably copy-paste from your browser to a spreadsheet program and go from there rather than dealing with the XML.

@MonicaCellio While CSV may be strictly comma-separated, programs that interpret it will accept backslash-comma as nonseparator, or "...,..." (with the quote marks) as not containing a separator. Check your program (e.g. mysql's load‌​) specs.

Oh, and now I see @TRiG has said the same thing.

@msh210 I want to load the whole data dumps -- no way cut/paste scales for that. :-)

@MonicaCellio If you know what you're planning to do with it, it may be possible to run queries on each site and only then combine them.

@MonicaCellio If you don't have an XML parser, perl (or sed) works: change every instance of foo to a comma, etc.

@msh210 I have a new toy and am exploring possibilities. SEDE does a good job for per-site queries, but it can't answer cross-site queries. I don't know what those queries might all be yet -- thinking about all the various aspects of this. (And, um, learning enough SQL to do it.) [cont]

One thing I've been wondering about is how users' participation on multiple sites clusters -- where are Mi Yodeya's most-active users also active, stuff like that. Tags used on multiple sites (or related tags, which would require human analysis first of course) could also be interesting hunting grounds -- given that X is on topic on multiple sites, where does it tend to do best? Etc. Definitely thinking out loud & open to input.

(And with that, ducking out again for the next meeting...)

@MonicaCellio It'd be nice if SE allowed a database dump download in addition to the SEDE. :-)

Oh -- it does!!

@MonicaCellio archive.org/details/stackexchange

(H/t Smi.)

@msh210 yes, those are the data dumps I'm talking about. What if you loaded them all into a DB...? What could you do then?

@MonicaCellio Join them on network ID as you said (if it includes network ID).

... which it does not. So you'd need to download the Users table from the SEDE also.

Or the ID and network ID columns from it, anyway.

You could union/append the post tables, I suppose, to make one big table, so long as you add a column for site.

But I doubt that'd be so useful. Probably better to keep them separate.

@msh210 every site in the data dump has a users table that includes the network profile ID. So one could load all of them into one uber-table (so long as the load step includes adding a column for the site of origin), and then could look at some user patterns just from that (though you'd want other tables, like posts, for some queries). A good first exercise might be: "high-rep users on site X also liked..." -- that can be done just from site, network ID, and rep.

Is this room a god place to ask about a post?

*good

@BrianTkatch sure, what's up?