« first day (1627 days earlier)      last day (3335 days later) » 

12:28 AM
could I get advice on small-scale/cheap consulting? or pointers to better places to ask such a question?
everything (on my end) is running on Linux (specifically Debian)
I need to tunnel one VPN (hard requirement) over another (just my idea for a cheap way to meet some other hard requirements--might be other ways to do this) in order to SSH to research computing I need to finish my thesis project.
if I can't get this working, I'll hafta punt on the project--it's currently unfunded, and my savings burn rate is significant
I posted the problem @ serverfault.com/questions/674476/… but got no response
it's definitely too weird for the support folks @ the site hosting the clusters to which I need to VPN--they only run/support XP (still! in 2015!)
and the folks at my university are not interested, since this is not on their network
so I need help from someone who knows more about TCP/IP than I do, but I'm currently unfunded, so my current income==0, but very much need to solve this problem, so would pay what I could.
I'd appreciate help pro-bono, and am not embarrassed to ask because
(1) all my work is open: open-source, open-science code and docs on open repositories--I'm even [openly documenting this problem/solution](https://bitbucket.org/tlroche/linode_jumpbox_config/wiki/Home)--and everything's running on Linux
(2) [my research](https://bitbucket.org/tlroche/aqmeii-na_n2o/wiki/Home) (on modeling N2O, an ozone-depleting greenhouse gas) serves "the public good"--this is not about helping me launch my startup
 
 
5 hours later…
6:18 AM
@Gilles - This was good advice: But don't make it about Bob; as soon as someone thinks it's about particular people, you lose the ability to have the general discussion. Talking about specific users like that is uncomfortable for a lot of people, and if the moderators are already taking steps, they probably can't reveal that. You should have followed it.
You complain that I have latched-on or something, but I'm not writing a lot of stories about you.
Nor do I make a lot of baseless accusations, either. If you have an issue - cite the content publicly - as cuonglm suggested. That's the point of this place, is it not?
I apologize, if, unlike you, I don't always write something that is perfectly correct. I will do my best to rectify that. But seriously, get a new joke, yeah? You're wearing out that xkcd link.
 
6:40 AM
@Gilles - I can say for certain, though, that I particularly object to your (almost ridiculously ironic) claim that I make ad-hominem attacks on you, when - as it seems to me - as your post and as this very chat thread can plainly witness - the opposite is true. If you have a complaint, please, just cite evidence as to why and stop perpetuating baseless claims. Please.
 
7:03 AM
@Gilles - It's interesting to me that you didn't bother to post the comment above that one - the one which actually had evidence to the contrary. And I did read the manual Gilles - at your suggestion, and it was a good one - only to learn that while there is a readline variable you can set to configure bash to follow tty special chars, it matters not.
Because if you unset that var and set, for example, \c-w to something else, then it will still do the word kill function if stty has it configured to do so.
But those were minor things - the answer was a good one, I thought, and I wanted to upvote it - which is why I posted the comment in the hopes you would correct it. Instead, it would seem, you had a fit, I guess.
I'm sorry.
 
7:22 AM
And the last thing I have to say is this: Gilles, if you really think I'm out to show you up for an incompetent or whatever paranoid thing you said, then why do I delete them on the occasions that you (or anyone else) agrees with the comment and makes an to his/her q/a to some effect? It's about the information, man, it's not about you. So stop whining, already.
@slm @MichaelMrozek @terdon and some others - can you guys please suspend gilles if he bitches in chat about me again?
 
8:14 AM
@TomRoche ask a question on the site. If you aren't getting any responses, feel free to come back here. But note only a small proportion of the site users are in chat at any given time. The moderators do keep an eye on things here though. Well, some of them.
 
 
1 hour later…
9:17 AM
@TomRoche Your question on SF has received 36 views in 11 days. It is also, I think, not quite on topic there since they deal with professional sysadmins only. I recommend you flag your question for mod attention and ask them to migrate it to this site. You might have better chances of getting an answer here.
 
 
4 hours later…
1:15 PM
Hi i'm not sure if we can post something like, 'what does this perl one liner do' on this forum. Some forums dont allow that. Can someone please post the URL that informs us if we can post such questions.
 
@user3146086 Well, that one is borderline. Strictly speaking, that's not about *nix but we do consider simple Perl scripts on topic. A one-liner would probably be OK. Make sure you also include what you have tried. Mention that you checked the documentation etc.
What is the perl command in question?
 
perl -ne 'BEGIN { $/ = q[|] } s/\n//g; print $_, $. % 9 == 0 ? "\n" : ""' test.txt > xyz.txt
thats from
http://www.unix.com/shell-programming-and-scripting/186555-concatinating-lines-based-number-delimiters.html
except in my case the row does not end with a delimiter
So I have 212 delimiters and 213 columns
I tried decrypting this on my own, but have not been able to, specially because each character is a command that does more than one job
 
@user3146086 Well, not quite.
BEGIN{} : this is run once at the beginning. Before the input files are read.
$/ = q[|] : this sets the input record separator to |. In other words, a "line" is now defined by | and not a newline (\n)
s/\n//g; : this removes all newline characters
The print command is a bit more complex. It will print $_ (the current "line") then:
If the line number is a multiple of 9 it will print a newline. If it isn't, it will print nothing.
It is equivalent to this:
if ($. % 9 == 0){print "$_\n"} else { print "$_"}
 
1:30 PM
what is the comma after dollar slash supposed to do?
I could not resolve that
 
@user3146086 You mean dollar underscore?
print a,b,c will print a and b and c, separating them with the current value of $,.
For example: `perl -le '$,=":"; print "a","b","c"'
Will print a:b:c
And sorry, I meant $, and not $". Stupid default variables.
Anyway, I think that would probably have been considered off topic. I'm not sure oddly enough. Our scope is not 100% clear on perl programming. In general, if it a task that can interest *nix users/admins that's fine. Asking for a one-liner to be explained... I'm not sure.
@user3146086 does that make sense?
 
Immense thanks.
 
@user3146086 You're welcome.
 
1:59 PM
any ideas about how I can change it for files whose lines dont end with delimiter?
 
@user3146086 Now that's a question you can ask on the site. Be sure to include an example of your input file, the output you would like to see based on that example and the perl command you have tried.
I can't answer without seeing all those and those make it into a bona fide question, 100% on topic.
 
sorry my mistake, I meant to ask what does the comma after dollar underscore supposed to do? last question to you.. there is no space after underscore. And a space after comma.
that is the last question
 
@user3146086 The spaces are there for readability, they're not strictly needed. As for the comma, that's what I answered. print a,b,c means "print a b and c separated with whatever is the current value of $, ( a space by default)"
And print $a,$b,$c and print $a, $b, $c and print $a , $b , $c are all equivalent.
 
2:20 PM
Thanks again
 
@user3146086 You're welcome.
 
2:53 PM
@terdon: Thanks for the advice. I just now flagged the SF question for migration.
 
@TomRoche Great! No promises though, I have absolutely no experience with what you're describing but it sounds like a fringe case. I hope someone here can help you.
By the way, you could always just delete it and repost here if the mods are busy.
 
3:36 PM
@TomRoche you have to use VPN to ssh to your university computers?
Ugh, that looks nasty. My initial suggestion would be to try to simplify the setup. The OS on the compute cluster is Windows?
Amazing that there are people dumb enough to run clusters on Windows. The licensing itself must be quite something.
 
4:37 PM
@jasonwryan my bad, I was just informed about the specific term newfag and how it is, apparently, offensive. Thanks for pointing it out, dealing with it now.
 
5:06 PM
So, I was wondering if anyone can comment - are there any generally available phones (or classes of phones) for which free sw is available for syncing with Linux systems (I use Debian). I'm not sure this if this is a precise enough question for the site - I think it probably isn't. Would it be on-topic at Software Recommendations?
Doing a quick search suggests Android is the preferred platform for this, not surprisingly.
 
 
2 hours later…
Tim
7:37 PM
I am running following to recover a deleted text file on a 600GB partition with 190GB used:
sudo grep -a -B500 -A500 "Javaesque-style code" /dev/sda4 > /tmp/recovered
How long will that take roughly?
Similar to the time to backup 110GB files by rsync without writing out stdout?
 
@Tim Will that even work?
But yes, if it does work, it will take ages. grep needs to process the entire partition.
 
@Tim Why would grep pick up a deleted file?
 
@FaheemMitha I'm guessing it will actually read the disk's data but there's no reason to assume the file's contents will be in contiguous blocks.
 
@terdon I don't think it does that, actually.
i think it searches files. With proper inodes and stuff.
if it just randomly trawled disk data it would be pretty much useless.
 
@FaheemMitha That was my first thought but it probably can actually. Devices are files and the -a switch makes it treat binary as text.
@FaheemMitha Yeah, that's the problem. It might recover some bits and pieces of the file but I doubt it will get all of it and, in any case, it will be all mixed up with other files.
 
7:43 PM
@terdon Hmm. That assumes that when you are reading the device it dumps all the raw data, but I don't know if it actually does that. I kind of doubt it.
This is probably something one could find out with a little googling, I imagine.
 
shrugs
 
@Tim I recommend you start using distributed version control, if you are not already.
You'll still lose data, but only if you have not committed it.
And there are incremental commit schemes available that help cut down on the incidence of that.
 
@Tim I suggest you use a dedicated tool for this instead of grep. Something like extundelete
3
Q: Recovering files using extundelete

FallenI've accidentally deleted some files in my package managers's cache after an update (so I've deleted all packages that were downloade and unforunately they weren't even installed). Here's what I've done: Booted to a Ubuntu System that's on the same HDD as the Arch system. Now that I booted int...

 
@terdon Seconded.
They are having elections right now in Cross Validated. Not a site I pay much attention to, but was just reading the election related discussion there.
Apparently their volume has gone up sharply in recent years. Anyone tracking volume here? I don't see any big changes, but I haven't looked at any records.
 
@FaheemMitha Not much. We're growing steadily but nothing particular in recent months.
 
7:51 PM
@terdon Ok. I would actually expect any halfway reasonable statistics site to be very busy. Since statistics is about the broadest of the sciences, and applied everywhere.
 
@terdon that's general SE statistics.
India 16.9? Wow, really?
Well, I guess they are not counting Europe as one entity.
 
@FaheemMitha No it's not, that's us
 
@terdon ever spent any time on stats.sx?
@terdon only in the actual url. everywhere else is says stackexchange.com.
If they mean unix.sx their ui could use work.
 
Oh, so it does.
OK, hang on then
 
7:56 PM
Where do visitors go on stackexchange.com? -> unix.stackexchange.com 8.35%
Hmm. Higher than I would expect, actually.
There must be dozens of sites now.
 
@terdon thanks.
I don't see significant changes during the covered period.
A fair amount of what looks like seasonal fluctuation.
Ok, that does look like an upward trend.
Balls, I put in 120, but when I cut and pasted the link it gave 60
Over 180 weeks the trend is even more noticeable.
 
Yes, slow but steady growth. Good.
 
8:23 PM
@terdon you probably don't care about this question any more, but there are certainly dedicated packages for handling this. I use listings. mdframed is also a useful auxiliary.
4
Q: How can I show codeboxes?

terdonI am in the process of writing a simple manual. In it, I explain how to set up the files and programs necessary to run a pipeline I have written. I would like to include command line commands in that file in a way that makes them stand out and easy to read. Basically, how can I get this effect ...

Oh, I guess I use mdframed more for quote boxes, listings for commands. You can do
something like
\lstnewenvironment{cmd}[1][]{%
\lstset{basicstyle=\small, breaklines=true, breakatwhitespace=false, language=sh, backgroundcolor=\color{LightGray}, upquote=true, showstringspaces=false, #1}}
{}
 
@FaheemMitha Off-topic here; “what software can I use to synchronize <type of data> with <type of phone>” would be on-topic
What phone to choose would be off-topic on Software Recommendations, what phone OS to choose would be on-topic but asking about OSes is delicate because there are so many variables.
 
@Gilles here meaning unix.sx? how would one specify <type of data> and <type of phone>? Would Android be suitable for the latter?
 
What apps to use on <type of phone> and on Linux to do <…> that can synchronize their data would be a good question for SR
@FaheemMitha I'm not familiar with Android.SE but I doubt that they accept “what phone should I buy”. It's very difficult to give useful criteria.
What do you mean by synchronize anyway?
 
@Gilles No, that wasn't my question. I was asking about software to sync.
 
Synchronizing files is easy but if you have a file that you can't view/edit it doesn't help except as a backup
 
8:28 PM
@Gilles Update data on the computer, without deleting the existing data there. So I guess sync isn't quite the right term.
 
well, at least it's easy on a rooted phone
 
I didn't know there was an android site. There are so many it is hard to keep track.
 
in any case, if you want to keep your software options open and you want a non-ridiculous choice of hardware, get a phone that's sold with Android and install cyanogenmod on it.
 
If I'm thinking of posting on software rec, I'll ask in chat first. Otherwise it will probably be closed/deleted.
@Gilles Ok.
 
Or maybe something with Firefox OS but I don't know how rootable they are
 
8:30 PM
Will your average telephone company work with this cyano thingy?
I don't know how proprietary the protocols are. Would it work on any Android phone?
 
@FaheemMitha what OS(es) you run is independent of what phone company(ies) you connect to the network with
 
@Gilles ok
 
Tim
8:55 PM
@terdon. @Faheem: My firefox froze, cpu is not full but disk reading is the reason?

$ sudo grep -a -B500 -A500 "Javaesque-style code" /dev/sda4 > /tmp/recovered

^grep: memory exhausted
anyone tried ext3grep or extundelete before?
 
@Tim sounds like you're swapping a lot
could also be heavy file I/O
@Tim that's not going to be very useful
 
Tim
@Gilles: Yes, 6~7 GB out of my 8GB ram is used and 3GB of swap is used
 
you'll find a lot of false positives (previous versions of files, disparate chunks of files...)
recovering files is difficult, but you stand a better (less worse really) chance with specialized tools
 
Tim
@Gilles Does what you say also apply to ext3grep (also applies to ext4) and extundelete?
 
@Tim yes, you're more likely to get mileage out of them than out of plain grep
 
Tim
9:01 PM
do they all require me to make the partition offline?
that is my home partition.
 
@Tim TAKE YOUR PARTITION OFFLINE IMMEDIATELY
at least make it read-only
the longer you keep it mounted, the less chance you have of recovering anything
 
Tim
I guess some programs already write to my home for their hidden files. such as firefox
 
Make a copy of the partition and work on the copy
 
Tim
I probably will give it up. But this is another lesson
Would you make a partition for your own files, rather than putting them in home? @Gilles?
 
@Tim I often make /home a separate partition
 
Tim
9:06 PM
I meant given /home is a partition, would you put your self-created files in that partition, or another dedicated partition?
 
well, yes, my home is primarily for my own files
I tend to use other partitions for large amounts of data obtained from elsewhere
e.g. I do compilations in /usr/local/src
 
9:38 PM
"you have to use VPN to ssh to your university computers?"

Yes, but that's not my problem, since I'm not using my university's cluster (since that would cost me a *lot* more money). My problem is, I'm trying to ssh to a government cluster. That formerly required only one VPN :-) but now requires 2 VPNs (due to some new requirements) :-(
 
Tim
@FaheemMitha What is "distributed version control"? git? I use rsync for backup to external drive.
 
@FaheemMitha @FaheemMitha "The OS on the compute cluster is Windows?" Fortunately, no--I don't know of any scientific-computing organization that insane :-) But both the edu and gov clusters' admins only directly support Windows clients, and one of the VPNs which I am required to use (an F5 VPN) is barely supported on Linux.
 
@Tim git and mercurial.
@TomRoche File a wishlist bug for oher OSs. Even if it doesn't help you, maybe some day it will help someone else.
@Tim There are others, but nobody really uses them. E.g. monotone, darcs.
 
Tim
I though version control is only used for project? Isn't it too much overhead for backing up my large amount of files, compared to rsync?
 
@Tim you can use version control for any group of files, as long as the individual files aren't too large. If they are, there are workarounds.
In general, text files should be kept under version control if they are of any importance at all.
 
Tim
9:52 PM
what do you use for backing up many files in total 150GB size?
text files are scattered in directories with pdf, html files
 
@Tim you can selectively version control some files in a directory. You don't have to version control everything.
The important thing is to push the repos somewhere remote, so there is a backup if you accidentally nuke your hard drive.
Most of my important work, except, sometimes, for data, is in text files.
 
Tim
I have too many dirs with plain text files in them. The dirs are at different levels in file hierarchy. Shall I do version control for each dir respectively? That sounds a lot of work
 
@Tim that's doable. I don't know the context, but you can have one big repos or a number of smaller repositories. Whether you do it is, of course, up to you, but would you prefer to lose stuff?
 
Tim
no. I think I have to do backup more frequently.
 
10:49 PM
@FaheemMitha Thanks, yes, someone mentioned listings in the answers and I've been using that.
 
11:37 PM
@Tim - thanks, Tim, that's appreciated. Repute is, of course, measured only by the thoughts of others, and so if you think it, it must be true.
@Tim - if you're backing up a swathe of files/directories you might consider mksquashing them into a mountable image. Or else, you could create a btrfs partition on which to store them, and actively compress them w/ gzip or lzo. Both solutions would result in a mountable, compressed file-system, though the btrfs solution would be rw - while the squashed partition would be smaller.
 
1
Q: list all symbolic links to valid directories only with find

muffelI can use find /search/location -type l to list all symbolic links inside /search/location. How do I limit the output of find to symbolic links that refer to a valid directory, and exclude both, broken symbolic links and links to files?

7
Q: How do I check if a file is a symbolic link to a directory?

rubo77I can check, if a file exists and is a symbolic link with -L for file in *; do if [[ -L "$file" ]]; then echo "$file is a symlink"; else echo "$file is not a symlink"; fi done and if it is a directory with -d: for file in *; do if [[ -d "$file" ]]; then echo "$file is a directory"; el...

are these duplicates?
Hmm, no, one is testing a particular file, the other one is enumerating files. That's pretty different.
 
@Gilles - perhaps. The test, as you say, does mark a pointed difference, though.
Overall, I think, if an action were to be taken it should be a merge.
 

« first day (1627 days earlier)      last day (3335 days later) »