@Akater Shell is a small swiss army tool. When you are doing something small(ish), it's useful. If you want to do something bigger, you probably can do it with the generic tool, but using a specialised one is going to make the work a whole lot easier.
Whenever I need to handle any data structure more complex than an array, shell is insufficient.
@WilliamHilsum Not answering this since your comment implies you found a solution, but what did you mean yesterday in the AU chat room about a scalable solution to ssh into multiple machines? What's wrong with ssh server?
Basically, we are demoing a tech that we built in house, currently we have about 20 units, but, shortly we will have hundreds, possibly thousands... They have automation scripts for most things, but, worst case scenario will require us to ssh in to diagnose
they have a built in 3g modem, and, I am looking at getting a custom APN that will terminate on our network... but, until then, I'm stuck with NAT, and no good way of accessing them
it is a really big problem that I need to get solved for demos (in fact one today!), as the APN solution will take months to get ready
@WilliamHilsum Ookay. I guess the fact that I only understood about half of that would suggest I'm not really qualified to help you. Is my answer here even remotely relevant?
at this point in time, as it is only 20 units, I have reverse SSH working at last... but, it requires me to manually set a port for each unit, and, not sure how much I trust it (complete encryption vs just user data is coming soon) and I just haven't built client devices like this that need to scale so rapidly...
It won't scale, but works for now... I just need to replace the pidof part, or, find someway of getting ssh to detect when the connection dies (Large event, thousands of people... 3g is very flakey)
@WilliamHilsum no time to dive into your problem now. If the problem is that you need to detect a dead connection when idle, reduce the keepalive interval
if you have a very flakey link and can make some security compromises, look at mosh
@Gilles were you talking about client or server? I've stuck autossh in to rc.local and it seems to connect on boot, but then doesn't reconnect... i think the session is kept open on the server or causing issues... I wonder if it is this now :/
@JennyD Well, hence the expression design flaw. :-) shell is the fundamental mean of interaction with OS / device in Unix, will always remain one by design, and alas — it's bad for almost any job.
@JennyD NB: I'm not really trying to prove anything. I joined unix.sx, looked if the chat is active, immediately noticed an interesting topic people usually don't discuss, couldn't resist dropping a comment. I don't expect people in Unix chat to be fond of UNIX-HATERS book, or something. My point is, people want to use shell as a universal tool for reason, and that's it.
@casey Exactly! Why should I write a simple data parser that's dependent on some big cluttered language that may not be installed at the moment of evaluation? A pretty rational argument, it seems.
@Akater No, it's great for launching commands and manipulating files. There are certain it things it does better than any language. It just isn't designed for and shouldn't be used to create complex programs.
But, as I said before, sometimes there simply is no other choice.
If you're writing data parsers in shell, and are not forced to because of a limited toolset, you deserve what you get.
However, "run this command and, if it succeeds, pass its output through this one or, if it fails, email me" is simpler in shell than in just about anything you care to name.
@Akater No, it can. It just isn't good at it. Why should it be? It's like complaining that your car is not very good at driving nails into a wall. It can do it of course, but you'd be crazy to try it.
The shell is a tool for specific jobs, you can't complain that it's not good at what it was never meant to be used for.
@casey oh right, 'awk', a very descriptive name. I didn't come here for an argument, really. :-) I don't want to waste anyone's time. Our positions are entirely clear to everyone and all interesting conclusions can be inferred.
@Akater it may not be descriptive, but it is well known. How many people can idenify coretools / perl /etc just based on the animal on the O'reilly book that covers it
@terdon In Lisp, names are usually long and descriptive, I think. They definitely are in my favourite quasi-Lisp, Mathematica (which most people would consider scripting langauge since there's no compiler)
Basically, you use the shell for what it's good for. And it is )by far_ the simplest and easiest way to do file manipulation and command launching.
@Akater So? As I said, awk is equivalent to Lisp, not to some lisp function. It is not part of the shell, it is a standalone scripting language.
And I won't take any flak from someone who thinks that Lisp's syntax makes sense! It allows hanging quotes ans parentheses FFS and that's just against the laws of nature! :P
There are different tools for different jobs. Nobody sane chooses to parse text in the shell. It is simply not designed for it. It can do it, but it is usually slow and painful.
The main point is that, just like any other tool, the shell excels at some things but not in others. I sure wouldn't want to use Lisp to, say, rename a file. I'm sure it can, but it won't be simpler than running mv from the shell.
@Akater No, functions are part of the language you are using. Either because you wrote them or because the language makes them available. System calls (which is what you're doing) use an external executable that is present on your system and have absolutely nothing to do with the language you are using to make them.
So, grep is not a function of the shell. It is an executable located at /bin/grep, written in C and compiled. You can call it from the shell, perl, lisp, C, Java, anything you like. It is not, however, part of those languages.
yep, the shell doesn't invoke grep as it would a function. it forks and uses its child to exec grep, which completely replaces the shell in the child, obliterating it. The parent shell only interacts with the child (now grep) by examining its return code and potentially its output to standard out or standard error
@Akater Yes, and that's a good thing. That's precisely what makes the shell great for this sort of thing. It allows you to call external executables in the same syntax as functions (which would be shell builtins like cd, and not mv which is also an external binary).
However, any gripes you may have about the output cannot be laid at the shell's feet.
@Akater how so? I have bash, tcsh and zsh, common lisp, elisp, gnu c,c++,fortran,go,objc,objc++, intel c,c++,fortran, llvm c++, julia, rust, R, python, perl, and plenty of others on this machine
You seem to expect that a single tool will do everything when the entire Unix philosophy is based on having multiple simple tools and chaining them together using redirections and pipes.
@WilliamHilsum So really your problem here is NAT, not SSH, right? Are they all behind the same NAT? Because if so, I'd suggest setting up some sort of port forwarding through the NAT to one server, and then use that one to jump on to the others. (That is, unless you can set up actual forwarding through the NAT.)
@terdon Look, I really feel bad about it. I didn't come to Unix chat to talk about how awful Unix philosophy is and how Worse is Better is not really better.
they only shell features in use are pipes, redirection and looping. All of the work is done by individual programs, ldd, awk, equery, grep, sort and uniq
@Akater you dont need to apologize, we are just curious how you justify such a statement
@terdon I respect GNU as a Human Rights project, a lot, but not Unix, on a technical level. I just tried to explain why you might want to stick to one tool, even if you're not a sysadmin
@Akater Yeah, seriously, don't worry. We're not about to take offense. If you don't like it, you don't like it. You're being perfectly polite about it. You just seem to be laboring under some misapprehensions.
@Akater There's no system in the world that has one tool for everything.
@Akater I quite understand why people want to use only one tool. And I also think that unless they are willing to learn to use more than one tool, they have nobody to blame but themself.
It's like, I have two hammers - one big and one small. An actual carpenter will have lots of different hammers, plus a whole lot of other tools, because they have more specialized needs than I do. If I were to try to build furniture instead of just banging in the occasional nail, I'd have to learn to use carpenter's tools instead of just the basic hammer.
@terdon Well, that's a classical juxtaposition: something simple vs something complete, Worse is Better vs The Right Thing. I don't think we'll tell something original to each other in that regard. Also, I have to return to configuring emacs. Thank you all for your time.
@Akater why do you blame unix for that? Historically the unix people did their own chips (AIX->power, Solaris->Sparc, Digital unix->Alpha, etc). Our current mass market of intel chips that most of the unixes run on now was historically tied to DOS and Windows
@Akater We might if you didn't come out with such loaded statements. "Worse is Better vs The Right Thing" indeed! So, "my way is better, your way is wrong"! Come on, you were doing so well up to that!
@JennyD Beauty is in the eye of the beholder, I guess. I understand that lisp is a brilliant language in many ways, all I know is that the syntax makes no sense to me at all.
@terdon The Right Thing is not my expression, it's cultural. It sounds arrogant, I guess. :-\ I only wish to remind The Right Thing guys are a (really small) minority.
@Akater Yes, but you are assuming you know what The Right Thing is. As far as I'm concerned, simple, lightweight tools that do one thing and do it great which can easily communicate with one another is The Right Thing.
@terdon i can handle the syntax, I am just not well versed in the grammar and certainly not in the huge berth of functions you have access to... and debugging elisp is painful
We've had this conversation before. I don't know Lisp, so I don't have strong opinions on the subject. All I know is that emacs-lisp seems to allow dangling quotes and parentheses and that just bugs me.
@FaheemMitha I'm not saying it's bad. By all accounts it is actually excellent. It's just not one of those languages that you can grok just by reading the code. I have no idea what lisp code is supposed to be doing. I don't know Python either, for example, but I can get an idea of what's going on.
And I just can't get my head around orphan quotes. That really gets to me :)
Nope. Why would I learn Lisp instead of the others? Python I need for work, Java I need because I want to learn a lower level language that I can write portable software in. Lisp and any other language is perforce, lower down in my priorities.
@terdon: IMO, you should learn Lisp, it gave you another way to look at problem, to programming. I'm trying myself with it by switching from vim to emacs :)
@eyoung100 Oh, OK, I saw the question. No, I don't really think that's very useful. EOL or not, the questions are welcome here.
@cuonglm Perhaps. Remember, though, I'm not a programmer or sysadmin. I'm a bioinformatician. If I learn a new language now it's going to be one that is common in my field and Lisp isn't. I might learn it at some point but not just now.
@terdon for distribution specific questions where the distribution is at EOL, ie Ubuntu 10,11,12 Debian 6 - Squeeze etc I was just about to show someone on Squeeze how to include a kernel driver for a kernel when it hit me that Squeeze is EOL
@eyoung100 I don't like forcing people to upghrade, personally. If they want to use the EOL, let them. Tagging as such doesn't really help though, in any case.
@eyoung100 Looks fine to me. Your comments are always civil :)
And I quite agree that upgrading is the way to go but that's often not feasible for whatever reason. A tag specifically for eol seems pointless. Who would search for it? Why? Also, it's a classic meta-tag. It doesn't tell us anything by itself (eol what? Ubuntu? Debian? Some other software?).
@eyoung100 Don't worry, I already removed it. And no, I'd rather you didn't :)
Feel free to bring it up on meta if you feel the tags add value though. You've been around for a while and know how the site works, that I have a different opinion doesn't mean I'm right.
@cuonglm and @FaheemMitha heh, I seem to live up to the stereotype:
> This is easier said than done. Hardly anyone wants to listen to Lisp programmers. Perl folks have a deep suspicion of Lisp, as demonstrated by Larry Wall's famous remark that Lisp has all the visual appeal of oatmeal with fingernail clippings mixed in.
@WilliamHilsum both, I think, since you're doing a reverse tunnel
alternatively, you could try setting up a VPN
this introduces overhead, so you'll waste some bandwidth and latency, but it can make connection management immensely easier
have the device behind a NAT launch a VPN. Now it's no longer behind a NAT, it's on your network
network goes down and back up? restart the VPN, your existing inner connections keep working
@terdon you should learn a functional language (Lisp, Scheme, ML, F#, Haskell, Clojure, …) because many problems are best solved by using a function as a first-class value, and the other languages you mention are terrible at that (they make you believe that everything has to be in a class)
the point of learning lisp is not to program in lisp, it's to write lisp programs in $language syntax
@terdon did you mix versions of svn? it's pretty sensitive about that
@Braiam thats why I posted the question , it doesn't, see realtek.com.tw/downloads/… which is why I will answer if he says he wont upgrade but its the principle
@Gilles got one unit working and the other isn't reestablishing correctly... And yep, VPN is long term strategy, overhead isn't really an issue... only transmitting a few meg a day... but, it requires so much planning - and not really sure where to begin
@JennyD Sorry, missed your message earlier... Nope, all different units all with their own 3g cards - the APN uses CGNAT, so, not really a lot I can do... and the problem sort of is with SSH, but more my fault for not knowing the correct commands - e.g. I made more progress with autossh, but, I don't think it fully reconnects/key file error... I just need a lot of trial and error