@mikeserv will you please get over this obsession of yours that I have an agenda? What is it even supposed to be? I'm really fed up with waking up to comments whining about some kind of perceived agenda you think I have.

What, do you think I have anything against sed? I love sed! I think it's a great tool and wish I knew it as well as you do.

What the hell is my agenda supposed to be?

Of course sed isn't designed to work on fields! Come on! Of course it can be made to as it can be made to do just about anything, all I said there is that it is not designed for fields in the way that awk so obviously is.

To make sed understand fields, you need to build a regex that lets it do so. Awk and perl will automatically split on whitespace and give you direct access to the fields.

Anyway, none of this is remotely relevant. I have repeatedly asked you to post an answer showing the field capability of sed. Yet you refuse.

I'm sure that in the hands of an expert such as yourself it can work wonders, why do you insist on putting this in my answer instead of posting your own?

Quite frankly, it looks to me like you have an agenda against me and my answers and I have no idea why you're so obsessed about it.

On a separate note, jesus man, how may times do we have to ask you not to post so many comments!? They raise automatic flags and just create work for everybody. Why don't you use the chat instead? That's what it's there for.

sed is designed to work on whatever the fuck you want it to. It's a tool that does as its told. If you want it to work on fields - whatever that means - then thats what its designed to work on. You shouldn't post stuff like that - why spread ignorance like that? I can't get over it - it's appalling.

And post so many comments? Every comment was a reply.

If you want sed to split whitespace, you just tell it to: s/[^ ][^ ]*/new value/g

I don't like chat either.

@mikeserv What is it you object to so much? I honestly don't know. I said "use a tool that is designed for fields, for example perl or awk or use regular expressions"

You seem to think I'm referring to sed exclusively.

@mikeserv I don't care if you like it or not. You're abusing the comments, stop it.

@mikeserv No it wasn't. In my original question, you kept coming back and posting another rant every 10 minutes or so. I'd stopped replying for ages.

The entire point of that post was to highlight that there are two basic approaches to dealing with field-based data: dedicated tools like awk or perl and more versatile things that use regexes for which I used sed as an example.

If you have a way to use sed to manipulate fields without regular expressions I'd love to see it. \

Just post it on your own answer. I don't understand why that's so hard for you to grok. Comments are not for extended discussions. 9 times out of 10, when we get an automatic flag about too many comments, it's because of you. "I don't like chat" is not good enough, that's no excuse to abuse the system dammit!

Anyway, I never said nor do I believe that sed is not capable of dealing with fields. Hell, I even chose to use it as an example of regex-based field dealings.

All I said is that sed, unlike awk, is not designed to work on fields. That is, very simply, a fact. As you very well know, sed is designed to work on streams, not fields. It has no knowledge or concept of fields. Of course it can be made to grok them but only because it is so versatile and can use regexes.

@mikeserv You left 7 comments to my 4. Do the math. Plus, you started the whole thing by flinging unfounded accusations and pure paranoia. Hell, I haven't cast a single close vote in favor of that post nor was I planning to. Jesus!

So, unless you can come up with actual proof (or even the slightest evidence) all you are doing is pure slander and I don't see why you expect me to stand for it.

''@terdon - awk doesn't provide a way to do that either - have you looked at its source code? you want me to prove what exaCtly?

@terdon - left them where?

@mikeserv What? A way to do what? Deal with fields? And what does the source have to do with it? Of course it uses regexes in the background, the point is that the user doesn't need to.

@mikeserv On the answer you so object to.

In any case, whatever you do, you will never convince me that whatever brilliant esoteric sed thing you come up with to add 3 to the 5th field of every line will be simpler than `awk '{$5+=3}1;'. So yeah, sorry, but there are tools that are designed to work on on fields and tools that can be made to.

Oh, yeah, i did. A couple of those were to tell you how to fix the stuff in the answer - to which you did not reply, but still acted in reply.

It may not be more simple - but it usually is faster and more efficient.

@mikeserv Yes, I stopped replying because I was hoping you might eventually shut up or post an answer. You didn't.

@mikeserv Now that is both wrong and ignorant.

Time it and come back to me. I have to go.

The problem w/ a tool that understands fields is that every single line has to be split - from to finish.

Time it and come back to me. Add 3 to every 5th field. You're the one who keeps slinging accusations, you prove it.

You must read the entire thing - every time.

No - i don't even know what the fuck that means!

@mikeserv Write me a sed script that adds 3 to the value of the, say, 112th field of a line. Time it on a large file and show me that it's a) possible, b) easier than awk '{$112+=3}1; and c) faster than that awk. You're the one making claims here, not me, so you can bloody well prove them. If not, stop bugging me about this.

Here's an example. I don't know if this is the most efficient way possible in sed but it's the best I got and seems to be the same as what you had added to my answer. First, let's create a file with 100,000 lines each of which has 200 fields (the numbers from 1 to 200):

perl -e '$x=join " ",1..200; print "$x\n" x 100000' > file

Now, try replacing the 112th field on each line with foo:

$ time awk '{$112="foo"}1;' file > /dev/null

real	0m1.626s
user	0m1.568s
sys	0m0.056s

$ time perl -lane '$F[111]="foo"; print "@F"' file > /dev/null

real	0m11.115s
user	0m11.052s
sys	0m0.052s

OK, Perl is not too good here (though it still beats sed) precisely because we have to split the whole line but awk which is specifically designed for this type of thing is way faster than both perl and sed:

$ time sed 's/[^ ]*/foo/112' file > /dev/null

real	0m16.903s
user	0m16.848s
sys	0m0.040s

I guess you might find me a sed version that actually manages to beat the perl one but I very much doubt you'll be able to do that for the awk one. Because, surprisingly enough, awk is specifically designed to deal with field-based data.

@mikeserv ^^

I would also like to see how you get sed to add 3 to the value of the 112th field instead of replacing it. I have no idea if it's even possible and as far as I know it's not but hey,. the other two can do it because they know about fields:

$ time awk '{$112+=3}1;' file > /dev/null

real	0m1.350s
user	0m1.316s
sys	0m0.032s

$ time perl -lane '$F[111]+=3; print "@F"' file > /dev/null

real	0m9.951s
user	0m9.916s
sys	0m0.032s

I have no clue how to do this in sed. I would have to use external programs which defeats the purpose.

And here, since you were so sure of yourself you wanted to add this one to my answer, compare perl, awk and sed when truncating at the 14th field (copied from your edit verbatim). I actually had to stop sed after the first 10 minutes since it was taking too long. The other 2 took seconds:

 time perl -lane 'print "@F[0..13]"' file > /dev/null

real	0m10.341s
user	0m10.288s
sys	0m0.028s

$ time awk 'NF=14' file > /dev/null

real	0m0.696s
user	0m0.660s
sys	0m0.032s

$ time sed 's/\([^:]*:\)\{14\}/\1/' file > /dev/null
^C

real	10m14.100s
user	10m12.836s
sys	0m0.532s

So, @mikeserv enough of your agenda with sed, get your facts straight and stop molesting me with your obsession. Deal?