@a20 Usually, when a user is removed, I lose a few points. Rarely, a user gets removed and my points go up -- an interesting indication that some users, at least, tend to be more negative than others. :)
My thought is, if you're downvoting, you should also comment to explain what improvements are needed. Otherwise you're not doing much useful.
@EiríkrÚtlendi well, I had noticed several times that I suddenly received down-votes within minutes on several of my answers, sometimes old ones, sometimes new ones, both simple answers and more complicated answers where I actually put in quite a bit of effort -- in total almost half of my contributions. So I already suspected that someone had some issue with me for whatever reason.
Anyway, one day I saw that my points had gone up significantly, and then I knew for sure what had happened...
I completely agree, just down-voting without providing any sort of feedback, it does not help anyone.
I also think it is important for people to be acknowledged. After all, we are all putting our free time in here without getting any sort of compensation. Points do not matter much in the end, but still, one wants to get that simple "thank you". Too much negativity and I think it will just make people stop putting in an effort.
Some people lose a lot of points because you get points voted whenever you answer by the long-term contributor of this site. I think he wanted to motivate questioner to come here.