@NikeDattani To me, flags are for pointing out issues with questions/answers/comments/chat that can/should/need to be addressed by a mod as part of their role as a moderator. I would consider how I (or the other mods) vote as mostly outside the scope of the our mod role. As a user, I think voting is important for developing the site and cultivating a culture of voting is important, but I see that as distinct from my "job" as a mod.
@NikeDattani I understand that you do this with the best of intentions and I can't necessarily speak for other user's opinion, but the idea of tracking the individual votes of users makes me a bit uneasy. While this can be possible due to the smaller size of our site and the relative infrequency of voting (particularly downvoting), I believe the intention of SE is to have only aggregate voting be public and individual votes private (unless the user chooses to say how they voted).
