@CaptainBohemian Your initial question was 'can On one hand -not- have a corresponding 'on the other hand'?' My snap answer is 'No, that would be crazy to leave it out.' But snap answers are too fast. Maybe there's some example somewhere where it works out OK.
Instead of being so theoretical, it works better with actual examples. Do you have a full couple sentences, one starting with On the one hand' and another where you could have 'on the other hand'(but possibly do something else)?
@CaptainBohemian The phrase "on the one hand" may be viewed as merely setting up a dichotomy, and is often used to set up a counter-argument. It doesn't have to be followed by "on the other hand" and often that is deemed to be stylistically fussy, even obsessive. My sentence is an illustration of how the one can be used without the other.
@Cerberus The fact that it is related to the context is perfectly fine to me. I'm not sure if there are situations when using some adjectives (related to counting) may sound strange, no matter the context.