Hey people. Sorry for disturbing again.

A small doubt regarding the use of la and l'

When talking about an apple: We use la pomme

But, when an orange is being referred to: We use l'orange

Why is it so? Is it true that la is used for feminine, like "la femme", and l'- for men like l'homme?