We're looking for the cardinality of the set containing the subsets of the power set of $A=\{ x_1, x_2, \dots, x_m \}$ with one or less elements, so we're essentially just looking for the cardinality of the set $\{ \{ \{ x_1 \} \}, \{ \{ x_2 \} \}, \{ \{ x_3 \} \}, \dots, \{ \{ x_m \} \} \}$, right?
Your explanation does make sense, so thank you! I'm just trying to make sense of my own more explicit argument. ^_^