The armor damage isn't permanent - as long as the armor isn't explicitly destroyed (by a destroy spell or explosive trap, or a shatter spell if it's worth 20 points or less) the armor can still be refit by anyone with the blacksmith skill. This returns it to its original maximum value.
It doesn't have to hit the armor - you always take damage to your armor before taking body damage regardless of where you get hit.
The obvious exception to this is effects that bypass armor...not super common but something to be aware of. The plus is if something bypasses your armor it doesn't do any damage to it that must be repaired;D