Yes they are removed automatically.
If you see people on 5, it's probably because the limit was previously on 5 and you've now changed it down to 3. If they post again, their next approval should take them out of the group.
You can resynchronise the members with the
Support Toolkit