Correct, but if you are not logged in you will see the features that the Anonymous user has.
That settings should work for all non logged in users. This won't affect users that already registered and logged in.
Setting the default language to French will make that the default language for anyone registering from that moment on. But it doesn't change the language of the current members (anonymous is also a member).
That seems to work! At actually looked at that place, but did not spot the form options.