These are different parts of the same issue.
In the ACP Settings page, you may specify that the poll author may specify whether that particular poll will show votes or not. And if you choose to show this option, you may as well specify the default value. This is controlled together with the "Can create polls" forum permission. If this is not provided, NO author will be able to specify that the voters are public. So no regular user will be able to see the votes.
On top of this, there are THREE different (related) permissions that have to do with viewing poll votes.
One is the immediate complement to the above: the "Can see poll voters" forum permission controls which users will be able to see the voters IN A POLL WHOSE AUTHOR HAS SPECIFIED THAT VOTERS ARE PUBLIC. Even if the author does make the voters public, if the user does not have the forum permission, the voters will not be shown. And even if the user has the permission to see poll voters, if the poll author did not specify this option, the votes will not be visible.
Finally, there are other TWO moderation permissions. Well, really it is one permission, but applied either to GLOBAL scope (for Global Moderators, affects ALL forums) or at LOCAL scope (for Forum Moderators, affects specific forums). With this permission "Can view poll voters" (yes, almost identical in name to the previous one), moderators are able to see the poll voters REGARDLESS of the choice specified in the poll creation by the author and the forum permission. So, a Global Moderator, with the "Can view poll voters" permission, will be able to see the voters in ALL polls, regardless of the author's configuration. Local Moderators can do the same, limited to the forums they moderate (if they have this permission).
Hope this helps and clarifies... It is a bit complex, but the feature is important, and must be protected adequately for privacy.