It shouldn't be, doesn't phpbb dynamically load most classes as they're needed? In which case, only the classes required for authentication should be loaded?
If you did want to rewrite it I think you'd need to pull a lot of the code from auth and session classes, just not sure it'd be worth all the effort and potential security risks to gain a slight performance improvement. Maybe someone else can give a bit more info, but personally I wouldn't waste my time rewriting core phpbb code
