Find php.ini, probably in your Windows directory. Open it in an editor like NOTEPAD, and find the "extensions" section. Uncomment the line for MySQL and/or whatever other databases and features you want (GD and GZIP are handy, too). Save the file.
If you're running PHP as a CGI program, you should be able to access MySQL now; otherwise, stop and restart the web server.
Oh, I forgot - make sure the DLLs listed where you uncomment the extensions are somewhere that Windows can find them. They're probably in your PHPx\extensions directory; just copy them to the same place php.exe is.