You can edit or delete a PM when it's still in outbox. If you delete at that point it's deleted for both you and recipient. Since a notification may have been sent if I recall correctly there will be message to recipient it was deleted.
Once the recipient opens either their inbox or the message itself it's considered sent and goes into sentbox. Deleting it from there only deletes it from your list. It's not fully deleted until both the recipient and and sender delete it.
You can answer questions like this yourself pretty easily. Create a test account as regular user and send it a PM, delete it.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”
Attributed - Thomas Edison