Yes. Deleting a player will delete all their progress including any charms retrieved for your Charm Bracelet.  

To delete a player, tap on their name.

Once the name is highlighted, tap on the red button with the garbage can to delete their name and game.