Reversals and payments which have been issued to an invoice cannot be deleted. Issued payments can be reversed however.
Under the Payments heading on the left select the payment you wish to delete
Under the same Payments heading click the icon
On the confirmation dialog that appears select Yes