If try to revoke and vassal rebels, and then your current ruler dies while rebelion is still on, you put down rebelion with your new ruler and have vassal in prison. This way you don't have to bother with rebelion. If vassal is already in your prison for whatever reason (can be event if you spy on him f.e.), he must acceppt your revoke although you still get opinion penalty if you have no reason to revoke. And he must accept because he is in your prison. When you defeat him, he ends up in your jail and you have to do revoking again, but you don't get penalty this time because he rebeled.
To expand on Alien-47 answer: yes, you still get the opinion penalty for unsuccessful attempt.