That's why you don't wear fins when you swim! lol.
Anyway, this is fairly common because you're swimming with fins. It's quite easy to cramp in the water especially if you're "giving it your all."
The trick abotu swimming is technique rather than muscle power. It's quite easy to do the wrong technique and tire yourself easy.
Usually with fins I do side strokes back and fort the pool. I used to get the cramps like you described. What I found out is that I gave too much effort. Just relax and swim without tiring yourself out.
If you ever go in a lake or ocean, be sure to wear a life vest in case this happens out in the middle.