Then he comes from a shitty world or has terrible friends and allies.

In my world, you don't leave a nakama behind for anything. We fought the whole damn world to try to get back just one of our own even though it was impossible, even though we couldn't do it. And there's not a single one of us who wouldn't do it all over again for the slightest chance, or just to make that stand.

A pirate crew who won't risk that to protect one another isn't strong enough to make it to the top. So they're not even a threat.

[He's not sure that's a pretty picture, but he is extremely convicted about it.]