Ahhh... Shame you were on Tortuga. When that happened to me on other islands, I was invincible. If you remember where the buildings are, you will get a lot of questions how you can walk through them (f the people can see you, you are not invincible). Also, by far, the easiest way into the shipwright.
Loot Hard!