[laurenheartskull]

Fort Charles is that thing at the highest part of Port Royal.

And yeah, like those people said, be sure you are 'tracking' the quest you are on in your journal. If you are a yellow arrow will be on the compass along with a ray of light to guide you.