Get Governor's Seal
 

Where on earth is the Governor's Mansion and how do I get his seal?

the best way to do it is go through the town and there's a certain exit where you can go to it, or a little north of the dock behind a house you can get to it

Any landmarks? I cannot find it LOL.

On Port Royal, stand with your back in front of the half-ship. Now look straight ahead and to the right, there's a small house/shop with a man in front, he's the Medicine Man. Now look a bit to the left and straight ahead, that path takes you to the mansion.