I am wondering of the best way to remove the mini icon menu that sits either just under the header or just to the left of the content.

Are these menus options in the Menu Builder or do I need to remove them in the code?

It's called "button menu", and it can be removed (or relocated) simply by modifying the button menu section in the page header under skin_wrappers.php.