when i remove my 454 from the car do i need to remove the starter. And i have hooker headers if that madders?

It will make it a little easier.

For sure, unhook the wires going to it after disconnecting the battery.

Do yourself a favor. While engine is out get a high torque mini starter and install in place of stock unit. Gives you more clearance and wont overheat from headers as many stock starters do.

Yes it can be done. No harder than the starter is to remove it would pay dividends over removing the engine with it in place.