I suggest doing a lot of reading here:
http://www.theturboforums.com/smf/index.php
There are pages, and pages, and pages of debate and experience over what material to use and what lasts and what doesn't.
Heavy gauge (schedule 10, .109" wall) mild steel will hold up if coated and schedule 10 mild weld el's are cheap. Lighter gauge stuff will work OK too but won't last as long.
As far as stainless, 304 or 316 will work but you have to leave room for the pipes to move, stainless has a much higher CTE than mild steel. If the headers are short runner then things will probably be OK, but with longer runners and their proportionally greater growth under temperature you're best using double-slip collectors to allow the pipes to move around. 321 is the best grade of stainless to use but much more expensive-- i.e. a single 321 1.75" J-bend is about $75-80, compared to a mild Weld El at about $3, a 304L weld El at $5, and a 316L weld L at about $8.
Myself, I'm building my turbo headers from schedule 10 316 Weld El's. They're going to be unequal length and close to the block so I think they'll be OK. 316L stainless is rated at a slightly higher working temp than 304L. 316L isn't as good as 321, but far, far cheaper. If they crack I won't have much money tied up in them and I'll rebuild them out of 321 at that point.
If I was building long equal length headers, I'd use 321 for sure and use double-slip collectors from SPD to allow the longer runs of piping to move around under the heat load.
Another guy I know built his equal length turbo headers out of mild steel and had them coated with the black super high temp jet hot, they're holding up fine so far although the coating has significantly discolored.
With the high heat load seen from the exhaust ports to the turbos, you can't cut corners. Get your material fitted well, get it clean, and backpurge when you weld-- especially with the stainless.
Downpipes (from the turbo out the back) can be either stainless or mild. I'm going to use 304 downpipes to the bellhousing area and then switch over to standard aluminized steel.