You'll find that the engine is off center and twisted/cocked pretty good. If you were to place a straight edge on the fan and compare it to the radiator support, you'll see that the passenger side has a bigger gap than the driver's side. Also from top to bottom, there's more of a gap at the top.
Looking at the original radiator brackets, you'll see that the passenger side is wider than the driver's side in order to try and square the radiator to the fan. Also the radiator is off center. Even the tunnel in the car is off center. Just look at the amount of room the left seat has compared to the right side. Three inches off center at the crank bolt from one frame rail to the other!
Your right header will probably only have a 1/4" clearance to the shock tower if that much. Ma did this off set to make enough room for the steering box. When making your own exhaust system, leave your tape measure in the tool box. Both halfs will be different.
What you are describing sounds correct. You don't notice it as much with a small block or even a 361/383 sitting in there but drop in the big block and it really stands out.
I can't believe you're actually working on it again!