When you add caster shims to the bottom of the beam, and it pulls away from the frame head, you VERY subtly turn the castor into camber. It's hugely subtle, but it is there.
What was once a degree of toe-in becomes an element of negative castor adjustment and effects the alignment minutely.
For me, I really like a LOT of extra caster for high speeds and even more negative camber for high . . . well . . . high speeds; especially in deep turns, like the sweeping on and off ramps. I like to sling shot into traffic and end up in the passing lane already at speed or greater and it takes a lot of negative camber to steady the car and allow for rapid acceleration through the ramp apex. A quick check of the side view and across four lanes to settle into the high speed groove.
Then the additional caster come into play, slows the steering a bit and removes all of the twitchiness.
SWEET.
So, anyway, yes. The caster, as it translates into camber, can effect the alignment, but for the better. I'd totally leave it alone. And drive WICKED fast. And if a cop stops you, hit him in the face with a shovel and tell him it's from me, would you?