There is a reason to be concerned that balance could be off, and for racing purposes, it's best to check balance as a pair.
However, replacing wheels and covers is VERY common, and some do it at the race track. In my case, I've swapped compressor wheels with no ill effect. I would not do that with a customer turbo though.