You say the noise is there with or without the shaft but quieter with the shaft out, I'm wondering if the shaft was just transferring and/or amplifying the sound. The front shaft runs directly below the drivers seat so it could make the noise more noticeable. Could be an issue anywhere in the driveline (rear end, transfer case, etc.) and transmitting up through. Put the front shaft back in and remove the rear one. It would tell you if the noise is from the transfer case forward.
If I had to place a guess, I'd say something in the rear locker let go when you hit the pothole and now the sound is transmitting through the driveline. Be interesting if you pull the rear cover and see any pieces.