Not sure it matters whether it's frontwards or backwards. If it's frontwards, the vacuum at the exh tip with the pressure at the air box should make it spin also.
Can it damage the charger? IMO, no. There is very little air volume at best. Air can't flow through unless there is a cyl at overlap. Overlap has the valves nearly shut, and overlap is < 30 deg x 4 = 120, per 360 deg, or at best a 1/3 chance. Now let's say you are unlucky and the engine stops at overlap. The air must go through the exh valve that are only open <.05" and then the intake, <.05" the through the intercooler, then compressor (which wants to turn the turbine the other direction) then the air filter.
OK, assuming that small amount of air could actually keep the turbo spinning. A turbo is a high powered device. It consumes over 50HP under load. And everytime you shut off your engine, it coasts for upwards of 2 minutes. It's not fragile. But should the oil film somehow go away, the turbo would just stop spinning. The air going through those little valve gaps could not apply enough torque to burn off the bearing material.