Braking systems depend on having a hydraulic fluid that does not compress and when brake fluid absorbs moisture, it does compress. When it compresses, little braking pressure is exerted on the brake discs/shoes. The brakes are said to fail.
Brake fluid that has absorbed moisture boils that moisture at a much lower temperature that fresh fluid that has no moisture in it. When your braking system heats up, particularly when towing, the moisture contained within the fluid vaporizes, and you have a compressible gas in your braking system. And your brakes initially feel quite spongy, then they just don't work. I have personal experience with this pulling a little 5th wheel trailer to a camp site near Mt St Helens, having my brakes 'boil' and loosing all braking from the truck. Fortunately, the trailer brakes were enough, but barely, to allow me to stop.
When you used DOT 4 fluid, you may have caused the problem to be somewhat worse than it was originally as DOT 4 fluid absorbs more moisture that DOT 3 fluid, given same operating conditions. While new DOT 4 fluid has a higher boiling point than new DOT 3 fluid, the downside is that you must completely change your DOT4 fluid more often than DOT 3 to eliminate the moisture from your braking system. Perhaps a 2 year change cycle would be appropriate for DOT 4 fluid.
Personally, I use only DOT 3 fluid, and at every oil change, I suction out the master cylinder and replace it with new, dry, brake fluid. Only takes a couple of minutes and helps to keep the moisture level low in the braking system.