Identify your "problem" window. From inside the RV, unscrew the window. Have an assistant outside hold the window, making sure it does not fall out.
Remove the window. Use a putty knife to remove all old sealant. Pull on the old weatherstripping to release the adhesive; remove the weatherstripping.
Measure the window frame to determine how much new weatherstripping you'll need. Cut an appropriately sized piece with a knife or scissors.
Attach the weatherstripping where the old weatherstripping used to be.
Use a caulking gun to apply fresh sealant around the window frame. Put the window in place and screw it back in.