Border travel must be noted in the rental agreement for insurance cover to apply. Should you spontaneously decide to do so during the rental period, please call the rental company and discuss what to do next.
From experience, you can also go to any rental company location in your area and pay the fee for border travel there. You are then insured abroad.