You’ve had your mortgage for a long time.
The amortization chart shows that the proportion of your payment that is credited to the principal of your loan increases each year, while the proportion credited to the interest decreases each year. In the later years of your mortgage, more of your payment applies to principal and helps build equity. By refinancing late in your mortgage, you will restart the amortization process, and most of your monthly payment will be credited to paying interest again and not to building equity.
Your current mortgage has a prepayment penalty
A prepayment penalty is a fee that lenders might charge if you pay off your mortgage loan early, including for refinancing. If you are refinancing with the same lender, ask whether the prepayment penalty can be waived. You should carefully consider the costs of any prepayment penalty against the savings you expect to gain from refinancing. Paying a prepayment penalty will increase the time it will take to break even, when you account for the costs of the refinance and the monthly savings you expect to gain.
You plan to move from your home in the next few years.
The monthly savings gained from lower monthly payments may not exceed the costs of refinancing if you are planning to move in the near future.