I recently refinanced my house. The loan is in my name only, but was told my husband had to be on the deed...

when I originally purchased the house, I was told the same thing when I signed the loan papers, but when I divorced my 1st husband, the judge told me that if he wasn't on the loan, he didn't have to be on the deed. Now the broker is telling me that yes he does. Seems to me that if I'm the only one responsible for the house payments, I should be the only name on the deed. Who is correct?