The answer to your question depends on if you and your spouse are co-borrowers on both mortgages where the homes are located. In short, spouses usually cannot get a mortgage for their own primary residence unless they are the sole borrower on the loan.
If you and your spouse apply for a mortgage as co-borrowers and the home is not the primary residence for one of the borrowers, the lender includes the total monthly housing expense -- including your mortgage payment, property tax and homeowners insurance -- for the applicant's primary residence in the loan application. This means the applicants need to earn enough money to qualify for two mortgages instead of one, which makes it more challenging to qualify for the loan.
If you and your spouse apply for the loan as co-borrowers and you both move into the home, then the home is considered the primary residence for both applicants. If both of you move out of that home and want to buy a new home and again apply for a mortgage as co-borrowers, the lender is likely to consider that property the primary residence for both of you. In this scenario, your prior home would become your second or vacation home but the terms of that mortgage do not change.
Additionally, property location is a factor. If one of the homes is located in a different county then you could qualify for an owner occupied mortgage on a second or vacation home, even if you both do not plan to use the property as your primary residence. Although this new home is not classified as your primary residence, the mortgage rate should be lower than if the loan is classified as non-owner occupied.
If the home you want to buy is located in the same county as your existing home and both of you do not intend to live in the property, then you would apply for a non-owner occupied mortgage. In short, non-owner occupied mortgage terms tend to be less favorable. Specifically, you may be required to pay a higher mortgage rate or make a higher down payment, which may make it more challenging to qualify for the loan.
Please note that the scenarios we reviewed above reflect general mortgage industry guidelines but you may be able to find a lender that applies their own policies and makes an exception in your case. Your question is pretty unique so we recommend that you contact multiple lenders to understand how they would handle your situation.
"B2-2-04, Guarantors, Co-Signers, or Non-Occupant Borrowers on the Subject Transaction." Selling Guide: Fannie Mae Single Family. Fannie Mae, June 5 2018. Web.
"B2-1.1-01, Occupancy Types." Selling Guide: Fannie Mae Single Family. Fannie Mae, May 1 2019. Web.