Yes.
However, this situation will require more diligence on your behalf than if your spouse was a law biding, LTC licensed citizen. For example, if your spouse has their LTC, when in public places, there may be times when you could leave your firearm with your spouse and not have to worry about negative legal ramifications.
Concerning those with felony convictions, until certain time-frames have elapsed after a felony conviction (5 years after conviction and fulfillment of all encarceration, supervision, probation, etc,) one is not allowed to have posession of a firearm at all.
Once the 5 years has passed after fullfilment of all requirements pertaining to the conviction of a felony, then one may possess a firearm inside his / her residence only, but never again in a place that is not their residence.
As an LTC licensee, you are responsible for your firearms. The best way to make this work legally is to keep the firearm on your person or locked up AT ALL TIMES, never to be left in a place where the spouse could / would have access to it, or else you both might be breaking the law, depending on where the access to the firearm takes place and if the five year prohibition has passed.
This means, that if you are carrying concealed in your purse in a public place such as a restaurant, then you cannot leave your purse with your husband while you go to the restroom, or else you could be an accessory to a felon, as well as he would be a felon in possession of a firearm in public, which would be another felony conviction for him / her.