When a guy said “It’s not my priority at the moment” – it means he thinks that there are more things that he should consider first before getting involved with someone. Maybe he wanted to be stable (financially, physically, emotionally and spiritually).