It is time for you to pick your side wisely. You may become part of a triangle at your workplace where you have to choose a side. Make your choice with a logical explanation.
There may be other things affecting you personally, like your friendships or finances. However, do not let this have an impact on your relationship and talk to your partner about your problems.
You have chosen the right path. It may take some time for you to succeed and get all the fruit from your long term financial plans but it will be worth it.