I have been on Tamoxifen for 11 months and, apart from odd 'senior moments', thinning hair and mild flushing, don't seem to be suffering too badly with SEs. However, as I declined chemo, I was also put on Zoladex injections into the stomach. I have mine every 12 weeks, but many people have them every 4. This switches off the production of oestrogen, so it should make me menopausal within a couple of years, then I will switch to an aromatase inhibitor instead of the Tamoxifen. As there are several AIs, it's easier to find one which causes least SEs.
If it is the blocking of the oestrogen that is giving you the SEs, obviously Zoladex will only make matters worse, but if it is the Tamoxifen itself, then Zoladex and AIs may be the way to go for you. It's certainly worth discussing your options with your BCN and onc. Although I strongly believe in quality of life over quantity, you don't want to put yourself in a position where you regret not taking prevenative medication.