Yes of course we should be concerned. We each have to read up about it and all the side effects and make an informed decision, taking all this into account. The degree to which people experience the side effects varies considerably so don't assume you will get all of them.
While you're waiting for some replies to your post I wondered if you'd seen our fact sheet on Tamoxifen. I've attached a link:
You might also find it helpful to talk this over with one of our Helpline staff. They are open from 9-5 on weekdays and 10-2 on Saturdays. The number is 0808 800 6000
Very best wishes