You have to pick what you're going to be worried about. Markets are volatile, but retirement is certain.
Long-term investment success is almost totally a function of how one emotionally handles declines in the equity market, as opposed to how one's portfolio handles them.