It is definitely not every client that we get to work with for a long time. There are times when we have to let go due to reasons that are beyond our control as freelancers.
Some might be because of payment, some character, and other reasons.
So many more than I can remember. I've let go more than 20 clients over the years and it doesn't bother me at all. The most important thing is being able to get new clients from time to time.