This happens in life, too. When your counterparts lose continuity with your work, they turn to other work and the context-switch to return to working with you becomes increasingly hard.
I recently went through flat remodeling which took a month longer than it should and resulted in lower quality, because at the very beginning, I couldn't supply enough work for the contractors (we had to rush into it because ${life reasons}, so not everything was specced out when the work started), so the initial team of workers ended up taking better jobs abroad, and our general contractor had to find people to fill in.
Helps to be quick.