Is there a limited staff positions?
I don't see it that having a few more members there that slack from time to time as an inconvenient, as long as they do shit from time to time. The more trustworthy people there are the better imo.
Kicking should be in case of death or toxic behaviour, power abuse or stucf like that. If things are not good hire one or two more. If people go afk for long periods and it is a recurring think then kick em. Afterall this is a volunteer work.