It doesn't download new client versions, it's mod files. It does a full check of the mods it has & the ones on the server, then it downloads any it doesn't have.

If your net's slow, or the server's busy it takes longer to do thus longer slowdown during.