There seems to be a bug when you play against unranked players.

I will most likely reset the ranking from the beginning when I find it.