This issue is caused by low internet connection. Try to use an ethernet cable if you are on Wi-Fi or try to use less background connection/downloads when playing Toribash.