I was honestly tthinking the same thing, I wanted to changed the drums to something kinda LoFi, but I ended up getting lazy, next time when I think I notice something that is right, ill be sure to try it though, I for sure had that feeling as well when that part of the song comes in at 2:24. That's kinda my only problem with the track too.
Thanks for the feedback though!