Hi there, sorry for the super late reply. Turns out it is because of a pushable tile entities mod, I think it is Quark. (You can disable other features if you don't like them, Quark is very configurable)
Sorry for the reaaaalllly late reply, hope you use this in the future!