The issue was in the size of the firmware I was pushing.
Having both a large firmware on the device and updating it with another large firmware caused the device to run out of memory.
I solved the issue by pushing a small firmware update with only the ota library in it and then pushing the actual firmware update.