Pay is still in yen, and pay can be very low. Great for when you're in Japan, not necessarily for saving up.
Every situation is different. Some BoEs are really helpful, some really couldn't care less.
Hours are long, and there can be many situations where you just have nothing to do.
For most places, only a max 20 to 25 vacation days available. Some places have less. And you need to use your vacation days as sick days.
You will have to work during the summer and winter holidays when there are no classes.
You'll be working alongside Japanese teachers of English. There is no guarantee that they will actually be able to speak English.