There's not really much point, the monsters give free stats and drop some items, at zero adventure cost. The only downside would be if you could not beat them up, which I guess could understand but you should be either A) Ascending, so you don't want to waste adventures finding new blocks, or B)...