The later books punished you rather harshly for having it. I think it was intentional so that people with more "powered up" characters (i.e. had been playing since book 1) were given a bigger challenge.