Just an extreme idea. Whats about losing one level of notoriety over all to gain a skill point. In other words, here I am a level 38 and "D'OH" I didn't want my staff point on banish. I can take a Notoriety penalty dropping me back to a level 37 but alas I can move that skill point to a new place. Too extreme? It would certainly make someone think about moving a skill point when the penalty is so steep.
