Well, I installed Emacs 21.1 from sources and it is working great now. I configured with --with-x=no and --prefix=/usr and all is well now. It behaves like I expect Emacs to. And I did all this without having to install any X libraries:).