Even if you only use terminals, and you have to have a framebuffer that supports that native resolution of your screen, you'll find that Xorg is nearly always more performant.
Using tmux in an X terminal emulator really gets you the best of all worlds. Good mouse support, performant as you mention, and if X really starts to give you hassle for some reason you can always seamlessly continue working in a VT or with putty if you really want to.
Could someone elaborate how x11/xterm compares to iTerm2? I've been using tmux in iTerm. After reading this discussion, i installed x11 and quartz-wm. Why would I switch to X11 ?
(I do notice that within tmux on iterm, i can use the mouse for selection (using Alt), but the mouse does not work in tmux in xterm.)
Even if you only use terminals, and you have to have a framebuffer that supports that native resolution of your screen, you'll find that Xorg is nearly always more performant.
If you have issues with X11's security model that's a different story, I'd suggest trying http://wayland.freedesktop.org/