No, I've definitely had both crash. I kept using just urxvt so that I only lose one terminal window, but eventually found st to be much more stable, and better overall.
Interesting. You might want to enable core dumps, and the next time you get a crash of a program, load it up in `gdb` and at least get a stack trace. That probably won't give instant blame among a C program and libraries, but it might at least point in the right direction for why it's crashing.