Maybe… SSH being blocked is an interesting side-effect, almost if suddenly interrupts aren't being serviced.
Does the machine respond to pings during the "blockage"?
The hypothesis: some low-level component (interrupt controller, DMA, PCI… etc) is getting tickled the wrong way by X11 and thus, things seize up because interrupts aren't getting processed anymore.
Another machine will certainly be a way to find your feet with NetBSD, then with more knowledge, you can return to this one.
