Endgame:

So now the system is running from the sata ssd drive, but the nvme ssd is plugged into the M.2->pcie adapter with the failed ssd now working again.

So I ask, what's the difference in speed between the nvme ssd  plugged into the pcie and the sata?

I run bonnie++ on the nvme and then:

Feb 24 16:59:11 nbecker8 kernel: nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0x10

Oh, look at that!  Just as I suspected, the nvme controller fails when heavily used.  The thing that surprised me is that I would have thought the nvme controller which just went down would have been on the pcie interface card.  But now it seems that the nvme controller is on the motherboard and even when the nvme is plugged into the pcie the same motherboard nvme controller is still being used.  That's the only explanation I can think of.