@thomasotto asked me to copy my response to his bug report on this thread. Here it is, though it looks like @PeterO already expressed a similar idea:
"I don’t work for Neural DSP, but I think I can make an educated guess on why we are observing this behavior.
It has been confirmed that Rows 3 & 4 are handled by a different DSP core. Based on your measurements, that second DSP is processing the output of the first DSP in buffers of 128 samples. Given the sample rate of 48 kHz, we’re getting:
128/48000 = 0.00266(6), which precisely matches the additional 2.67 ms of latency you’re measuring.
Some additional latency is inevitable whenever buffering is involved, but it is a mystery to me why that buffer has to be that large considering that Rows 1 & 2 clearly work with a smaller buffer (otherwise it wouldn’t be possible to get the 1.75 ms number you’re quoting, and what the other people have measured on simple presets utilizing only Rows 1 & 2)."