I think I’ve found the problem with looper drift. If the length of the loop is a repeating number then the QC must round it which can make it a different length than the drum track. For example, the length of a QC loop with a tempo of 66 and a quantize of 4 is 60/66*4 = 3.63636363636363…. So, QC likely rounds it down which ends up being shorter than the drum track.
Tempos that work for me with a quantize of 4 are 50, 60, 64, 75, 80, 96, 100, 120, 125, 128 and 150. I’ve never had drift with these tempos. It does depend on the quantize you’re using. For example, a tempo of 72 doesn’t work for quantize of 4 (3.3333…) but does work for 3 (2.5).