Make sure your trigger finger only touches the trigger when shooting. Your finger or gloved-finger dragging against the trigger guard or stock moves the rifle a slight amount when pulling the trigger. If you wear a glove, make sure it fits tightly.
With my index finger only on the trigger, a bolt-action CZ 527 6.5 mm Grendel shot several five-shot groups at 100 yds., about 10-percent tighter than groups shot with my finger dragging on the stock. The stock-drag groups had a definite horizontal spread.