A micro-step is one forward/loss/backward pass for a micro batch. One optimizer step updates weights after accumulation completes.

Effective batch = micro batch × accumulation × GPU count. For OOM, reduce micro batch or sequence length; increase accumulation if the target effective batch must be preserved.

First thought

Accumulation 8 means the weights update eight times.

Correction

Eight backward passes accumulate gradients, then one optimizer step updates weights once.

Decision rule

Find the largest safe micro batch first; record step counts as optimizer steps.

Epochs, Steps, and Gradient Accumulation