This is a case where an easy calculation really means a tedious but straightforward calculation that I don't want to write down, and not an obvious and simple calculation. There is loads of cancellation that is occurring.
Examine the first few terms
$$f(x) - f(t) - f'(t)(x - t) - f''(t)(x-t)^2/2.$$
Differentiating with respect to $t$, this becomes$$ [0] - [f'(t)] - [f''(t)(x-t) - f'(t)] - [f'''(t)(x-t)^2/2 - f''(t)(x-t)].$$I have written brackets around the derivatives of each term in the previous expression, to better see where each term comes from.
The first term is $0$. Note that the $-f'(t)$ in the second term cancels with the $f'(t)$ in the third term. Similarly, the $f''(t)(x-t)$ cancels with the $f''(t)(x-t)$ in the fourth term.
You could prove that this happens more generally with either an inductive argument or with careful use of precise summation notation. The only terms that don't cancel with other terms are the first (which is $0$ on its own) and the last (which is the term Bartle claims).
