By Brigid Riley and Kevin Buckland

TOKYO (Reuters) - A day after loosening its grip on long-term interest rates, the Bank of Japan intervened in the government bond market to rein in a jump in yields to fresh decade highs, reminding the market that it should avoid moving too fast.

The 10-year Japanese government bond yield rose 2 basis points (bps) to 0.970% on Wednesday, a level last seen in May 2013, before retreating immediately after the BOJ announced an emergency bond-purchase operation. It stood at 0.955% as of 0605 GMT.

"They've said okay, let's let the market find a new equilibrium - but let's remind the market that about the upper bound, we can intervene," said Claudio Irigoyen, global head of economics at BofA Global Research.

Japan's central bank on Tuesday took another small step away from its decade-long commitment to ultra-easy stimulus by changing the 1% ceiling for the 10-year yield to a reference point rather than a hard cap.

It also removed a pledge to defend the level with offers to buy unlimited amount of bonds, nodding to market forces that have continued to push yields up in line with global moves and domestic inflationary pressures.

There's "a continued sense of caution in the market that we're moving in the direction of policy normalisation," said Keisuke Tsuruta, fixed income strategist at Mitsubishi UFJ (NYSE:MUFG) Morgan Stanley Securities.

While the 10-year yield's rise was halted by the BOJ's intervention, other parts of the curve continued to climb.

The five-year yield reached 0.485% after the announcement, a level not seen since April 2011.

The 20-year JGB yield touched 1.745% for the first time since July 2013, and the 30-year yield reached 1.91%, a level last seen in May 2013.

The two-year JGB had not traded yet following the intervention, but the yield ticked up to 0.160% earlier in the day for the first time since July 2011.

Yield curve controls are "simplified but effectively dead," said James Malcolm, UBS currency strategist based in London.

"The positive spin is that less overt control should help market function recover."