Its the syncro's. Though now that you have driven it a long way since the problem first started, its likely to be that and a lot more as well too. Rev-matching gets you the gear as it 'removes' the job the syncro's do, matching the shaft spinning speed with the gear turning speed.....a-la a 'dog-box' (non-syncromesh g.box) in a race or rally car.
Drop the box and rebuild/swap in another.
Mitch.