https://www.youtube.com/shorts/6gJ_NO5BL0g apologies the embed YT video didn't work for me, it never does

you can put a bit of oil or grease in the nylon bush and seal and in the case to lubricate the parts, you can polish the shaft (say with a green washing up scotch pad or fine emery) where it passes through the bush and seal if it is very rusty
I suspect the 3rd 4th fork is the issue, it may appear to be in neutral but may be very slightly out and is putting pressure on the cage at the point I am pointing at in this pic

When fully in the hammer (to the right of my finger tip) is in the middle of the cut out in the 3rd 4th fork and can move the selector fork when the shaft is rotated and the cage slides through the selector fork when the shaft is pushed in to the other 2 planes so the hammer either interacts with the relay lever or the reverse selector fork to engage those gears
I suspect that if you try and engage 3rd or 4th gears by moving their selector fork that you may not be able to engage either gear as easily as you can 1st and 2nd, or if you do that you can not get back into neutral
If this is the case then I would diagnose a broken 3rd4th synchroniser hub that has cammed and is preventing the slider part of the synchroniser moving easily and preventing the fork being relaxed in neutral and thus rubbing on the cage of the selector shaft preventing free movement