I've eliminated Fences as the culprit to this problem by checking on a different Windows 10 computer which has never had Fences installed.
To reproduce this issue: On a 3 monitor setup, right-click to refresh the desktop and the shortcut arrows will now be hidden on secondary monitors. Drag any shortcut to the primary monitor and suddenly the shortcut arrow will be visible for this individual shortcut.
I've hunted high and low for a solution, but it doesn't seems like an obvious Windows 10 setting that I'm missing. Hard to believe that it's a Windows 10 (Build 17134) bug, but it's possible.