Hi noahkruger, yes there are a couple options. One is the method above, and it's the recommended one if you'd like to continue to use Fences 3. It'll ensure icons don't get placed behind a Fence in Win11 24H2.
If the above method isn't suitable, unfortunately Fences 3 is out of support, so you'll need to upgrade to v5 (trial is available so you can be sure it fixes it for you), which has compatibility updates with Win11 24H2 that prevent this from happening.