I guess I found when the issue is triggered. It is simply related to the Windows scaling used for a monitor (integrated or discrete).
The issue exists in Windows 11 as well. I have tested the issue on my personal laptop (Win10 FHD) as well as a school laptop (Win11 23H2 4K).
Win10:
1. Laptop monitor: FHD @ <=150% scale + monitor 4k @ <=150% scale -> The issue does not happen.
2. Laptop monitor: FHD @ <=150% scale + monitor 4k @ >150% scale -> The issue happens only when the windows explorer in viewed in the discrete monitor.
3. Laptop monitor: FHD @ >150% scale + monitor 4k @ >150% scale -> The issue happens when the windows explorer in viewed in either monitors.
4. Laptop monitor: FHD @ 175% scale + monitor 4k @ <=150% scale -> The issue happens only when the windows explorer in viewed in the laptop monitor.
5. Laptop monitor: FHD @ <175% scale + monitor FHD @ <175% scale -> The issue does not happen.
6. Laptop monitor: FHD @ 100% scale + monitor FHD @ 175% scale -> The issue happens only when the windows explorer in viewed in the discrete monitor.
7. Laptop monitor: FHD @ 175% scale + monitor FHD @ 175% scale -> The issue happens when the windows explorer in viewed in either monitors.
8. Laptop monitor: FHD @ <175% scale -> The issue does not happen.
9. Laptop monitor: FHD @ 175% scale -> The issue happens.
Win11:
10. Laptop monitor: 4k @ 150% scale + monitor FHD @ 175% scale -> The issue happens only when the windows explorer in viewed in the discrete monitor.
11. Laptop monitor: 4k @ 250% scale + monitor FHD @ 175% scale -> The issue happens when the windows explorer in viewed in either monitors.
12. Laptop monitor: 4k @ <225% scale -> The issue does not happen.
13. Laptop monitor: 4k @ >=225% scale -> The issue happens.