Noticed this and figured I would point it out.

Windows 11 (22H2) + Start11 1.25.  I have the Start menu set to "Windows 10" style, not sure if that matters.  I also have the Start menu animations turned off.

In "Start button" settings, turn "Use a custom start button image" OFF to have the standard Windows 11 Start button appear.  Click the Start button and the Start11-powered Windows 10 style Start menu appears.  Click the Start button again, while the Start menu is still open, and it disappears.  (I would argue this is the appropriate behavior.)

In "Start button" settings, turn "Use a custom start button image" ON to have a custom Start button appear.  It behaves a little differently with this setting.  Click the Start button and the Start11-powered Windows 10 style Start menu appears.  Click the Start button again and the Start menu does not disappear, it sort of blinks and comes right back.  To get the Start menu to disappear, you have to click somewhere else, like an empty space on the taskbar.

(This isn't consistently reproducible but it seems that repeated clicks on the Start button cause the menu to stay open more often than disappear.)