Never mind, it took me 30m to find the setting. (Tools > Layout Inspector > "Enable embedded Layout Inspector")
The issue was that the "opt-out" button in the notification thats shown when first starting the layout inspector links to the wrong settings page ("Experimental", where it no longer exists)