Any idea why my buttons are so huge, I expected them to match the icon size kinda, but they seem to be taking as much room as possible

I think you need

IconButton

Yeh that works, it's still feels 'huge' though, but maybe that's just the preview

The buttons include some padding so you can try making a custom button with just

Icon

and

Surface

if you want to reduce the padding