It usually only provides a significant performance increase for higher order functions, because the lambdas passed to the function can be inlined as well.
It is not just for performance, though. When writing custom control structures using higher order functions, inlining can be great as it opens for non-local jumps. This is why all the functional collection operations are inlined higher order functions.