Can I prevent friends from joining my world or kick players?

Friends can only join your world through an invite.

There is also a feature to kick other players from your session.