Steam
Click on the + Neighbors button on the bottom game panel.
Once you click on the mentioned option, a new player’s Avatar will appear on the screen. This actually shows you that which Avatar you will pick for someone who you want to invite. As a player, you need to allow the system to decide the avatar for someone you are inviting.
Now, you only need to make a click on the button to send the invitation. This will be the final procedure that you have to follow for inviting someone to my secret neighbor without any kind of uncertainty.
Xbox, Gamepass
Click on the 
button on the bottom game panel.
Once you click on the mentioned option, a new player’s Avatar will appear on the screen. This actually shows you that which Avatar you will pick for someone who you want to invite. As a player, you need to allow the system to decide the avatar for someone you are inviting.
Now, you only need to make a click on the button to send the invitation. This will be the final procedure that you have to follow for inviting someone to my secret neighbor without any kind of uncertainty.
Alternatively, you can press Xbox button on your controller > open Friends section > find the user you want to invite > choose Invite.
PlayStation, iOS
There is no direct option to invite a friend. However, you can create a custom lobby and tell the friends you want to play with its name. They will be able to scan through the available lobbies and find yours 😄