Groomsman or groomsmen

man = one
men = as a group


if you're asking one person, it would be a groomsman.

If you are talking to one person, for example if your FH is asking his friend, he would say would you like to be my groomsman (singular). If he's talking to more than one person at a time, like the whole group he would say would you like to be my groomsmen (plural).

HTH!