Not just irc, almost any internet "aware" app can and most of the time does share your unmasked IP for anyone , including those that probbly know you had a server running , to see.
Might try some programs like zonealarm or other firewall / proxie type software , or even better yet hardware, but the bottom line is rember even though this is a game you are running a SERVER now, just that word allone will make script kiddies the world round flock to see anyway possible to get into your system , via the iis, icq,anything they can. It is part of your job as a "Server Administrator" to keep these unwanted people out.
Hope that gave ya some insight.
-Eagle Out