Run a decent firewall that logs when someone is trying to get in, then you'll know. Kerio Personal Firewall has a "trainer" mode where it'll prompt you every time something happens, and it'll learn from your answers what to do next time. Logging's pretty good on Kerio too iirc; I stopped using it a while ago cos you can't use the free one on a PC that is connected to the internet with other PCs connecting through it.
These days I just use Windows Firewall; it seems good enough and works in my configuration.
Kerio blocks inbound and outbound connections so if you've got something nasty on your computer communicating with the outside world, Kerio will let you know about it. Windows firewall is inbound only; it doesn't block any outbound connections.