It is entirely possible. From the IP address you can determine the latitude and longitude of the persons connection (really only valid if they have a static IP). I'll leave the how of that to you, but it's a very simple process. *cough ip-adress.com cough* And it doesn't just show the ISP location either.

From there, google maps will show you a satellite image of the location, and a quick phone book search for the persons name in that location will turn up their physical address.

Done and done.