It depends on how tightly the atoms are packed and how joined-toegther they are. Atoms in the air are lightly packed and not joined-together - so we can walk through them. Atoms in walls are tightly packed and joined-together - we can't walk through them.

This is do to with atomic effects. When two atoms come close together (they don't infact need to touch at all) they are repelled because of the charges (two electrons coming close together repel and the same for two protons). This force (called the Electromagnetic force) increases dramatically the closer they get together (there is a point where they start to attract but I won't go into that). Now you are very dense and so is the wall so as you touch it some atoms won't come close together and repel, but others will. These atoms are linked together and so the ones that are repelled also move the ones that aren't. This is what pushes your hand away stopping it passing through a wall. Since atoms orbit the nucleus at very long radii (in the atoms frame of reference), the ElectroMangetic force will be from two electrons coming close together rather than nuclei. From this, there is a small probability you could pass through a wall and never have two atoms get close enough to repel one another, but it is so infintesimally small (for the example in question) you might as well look for ice skating mongooses than that to actually occur.