It is just because woolen clothes have fibres and between those fibres air is trapped which reduces heat loss. Air reduces heat loss because it is an insulator i.e. poor conductor of heat. Hence, all the heat from our body gets trapped inside the clothes which makes us feels warmer with the clothes.