Thermal Conductivity

Metal feels cold because it's an excellent conductor of heat. It pulls heat away from your skin quickly, making it feel cold even when it's the same temperature as the air.

Wood, plastic, and cloth are poor conductors, so they don't pull heat away as fast.