The hair shaft is the part of the hair which extends from the pore at the top of the skin surface and sits in the open air.
The hair follicle is an invagination of the epidermis that extends from the skin surface to the dermis or hypodermis.
The hair bulb is the deepest part of the hair follicle and helps to form the hair follicle by a dermal cell mound called papilla.
The arrector pili muscle (spelled wrong in the first figure below) is the muscle which allows the hair to "stand straight up". This allows the sebaceous gland to squeeze some sebum through and help coat the hair to make it more glossy.

See the figures below to help