In 1929 she started her own dressmaking business run from her sitting room at home and was so successful that she was soon running a busy factory. She opened her first shop on Athol Mount in 1930 and a decade later opened another shop in Keighley Road in Ovenden. She was set apart from the rest by her hands-on training as a machinist, a passion for clothes design, an eye for making her own money and sheer determination.

Her home town of Halifax was at the forefront of machine engineering and had long been a thriving clothing industry, so it was the ideal location to launch the ‘Elsie Whiteley’ label from. The label specialised in stylish ladies blouses and became so popular that Elsie was producing from over six mills across Calderdale during it's peak and selling to 1000 retailers across the UK. It had it's own premises on London’s famous Regent Street, but also retailed from major department stores such as John Lewis, Fenwick and Selfridges.

In 1959 her two sons began helping her to manage her growing business and helped her to keep up with the ever changing labour market and fashion trends. Elsie herself never retired and was still altering garments from home right up until her death in 1972.