Wheelers Bay was originally called Hudsons Bay and its local name was Tatey Bay. The name changed when Charles and Thomas Wheeler came to the bay in 1842. There was much competition between the Wheelers and the Spencers of Ventnor vying with each other to secure business. In 1868 the Wheelers helped to demolish Devonshire Terrace before it was relocated further back from the cliff edge and in 1885 a John Wheeler lived at High Port. The Wheelers helped to supply Niton when the village was cut off by snow in 1881 and Wheeler Bros ferried pilots to and from vessels in the Channel.
Wheelers Bay Road was earlier named Gas House Hill, dating from the time when the Gas Works were built on the site, and coal and other merchandise were unloaded there.
Records show fresh herrings selling 13 for one shilling in the early 1900s. Local children were discouraged from bathing at Ventnor, and sent round to Wheelers Bay to use the beach there; modern sea defence work has filled in the beach area.