The earl (sic), it is said, made a contract with a Mr. Walter Taylor, a builder of Southampton, for the complete demolition of the Abbey; it being intended by Taylor to employ the materials in erecting a town house at Newport and other buildings. After making this agreement, however, Taylor dreamed that, as he was pulling down a particular window, one of the stones forming the arch fell upon him, and killed him. His dream impressed him so forcibly that he mentioned the circumstance to a friend, who is said to have been the father of the well-known Dr. Isaac Watts, and in some perplexity asked his advice. His friend thought it would be the safest course for him to have nothing to do with the affair, respecting which he had been so alarmingly forewarned, and endeavoured to persuade him to desist from his intention. Taylor, however, at last decided upon paying no attention to his dream, and accordingly began his operations for the pulling down of the building; in which he had not proceeded far, when, as he was assisting at the work, the arch of one of the windows, but not the one he had dreamed of (which was the east window still standing), fell upon his head and fractured his skull. It was thought at first that the wound would not prove mortal; but it was aggravated through the unskilfulness of the surgeon, and the man died.

– Browne Walters

The abbey is alleged to be home to two ghosts. The first is that of a monk who is seen as a white figure; the second is said to be a former abbot who appears as a dark figure.[59]

The story of the nun walled up in a small room recounted in Barham's Ingoldsby Legends was a creation of the author and has no basis in fact or genuine folklore, as the author himself admits with a smile in his notes to the poem, attributing his story to one James Harrison:

a youthful but intelligent cab driver of Southampton, who "well remembers to have heard his grandmother say that 'Somebody told her so'."

– Richard Harris Barham