Sugar Ray Robinson did lose 19 times mostly at the end of his career, including his loss to Downes.
He did win 173 of his bouts though, so obviously this was a different era in boxing, and why SRR is considered maybe the greatest boxer of all time.
RIP Terry Downes