Old ship design

I don't think you can delete old designs, but you can hide them. Use the "Mark as Obsolete" button on the old designs, and most list have an option to show only the non-obsolete designs. The design will stay around (although now hidden for most purposes) as long as there is at least one ship still using it, and possibly longer.

Due to the way SE:V handles things, an upgrade is always a new design. You can use the "upgrade" button in the ship designer to make a copy with all the components improved to the latest version, but this isn't always helpful. For instance, if you wanted to upgrade the weapons to your latest versions but not spend the resources and repair time to replace all the engines for a trivial improvement, you'll have to do that by hand.