sim: Move destructor of Port to public

sim: Move destructor of Port to public

To preventing from instantiating an abstract class, hiding its
constructor is enough. Moving destructor to public doesn't break this
intention. This also makes us can use smart pointer to manage derived
Port class.

Change-Id: Ic9cf97e90a6c26108d359eb459df48cd23eaf15c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/23925
Reviewed-by: Gabe Black <gabeblack@google.com>
Maintainer: Gabe Black <gabeblack@google.com>
Tested-by: kokoro <noreply+kokoro@google.com>