Typically, a web hosting service gives you the option of selecting either a traditional hard drive or a solid-state drive as your website's storage medium. Traditional hard drives have large capacities and lower prices, but they aren't quite as resilient as their SSD counterparts. Solid-state drives, on the other hand, are often faster and more reliable than HDDs, but they cost more and have smaller storage capacities. Unless you truly need blazing speed, a traditional hard drive should get the job done.
A VPS runs its own copy of an operating system (OS), and customers may have superuser-level access to that operating system instance, so they can install almost any software that runs on that OS. For many purposes they are functionally equivalent to a dedicated physical server, and being software-defined, are able to be much more easily created and configured. They are priced much lower than an equivalent physical server. However, as they share the underlying physical hardware with other VPSes, performance may be lower, depending on the workload of any other executing virtual machines.