1. Is it going to be a static HTML site or do you need a website builder or a CMS? If you only need a simple static website, any host will do the job just fine, so you can concentrate on the cheapest pricing. However, if you need a CMS or website builder solution, look at the best providers that include auto-installers or built-in website builders.
If you aim to have a web presence, you've got to have email. It's a convenient way for potential customers and clients to send you a message, Word document, or other files. Thankfully, most web hosts include email in the price of their hosting plans. Some web hosts offer unlimited email account creation (which is great for future growth), while others offer a finite amount. You, naturally, should want unlimited email.
A VPS runs its own copy of an operating system (OS), and customers may have superuser-level access to that operating system instance, so they can install almost any software that runs on that OS. For many purposes they are functionally equivalent to a dedicated physical server, and being software-defined, are able to be much more easily created and configured. They are priced much lower than an equivalent physical server. However, as they share the underlying physical hardware with other VPSes, performance may be lower, depending on the workload of any other executing virtual machines.