There are several AE/BE differences here. Evidently
dispensing chemist BE = pharmacist AE
(= the person who is qualified to fill prescriptions and works in / manages the ...)
(= the department or counter where you order and pick up prescriptions, e.g., inside a hospital or clinic, or inside a larger store, whether a supermarket or a ...)
chemist's BE / drugstore AE
(= a store such as Boots (BE) or Walgreen's or CVS (AE), which originally existed in the first instance for medicines, both prescription and nonprescription; many were once locally owned, but now they're nearly all national chains that also sell a lot of other stuff)
The section inside a supermarket or drugstore that sells a similar range of nonprescription health products, however, is often called the pharmacy aisle or pharmacy section (AE), because it's usually located next to the actual pharmacy.
I agree that for household cleaning products, you would look instead at a supermarket aka (AE) grocery store, or at a big-box store like (AE) Wal-Mart or Target. The drugstore would probably have a few items, but not as big a selection, and not as good a price.
And I agree that 'apothecary' is not still in use; it's archaic.