The music stores nearby are a bit out of the way, for me, and selections are limited in each one, so if I want something in particular, I need to choose a specific store and plan for that journey.
I think you could make something work with the frequent maintenance items like strings and picks.
The problem would be shipping costs.
If a pack of strings costs about $6 in a store, I'm not likely going to want to pay another $4-5 to have them mailed to me.
If you can find a way to make that $6 pack of strings appear in my mailbox for $7, you've got a business, I think.
Same for the picks I use all the time. About $4 a dozen in the store. If they arrived in my mailbox for $5 a dozen, I think I'd likely go that route. If it was $9, probably not.
On a higher-end scale, most of the retailers seem to be chained to specific distribution houses, or product lines. For example, my local store chose Yorkville over Peavey, so they have Traynor amps and Yorkville mixers and speakers, but you can't find a Peavey Classic 30 anywhere in the store. Peavey, like Gibson, wants a shop to carry the whole product line - PA stuff, drums, etc - and that particular store already has the majority of that stuff covered through Yorkville. They don't carry EHX either, not sure of the reasoning there.
If you did some research into which lines the majority of stores are carrying, you might find some holes in the market that you could fill.
Pricing is a major concern, too. One of the bigger Canadian online shops is Axe Music, and I often check their site when I'm looking for a commodity item that doesn't need a play-test (a pedal I've already tried, or a Line 6 device, or similar). Their pricing is sometimes HIGHER than my local shop, though, and it would be a very rare day that I'd be ready to pay a higher price, plus shipping, to get something I can buy from a real live person, face-to-face.
One spot I'd stay away from is speakers, but that's just because there's a GREAT source for speakers online -
Q-Components loudspeakers & audio products
Friendly guys, very helpful, efficient, good selection, fair prices, quick shipping, in my opinion one of the best online experiences I've had. You might want to take a look at them as an example of how it ought to be done.