There are a number of physical formats you can consider when buying silver:
As a general rule, you should look at buying legal tender silver coins wherever possible. This will almost certainly include less overhead and premium than other formats, and be closer to the true spot price of silver.
Purchasing rare silver coins should be avoided unless you really know what you're doing. Some subjective measures of value are easy to misjudge, and may result in you getting taken advantage of.
Governments around the world have seized noble metals from time to time, so keeping your entire stash locally could be a bad idea depending on where you live.
Safe jurisdictions like Switzerland should be considered when you get to the point of holding a significant (tens of thousands of dollars worth generally) amount.
Physical precious metals like silver are a hedge to the traditional banking system. In a crisis, if the system collapses, it may very well take your property down with it (if it can).
If there is a run on the banks, having your silver under your direct control means they can't confiscate it, block access to it, or have loaned it out in a leveraged fashion.
Having liquid silver on-hand and ready in the event of a crisis is the best hedge. Being invested in silver ETFs is fine depending on your objectives, but if you're truly hedging against unforeseen market collapses, you want physical metal in your possession.