There are a number of physical formats you can consider when buying silver:
As a general rule, you should look at buying legal tender silver coins wherever possible. This will almost certainly include less overhead and premium than other formats, and be closer to the true spot price of silver.
Purchasing rare silver coins should be avoided unless you really know what you're doing. Some subjective measures of value are easy to misjudge, and may result in you getting taken advantage of.
Silver owned through a commodities exchange, or "on paper" is no substitute for physical bars, coins, or jewelry. Exchanges are heavily leveraged, which means if everyone tried to "cash out" at once, there isn't enough physical metal to go around.
Moreover, precious metals are typically meant as a hedge against the complete collapse of the traditional banking infrastructure. If you wind up needing it, you'll want to have it close at hand!
It's important to comply with the law when it comes to buying silver, and knowing what they are goes a long way.
Under a certain dollar amount, there's no need to declare (which may be attractive to some), but once you hit a threshold you will need to (and should) declare your holdings.
Having liquid silver on-hand and ready in the event of a crisis is the best hedge. Being invested in silver ETFs is fine depending on your objectives, but if you're truly hedging against unforeseen market collapses, you want physical metal in your possession.