“Secured by math” means the cryptography is strong – but it says nothing about everything else you need to use bitcoins safely in practice. “Be your own bank” means you take on the job of providing all the security and technical knowledge that a regulated professional institution normally would.
The Bitcoin Wiki offers a page with step-by-step instructions on how to secure your personal Bitcoin wallet that would dismay even a typical IT professional,
let alone a casual computer user. You will need a security specialist’s understanding of the possible modes of attack on a modern operating system, how to encrypt all data securely and yet accessibly, password strength, backup procedures, how to securely erase a disk, the quirks of whatever Bitcoin wallet software you’re using …
This is why the vast majority of users store their bitcoins on an exchange like it’s an unregulated and uninsured savings bank, even though the exchanges’ security and reliability record is dismal. (Keeping your money in a sock under someone else’s bed.)