When you want to buy a house, you don't peruse blue prints of pipes and air ducts. Yet their location will determine where you can build the extra bathroom or a spare kitchen for the MIL apartment. A good security design will make it easy to add new features to your product. A bad one will require a full redesign/security inspection when all you want to do is add a new button or add extra servers for load balancing. And a really bad design will result in a flood of unexpected costs as you deal with a security breach.
If you want to deal as little with security as possible in your development lifecycle, then you need to design the security architecture right from the beginning.