If you want to access Safe Mode, power down your phone completely — hold the power button, then select Power off from the menu. Once the phone is off, press and hold the power button to turn the phone on, and as soon as the "Samsung Note 4" logo pops on screen, release the power button and then press and hold the volume down button until the phone boots up completely.

The phone will clearly display a "Safe Mode" badge in a black box on the bottom-left corner of the screen (as seen below), which is visible no matter what screen or app you're in. You'll also see a notification arrive telling you that your phone is in Safe Mode.

To turn off safe mode, drag down the status bar, and tap the Safe mode enabled status.