With FileVault disabled, someone with access to your computer can restart it into recoveryOS, typically used for Disk Utility or reinstalling macOS, and easily retrieve all your stored files.