Should I remove the battery of my Whisper Flex (Ultimate) when I am not using my fan?

You don't have to do this, but it is recommended. Your battery will last longer than if you always leave it in while it is plugged in.