Sensitive data held in long-term storage should always be encrypted.
Hold the key separately so it cannot be stolen along with the data. You
could keep it on a USB stick in a safe (with a secure backup copy held
offsite) and change it regularly.
Data actually being processed will need to be decrypted and held as
plaintext. Ensure that any such data, and the key, is explicitly
overwritten before you release the memory. You may be limited in what you
can do by your operating system. For example, a secure OS will wipe any
memory images swapped to disc, which an ordinary OS will not.
Some languages provide secure storage classes for holding things like
keys. Read the documentation to see what is there already.