Clean up when exactly we should save settings, and add in checks to make sure we do not do it incorrectly