When changing mod settings in-game, it doesn't actually visually retain the change until you quit the menu all the way, meaning to the game world. The setting is actually saved and takes effect, but it doesn't show up as changed until you fully re-open that menu, which can be confusing. Here's a video that shows the problem.
[0.17.78] Mod settings not retained until menu is fully closed
Re: [0.17.78] Mod settings not retained until menu is fully closed
Thanks for the report. That's currently how it works due to single player pausing the game while you're in the menu.
If you want to get ahold of me I'm almost always on Discord.