What happens if we disable "Everyone, all users" permission level using Set-SPOTenant command (https://technet.microsoft.com/EN-US/library/fp161390.aspx ) to the existing folders permission if they are already shared with Everyone / All users?
Thanks,
Abhilash