Pulling things out into a configuration file (or environment variable) is valuable when either (a) the value needs to be set per-environment and you have a variable number of environments or (b) the value needs to be changeable by somebody other than the dev team.
Also it's worth considering for constants whether what you actually want is a 'resource file' (which could easily just be a file containing all your constants written in your programming language, no need to use a different syntax if that doesn't gain you anything) - learning and internalising (and also learning how to explain to other developers) that a resource file is -not- the same as a configuration file has been a significant quality of life improvement for me.
Also it's worth considering for constants whether what you actually want is a 'resource file' (which could easily just be a file containing all your constants written in your programming language, no need to use a different syntax if that doesn't gain you anything) - learning and internalising (and also learning how to explain to other developers) that a resource file is -not- the same as a configuration file has been a significant quality of life improvement for me.