Antipattern settings.py

Hello to Habrapitoners!

From time to time I come across development patterns that exist not because they solve a problem well, but because it is done in the popular X framework, therefore, many think that is good.

Now I want to complain about the “all settings are in settings.py” pattern. It is clear that he gained popularity thanks to Django. I’ve met every now and again in projects that are not tied to the same story in any way: a large code base, small, pretty components that are not connected to each other, and here you are: all together from arbitrary places crawl into the magic settings non-module for their constants.

So, why is such an approach disgusting in my opinion.

Cascading issues

In real-life projects, as a rule, you need at least three sets of settings: to run the project on localhost, to run unittest, and to turn everything on the battle servers. In this case, most of the settings usually coincide in all cases, and some differ.

For example, you use MongoDB as storage. In general, you need to connect to it on localhost and use a DB named my_project. However, to run unittest, you need to take a DB with a different name so as not to affect the combat data: let's say unittests. And in the case of production, you need to connect not to localhost, but to a very specific IP, to a server given under mongu.

So how, depending on external conditionssettings.MONGODB_ADDRESSfrom settings.py should take different values? Usually the course is voodoo-construction at the end of which consists of __import__, __dict__, vars(), try/except ImportError, which is trying to complete and close the namespace all the guts of another module kind settings_local.py.

What additionally needs to be loaded is _local.pyspecified either by hardcode or through an environment variable. In any case, in order for the same unittests to turn on their settings only at launch time, you have to dance with a tambourine and violate Zen of Python: Explicit is better than implicit.

In addition, such a solution is associated with another problem, described below.

Executable code

Storing settings as executable py-code is creepy. In fact, the whole pattern, apparently, initially appeared as a supposedly simple and elegant solution: “Why do we need some kind of CFG parsers, if you can do everything right on python? And there are more opportunities! ” In scenarios a little more complicated than trivial, the solution turns sideways. Consider, for example, such a snippet:

# settings.py
BASE_PATH = os.path.dirname(__file__)
PROJECT_HOSTNAME = 'localhost'
SOME_JOB_COMMAND = '%s/bin/do_job.py -H %s' % (BASE_PATH, PROJECT_HOSTNAME)
# settings_production.py
PROJECT_HOSTNAME = 'my-project.ru'

Do you understand what the problem is? That we blocked the value PROJECT_HOSTNAMEabsolutely on the drum for the final value SOME_JOB_COMMAND. We could grit our teeth SOME_JOB_COMMANDafter copying the definition after overlapping, but even this is not possible: BASE_PATHthen, in another module. Copy and paste it? Is it too much?

I'm not talking about the fact that executable code as a configuration can simply lead to hard to debug ImportErrorwhen you start the application in a new environment.

Therefore, I am sure that flies should be separate, cutlets separately: basic values ​​in a text file, calculated in a py-module.

High coupling

A good project is one that can be broken into small cubes, and put each cube on github as a full-fledged open-source project.

When everything is so, but with one BUT: "please be settings.py in the root of the project and that it had settings FOO_BAR, FOO_BAZand FOO_QUX" it looks somehow ridiculous, doesn't it? And when something sounds ridiculous, it usually means that there are situations in which this absurdity comes around.

In our case, the example does not force itself to be invented for a long time. Let our application work with the VKontakte API, and we have something like VKontakteProfileCachethat settings.VK_API_KEYand uses forehead settings.VK_API_SECRET. Well, he uses and uses it, and then again, and our project should start working immediately with several VKontakte applications. And all,VKontakteProfileCachedesigned so that it works with only one pair of credentials.

Therefore, it is more harmonious and more expedient to never access the settings module directly. Let all consumers accept the necessary settings through the constructor parameters, through the dependency injection framework, as you wish, but not directly. And let the very-lowest level like code in pull out specific settings if __name__ == '__main__'. And where did he get them from - his personal problems. With this approach, unit testing is also greatly simplified: with what settings you need to run, with those we create.

Possible Solution

So, the “settings.py” pattern I poured mud. I feel better, thanks. Now about a possible solution. I used a similar approach in several projects and find it convenient and devoid of the listed problems.

Settings are stored in text ini-style files. We use ConfigObj for parsing : it has richer features compared to the standard ConfigParser, in particular, it is very simple to make cascades with it.

In the project, we start a basic settings file default_settings.cfgwith all possible settings and their values ​​with a reasonable default.

We create the utils.config module with functions like configure_from_files(), configure_for_unittests()which return an object with settings for different situations. configure_from_files()organizes cascade search for files: default_settings.cfg, ~/.my-project.cfg,/etc/my-project.cfgand probably somewhere else. It all depends on the project.

The calculated settings are evaluated as the last step in the assembly of the configuration object.

The module itself is used only by launching processes or tests. All classes interested in settings get ready-made values ​​through injections, that is, they don’t communicate with settings directly. Actually, it’s not always convenient when it’s better to transfer the darkness to the configuration object as a whole, but this does not negate the fact that it needs to be transferred - no face-to-face contact.

Perhaps I wrote too much about such a "trifle" as settings. But if at least someone, after reading, thinks before blindly copying a far from perfect approach to something and does it better, easier, more fun, I will consider the mission of this post to be completed.