How To Mix Logging Handlers (file & Timed) And Compress Log In The Same Config File?

I need to prepare a logging config file in json which rotate it by time, size and compress de file rotated for all modules in my app (I'm stuck now). I want to do it using a sigle

Solution 1:

This is to be expected. If you add a handler this handler either wont produce messages (due to filtering) and so nothing will change or it will write those messages (thus duplicating them).

The TimedRotatingFileHandler and the RotatingFileHandler only support, respectively, rotating by time and by size. Not both at the same time.

AFAIK there is no built-in way to achieve what you want, so using only the configuration file you wont be able to achieve what you want, you have to code something to put together the functionality.

Consider reading how to create a new rotating handler in the Logging Cookbook. If you save this handler class in a file mypackage.myrotatinghandler you can then specify:

class: 'mypackage.myrotatinghandler.MyHandler'

in the configuration file.

Note that, to add compression when saving a file it is sufficient to set the rotator attribute of the rotating handler with a function that saves a compressed file. Taken from the link above:

defnamer(name):
    return name + ".gz"defrotator(source, dest):
    withopen(source, "rb") as sf:
        data = sf.read()
        compressed = zlib.compress(data, 9)
        withopen(dest, "wb") as df:
            df.write(compressed)
    os.remove(source)

rh = logging.handlers.RotatingFileHandler(...)
rh.rotator = rotator
rh.namer = namer

The rh handler will behave as a normal RotatingFileHandler but also compress the logs.

However setting the conditions to do a rotation require that you re-write parts of the handler. You may read the sources for the logging.handlers module to see how the built-in handlers are implemented.