How To Log Messages From Different Threads To Different Files?

I have a Driver.py scripts where it calls multiple threads based on the given inputs. Threads are basically runs a module of a selected object. So Driver.py may call thread_1.run()

Solution 1:

Yes, you can direct log entries from different threads to different files. You'll need to:

• Create a log filter that can filter records by their LogRecord.thread or LogRecord.threadName attribute
• Create a filter that does not accept records with specific or all thread ids.
• Create a log handler per thread, giving it a log filter that only accepts logrecords for their specific thread.
• Attach the filter that ignores log records for your threads to any other handlers.

When filtering, you have the choice between filtering on thread id (the value returned by threading.get_ident()) or thread name (whatever you passed in as the name argument to the Thread() object). If you have a pattern for your thread names, this is where you'd use it.

Creating a custom filter is easy enough:

import threading
from logging import Filter

classThreadFilter(Filter):
    """Only accept log records from a specific thread or thread name"""def__init__(self, threadid=None, threadname=None):
        if threadid isNoneand threadname isNone:
            raise ValueError("Must set at a threadid and/or threadname to filter on")
        self._threadid = threadid
        self._threadname = threadname

    deffilter(self, record):
        if self._threadid isnotNoneand record.thread != self._threadid:
            returnFalseif self._threadname isnotNoneand record.threadName != self._threadname:
            returnFalsereturnTrueclassIgnoreThreadsFilter(Filter):
    """Only accepts log records that originated from the main thread"""def__init__(self):
        self._main_thread_id = threading.main_thread().ident

    deffilter(self, record):
        return record.thread == self._main_thread_id

If you want to match a specific pattern, adjust the code accordingly.