详解Python logging调用Logger.info方法的处理过程

本次分析一下Logger.info的流程

1. Logger.info源码:

 def info(self, msg, *args, **kwargs):
  """
  Log 'msg % args' with severity 'INFO'.

  To pass exception information, use the keyword argument exc_info with
  a true value, e.g.

  logger.info("Houston, we have a %s", "interesting problem", exc_info=1)
  """
  if self.isEnabledFor(INFO):
   self._log(INFO, msg, args, **kwargs)

注释中反应了可以通过 msg和不定参数args来进行日志的格式化。
真实的调用为:_log方法:

2. Logger._log方法:

 def _log(self, level, msg, args, exc_info=None, extra=None, stack_info=False):
  """
  Low-level logging routine which creates a LogRecord and then calls
  all the handlers of this logger to handle the record.
  """
  sinfo = None
  if _srcfile:
   #IronPython doesn't track Python frames, so findCaller raises an
   #exception on some versions of IronPython. We trap it here so that
   #IronPython can use logging.
   try:
    fn, lno, func, sinfo = self.findCaller(stack_info)
   except ValueError: # pragma: no cover
    fn, lno, func = "(unknown file)", 0, "(unknown function)"
  else: # pragma: no cover
   fn, lno, func = "(unknown file)", 0, "(unknown function)"
  if exc_info:
   if isinstance(exc_info, BaseException):
    exc_info = (type(exc_info), exc_info, exc_info.__traceback__)
   elif not isinstance(exc_info, tuple):
    exc_info = sys.exc_info()
  record = self.makeRecord(self.name, level, fn, lno, msg, args,
         exc_info, func, extra, sinfo)
  self.handle(record)

最后两行:

生成日志记录:

record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info, func, extra, sinfo)

处理日志记录

self.handle(record)

2 生成日志记录:

 def makeRecord(self, name, level, fn, lno, msg, args, exc_info,
     func=None, extra=None, sinfo=None):
  """
  A factory method which can be overridden in subclasses to create
  specialized LogRecords.
  """
  rv = _logRecordFactory(name, level, fn, lno, msg, args, exc_info, func,
        sinfo)
  if extra is not None:
   for key in extra:
    if (key in ["message", "asctime"]) or (key in rv.__dict__):
     raise KeyError("Attempt to overwrite %r in LogRecord" % key)
    rv.__dict__[key] = extra[key]
  return rv

调用_logRecordFactory初始化一个日志记录实例,_logRecordFactory 其实就是LogRecord类,初始化时,可能包含logger的name, level、调用的函数、行号、日志字符串、模板参数、堆栈信息等。

再看extra信息,extra到底有何用?现在从代码中可以看到,只是更新到生成的日志记录实例的__dict__中去.猜测:肯定会在生成最终的日志字符串的时候会用到。继续往下看。

3 处理日志记录self.handle(record):

Logger继承自Filterer,

 def handle(self, record):
  """
  Call the handlers for the specified record.

  This method is used for unpickled records received from a socket, as
  well as those created locally. Logger-level filtering is applied.
  """
  if (not self.disabled) and self.filter(record):
   self.callHandlers(record)

3.1 if语句中有一self.filter(record)的判断,看函数名,是来筛选是否要继续处理消息的,其核心源码如下:

 def filter(self, record):
  """
  Determine if a record is loggable by consulting all the filters.

  The default is to allow the record to be logged; any filter can veto
  this and the record is then dropped. Returns a zero value if a record
  is to be dropped, else non-zero.

  .. versionchanged:: 3.2

   Allow filters to be just callables.
  """
  rv = True
  for f in self.filters:
   if hasattr(f, 'filter'):
    result = f.filter(record)
   else:
    result = f(record) # assume callable - will raise if not
   if not result:
    rv = False
    break
  return rv

可以看到, 如果在handler中的filter中如果有返回为False或空,则会屏蔽对应的record,返回True或部位空的值,则会将record放行。那么我们就可以自定义自己的filter。

3.2 让Logger中所有的handles去处理record:

def callHandlers(self, record):
  """
  Pass a record to all relevant handlers.

  Loop through all handlers for this logger and its parents in the
  logger hierarchy. If no handler was found, output a one-off error
  message to sys.stderr. Stop searching up the hierarchy whenever a
  logger with the "propagate" attribute set to zero is found - that
  will be the last logger whose handlers are called.
  """
  c = self
  found = 0
  while c:
   for hdlr in c.handlers:
    found = found + 1
    if record.levelno >= hdlr.level:
     hdlr.handle(record)
   if not c.propagate:
    c = None #break out
   else:
    c = c.parent
  if (found == 0):
   if lastResort:
    if record.levelno >= lastResort.level:
     lastResort.handle(record)
   elif raiseExceptions and not self.manager.emittedNoHandlerWarning:
    sys.stderr.write("No handlers could be found for logger"
         " \"%s\"\n" % self.name)
    self.manager.emittedNoHandlerWarning = True

代码中会去循环调用当前logger的所有handlers去处理record,for循环部分,之后,如果当前的logger的propagate的值为False或空,则不向logger的父logger传递,即向上传递。

4. Handler 中的 handler(record) 部分:

def handle(self, record):
  """
  Conditionally emit the specified logging record.

  Emission depends on filters which may have been added to the handler.
  Wrap the actual emission of the record with acquisition/release of
  the I/O thread lock. Returns whether the filter passed the record for
  emission.
  """
  rv = self.filter(record)
  if rv:
   self.acquire()
   try:
    self.emit(record)
   finally:
    self.release()
  return rv

可以看到, Handler在处理record时, 会去加锁,然后调用self.emit(record)方法去处理。

4.1 emit(record)

def emit(self, record):
  """
  Do whatever it takes to actually log the specified logging record.

  This version is intended to be implemented by subclasses and so
  raises a NotImplementedError.
  """
  raise NotImplementedError('emit must be implemented '
         'by Handler subclasses')

看到需要由子类去实现,以StreamHandler为例子:

def emit(self, record):
  """
  Emit a record.

  If a formatter is specified, it is used to format the record.
  The record is then written to the stream with a trailing newline. If
  exception information is present, it is formatted using
  traceback.print_exception and appended to the stream. If the stream
  has an 'encoding' attribute, it is used to determine how to do the
  output to the stream.
  """
  try:
   msg = self.format(record)
   stream = self.stream
   stream.write(msg)
   stream.write(self.terminator)
   self.flush()
  except Exception:
   self.handleError(record)

4.2 Handler.format(record):

 def format(self, record):
  """
  Format the specified record.

  If a formatter is set, use it. Otherwise, use the default formatter
  for the module.
  """
  if self.formatter:
   fmt = self.formatter
  else:
   fmt = _defaultFormatter
  return fmt.format(record)

如果handler有自定义的formatter就用自定义的,如果没有则用默认的Formatter的实例, 初始化元源码为:

 def __init__(self, fmt=None, datefmt=None, style='%'):
  """
  Initialize the formatter with specified format strings.

  Initialize the formatter either with the specified format string, or a
  default as described above. Allow for specialized date formatting with
  the optional datefmt argument (if omitted, you get the ISO8601 format).

  Use a style parameter of '%', '{' or '$' to specify that you want to
  use one of %-formatting, :meth:`str.format` (``{}``) formatting or
  :class:`string.Template` formatting in your format string.

  .. versionchanged:: 3.2
   Added the ``style`` parameter.
  """
  if style not in _STYLES:
   raise ValueError('Style must be one of: %s' % ','.join(
        _STYLES.keys()))
  self._style = _STYLES[style][0](fmt)
  self._fmt = self._style._fmt
  self.datefmt = datefmt

有三个参数:

  • fmt: 格式化模板
  • datefmt: 时间格式化参数
  • style: 日志格式化的样式。

style有三种:

_STYLES = {
 '%': (PercentStyle, BASIC_FORMAT),
 '{': (StrFormatStyle, '{levelname}:{name}:{message}'),
 '$': (StringTemplateStyle, '${levelname}:${name}:${message}'),

可以看出对应到:% 操作符的格式化, format方法的格式化以及Template的格式化。

Formatter的format方法源码为:

 def format(self, record):
  """
  Format the specified record as text.
  The record's attribute dictionary is used as the operand to a
  string formatting operation which yields the returned string.
  Before formatting the dictionary, a couple of preparatory steps
  are carried out. The message attribute of the record is computed
  using LogRecord.getMessage(). If the formatting string uses the
  time (as determined by a call to usesTime(), formatTime() is
  called to format the event time. If there is exception information,
  it is formatted using formatException() and appended to the message.
  """
  record.message = record.getMessage()
  if self.usesTime():
   record.asctime = self.formatTime(record, self.datefmt)
  s = self.formatMessage(record)
  if record.exc_info:
   # Cache the traceback text to avoid converting it multiple times
   # (it's constant anyway)
   if not record.exc_text:
    record.exc_text = self.formatException(record.exc_info)
  if record.exc_text:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + record.exc_text
  if record.stack_info:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + self.formatStack(record.stack_info)

看到会调用record.getMessage(),这里仅仅是获取我们需要的日志信息。

之后会调用s = self.formatMessage(record):

 def formatMessage(self, record):
  return self._style.format(record)

其实是调用了当前style的format方法,以%这一类型为例PercentStyle:

class PercentStyle(object):

 default_format = '%(message)s'
 asctime_format = '%(asctime)s'
 asctime_search = '%(asctime)'

 def __init__(self, fmt):
  self._fmt = fmt or self.default_format

 def usesTime(self):
  return self._fmt.find(self.asctime_search) >= 0

 def format(self, record):
  return self._fmt % record.__dict__

从其中的format方法可以看出,是针对record的__dict__属性中的所有参数进行格式化,这下,就清楚了之前的extra参数是干嘛用的了:可以在formatter中加入自己自定义的一些参数,如固定的用户信息等等。

之后,将最终的message flush到对应的Stream里面去就行了,就是整个流程:

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持我们。

(0)

相关推荐

  • python 读取文件并把矩阵转成numpy的两种方法

    在当前目录下: 方法1: file = open('filename') a =file.read() b =a.split('\n')#使用换行 len(b) #统计有多少行 for i in range(len(b)): b[i] = b[i].split()#使用空格分开 len(b[0])#可以查看第一行有多少列. B[0][311]#可以查看具体某行某列的数 import numpy as np b = np.array(b)#转成numpy形的 type(b) # 输出<输出clas

  • Python魔法方法详解

    据说,Python 的对象天生拥有一些神奇的方法,它们总被双下划线所包围,他们是面向对象的 Python 的一切. 他们是可以给你的类增加魔力的特殊方法,如果你的对象实现(重载)了这些方法中的某一个,那么这个方法就会在特殊的情况下被 Python 所调用,你可以定义自己想要的行为,而这一切都是自动发生的. Python 的魔术方法非常强大,然而随之而来的则是责任.了解正确的方法去使用非常重要! 魔法方法 含义 基本的魔法方法 __new__(cls[, ...]) new 是在一个对象实例化的时

  • python 实现读取一个excel多个sheet表并合并的方法

    如下所示: import xlrd import pandas as pd from pandas import DataFrame DATA_DIR = 'E:/' excel_name = '%s2017.xls' % DATA_DIR wb = xlrd.open_workbook(excel_name) # print(wb) # 获取workbook中所有的表格 sheets = wb.sheet_names() # print(sheets) # 循环遍历所有sheet df_28

  • Python把对应格式的csv文件转换成字典类型存储脚本的方法

    该脚本是为了结合之前的编写的脚本,来实现数据的比对模块,实现数据的自动化!由于数据格式是定死的,该代码只做参考,有什么问题可以私信我! CSV的数据格式截图如下: readDataToDic.py源代码如下: #coding=utf8 import csv ''' 该模块的主要功能,是根据已有的csv文件, 通过readDataToDicl函数,把csv中对应的部分, 写入字典中,每个字典当当作一条json数据 ''' class GenExceptData(object): def __ini

  • Python函数中不定长参数的写法

    1.不定长参数的写法,用 *变量名 表示 2.不定长参数累加 3.不定长参数,使用**c接受m=23,n=56的值: 传参时,a必写,b.c可以缺省 def fun(a, b, *args): print(a) print(b) print(args) print("="*30) ret = a + b for i in args: ret += i return ret print(fun(1,2,3,4)) 结果: 1 2 (3, 4) ======================

  • Python numpy中矩阵的基本用法汇总

    Python矩阵的基本用法 mat()函数将目标数据的类型转化成矩阵(matrix) 1,mat()函数和array()函数的区别 Numpy函数库中存在两种不同的数据类型(矩阵matrix和数组array),都可以用于处理行列表示的数字元素,虽然他们看起来很相似,但是在这两个数据类型上执行相同的数学运算可能得到不同的结果,其中Numpy函数库中的matrix与MATLAB中matrices等价. 直接看一个例子: import numpy as np a = np.mat('1 3;5 7')

  • Python多线程同步---文件读写控制方法

    1.实现文件读写的文件ltz_schedule_times.py #! /usr/bin/env python #coding=utf-8 import os def ReadTimes(): res = [] if os.path.exists('schedule_times.txt'): fp = open('schedule_times.txt', 'r') else: os.system('touch schedule_times.txt') fp = open('schedule_ti

  • python调用c++ ctype list传数组或者返回数组的方法

    示例1: pycallclass.cpp: #include <iostream> using namespace std; typedef unsigned char BYTE; #define MAX_COUNT 20 struct tagOutCardResult_py { BYTE cbCardCount; BYTE cbResultCard1; BYTE cbResultCard2; BYTE cbResultCard3; BYTE cbResultCard4; BYTE cbRes

  • python调用c++传递数组的实例

    如下所示: INPUT = c_int * 4 # 实例化一个长度为2的整型数组 input = INPUT() # 为数组赋值(input这个数组是不支持迭代的) input[0] = 11 input[1] = 2 input[2] = 3 input[3] = 4 dll.teststring.restype = c_char_p # bytes(aaaa, encoding="utf-8") a = dll.teststring(input,4) MYLIBDLL char*

  • python读取csv和txt数据转换成向量的实例

    最近写程序需要从文件中读取数据,并把读取的数据转换成向量. 查阅资料之后找到了读取csv文件和txt文件两种方式,下面结合自己的实验过程,做简要记录,供大家参考: 1.读取csv文件的数据 import csv filtpath = "data_test.csv" with open(filtpath,'r') as csvfile: reader = csv.reader(csvfile) header = next(reader) data = [] for line in rea

随机推荐