1 :mod:`email`: Parsing email messages

     2 ------------------------------------

     3 

     4 .. module:: email.parser

     5    :synopsis: Parse flat text email messages to produce a message object structure.

     6 

     7 

     8 Message object structures can be created in one of two ways: they can be created

     9 from whole cloth by instantiating :class:`Message` objects and stringing them

    10 together via :meth:`attach` and :meth:`set_payload` calls, or they can be

    11 created by parsing a flat text representation of the email message.

    12 

    13 The :mod:`email` package provides a standard parser that understands most email

    14 document structures, including MIME documents.  You can pass the parser a string

    15 or a file object, and the parser will return to you the root :class:`Message`

    16 instance of the object structure.  For simple, non-MIME messages the payload of

    17 this root object will likely be a string containing the text of the message.

    18 For MIME messages, the root object will return ``True`` from its

    19 :meth:`is_multipart` method, and the subparts can be accessed via the

    20 :meth:`get_payload` and :meth:`walk` methods.

    21 

    22 There are actually two parser interfaces available for use, the classic

    23 :class:`Parser` API and the incremental :class:`FeedParser` API.  The classic

    24 :class:`Parser` API is fine if you have the entire text of the message in memory

    25 as a string, or if the entire message lives in a file on the file system.

    26 :class:`FeedParser` is more appropriate for when you're reading the message from

    27 a stream which might block waiting for more input (e.g. reading an email message

    28 from a socket).  The :class:`FeedParser` can consume and parse the message

    29 incrementally, and only returns the root object when you close the parser [#]_.

    30 

    31 Note that the parser can be extended in limited ways, and of course you can

    32 implement your own parser completely from scratch.  There is no magical

    33 connection between the :mod:`email` package's bundled parser and the

    34 :class:`Message` class, so your custom parser can create message object trees

    35 any way it finds necessary.

    36 

    37 

    38 FeedParser API

    39 ^^^^^^^^^^^^^^

    40 

    41 .. versionadded:: 2.4

    42 

    43 The :class:`FeedParser`, imported from the :mod:`email.feedparser` module,

    44 provides an API that is conducive to incremental parsing of email messages, such

    45 as would be necessary when reading the text of an email message from a source

    46 that can block (e.g. a socket).  The :class:`FeedParser` can of course be used

    47 to parse an email message fully contained in a string or a file, but the classic

    48 :class:`Parser` API may be more convenient for such use cases.  The semantics

    49 and results of the two parser APIs are identical.

    50 

    51 The :class:`FeedParser`'s API is simple; you create an instance, feed it a bunch

    52 of text until there's no more to feed it, then close the parser to retrieve the

    53 root message object.  The :class:`FeedParser` is extremely accurate when parsing

    54 standards-compliant messages, and it does a very good job of parsing

    55 non-compliant messages, providing information about how a message was deemed

    56 broken.  It will populate a message object's *defects* attribute with a list of

    57 any problems it found in a message.  See the :mod:`email.errors` module for the

    58 list of defects that it can find.

    59 

    60 Here is the API for the :class:`FeedParser`:

    61 

    62 

    63 .. class:: FeedParser([_factory])

    64 

    65    Create a :class:`FeedParser` instance.  Optional *_factory* is a no-argument

    66    callable that will be called whenever a new message object is needed.  It

    67    defaults to the :class:`email.message.Message` class.

    68 

    69 

    70    .. method:: feed(data)

    71 

    72       Feed the :class:`FeedParser` some more data.  *data* should be a string

    73       containing one or more lines.  The lines can be partial and the

    74       :class:`FeedParser` will stitch such partial lines together properly.  The

    75       lines in the string can have any of the common three line endings,

    76       carriage return, newline, or carriage return and newline (they can even be

    77       mixed).

    78 

    79 

    80    .. method:: close()

    81 

    82       Closing a :class:`FeedParser` completes the parsing of all previously fed

    83       data, and returns the root message object.  It is undefined what happens

    84       if you feed more data to a closed :class:`FeedParser`.

    85 

    86 

    87 Parser class API

    88 ^^^^^^^^^^^^^^^^

    89 

    90 The :class:`Parser` class, imported from the :mod:`email.parser` module,

    91 provides an API that can be used to parse a message when the complete contents

    92 of the message are available in a string or file.  The :mod:`email.parser`

    93 module also provides a second class, called :class:`HeaderParser` which can be

    94 used if you're only interested in the headers of the message.

    95 :class:`HeaderParser` can be much faster in these situations, since it does not

    96 attempt to parse the message body, instead setting the payload to the raw body

    97 as a string. :class:`HeaderParser` has the same API as the :class:`Parser`

    98 class.

    99 

   100 

   101 .. class:: Parser([_class])

   102 

   103    The constructor for the :class:`Parser` class takes an optional argument

   104    *_class*.  This must be a callable factory (such as a function or a class), and

   105    it is used whenever a sub-message object needs to be created.  It defaults to

   106    :class:`Message` (see :mod:`email.message`).  The factory will be called without

   107    arguments.

   108 

   109    The optional *strict* flag is ignored.

   110 

   111    .. deprecated:: 2.4

   112       Because the :class:`Parser` class is a backward compatible API wrapper

   113       around the new-in-Python 2.4 :class:`FeedParser`, *all* parsing is

   114       effectively non-strict.  You should simply stop passing a *strict* flag to

   115       the :class:`Parser` constructor.

   116 

   117    .. versionchanged:: 2.2.2

   118       The *strict* flag was added.

   119 

   120    .. versionchanged:: 2.4

   121       The *strict* flag was deprecated.

   122 

   123    The other public :class:`Parser` methods are:

   124 

   125 

   126    .. method:: parse(fp[, headersonly])

   127 

   128       Read all the data from the file-like object *fp*, parse the resulting

   129       text, and return the root message object.  *fp* must support both the

   130       :meth:`readline` and the :meth:`read` methods on file-like objects.

   131 

   132       The text contained in *fp* must be formatted as a block of :rfc:`2822`

   133       style headers and header continuation lines, optionally preceded by a

   134       envelope header.  The header block is terminated either by the end of the

   135       data or by a blank line.  Following the header block is the body of the

   136       message (which may contain MIME-encoded subparts).

   137 

   138       Optional *headersonly* is as with the :meth:`parse` method.

   139 

   140       .. versionchanged:: 2.2.2

   141          The *headersonly* flag was added.

   142 

   143 

   144    .. method:: parsestr(text[, headersonly])

   145 

   146       Similar to the :meth:`parse` method, except it takes a string object

   147       instead of a file-like object.  Calling this method on a string is exactly

   148       equivalent to wrapping *text* in a :class:`StringIO` instance first and

   149       calling :meth:`parse`.

   150 

   151       Optional *headersonly* is a flag specifying whether to stop parsing after

   152       reading the headers or not.  The default is ``False``, meaning it parses

   153       the entire contents of the file.

   154 

   155       .. versionchanged:: 2.2.2

   156          The *headersonly* flag was added.

   157 

   158 Since creating a message object structure from a string or a file object is such

   159 a common task, two functions are provided as a convenience.  They are available

   160 in the top-level :mod:`email` package namespace.

   161 

   162 .. currentmodule:: email

   163 

   164 .. function:: message_from_string(s[, _class[, strict]])

   165 

   166    Return a message object structure from a string.  This is exactly equivalent to

   167    ``Parser().parsestr(s)``.  Optional *_class* and *strict* are interpreted as

   168    with the :class:`Parser` class constructor.

   169 

   170    .. versionchanged:: 2.2.2

   171       The *strict* flag was added.

   172 

   173 

   174 .. function:: message_from_file(fp[, _class[, strict]])

   175 

   176    Return a message object structure tree from an open file object.  This is

   177    exactly equivalent to ``Parser().parse(fp)``.  Optional *_class* and *strict*

   178    are interpreted as with the :class:`Parser` class constructor.

   179 

   180    .. versionchanged:: 2.2.2

   181       The *strict* flag was added.

   182 

   183 Here's an example of how you might use this at an interactive Python prompt::

   184 

   185    >>> import email

   186    >>> msg = email.message_from_string(myString)

   187 

   188 

   189 Additional notes

   190 ^^^^^^^^^^^^^^^^

   191 

   192 Here are some notes on the parsing semantics:

   193 

   194 * Most non-\ :mimetype:`multipart` type messages are parsed as a single message

   195   object with a string payload.  These objects will return ``False`` for

   196   :meth:`is_multipart`.  Their :meth:`get_payload` method will return a string

   197   object.

   198 

   199 * All :mimetype:`multipart` type messages will be parsed as a container message

   200   object with a list of sub-message objects for their payload.  The outer

   201   container message will return ``True`` for :meth:`is_multipart` and their

   202   :meth:`get_payload` method will return the list of :class:`Message` subparts.

   203 

   204 * Most messages with a content type of :mimetype:`message/\*` (e.g.

   205   :mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also be

   206   parsed as container object containing a list payload of length 1.  Their

   207   :meth:`is_multipart` method will return ``True``.  The single element in the

   208   list payload will be a sub-message object.

   209 

   210 * Some non-standards compliant messages may not be internally consistent about

   211   their :mimetype:`multipart`\ -edness.  Such messages may have a

   212   :mailheader:`Content-Type` header of type :mimetype:`multipart`, but their

   213   :meth:`is_multipart` method may return ``False``.  If such messages were parsed

   214   with the :class:`FeedParser`, they will have an instance of the

   215   :class:`MultipartInvariantViolationDefect` class in their *defects* attribute

   216   list.  See :mod:`email.errors` for details.

   217 

   218 .. rubric:: Footnotes

   219 

   220 .. [#] As of email package version 3.0, introduced in Python 2.4, the classic

   221    :class:`Parser` was re-implemented in terms of the :class:`FeedParser`, so the

   222    semantics and results are identical between the two parsers.

   223