.. _expressions:

***********

Expressions

***********

.. index:: single: expression

This chapter explains the meaning of the elements of expressions in Python.

.. index:: single: BNF

**Syntax Notes:** In this and the following chapters, extended BNF notation will

be used to describe syntax, not lexical analysis.  When (one alternative of) a

syntax rule has the form

.. productionlist:: *

   name: `othername`

.. index:: single: syntax

and no semantics are given, the semantics of this form of ``name`` are the same

as for ``othername``.

.. _conversions:

Arithmetic conversions

======================

.. index:: pair: arithmetic; conversion

When a description of an arithmetic operator below uses the phrase "the numeric

arguments are converted to a common type," the arguments are coerced using the

coercion rules listed at  :ref:`coercion-rules`.  If both arguments are standard

numeric types, the following coercions are applied:

* If either argument is a complex number, the other is converted to complex;

* otherwise, if either argument is a floating point number, the other is

  converted to floating point;

* otherwise, if either argument is a long integer, the other is converted to

  long integer;

* otherwise, both must be plain integers and no conversion is necessary.

Some additional rules apply for certain operators (e.g., a string left argument

to the '%' operator). Extensions can define their own coercions.

.. _atoms:

Atoms

=====

.. index:: single: atom

Atoms are the most basic elements of expressions.  The simplest atoms are

identifiers or literals.  Forms enclosed in reverse quotes or in parentheses,

brackets or braces are also categorized syntactically as atoms.  The syntax for

atoms is:

.. productionlist::

   atom: `identifier` | `literal` | `enclosure`

   enclosure: `parenth_form` | `list_display`

            : | `generator_expression` | `dict_display`

            : | `string_conversion` | `yield_atom`

.. _atom-identifiers:

Identifiers (Names)

-------------------

.. index::

   single: name

   single: identifier

An identifier occurring as an atom is a name.  See section :ref:`identifiers`

for lexical definition and section :ref:`naming` for documentation of naming and

binding.

.. index:: exception: NameError

When the name is bound to an object, evaluation of the atom yields that object.

When a name is not bound, an attempt to evaluate it raises a :exc:`NameError`

exception.

.. index::

   pair: name; mangling

   pair: private; names

**Private name mangling:** When an identifier that textually occurs in a class

definition begins with two or more underscore characters and does not end in two

or more underscores, it is considered a :dfn:`private name` of that class.

Private names are transformed to a longer form before code is generated for

them.  The transformation inserts the class name in front of the name, with

leading underscores removed, and a single underscore inserted in front of the

class name.  For example, the identifier ``__spam`` occurring in a class named

``Ham`` will be transformed to ``_Ham__spam``.  This transformation is

independent of the syntactical context in which the identifier is used.  If the

transformed name is extremely long (longer than 255 characters), implementation

defined truncation may happen.  If the class name consists only of underscores,

no transformation is done.

.. _atom-literals:

Literals

--------

.. index:: single: literal

Python supports string literals and various numeric literals:

.. productionlist::

   literal: `stringliteral` | `integer` | `longinteger`

          : | `floatnumber` | `imagnumber`

Evaluation of a literal yields an object of the given type (string, integer,

long integer, floating point number, complex number) with the given value.  The

value may be approximated in the case of floating point and imaginary (complex)

literals.  See section :ref:`literals` for details.

.. index::

   triple: immutable; data; type

   pair: immutable; object

All literals correspond to immutable data types, and hence the object's identity

is less important than its value.  Multiple evaluations of literals with the

same value (either the same occurrence in the program text or a different

occurrence) may obtain the same object or a different object with the same

value.

.. _parenthesized:

Parenthesized forms

-------------------

.. index:: single: parenthesized form

A parenthesized form is an optional expression list enclosed in parentheses:

.. productionlist::

   parenth_form: "(" [`expression_list`] ")"

A parenthesized expression list yields whatever that expression list yields: if

the list contains at least one comma, it yields a tuple; otherwise, it yields

the single expression that makes up the expression list.

.. index:: pair: empty; tuple

An empty pair of parentheses yields an empty tuple object.  Since tuples are

immutable, the rules for literals apply (i.e., two occurrences of the empty

tuple may or may not yield the same object).

.. index::

   single: comma

   pair: tuple; display

Note that tuples are not formed by the parentheses, but rather by use of the

comma operator.  The exception is the empty tuple, for which parentheses *are*

required --- allowing unparenthesized "nothing" in expressions would cause

ambiguities and allow common typos to pass uncaught.

.. _lists:

List displays

-------------

.. index::

   pair: list; display

   pair: list; comprehensions

A list display is a possibly empty series of expressions enclosed in square

brackets:

.. productionlist::

   list_display: "[" [`expression_list` | `list_comprehension`] "]"

   list_comprehension: `expression` `list_for`

   list_for: "for" `target_list` "in" `old_expression_list` [`list_iter`]

   old_expression_list: `old_expression` [("," `old_expression`)+ [","]]

   list_iter: `list_for` | `list_if`

   list_if: "if" `old_expression` [`list_iter`]

.. index::

   pair: list; comprehensions

   object: list

   pair: empty; list

A list display yields a new list object.  Its contents are specified by

providing either a list of expressions or a list comprehension.  When a

comma-separated list of expressions is supplied, its elements are evaluated from

left to right and placed into the list object in that order.  When a list

comprehension is supplied, it consists of a single expression followed by at

least one :keyword:`for` clause and zero or more :keyword:`for` or :keyword:`if`

clauses.  In this case, the elements of the new list are those that would be

produced by considering each of the :keyword:`for` or :keyword:`if` clauses a

block, nesting from left to right, and evaluating the expression to produce a

list element each time the innermost block is reached [#]_.

.. _genexpr:

Generator expressions

---------------------

.. index:: pair: generator; expression

A generator expression is a compact generator notation in parentheses:

.. productionlist::

   generator_expression: "(" `expression` `genexpr_for` ")"

   genexpr_for: "for" `target_list` "in" `or_test` [`genexpr_iter`]

   genexpr_iter: `genexpr_for` | `genexpr_if`

   genexpr_if: "if" `old_expression` [`genexpr_iter`]

.. index:: object: generator

A generator expression yields a new generator object.  It consists of a single

expression followed by at least one :keyword:`for` clause and zero or more

:keyword:`for` or :keyword:`if` clauses.  The iterating values of the new

generator are those that would be produced by considering each of the

:keyword:`for` or :keyword:`if` clauses a block, nesting from left to right, and

evaluating the expression to yield a value that is reached the innermost block

for each iteration.

Variables used in the generator expression are evaluated lazily in a separate

scope when the :meth:`next` method is called for the generator object (in the

same fashion as for normal generators).  However, the :keyword:`in` expression

of the leftmost :keyword:`for` clause is immediately evaluated in the current

scope so that an error produced by it can be seen before any other possible

error in the code that handles the generator expression.  Subsequent

:keyword:`for` and :keyword:`if` clauses cannot be evaluated immediately since

they may depend on the previous :keyword:`for` loop.  For example:

``(x*y for x in range(10) for y in bar(x))``.

The parentheses can be omitted on calls with only one argument. See section

:ref:`calls` for the detail.

.. _dict:

Dictionary displays

-------------------

.. index:: pair: dictionary; display

.. index::

   single: key

   single: datum

   single: key/datum pair

A dictionary display is a possibly empty series of key/datum pairs enclosed in

curly braces:

.. productionlist::

   dict_display: "{" [`key_datum_list`] "}"

   key_datum_list: `key_datum` ("," `key_datum`)* [","]

   key_datum: `expression` ":" `expression`

.. index:: object: dictionary

A dictionary display yields a new dictionary object.

The key/datum pairs are evaluated from left to right to define the entries of

the dictionary: each key object is used as a key into the dictionary to store

the corresponding datum.

.. index:: pair: immutable; object

Restrictions on the types of the key values are listed earlier in section

:ref:`types`.  (To summarize, the key type should be :term:`hashable`, which excludes

all mutable objects.)  Clashes between duplicate keys are not detected; the last

datum (textually rightmost in the display) stored for a given key value

prevails.

.. _string-conversions:

String conversions

------------------

.. index::

   pair: string; conversion

   pair: reverse; quotes

   pair: backward; quotes

   single: back-quotes

A string conversion is an expression list enclosed in reverse (a.k.a. backward)

quotes:

.. productionlist::

   string_conversion: "'" `expression_list` "'"

A string conversion evaluates the contained expression list and converts the

resulting object into a string according to rules specific to its type.

If the object is a string, a number, ``None``, or a tuple, list or dictionary

containing only objects whose type is one of these, the resulting string is a

valid Python expression which can be passed to the built-in function

:func:`eval` to yield an expression with the same value (or an approximation, if

floating point numbers are involved).

(In particular, converting a string adds quotes around it and converts "funny"

characters to escape sequences that are safe to print.)

.. index:: object: recursive

Recursive objects (for example, lists or dictionaries that contain a reference

to themselves, directly or indirectly) use ``...`` to indicate a recursive

reference, and the result cannot be passed to :func:`eval` to get an equal value

(:exc:`SyntaxError` will be raised instead).

.. index::

   builtin: repr

   builtin: str

The built-in function :func:`repr` performs exactly the same conversion in its

argument as enclosing it in parentheses and reverse quotes does.  The built-in

function :func:`str` performs a similar but more user-friendly conversion.

.. _yieldexpr:

Yield expressions

-----------------

.. index::

   keyword: yield

   pair: yield; expression

   pair: generator; function

.. productionlist::

   yield_atom: "(" `yield_expression` ")"

   yield_expression: "yield" [`expression_list`]

.. versionadded:: 2.5

The :keyword:`yield` expression is only used when defining a generator function,

and can only be used in the body of a function definition. Using a

:keyword:`yield` expression in a function definition is sufficient to cause that

definition to create a generator function instead of a normal function.

When a generator function is called, it returns an iterator known as a

generator.  That generator then controls the execution of a generator function.

The execution starts when one of the generator's methods is called.  At that

time, the execution proceeds to the first :keyword:`yield` expression, where it

is suspended again, returning the value of :token:`expression_list` to

generator's caller.  By suspended we mean that all local state is retained,

including the current bindings of local variables, the instruction pointer, and

the internal evaluation stack.  When the execution is resumed by calling one of

the generator's methods, the function can proceed exactly as if the

:keyword:`yield` expression was just another external call. The value of the

:keyword:`yield` expression after resuming depends on the method which resumed

the execution.

.. index:: single: coroutine

All of this makes generator functions quite similar to coroutines; they yield

multiple times, they have more than one entry point and their execution can be

suspended.  The only difference is that a generator function cannot control

where should the execution continue after it yields; the control is always

transfered to the generator's caller.

.. index:: object: generator

The following generator's methods can be used to control the execution of a

generator function:

.. index:: exception: StopIteration

.. method:: generator.next()

   Starts the execution of a generator function or resumes it at the last executed

   :keyword:`yield` expression.  When a generator function is resumed with a

   :meth:`next` method, the current :keyword:`yield` expression always evaluates to

   :const:`None`.  The execution then continues to the next :keyword:`yield`

   expression, where the generator is suspended again, and the value of the

   :token:`expression_list` is returned to :meth:`next`'s caller. If the generator

   exits without yielding another value, a :exc:`StopIteration` exception is

   raised.

.. method:: generator.send(value)

   Resumes the execution and "sends" a value into the generator function.  The

   ``value`` argument becomes the result of the current :keyword:`yield`

   expression.  The :meth:`send` method returns the next value yielded by the

   generator, or raises :exc:`StopIteration` if the generator exits without

   yielding another value. When :meth:`send` is called to start the generator, it

   must be called with :const:`None` as the argument, because there is no

   :keyword:`yield` expression that could receive the value.

.. method:: generator.throw(type[, value[, traceback]])

   Raises an exception of type ``type`` at the point where generator was paused,

   and returns the next value yielded by the generator function.  If the generator

   exits without yielding another value, a :exc:`StopIteration` exception is

   raised.  If the generator function does not catch the passed-in exception, or

   raises a different exception, then that exception propagates to the caller.

.. index:: exception: GeneratorExit

.. method:: generator.close()

   Raises a :exc:`GeneratorExit` at the point where the generator function was

   paused.  If the generator function then raises :exc:`StopIteration` (by exiting

   normally, or due to already being closed) or :exc:`GeneratorExit` (by not

   catching the exception), close returns to its caller.  If the generator yields a

   value, a :exc:`RuntimeError` is raised.  If the generator raises any other

   exception, it is propagated to the caller.  :meth:`close` does nothing if the

   generator has already exited due to an exception or normal exit.

Here is a simple example that demonstrates the behavior of generators and

generator functions::

   >>> def echo(value=None):

   ...     print "Execution starts when 'next()' is called for the first time."

   ...     try:

   ...         while True:

   ...             try:

   ...                 value = (yield value)

   ...             except Exception, e:

   ...                 value = e

   ...     finally:

   ...         print "Don't forget to clean up when 'close()' is called."

   ...

   >>> generator = echo(1)

   >>> print generator.next()

   Execution starts when 'next()' is called for the first time.

   1

   >>> print generator.next()

   None

   >>> print generator.send(2)

   2

   >>> generator.throw(TypeError, "spam")

   TypeError('spam',)

   >>> generator.close()

   Don't forget to clean up when 'close()' is called.

.. seealso::

   :pep:`0342` - Coroutines via Enhanced Generators

      The proposal to enhance the API and syntax of generators, making them usable as

      simple coroutines.

.. _primaries:

Primaries

=========

.. index:: single: primary

Primaries represent the most tightly bound operations of the language. Their

syntax is:

.. productionlist::

   primary: `atom` | `attributeref` | `subscription` | `slicing` | `call`

.. _attribute-references:

Attribute references

--------------------

.. index:: pair: attribute; reference

An attribute reference is a primary followed by a period and a name:

.. productionlist::

   attributeref: `primary` "." `identifier`

.. index::

   exception: AttributeError

   object: module

   object: list

The primary must evaluate to an object of a type that supports attribute

references, e.g., a module, list, or an instance.  This object is then asked to

produce the attribute whose name is the identifier.  If this attribute is not

available, the exception :exc:`AttributeError` is raised. Otherwise, the type

and value of the object produced is determined by the object.  Multiple

evaluations of the same attribute reference may yield different objects.

.. _subscriptions:

Subscriptions

-------------

.. index:: single: subscription

.. index::

   object: sequence

   object: mapping

   object: string

   object: tuple

   object: list

   object: dictionary

   pair: sequence; item

A subscription selects an item of a sequence (string, tuple or list) or mapping

(dictionary) object:

.. productionlist::

   subscription: `primary` "[" `expression_list` "]"

The primary must evaluate to an object of a sequence or mapping type.

If the primary is a mapping, the expression list must evaluate to an object

whose value is one of the keys of the mapping, and the subscription selects the

value in the mapping that corresponds to that key.  (The expression list is a

tuple except if it has exactly one item.)

If the primary is a sequence, the expression (list) must evaluate to a plain

integer.  If this value is negative, the length of the sequence is added to it

(so that, e.g., ``x[-1]`` selects the last item of ``x``.)  The resulting value

must be a nonnegative integer less than the number of items in the sequence, and