| rest25/library/pickle.rst => rest262/library/pickle.rst | ||
|---|---|---|
| n | 1- | |
| 2 | :mod:`pickle` --- Python object serialization | |
| 3 | ============================================= | |
| 4 | ||
| 5 | .. index:: | |
| 6 | single: persistence | |
| 7 | pair: persistent; objects | |
| 8 | pair: serializing; objects | |
| 9 | pair: marshalling; objects | |
| 10 | pair: flattening; objects | |
| 11 | pair: pickling; objects | |
| 12 | ||
| 13 | .. module:: pickle | |
| 14 | :synopsis: Convert Python objects to streams of bytes and back. | |
| n | 15- | |
| n | 14+ | .. sectionauthor:: Jim Kerr <jbkerr@sr.hp.com>. |
| 16- | ||
| 15+ | .. sectionauthor:: Barry Warsaw <barry@zope.com> | |
| 17- | .. % Substantial improvements by Jim Kerr <jbkerr@sr.hp.com>. | |
| 18- | .. % Rewritten by Barry Warsaw <barry@zope.com> | |
| 19 | ||
| 20 | The :mod:`pickle` module implements a fundamental, but powerful algorithm for | |
| 21 | serializing and de-serializing a Python object structure. "Pickling" is the | |
| 22 | process whereby a Python object hierarchy is converted into a byte stream, and | |
| 23 | "unpickling" is the inverse operation, whereby a byte stream is converted back | |
| 24 | into an object hierarchy. Pickling (and unpickling) is alternatively known as | |
| 25 | "serialization", "marshalling," [#]_ or "flattening", however, to avoid | |
| 26 | confusion, the terms used here are "pickling" and "unpickling". | |
| 86 | Note that serialization is a more primitive notion than persistence; although | |
| 87 | :mod:`pickle` reads and writes file objects, it does not handle the issue of | |
| 88 | naming persistent objects, nor the (even more complicated) issue of concurrent | |
| 89 | access to persistent objects. The :mod:`pickle` module can transform a complex | |
| 90 | object into a byte stream and it can transform the byte stream into an object | |
| 91 | with the same internal structure. Perhaps the most obvious thing to do with | |
| 92 | these byte streams is to write them onto a file, but it is also conceivable to | |
| 93 | send them across a network or store them in a database. The module | |
| n | 94- | :mod:`shelve` provides a simple interface to pickle and unpickle objects on DBM- |
| n | 91+ | :mod:`shelve` provides a simple interface to pickle and unpickle objects on |
| 95- | style database files. | |
| 92+ | DBM-style database files. | |
| 96 | ||
| 97 | ||
| 98 | Data stream format | |
| 99 | ------------------ | |
| 100 | ||
| 101 | .. index:: | |
| 102 | single: XDR | |
| 103 | single: External Data Representation | |
| 117 | ||
| 118 | * Protocol version 0 is the original ASCII protocol and is backwards compatible | |
| 119 | with earlier versions of Python. | |
| 120 | ||
| 121 | * Protocol version 1 is the old binary format which is also compatible with | |
| 122 | earlier versions of Python. | |
| 123 | ||
| 124 | * Protocol version 2 was introduced in Python 2.3. It provides much more | |
| n | 125- | efficient pickling of new-style classes. |
| n | 122+ | efficient pickling of :term:`new-style class`\es. |
| 126 | ||
| n | 127- | Refer to PEP 307 for more information. |
| n | 124+ | Refer to :pep:`307` for more information. |
| 128 | ||
| 129 | If a *protocol* is not specified, protocol 0 is used. If *protocol* is specified | |
| 130 | as a negative value or :const:`HIGHEST_PROTOCOL`, the highest protocol version | |
| 131 | available will be used. | |
| 132 | ||
| 133 | .. versionchanged:: 2.3 | |
| 134 | Introduced the *protocol* parameter. | |
| 135 | ||
| 244 | :class:`Unpickler`: | |
| 245 | ||
| 246 | ||
| 247 | .. class:: Pickler(file[, protocol]) | |
| 248 | ||
| 249 | This takes a file-like object to which it will write a pickle data stream. | |
| 250 | ||
| 251 | If the *protocol* parameter is omitted, protocol 0 is used. If *protocol* is | |
| n | 252- | specified as a negative value, the highest protocol version will be used. |
| n | 249+ | specified as a negative value or :const:`HIGHEST_PROTOCOL`, the highest |
| 250+ | protocol version will be used. | |
| 253 | ||
| 254 | .. versionchanged:: 2.3 | |
| 255 | Introduced the *protocol* parameter. | |
| 256 | ||
| 257 | *file* must have a :meth:`write` method that accepts a single string argument. | |
| 258 | It can thus be an open file object, a :mod:`StringIO` object, or any other | |
| 259 | custom object that meets this interface. | |
| 260 | ||
| n | 261- | :class:`Pickler` objects define one (or two) public methods: |
| n | 259+ | :class:`Pickler` objects define one (or two) public methods: |
| 262 | ||
| 263 | ||
| n | 264- | .. method:: Pickler.dump(obj) |
| n | 262+ | .. method:: dump(obj) |
| 265 | ||
| n | 266- | Write a pickled representation of *obj* to the open file object given in the |
| n | 264+ | Write a pickled representation of *obj* to the open file object given in the |
| 267- | constructor. Either the binary or ASCII format will be used, depending on the | |
| 265+ | constructor. Either the binary or ASCII format will be used, depending on the | |
| 268- | value of the *protocol* argument passed to the constructor. | |
| 266+ | value of the *protocol* argument passed to the constructor. | |
| 269 | ||
| 270 | ||
| n | 271- | .. method:: Pickler.clear_memo() |
| n | 269+ | .. method:: clear_memo() |
| 272 | ||
| n | 273- | Clears the pickler's "memo". The memo is the data structure that remembers |
| n | 271+ | Clears the pickler's "memo". The memo is the data structure that remembers |
| 274- | which objects the pickler has already seen, so that shared or recursive objects | |
| 272+ | which objects the pickler has already seen, so that shared or recursive objects | |
| 275- | pickled by reference and not by value. This method is useful when re-using | |
| 273+ | pickled by reference and not by value. This method is useful when re-using | |
| 276- | picklers. | |
| 274+ | picklers. | |
| 277 | ||
| n | 278- | .. note:: |
| n | 276+ | .. note:: |
| 279 | ||
| n | 280- | Prior to Python 2.3, :meth:`clear_memo` was only available on the picklers |
| n | 278+ | Prior to Python 2.3, :meth:`clear_memo` was only available on the picklers |
| 281- | created by :mod:`cPickle`. In the :mod:`pickle` module, picklers have an | |
| 279+ | created by :mod:`cPickle`. In the :mod:`pickle` module, picklers have an | |
| 282- | instance variable called :attr:`memo` which is a Python dictionary. So to clear | |
| 280+ | instance variable called :attr:`memo` which is a Python dictionary. So to clear | |
| 283- | the memo for a :mod:`pickle` module pickler, you could do the following:: | |
| 281+ | the memo for a :mod:`pickle` module pickler, you could do the following:: | |
| 284 | ||
| n | 285- | mypickler.memo.clear() |
| n | 283+ | mypickler.memo.clear() |
| 286 | ||
| n | 287- | Code that does not need to support older versions of Python should simply use |
| n | 285+ | Code that does not need to support older versions of Python should simply use |
| 288- | :meth:`clear_memo`. | |
| 286+ | :meth:`clear_memo`. | |
| 289 | ||
| 290 | It is possible to make multiple calls to the :meth:`dump` method of the same | |
| 291 | :class:`Pickler` instance. These must then be matched to the same number of | |
| 292 | calls to the :meth:`load` method of the corresponding :class:`Unpickler` | |
| 293 | instance. If the same object is pickled by multiple :meth:`dump` calls, the | |
| 294 | :meth:`load` will all yield references to the same object. [#]_ | |
| 295 | ||
| 296 | :class:`Unpickler` objects are defined as: | |
| 304 | factory. | |
| 305 | ||
| 306 | *file* must have two methods, a :meth:`read` method that takes an integer | |
| 307 | argument, and a :meth:`readline` method that requires no arguments. Both | |
| 308 | methods should return a string. Thus *file* can be a file object opened for | |
| 309 | reading, a :mod:`StringIO` object, or any other custom object that meets this | |
| 310 | interface. | |
| 311 | ||
| n | 312- | :class:`Unpickler` objects have one (or two) public methods: |
| n | 310+ | :class:`Unpickler` objects have one (or two) public methods: |
| 313 | ||
| 314 | ||
| n | 315- | .. method:: Unpickler.load() |
| n | 313+ | .. method:: load() |
| 316 | ||
| n | 317- | Read a pickled object representation from the open file object given in the |
| n | 315+ | Read a pickled object representation from the open file object given in |
| 318- | constructor, and return the reconstituted object hierarchy specified therein. | |
| 316+ | the constructor, and return the reconstituted object hierarchy specified | |
| 317+ | therein. | |
| 319 | ||
| n | 319+ | This method automatically determines whether the data stream was written |
| 320+ | in binary mode or not. | |
| 320 | ||
| n | 322+ | |
| 321- | .. method:: Unpickler.noload() | |
| 323+ | .. method:: noload() | |
| 322 | ||
| n | 323- | This is just like :meth:`load` except that it doesn't actually create any |
| n | 325+ | This is just like :meth:`load` except that it doesn't actually create any |
| 324- | objects. This is useful primarily for finding what's called "persistent ids" | |
| 326+ | objects. This is useful primarily for finding what's called "persistent | |
| 325- | that may be referenced in a pickle data stream. See section | |
| 327+ | ids" that may be referenced in a pickle data stream. See section | |
| 326- | :ref:`pickle-protocol` below for more details. | |
| 328+ | :ref:`pickle-protocol` below for more details. | |
| 327 | ||
| n | 328- | **Note:** the :meth:`noload` method is currently only available on |
| n | 330+ | **Note:** the :meth:`noload` method is currently only available on |
| 329- | :class:`Unpickler` objects created with the :mod:`cPickle` module. | |
| 331+ | :class:`Unpickler` objects created with the :mod:`cPickle` module. | |
| 330- | :mod:`pickle` module :class:`Unpickler`\ s do not have the :meth:`noload` | |
| 332+ | :mod:`pickle` module :class:`Unpickler`\ s do not have the :meth:`noload` | |
| 331- | method. | |
| 333+ | method. | |
| 332 | ||
| 333 | ||
| 334 | What can be pickled and unpickled? | |
| 335 | ---------------------------------- | |
| 336 | ||
| 337 | The following types can be pickled: | |
| 338 | ||
| 339 | * ``None``, ``True``, and ``False`` | |
| 389 | conversions can be made by the class's :meth:`__setstate__` method. | |
| 390 | ||
| 391 | ||
| 392 | .. _pickle-protocol: | |
| 393 | ||
| 394 | The pickle protocol | |
| 395 | ------------------- | |
| 396 | ||
| n | 399+ | .. currentmodule:: None |
| 400+ | ||
| 397 | This section describes the "pickling protocol" that defines the interface | |
| 398 | between the pickler/unpickler and the objects that are being serialized. This | |
| 399 | protocol provides a standard way for you to define, customize, and control how | |
| 400 | your objects are serialized and de-serialized. The description in this section | |
| 401 | doesn't cover specific customizations that you can employ to make the unpickling | |
| 402 | environment slightly safer from untrusted pickle data streams; see section | |
| 403 | :ref:`pickle-sub` for more details. | |
| 404 | ||
| 405 | ||
| 406 | .. _pickle-inst: | |
| 407 | ||
| 408 | Pickling and unpickling normal class instances | |
| 409 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 410 | ||
| n | 411- | .. index:: |
| n | 415+ | .. method:: object.__getinitargs__() |
| 412- | single: __getinitargs__() (copy protocol) | |
| 413- | single: __init__() (instance constructor) | |
| 414 | ||
| n | 415- | When a pickled class instance is unpickled, its :meth:`__init__` method is |
| n | 417+ | When a pickled class instance is unpickled, its :meth:`__init__` method is |
| 416- | normally *not* invoked. If it is desirable that the :meth:`__init__` method be | |
| 418+ | normally *not* invoked. If it is desirable that the :meth:`__init__` method | |
| 417- | called on unpickling, an old-style class can define a method | |
| 419+ | be called on unpickling, an old-style class can define a method | |
| 418- | :meth:`__getinitargs__`, which should return a *tuple* containing the arguments | |
| 420+ | :meth:`__getinitargs__`, which should return a *tuple* containing the | |
| 419- | to be passed to the class constructor (:meth:`__init__` for example). The | |
| 421+ | arguments to be passed to the class constructor (:meth:`__init__` for | |
| 420- | :meth:`__getinitargs__` method is called at pickle time; the tuple it returns is | |
| 422+ | example). The :meth:`__getinitargs__` method is called at pickle time; the | |
| 421- | incorporated in the pickle for the instance. | |
| 423+ | tuple it returns is incorporated in the pickle for the instance. | |
| 422 | ||
| n | 423- | .. index:: single: __getnewargs__() (copy protocol) |
| n | 425+ | .. method:: object.__getnewargs__() |
| 424 | ||
| n | 425- | New-style types can provide a :meth:`__getnewargs__` method that is used for |
| n | 427+ | New-style types can provide a :meth:`__getnewargs__` method that is used for |
| 426- | protocol 2. Implementing this method is needed if the type establishes some | |
| 428+ | protocol 2. Implementing this method is needed if the type establishes some | |
| 427- | internal invariants when the instance is created, or if the memory allocation is | |
| 429+ | internal invariants when the instance is created, or if the memory allocation | |
| 428- | affected by the values passed to the :meth:`__new__` method for the type (as it | |
| 430+ | is affected by the values passed to the :meth:`__new__` method for the type | |
| 429- | is for tuples and strings). Instances of a new-style type :class:`C` are | |
| 431+ | (as it is for tuples and strings). Instances of a :term:`new-style class` | |
| 430- | created using :: | |
| 432+ | ``C`` are created using :: | |
| 431 | ||
| n | 432- | obj = C.__new__(C, \*args) |
| n | 434+ | obj = C.__new__(C, *args) |
| 433 | ||
| n | 434- | |
| 435- | where *args* is the result of calling :meth:`__getnewargs__` on the original | |
| 436+ | where *args* is the result of calling :meth:`__getnewargs__` on the original | |
| 436- | object; if there is no :meth:`__getnewargs__`, an empty tuple is assumed. | |
| 437+ | object; if there is no :meth:`__getnewargs__`, an empty tuple is assumed. | |
| 437 | ||
| n | 438- | .. index:: |
| n | 439+ | .. method:: object.__getstate__() |
| 439- | single: __getstate__() (copy protocol) | |
| 440- | single: __setstate__() (copy protocol) | |
| 441- | single: __dict__ (instance attribute) | |
| 442 | ||
| n | 443- | Classes can further influence how their instances are pickled; if the class |
| n | 441+ | Classes can further influence how their instances are pickled; if the class |
| 444- | defines the method :meth:`__getstate__`, it is called and the return state is | |
| 442+ | defines the method :meth:`__getstate__`, it is called and the return state is | |
| 445- | pickled as the contents for the instance, instead of the contents of the | |
| 443+ | pickled as the contents for the instance, instead of the contents of the | |
| 446- | instance's dictionary. If there is no :meth:`__getstate__` method, the | |
| 444+ | instance's dictionary. If there is no :meth:`__getstate__` method, the | |
| 447- | instance's :attr:`__dict__` is pickled. | |
| 445+ | instance's :attr:`__dict__` is pickled. | |
| 448 | ||
| n | 447+ | .. method:: object.__setstate__() |
| 448+ | ||
| 449- | Upon unpickling, if the class also defines the method :meth:`__setstate__`, it | |
| 449+ | Upon unpickling, if the class also defines the method :meth:`__setstate__`, | |
| 450- | is called with the unpickled state. [#]_ If there is no :meth:`__setstate__` | |
| 450+ | it is called with the unpickled state. [#]_ If there is no | |
| 451- | method, the pickled state must be a dictionary and its items are assigned to the | |
| 451+ | :meth:`__setstate__` method, the pickled state must be a dictionary and its | |
| 452- | new instance's dictionary. If a class defines both :meth:`__getstate__` and | |
| 452+ | items are assigned to the new instance's dictionary. If a class defines both | |
| 453- | :meth:`__setstate__`, the state object needn't be a dictionary and these methods | |
| 453+ | :meth:`__getstate__` and :meth:`__setstate__`, the state object needn't be a | |
| 454- | can do what they want. [#]_ | |
| 454+ | dictionary and these methods can do what they want. [#]_ | |
| 455 | ||
| n | 456- | .. warning:: |
| n | 456+ | .. warning:: |
| 457 | ||
| n | 458- | For new-style classes, if :meth:`__getstate__` returns a false value, the |
| n | 458+ | For :term:`new-style class`\es, if :meth:`__getstate__` returns a false |
| 459- | :meth:`__setstate__` method will not be called. | |
| 459+ | value, the :meth:`__setstate__` method will not be called. | |
| 460+ | ||
| 461+ | .. note:: | |
| 462+ | ||
| 463+ | At unpickling time, some methods like :meth:`__getattr__`, | |
| 464+ | :meth:`__getattribute__`, or :meth:`__setattr__` may be called upon the | |
| 465+ | instance. In case those methods rely on some internal invariant being | |
| 466+ | true, the type should implement either :meth:`__getinitargs__` or | |
| 467+ | :meth:`__getnewargs__` to establish such an invariant; otherwise, neither | |
| 468+ | :meth:`__new__` nor :meth:`__init__` will be called. | |
| 460 | ||
| 461 | ||
| 462 | Pickling and unpickling extension types | |
| 463 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 464 | ||
| n | 474+ | .. method:: object.__reduce__() |
| 475+ | ||
| 465- | When the :class:`Pickler` encounters an object of a type it knows nothing about | |
| 476+ | When the :class:`Pickler` encounters an object of a type it knows nothing | |
| 466- | --- such as an extension type --- it looks in two places for a hint of how to | |
| 477+ | about --- such as an extension type --- it looks in two places for a hint of | |
| 467- | pickle it. One alternative is for the object to implement a :meth:`__reduce__` | |
| 478+ | how to pickle it. One alternative is for the object to implement a | |
| 468- | method. If provided, at pickling time :meth:`__reduce__` will be called with no | |
| 479+ | :meth:`__reduce__` method. If provided, at pickling time :meth:`__reduce__` | |
| 469- | arguments, and it must return either a string or a tuple. | |
| 480+ | will be called with no arguments, and it must return either a string or a | |
| 481+ | tuple. | |
| 470 | ||
| n | 471- | If a string is returned, it names a global variable whose contents are pickled |
| n | 483+ | If a string is returned, it names a global variable whose contents are |
| 472- | as normal. The string returned by :meth:`__reduce__` should be the object's | |
| 484+ | pickled as normal. The string returned by :meth:`__reduce__` should be the | |
| 473- | local name relative to its module; the pickle module searches the module | |
| 485+ | object's local name relative to its module; the pickle module searches the | |
| 474- | namespace to determine the object's module. | |
| 486+ | module namespace to determine the object's module. | |
| 475 | ||
| n | 476- | When a tuple is returned, it must be between two and five elements long. |
| n | 488+ | When a tuple is returned, it must be between two and five elements long. |
| 477- | Optional elements can either be omitted, or ``None`` can be provided as their | |
| 489+ | Optional elements can either be omitted, or ``None`` can be provided as their | |
| 478- | value. The semantics of each element are: | |
| 490+ | value. The contents of this tuple are pickled as normal and used to | |
| 491+ | reconstruct the object at unpickling time. The semantics of each element | |
| 492+ | are: | |
| 479 | ||
| n | 480- | * A callable object that will be called to create the initial version of the |
| n | 494+ | * A callable object that will be called to create the initial version of the |
| 481- | object. The next element of the tuple will provide arguments for this callable, | |
| 495+ | object. The next element of the tuple will provide arguments for this | |
| 482- | and later elements provide additional state information that will subsequently | |
| 496+ | callable, and later elements provide additional state information that will | |
| 483- | be used to fully reconstruct the pickled date. | |
| 497+ | subsequently be used to fully reconstruct the pickled data. | |
| 484 | ||
| n | 485- | In the unpickling environment this object must be either a class, a callable |
| n | 499+ | In the unpickling environment this object must be either a class, a |
| 486- | registered as a "safe constructor" (see below), or it must have an attribute | |
| 500+ | callable registered as a "safe constructor" (see below), or it must have an | |
| 487- | :attr:`__safe_for_unpickling__` with a true value. Otherwise, an | |
| 501+ | attribute :attr:`__safe_for_unpickling__` with a true value. Otherwise, an | |
| 488- | :exc:`UnpicklingError` will be raised in the unpickling environment. Note that | |
| 502+ | :exc:`UnpicklingError` will be raised in the unpickling environment. Note | |
| 489- | as usual, the callable itself is pickled by name. | |
| 503+ | that as usual, the callable itself is pickled by name. | |
| 490 | ||
| n | 491- | * A tuple of arguments for the callable object. |
| n | 505+ | * A tuple of arguments for the callable object. |
| 492 | ||
| n | 493- | .. versionchanged:: 2.5 |
| n | 507+ | .. versionchanged:: 2.5 |
| 494- | Formerly, this argument could also be ``None``. | |
| 508+ | Formerly, this argument could also be ``None``. | |
| 495 | ||
| n | 496- | * Optionally, the object's state, which will be passed to the object's |
| n | 510+ | * Optionally, the object's state, which will be passed to the object's |
| 497- | :meth:`__setstate__` method as described in section :ref:`pickle-inst`. If the | |
| 511+ | :meth:`__setstate__` method as described in section :ref:`pickle-inst`. If | |
| 498- | object has no :meth:`__setstate__` method, then, as above, the value must be a | |
| 512+ | the object has no :meth:`__setstate__` method, then, as above, the value | |
| 499- | dictionary and it will be added to the object's :attr:`__dict__`. | |
| 513+ | must be a dictionary and it will be added to the object's :attr:`__dict__`. | |
| 500 | ||
| n | 501- | * Optionally, an iterator (and not a sequence) yielding successive list items. |
| n | 515+ | * Optionally, an iterator (and not a sequence) yielding successive list |
| 502- | These list items will be pickled, and appended to the object using either | |
| 516+ | items. These list items will be pickled, and appended to the object using | |
| 503- | ``obj.append(item)`` or ``obj.extend(list_of_items)``. This is primarily used | |
| 517+ | either ``obj.append(item)`` or ``obj.extend(list_of_items)``. This is | |
| 504- | for list subclasses, but may be used by other classes as long as they have | |
| 518+ | primarily used for list subclasses, but may be used by other classes as | |
| 505- | :meth:`append` and :meth:`extend` methods with the appropriate signature. | |
| 519+ | long as they have :meth:`append` and :meth:`extend` methods with the | |
| 506- | (Whether :meth:`append` or :meth:`extend` is used depends on which pickle | |
| 520+ | appropriate signature. (Whether :meth:`append` or :meth:`extend` is used | |
| 507- | protocol version is used as well as the number of items to append, so both must | |
| 521+ | depends on which pickle protocol version is used as well as the number of | |
| 508- | be supported.) | |
| 522+ | items to append, so both must be supported.) | |
| 509 | ||
| n | 510- | * Optionally, an iterator (not a sequence) yielding successive dictionary items, |
| n | 524+ | * Optionally, an iterator (not a sequence) yielding successive dictionary |
| 511- | which should be tuples of the form ``(key, value)``. These items will be | |
| 525+ | items, which should be tuples of the form ``(key, value)``. These items | |
| 512- | pickled and stored to the object using ``obj[key] = value``. This is primarily | |
| 526+ | will be pickled and stored to the object using ``obj[key] = value``. This | |
| 513- | used for dictionary subclasses, but may be used by other classes as long as they | |
| 527+ | is primarily used for dictionary subclasses, but may be used by other | |
| 514- | implement :meth:`__setitem__`. | |
| 528+ | classes as long as they implement :meth:`__setitem__`. | |
| 515 | ||
| n | 530+ | .. method:: object.__reduce_ex__(protocol) |
| 531+ | ||
| 516- | It is sometimes useful to know the protocol version when implementing | |
| 532+ | It is sometimes useful to know the protocol version when implementing | |
| 517- | :meth:`__reduce__`. This can be done by implementing a method named | |
| 533+ | :meth:`__reduce__`. This can be done by implementing a method named | |
| 518- | :meth:`__reduce_ex__` instead of :meth:`__reduce__`. :meth:`__reduce_ex__`, when | |
| 534+ | :meth:`__reduce_ex__` instead of :meth:`__reduce__`. :meth:`__reduce_ex__`, | |
| 519- | it exists, is called in preference over :meth:`__reduce__` (you may still | |
| 535+ | when it exists, is called in preference over :meth:`__reduce__` (you may | |
| 520- | provide :meth:`__reduce__` for backwards compatibility). The | |
| 536+ | still provide :meth:`__reduce__` for backwards compatibility). The | |
| 521- | :meth:`__reduce_ex__` method will be called with a single integer argument, the | |
| 537+ | :meth:`__reduce_ex__` method will be called with a single integer argument, | |
| 522- | protocol version. | |
| 538+ | the protocol version. | |
| 523 | ||
| n | 524- | The :class:`object` class implements both :meth:`__reduce__` and |
| n | 540+ | The :class:`object` class implements both :meth:`__reduce__` and |
| 525- | :meth:`__reduce_ex__`; however, if a subclass overrides :meth:`__reduce__` but | |
| 541+ | :meth:`__reduce_ex__`; however, if a subclass overrides :meth:`__reduce__` | |
| 526- | not :meth:`__reduce_ex__`, the :meth:`__reduce_ex__` implementation detects this | |
| 542+ | but not :meth:`__reduce_ex__`, the :meth:`__reduce_ex__` implementation | |
| 527- | and calls :meth:`__reduce__`. | |
| 543+ | detects this and calls :meth:`__reduce__`. | |
| 528 | ||
| 529 | An alternative to implementing a :meth:`__reduce__` method on the object to be | |
| 530 | pickled, is to register the callable with the :mod:`copy_reg` module. This | |
| 531 | module provides a way for programs to register "reduction functions" and | |
| 532 | constructors for user-defined types. Reduction functions have the same | |
| 533 | semantics and interface as the :meth:`__reduce__` method described above, except | |
| 534 | that they are called with a single argument, the object to be pickled. | |
| 535 | ||
| 536 | The registered constructor is deemed a "safe constructor" for purposes of | |
| 537 | unpickling as described above. | |
| 538 | ||
| 539 | ||
| 540 | Pickling and unpickling external objects | |
| 541 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| n | 558+ | |
| 559+ | .. index:: | |
| 560+ | single: persistent_id (pickle protocol) | |
| 561+ | single: persistent_load (pickle protocol) | |
| 542 | ||
| 543 | For the benefit of object persistence, the :mod:`pickle` module supports the | |
| 544 | notion of a reference to an object outside the pickled data stream. Such | |
| 545 | objects are referenced by a "persistent id", which is just an arbitrary string | |
| 546 | of printable ASCII characters. The resolution of such names is not defined by | |
| 547 | the :mod:`pickle` module; it will delegate this resolution to user defined | |
| 548 | functions on the pickler and unpickler. [#]_ | |
| 549 | ||
| 613 | In the :mod:`cPickle` module, the unpickler's :attr:`persistent_load` attribute | |
| 614 | can also be set to a Python list, in which case, when the unpickler reaches a | |
| 615 | persistent id, the persistent id string will simply be appended to this list. | |
| 616 | This functionality exists so that a pickle data stream can be "sniffed" for | |
| 617 | object references without actually instantiating all the objects in a pickle. | |
| 618 | [#]_ Setting :attr:`persistent_load` to a list is usually used in conjunction | |
| 619 | with the :meth:`noload` method on the Unpickler. | |
| 620 | ||
| n | 621- | .. % BAW: Both pickle and cPickle support something called |
| n | 641+ | .. BAW: Both pickle and cPickle support something called inst_persistent_id() |
| 622- | .. % inst_persistent_id() which appears to give unknown types a second | |
| 642+ | which appears to give unknown types a second shot at producing a persistent | |
| 623- | .. % shot at producing a persistent id. Since Jim Fulton can't remember | |
| 643+ | id. Since Jim Fulton can't remember why it was added or what it's for, I'm | |
| 624- | .. % why it was added or what it's for, I'm leaving it undocumented. | |
| 644+ | leaving it undocumented. | |
| 625 | ||
| 626 | ||
| 627 | .. _pickle-sub: | |
| 628 | ||
| 629 | Subclassing Unpicklers | |
| 630 | ---------------------- | |
| n | 651+ | |
| 652+ | .. index:: | |
| 653+ | single: load_global() (pickle protocol) | |
| 654+ | single: find_global() (pickle protocol) | |
| 631 | ||
| 632 | By default, unpickling will import any class that it finds in the pickle data. | |
| 633 | You can control exactly what gets unpickled and what gets called by customizing | |
| 634 | your unpickler. Unfortunately, exactly how you do this is different depending | |
| 635 | on whether you're using :mod:`pickle` or :mod:`cPickle`. [#]_ | |
| 636 | ||
| 637 | In the :mod:`pickle` module, you need to derive a subclass from | |
| 638 | :class:`Unpickler`, overriding the :meth:`load_global` method. | |
| 684 | # Pickle dictionary using protocol 0. | |
| 685 | pickle.dump(data1, output) | |
| 686 | ||
| 687 | # Pickle the list using the highest protocol available. | |
| 688 | pickle.dump(selfref_list, output, -1) | |
| 689 | ||
| 690 | output.close() | |
| 691 | ||
| n | 692- | The following example reads the resulting pickled data. When reading a pickle- |
| n | 716+ | The following example reads the resulting pickled data. When reading a |
| 693- | containing file, you should open the file in binary mode because you can't be | |
| 717+ | pickle-containing file, you should open the file in binary mode because you | |
| 694- | sure if the ASCII or binary format was used. :: | |
| 718+ | can't be sure if the ASCII or binary format was used. :: | |
| 695 | ||
| 696 | import pprint, pickle | |
| 697 | ||
| 698 | pkl_file = open('data.pkl', 'rb') | |
| 699 | ||
| 700 | data1 = pickle.load(pkl_file) | |
| 701 | pprint.pprint(data1) | |
| 702 | ||
| 707 | ||
| 708 | Here's a larger example that shows how to modify pickling behavior for a class. | |
| 709 | The :class:`TextReader` class opens a text file, and returns the line number and | |
| 710 | line contents each time its :meth:`readline` method is called. If a | |
| 711 | :class:`TextReader` instance is pickled, all attributes *except* the file object | |
| 712 | member are saved. When the instance is unpickled, the file is reopened, and | |
| 713 | reading resumes from the last location. The :meth:`__setstate__` and | |
| 714 | :meth:`__getstate__` methods are used to implement this behavior. :: | |
| n | 739+ | |
| 740+ | #!/usr/local/bin/python | |
| 715 | ||
| 716 | class TextReader: | |
| 717 | """Print and number lines in a text file.""" | |
| 718 | def __init__(self, file): | |
| 719 | self.file = file | |
| 720 | self.fh = open(file) | |
| 721 | self.lineno = 0 | |
| 722 | ||
| 729 | line = line[:-1] | |
| 730 | return "%d: %s" % (self.lineno, line) | |
| 731 | ||
| 732 | def __getstate__(self): | |
| 733 | odict = self.__dict__.copy() # copy the dict since we change it | |
| 734 | del odict['fh'] # remove filehandle entry | |
| 735 | return odict | |
| 736 | ||
| n | 737- | def __setstate__(self,dict): |
| n | 763+ | def __setstate__(self, dict): |
| 738 | fh = open(dict['file']) # reopen file | |
| 739 | count = dict['lineno'] # read from file... | |
| 740 | while count: # until line count is restored | |
| 741 | fh.readline() | |
| 742 | count = count - 1 | |
| 743 | self.__dict__.update(dict) # update attributes | |
| 744 | self.fh = fh # save the file object | |
| 745 | ||
| 746 | A sample usage might be something like this:: | |
| 747 | ||
| 748 | >>> import TextReader | |
| 749 | >>> obj = TextReader.TextReader("TextReader.py") | |
| 750 | >>> obj.readline() | |
| 751 | '1: #!/usr/local/bin/python' | |
| n | 752- | >>> # (more invocations of obj.readline() here) |
| 753- | ... obj.readline() | |
| 778+ | >>> obj.readline() | |
| 779+ | '2: ' | |
| 780+ | >>> obj.readline() | |
| 754- | '7: class TextReader:' | |
| 781+ | '3: class TextReader:' | |
| 755 | >>> import pickle | |
| n | 756- | >>> pickle.dump(obj,open('save.p','w')) |
| n | 783+ | >>> pickle.dump(obj, open('save.p', 'wb')) |
| 757 | ||
| 758 | If you want to see that :mod:`pickle` works across Python processes, start | |
| 759 | another Python session, before continuing. What follows can happen from either | |
| 760 | the same process or a new process. :: | |
| 761 | ||
| 762 | >>> import pickle | |
| n | 763- | >>> reader = pickle.load(open('save.p')) |
| n | 790+ | >>> reader = pickle.load(open('save.p', 'rb')) |
| 764 | >>> reader.readline() | |
| t | 765- | '8: "Print and number lines in a text file."' |
| t | 792+ | '4: """Print and number lines in a text file."""' |
| 766 | ||
| 767 | ||
| 768 | .. seealso:: | |
| 769 | ||
| 770 | Module :mod:`copy_reg` | |
| 771 | Pickle interface constructor registration for extension types. | |
| 772 | ||
| 773 | Module :mod:`shelve` | |
| Legends | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||