The Object Space

Introduction

The object space creates all objects in PyPy, and knows how to perform operationson them. It may be helpful to think of an object space as being a libraryoffering a fixed API: a set of operations, along with implementations thatcorrespond to the known semantics of Python objects.

For example, add() is an operation, with implementations in the objectspace that perform numeric addition (when add() is operating on numbers),concatenation (when add() is operating on sequences), and so on.

We have some working object spaces which can be plugged intothe bytecode interpreter:

• The Standard Object Space is a complete implementationof the various built-in types and objects of Python. The Standard ObjectSpace, together with the bytecode interpreter, is the foundation of our Pythonimplementation. Internally, it is a set of interpreter-level classesimplementing the various application-level objects – integers, strings,lists, types, etc. To draw a comparison with CPython, the Standard ObjectSpace provides the equivalent of the C structures PyIntObject,PyListObject, etc.
• various Object Space proxies wrap another object space (e.g. the standardone) and adds new capabilities, like lazily computed objects (computed onlywhen an operation is performed on them), security-checking objects,distributed objects living on several machines, etc.

The various object spaces documented here can be found in pypy/objspace.

Note that most object-space operations take and returnapplication-level objects, which are treated asopaque “black boxes” by the interpreter. Only a very few operations allow thebytecode interpreter to gain some knowledge about the value of anapplication-level object.

Object Space Interface

This is the public API that all Object Spaces implement:

Administrative Functions

• getexecutioncontext()
• Return the currently active execution context.(pypy/interpreter/executioncontext.py).

• getbuiltinmodule(name)
• Return a Module object for the built-in module given by name.(pypy/interpreter/module.py).

Operations on Objects in the Object Space

These functions both take and return “wrapped” (i.e. application-level) objects.

The following functions implement operations with straightforward semantics thatdirectly correspond to language-level constructs:

id, type, issubtype, iter, next, repr, str, len, hash,

getattr, setattr, delattr, getitem, setitem, delitem,

pos, neg, abs, invert, add, sub, mul, truediv, floordiv, div, mod, divmod, pow, lshift, rshift, and, or, xor,

nonzero, hex, oct, int, float, long, ord,

lt, le, eq, ne, gt, ge, cmp, coerce, contains,

inplace_add, inplace_sub, inplace_mul, inplace_truediv, inplace_floordiv, inplace_div, inplace_mod, inplace_pow, inplace_lshift, inplace_rshift, inplace_and, inplace_or, inplace_xor,

get, set, delete, userdel

• call(w_callable, w_args, w_kwds)
• Calls a function with the given positional (w_args) and keyword (w_kwds)arguments.

• index(w_obj)
• Implements index lookup (as introduced in CPython 2.5) using wobj. Will return awrapped integer or long, or raise a TypeError if the object doesn’t have an_index() special method.

• is(_w_x, w_y)
• Implements w_x is w_y.

• isinstance(w_obj, w_type)
• Implements issubtype() with type(w_obj) and w_type as arguments.

Convenience Functions

The following functions are used so often that it was worthwhile to introducethem as shortcuts – however, they are not strictly necessary since they can beexpressed using several other object space methods.

• eqw(_w_obj1, w_obj2)
• Returns True when w_obj1 and w_obj2 are equal.Shortcut for space.is_true(space.eq(w_obj1, w_obj2)).

• isw(_w_obj1, w_obj2)
• Shortcut for space.istrue(space.is(w_obj1, w_obj2)).

• hashw(_w_obj)
• Shortcut for space.int_w(space.hash(w_obj)).

• lenw(_w_obj)
• Shortcut for space.int_w(space.len(w_obj)).

NOTE that the above four functions return interpreter-levelobjects, not application-level ones!

• not(_w_obj)
• Shortcut for space.newbool(not space.is_true(w_obj)).

• finditem(w_obj, w_key)
• Equivalent to getitem(w_obj, w_key) but returns an interpreter-level Noneinstead of raising a KeyError if the key is not found.

• callfunction(_w_callable, *args_w, **kw_w)
• Collects the arguments in a wrapped tuple and dict and invokesspace.call(w_callable, …).

• callmethod(_w_object, 'method', …)
• Uses space.getattr() to get the method object, and then space.call_function()to invoke it.

• unpackiterable(w_iterable[, expected_length=-1])
• Iterates over w_x (using space.iter() and space.next())and collects the resulting wrapped objects in a list. If expected_length isgiven and the length does not match, raises an exception.

Of course, in cases where iterating directly is better than collecting theelements in a list first, you should use space.iter() and space.next()directly.

• unpacktuple(w_tuple[, expected_length=None])
• Equivalent to unpackiterable(), but only for tuples.

• callable(w_obj)
• Implements the built-in callable().

Creation of Application Level objects

• wrap(x)
• Deprecated! Eventually this method should disappear.Returns a wrapped object that is a reference to the interpreter-level objectx. This can be used either on simple immutable objects (integers,strings, etc) to create a new wrapped object, or on instances of W_Rootto obtain an application-level-visible reference to them. For example,most classes of the bytecode interpreter subclass W_Root and canbe directly exposed to application-level code in this way - functions, frames,code objects, etc.

• newint(i)
• Creates a wrapped object holding an integral value. newint creates an objectof type W_IntObject.

• newlong(l)
• Creates a wrapped object holding an integral value. The main difference to newintis the type of the argument (which is rpython.rlib.rbigint.rbigint). On PyPy3 thismethod will return an int (PyPy2 it returns a long).

• newbytes(t)
• The given argument is a rpython bytestring. Creates a wrapped objectof type bytes (both on PyPy2 and PyPy3).

• newtext(t)
• The given argument is a rpython bytestring. Creates a wrapped objectof type str. On PyPy3 this will return a wrapped unicodeobject. The object will hold a utf-8-nosg decoded value of t.The “utf-8-nosg” codec used here is slightly different from the“utf-8” implemented in Python 2 or Python 3: it is defined as utf-8without any special handling of surrogate characters. They areencoded using the same three-bytes sequence that encodes any char inthe range from '\u0800' to '\uffff'.

PyPy2 will return a bytestring object. No encoding/decoding steps will be applied.

• newbool(b)
• Creates a wrapped bool object from an interpreter-levelobject.

• newtuple([w_x, w_y, w_z, …])
• Creates a new wrapped tuple out of an interpreter-level list of wrapped objects.

• newlist([..])
• Creates a wrapped list from an interpreter-level list of wrapped objects.

• newdict()
• Returns a new empty dictionary.

• newslice(w_start, w_end, w_step)
• Creates a new slice object.

• newunicode(ustr)
• Creates a Unicode string from an rpython unicode string.This method may disappear soon and be replaced by :py:function::newutf8.

• newutf8(bytestr)
• Creates a Unicode string from an rpython byte string, decoded as“utf-8-nosg”. On PyPy3 it is the same as :py:function::newtext.

Many more space operations can be found in pypy/interpeter/baseobjspace.py andpypy/objspace/std/objspace.py.

Conversions from Application Level to Interpreter Level

• unwrap(w_x)
• Returns the interpreter-level equivalent of w_x – use thisONLY for testing, because this method is not RPython and thus cannot betranslated! In most circumstances you should use the functions describedbelow instead.

• istrue(_w_x)
• Returns a interpreter-level boolean (True or False) thatgives the truth value of the wrapped object w_x.

This is a particularly important operation because it is necessary to implement,for example, if-statements in the language (or rather, to be pedantic, toimplement the conditional-branching bytecodes into which if-statements arecompiled).

• intw(_w_x)
• If w_x is an application-level integer or long which can be convertedwithout overflow to an integer, return an interpreter-level integer. Otherwiseraise TypeError or OverflowError.

• bigintw(_w_x)
• If w_x is an application-level integer or long, return an interpreter-levelrbigint. Otherwise raise TypeError.

• ObjSpace.bytesw(_w_x)
• Takes an application level bytes(on PyPy2 this equals str) and returns a rpython byte string.

• ObjSpace.textw(_w_x)
• PyPy2 takes either a str and returns arpython byte string, or it takes an unicodeand uses the systems default encoding to return a rpythonbyte string.

On PyPy3 it takes a str and it will returnan utf-8 encoded rpython string.

• strw(_w_x)
• Deprecated. use text_w or bytes_w insteadIf w_x is an application-level string, return an interpreter-level string.Otherwise raise TypeError.

• unicodew(_w_x)
• Takes an application level :py:class::unicode and return aninterpreter-level unicode string. This method may disappear soon andbe replaced by :py:function::text_w.

• floatw(_w_x)
• If w_x is an application-level float, integer or long, return aninterpreter-level float. Otherwise raise TypeError (or:py:exc:_OverflowError_in the case of very large longs).

• getindexw(_w_obj[, w_exception=None])
• Call index(w_obj). If the resulting integer or long object can be convertedto an interpreter-level int, return that. If not, return a clampedresult if w_exception is None, otherwise raise the exception at theapplication level.

(If w_obj can’t be converted to an index, index() will raise anapplication-level TypeError.)

• interpw(_RequiredClass, w_x[, can_be_None=False])
• If w_x is a wrapped instance of the given bytecode interpreter class,unwrap it and return it. If can_be_None is True, a wrappedNone is also accepted and returns an interpreter-level None.Otherwise, raises an OperationError encapsulating a TypeErrorwith a nice error message.

• interpclassw(_w_x)
• If w_x is a wrapped instance of an bytecode interpreter class – forexample Function, Frame, Cell, etc. – returnit unwrapped. Otherwise return None.

Data Members

• space.builtin
• The Module containing the builtins.

• space.sys
• The sys Module.

• space.w_None
• The ObjSpace’s instance of None.

• space.w_True
• The ObjSpace’s instance of True.

• space.w_False
• The ObjSpace’s instance of False.

• space.w_Ellipsis
• The ObjSpace’s instance of Ellipsis.

• space.w_NotImplemented
• The ObjSpace’s instance of NotImplemented.

• space.w_int
• space.w_float
• space.w_long
• space.w_tuple
• space.w_str
• space.w_unicode
• space.w_type
• space.w_instance
• space.w_slice
• Python’s most common basic type objects.

• space.w_[XYZ]Error
• Python’s built-in exception classes (KeyError, IndexError,etc).

• ObjSpace.MethodTable
• List of tuples containing (method_name, symbol, number_of_arguments, list_of_special_names)for the regular part of the interface.

NOTE that tuples are interpreter-level.

• ObjSpace.BuiltinModuleTable
• List of names of built-in modules.

• ObjSpace.ConstantTable
• List of names of the constants that the object space should define.

• ObjSpace.ExceptionTable
• List of names of exception classes.

• ObjSpace.IrregularOpTable
• List of names of methods that have an irregular API (take and/or returnnon-wrapped objects).

The Standard Object Space

Introduction

The Standard Object Space (pypy/objspace/std/) is the direct equivalentof CPython’s object library (the Objects/ subdirectory in the distribution).It is an implementation of the common Python types in a lower-level language.

The Standard Object Space defines an abstract parent class, W_Objectas well as subclasses like W_IntObject, W_ListObject,and so on. A wrapped object (a “black box” for the bytecode interpreter’s mainloop) is an instance of one of these classes. When the main loop invokes anoperation (such as addition), between two wrapped objects w1 andw2, the Standard Object Space does some internal dispatching (similarto Object/abstract.c in CPython) and invokes a method of the properW_XYZObject class that can perform the operation.

The operation itself is done with the primitives allowed by RPython, and theresult is constructed as a wrapped object. For example, compare the followingimplementation of integer addition with the function int_add() inObject/intobject.c:

def add__Int_Int(space, w_int1, w_int2):
    x = w_int1.intval
    y = w_int2.intval
    try:
        z = ovfcheck(x + y)
    except OverflowError:
        raise FailedToImplementArgs(space.w_OverflowError,
                                space.wrap("integer addition"))
    return W_IntObject(space, z)

This may seem like a lot of work just for integer objects (why wrap them intoW_IntObject instances instead of using plain integers?), but thecode is kept simple and readable by wrapping all objects (from simple integersto more complex types) in the same way.

(Interestingly, the obvious optimization above has actually been made in PyPy,but isn’t hard-coded at this level – see Standard Interpreter Optimizations.)

Object types

The larger part of the pypy/objspace/std/ package defines andimplements the library of Python’s standard built-in object types. Each typexxx (int, float, list,tuple, str, type, etc.) is typicallyimplemented in the module xxxobject.py.

The W_AbstractXxxObject class, when present, is the abstract baseclass, which mainly defines what appears on the Python-level typeobject. There are then actual implementations as subclasses, which arecalled W_XxxObject or some variant for the cases where we haveseveral different implementations. For example,pypy/objspace/std/bytesobject.py defines W_AbstractBytesObject,which contains everything needed to build the str app-level type;and there are subclasses W_BytesObject (the usual string) andW_Buffer (a special implementation tweaked for repeatedadditions, in pypy/objspace/std/bufferobject.py). For mutable datatypes like lists and dictionaries, we have a single classW_ListObject or W_DictMultiObject which has an indirection tothe real data and a strategy; the strategy can change as the content ofthe object changes.

From the user’s point of view, even when there are severalW_AbstractXxxObject subclasses, this is not visible: at theapp-level, they are still all instances of exactly the same Python type.PyPy knows that (e.g.) the application-level type of itsinterpreter-level W_BytesObject instances is str because there is atypedef class attribute in W_BytesObject which points back tothe string type specification from pypy/objspace/std/bytesobject.py;all other implementations of strings use the same typedef frompypy/objspace/std/bytesobject.py.

For other examples of multiple implementations of the same Python type,see Standard Interpreter Optimizations.

Object Space proxies

We have implemented several proxy object spaces, which wrap another objectspace (typically the standard one) and add some capabilities to all objects. Tofind out more, see Transparent Proxies (DEPRECATED).