Pushy 0.5 Released

After just a few short months since the 0.4 release, I am pleased to announce Pushy 0.5. As usual, this release is mostly bug fixes, with a couple of features to make life easier. Instructions on downloading and installing can be found here:
    http://awilkins.id.au/pushy/install.php

On top of standard bug fixes and features, I have made an effort to beautify some of the code, and speed everything up in general, based on code profiling. This will be evident if your Pushy program performs a lot of proxying: 0.5 performs up to ~2x  faster than 0.4 in my tests.


New Features

There are two new features in Pushy 0.5: with-statement support, and a simple “execute” interface.

With-statement support
Pushy connections now support the “with” statement. This is functionally equivalent to wrapping a Pushy connection with “contextlib.closing”: it will close the connection when the with-statement exits. For example:

with pushy.connect(“local:“) as con:
    …

Compile/execute interface
Previously if you wanted to execute a statement in the remote interpreter, you would have to first obtain the remote “compile” built-in function, invoke it to compile a remote code object, and then evaluate that with the “connection.eval” method. For example, to execute “print ‘Hello, World!’”:

code_obj = con.eval(“compile”)(“print ‘Hello, World!’”, “<filename>”, “exec”)
con.eval(code_obj, locals=…, globals=…)

This is a bit unfriendly, so I thought it would be a good idea to add a simpler interface. There are two new methods: “connection.compile” and “connection.execute”. The former will compile a remote code object, and the latter executes either a string (statement) or a function. Continuing our very simple example, we get the much simpler:

con.execute(“print ‘Hello, World!’”)

Under the hood, this will do exactly what we would previously have had to do manually: remotely compile the statement and then evaluate the resultant code object. Now that suffices for a very simple use case like we have discussed above, but what if we want to execute a series of statements? Wouldn’t it be nice if we could remotely execute a locally defined function? Well, now you can, using “connection.compile”.

def local_function(*args, **kwargs):
    return (args, kwargs)
remote_function = con.compile(local_function)
remote_function(…)

The “connection.compile” method will take a locally defined function and define it in the remote interpreter. It will then return a reference to the remote function, so that we can invoke it as if it were a local function. This allows the user to define complex and/or expensive logic that will be executed entirely remotely, only communicating back to the peer when proxy objects are involved, or to return the result.


Bug Fixes

The most noteworthy bugs fixed in 0.5 are:

#738216 Proxied old-style classes can’t be instantiated
Old-style/classic classes (i.e. those that don’t inherit from “object”) previously could not be instantiated via a Pushy connection. This has been rectified by defining an additional proxy type for old-style classes.


#733026 auto-importer doesn’t support modules not imported by their parent
Previously, importing a remote submodule via “connection.modules” would only work if the parent of the submodule imported it. For example, “connection.modules.os.path” would work since os imports os.path. If the parent does not import the submodule, Pushy would fail to import the module. This has been fixed in 0.5; remotely imported modules will provide the same sort of automagical importing interface as “connection.modules”.


#734311 Proxies are strongly referenced and never deleted; del isn’t transmitted to delete proxied object
This is the most significant change in Pushy 0.5. Since inception, Pushy has never released proxy objects, meaning eventual memory bloating for objects that would otherwise by garbage collected. As of 0.5, Pushy will now garbage collect proxies by default. Every so often (by default, 5 seconds), Pushy will send a message to the peer to let it know which proxies have been garbage collected. This allows the peer to release the proxied objects, and reclaim resources.


#773811 Daemon/socket transport is slow
Due to the many small messages sent back and forth by Pushy, the “daemon” transport was found to be quite slow in stress testing. This is due to Nagle’s Algorithm, which delays sending network packets so that they can be combined. This has a considerable impact on the latency of Pushy’s operations, and so it is now disabled by Pushy.

Enjoy!