Double or nothing: Speed up your Geoprocessing with Parallel Python


If you’re not willing to wait for the next generation of CPUs and you’re not afraid to donate some of your spare time to the bleeding edge, then you might want to consider attempting to leverage the power of multiple processors by writing parallel code.

Python’s Global Interpreter Lock confines Python code to a single processing core. Erlang leverages processing in systems with multiple processors (SMPs) however its standard libraries are virtually useless for anything that needs to handle geospatial data.  Parallel Python (PP) however is a Python module which provides a mechanism for parallel execution of Python code on SMPs and clusters (computers connected via network). It is light and easy to install and integrate with other Python software.

This could mean that your app will potentially run twice as fast if you parallelize your code. Only time and testing will tell.

A good testing ground for parallel processing is any application that has to process many arrays of data and whose individual members need to be handled independently.The potentials for faster raster processing for example could be enormous.

For information, examples and downloads of PP:

http://www.parallelpython.com/

Ricardo A.

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s