Double or nothing: Speed up your Geoprocessing with Parallel Python
If you’re not willing to wait for the next generation of CPUs and you’re not afraid to donate some of your spare time to the bleeding edge, then you might want to consider attempting to leverage the power of multiple processors by writing parallel code.
Python’s Global Interpreter Lock confines Python code to a single processing core. Erlang leverages processing in systems with multiple processors (SMPs) however its standard libraries are virtually useless for anything that needs to handle geospatial data. Parallel Python (PP) however is a Python module which provides a mechanism for parallel execution of Python code on SMPs and clusters (computers connected via network). It is light and easy to install and integrate with other Python software.
This could mean that your app will potentially run twice as fast if you parallelize your code. Only time and testing will tell.
A good testing ground for parallel processing is any application that has to process many arrays of data and whose individual members need to be handled independently.The potentials for faster raster processing for example could be enormous.
For information, examples and downloads of PP: