Posted tagged ‘cluster’ need contribution

November 30, 2011


Introduction is a Python implementation of the MapReduce distributed computing framework. is:

  • Lightweight – All of the code is contained in a single Python file (currently weighing in at <13kB) that depends only on the Python Standard Library. Any computer with Python and can be a part of your cluster.
  • Fault tolerant – Workers (clients) can join and leave the cluster at any time without affecting the entire process. (Master checkpointing coming in future versions)
  • Secure – authenticates both ends of every connection, ensuring that only authorized code is executed. (TLS support coming in future versions)
  • Open source – is distributed under the MIT License, and consequently is free for all use, including commercial, personal, and academic, and can be modified and redistributed without restriction.


  • Just (v 0.1.2)
  • The full 0.1.2 release (includes documentation and examples)
  • Clone the git repository
    git clone



The following features will be included in by version 1.0:

  • On disk storage of worker data
  • Worker-to-worker transfer of the intermediate data
  • Master checkpointing
  • TLS


Get in touch with me at michaelfairley @

Patches are welcome, especially for the roadmapped features. It’s best to contact me to make sure that your potential work fits the goals of the project and has not already been started.