mincemeat.py is a Python implementation of the MapReduce distributed computing framework.
- Lightweight – All of the code is contained in a single Python file (currently weighing in at <13kB) that depends only on the Python Standard Library. Any computer with Python and mincemeat.py can be a part of your cluster.
- Fault tolerant – Workers (clients) can join and leave the cluster at any time without affecting the entire process. (Master checkpointing coming in future versions)
- Secure – mincemeat.py authenticates both ends of every connection, ensuring that only authorized code is executed. (TLS support coming in future versions)
- Open source – mincemeat.py is distributed under the MIT License, and consequently is free for all use, including commercial, personal, and academic, and can be modified and redistributed without restriction.
- Just mincemeat.py (v 0.1.2)
- The full 0.1.2 release (includes documentation and examples)
- Clone the git repository
git clone http://mincemeatpy.com/git/mincemeatpy.git
The following features will be included in mincemeat.py by version 1.0:
- On disk storage of worker data
- Worker-to-worker transfer of the intermediate data
- Master checkpointing
Get in touch with me at michaelfairley @ gmail.com.
Patches are welcome, especially for the roadmapped features. It’s best to contact me to make sure that your potential work fits the goals of the project and has not already been started.