Nairaland Forum

Maintaining a list of useful Python libraries here.

1. Numpy/Scipy (scientific programming)
2. Pandas (working with csv data files)

wmi (for managing windows OS)
Matplotlib (plotting simple to advanced graphs)

scikit-learn for machine learning

PIL - Python Image Library
Mutagen - advanced multimedia tagging system, including ID3v2, ID3v1

pygame for game development.

- Pip: python package manager
- Ipython: Development shell
- Sphinx: Generate documentation for your code.

Tutorial on creation python packages: http://guide.python-distribute.org/creation.html

ipython notebook...quite useful for presenting calculations you do to others.

pip, virtualenv, sqlalchemy, pyqt, django, tablib, requests, mongoengine, pymongo, ipython

boto for interacting with Amazon's web services.

Mrjob for writing map/reduce jobs

OK AWS is probably legit and probably what I'd go for if I was building a company from scratch.

But using a private hadoop cluster is so much more convenient

ekt_bear: OK AWS is probably legit and probably what I'd go for if I was building a company from scratch.

But using a private hadoop cluster is so much more convenient

Just curious, what kind of setup would you use for a minimal private hadoop installation (that would still be capable of performing optimally)?

You can run Hadoop on your own personal machine. But like I said, if you don't have access to some sort of computing cluster, but need to do large data processing, I'd suggest AWS.