Category Archives: infrastructure

Mapreduce in Search

Wrote about mapreduce in search in a presentation for next week. Mapreduce in Search (more up-to-date pdf version of the presentation) Best regards, Amund Atbrox

Posted in Atbrox, Hadoop and Mapreduce, infrastructure, search | Tagged , , | 2 Comments

Parallel Machine Learning for Hadoop/Mapreduce – A Python Example

Atbrox is startup providing technology and services for Search and Mapreduce/Hadoop. Our background is from from Google, IBM and Research. Update 2010-June-17 Code for this posting is now on github –http://github.com/atbrox/Snabler This posting gives an example of how to use … Continue reading

Posted in cloud computing, Hadoop and Mapreduce, infrastructure | Tagged , , , , , , , , | 14 Comments

How to combine Elastic Mapreduce/Hadoop with other Amazon Web Services

Elastic Mapreduce default behavior is to read from and store to S3. When you need to access other AWS services, e.g. SQS queues or database services SimpleDB and RDS (MySQL) the best approach from Python is to use Boto. To … Continue reading

Posted in cloud computing, Hadoop and Mapreduce, infrastructure | Tagged , , , , , , | 4 Comments

Mapreduce & Hadoop Algorithms in Academic Papers

The newest and most up-to-date version (May 2010) this blog post is available at http://mapreducebook.org An updated and extended version of this blog post can be found here. Motivation Learn from academic literature about how the mapreduce parallel model and … Continue reading

Posted in cloud computing, Hadoop and Mapreduce, infrastructure | Tagged , , , , , , | 1 Comment

How to get pip/virtualenv/Fabric working on Cygwin

If you are new to virtualenv, Fabric or pip is, Alex Clemesha’s excellent “Tools of the Modern Python Hacker” is a must-read. In short: virtualenv lets you switch seamlessly between isolated Python environments, Fabric automates remote deployment, while pip takes … Continue reading

Posted in infrastructure | Tagged , , , , , | 12 Comments