MapReduce近几年比较热的分布式计算编程模型，以C#为例简单介绍下MapReduce分布式计算。
阅读目录
背景
Map实现
Reduce实现
支持分布式
总结
背景
某平行世界程序猿小张接到Boss一项任务，统计用户反馈内容中的单词出现次数，以便分析用户主要习惯。文本如下：
const string hamlet = @"Though yet of Hamlet our dear brother's death The memory be green, and that it us befitted To bear our hearts in grief and our whole kingdom To be contracted in one brow of woe, Yet so far hath discretion fought with nature That we with wisest sorrow think on him, Together with remembrance of ourselves. Therefore our sometime sister, now our queen, The imperial jointress to this warlike state, Have we, as 'twere with a defeated joy,-- With an auspicious and a dropping eye, With mirth in funeral and with dirge in marriage, In equal scale weighing delight and dole,-- Taken to wife: nor have we herein barr'd Your better wisdoms, which have freely gone With this affair along. For all, our thanks. Now follows, that you know, young Fortinbras, Holding a weak supposal of our worth, Or thinking by our late dear brother's death Our state to be disjoint and out of frame, Colleagued with the dream of his advantage, He hath not fail'd to pester us with message, Importing the surrender of those lands Lost by his father, with all bonds of law, To our most valiant brother. So much for him. Now for ourself and for this time of meeting: Thus much the business is: we have here writ To Norway, uncle of young Fortinbras,-- Who, impotent and bed-rid, scarcely hears Of this his nephew's purpose,--to suppress His further gait herein; in that the levies, The lists and full proportions, are all made Out of his subject: and we here dispatch You, good Cornelius, and you, Voltimand, For bearers of this greeting to old Norway; Giving to you no further personal power To business with the king, more than the scope Of these delated articles allow. Farewell, and let your haste commend your duty."; |
小张作为蓝翔高材生，很快就实现了：
var content = hamlet.Split(new[] { " ", Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries);
var wordcount=new Dictionary<string,int>();
foreach (var item in content)
{
if (wordcount.ContainsKey(item))
wordcount[item] += 1;
else
wordcount.Add(item, 1);
}
作为有上进心的青年，小张决心对算法进行抽象封装，并支持多节点计算。小张把这个统计次数程序分成两个大步骤：分解和计算。
