diff --git a/02.01 基于搜索的问题求解(学生版).ipynb b/02.01 基于搜索的问题求解(学生版).ipynb index e1f9951..e714502 100644 --- a/02.01 基于搜索的问题求解(学生版).ipynb +++ b/02.01 基于搜索的问题求解(学生版).ipynb @@ -11,6 +11,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "现实世界中许多问题都可以通过搜索的方法来求解，例如设计最佳出行路线或是制订合理的课程表。当给定一个待求解问题后，搜索算法会按照事先设定的逻辑来自动寻找符合求解问题的答案，因此一般可将搜索算法称为问题求解智能体。" ] }, @@ -26,6 +33,13 @@ "metadata": {}, "source": [ "## 2.1.1 搜索算法基本概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -86,45 +100,35 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "观察上图，可以看到从起点 A 到目标点 G 距离最短的路径是 A -> B -> D -> G，其距离是 38，我们可以设计一个计算机程序，按照既定的规则，从起点 A 出发，不断尝试从一个节点移动到下一个节点，直到抵达目标点 G。\n", - "\n", - "在详细描述搜索算法之前，先了解下面四个重要的概念。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "+ **状态**。 在上面的例子中状态是什么？" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "+ **测试目标**。 在上面的例子中测试目标是什么？" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "+ **动作**。在上面的例子中动作是什么？" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "+ **路径**。在上面的例子中路径是什么？\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "更改下面的代码，设置一条路径并查看。" + "观察上图，可以看到从起点 A 到目标点 G 距离最短的路径是 A -> B -> D -> G，其距离是 38，我们可以设计一个计算机程序，按照既定的规则，从起点 A 出发，不断尝试从一个节点移动到下一个节点，直到抵达目标点 G。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "搜索算法四个重要的概念分别是什么？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "更改下面的代码，设置一条路径并查看，并结合此图说明搜索算法的四个重要概念。" ] }, { @@ -147,11 +151,30 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "搜索算法就是不断从某一状态转移到下一状态，直至到达终止状态为止。\n", - "\n", - "**搜索树**是什么？\n", - "\n", - "如何构造搜索树 ？\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "搜索算法就是不断从某一状态转移到下一状态，直至到达终止状态为止。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "搜索树是什么？\n", + "\n", + "如何构造搜索树？\n", "\n", "思考一下，路径搜索能出现回路吗？\n", "\n", @@ -179,6 +202,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "" ] }, @@ -218,6 +248,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "对于一个搜索问题，只要存在答案（即从初始节点到终止节点存在满足条件的一条路径），那么深度优先搜索和广度优先搜索都能找到一个答案吗？找到的答案一定是路径最短的吗？" ] }, @@ -225,6 +262,45 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "### 扩展内容\n", + "** 深度优先搜索 dfs ** 基础代码解读\n", + "\n", + "```\n", + "def iter_dfs(G, start, target):\n", + " '''\n", + " 深度优先搜索\n", + " :param G: 字典，存储每个点的相邻点\n", + " :param start: 初始点\n", + " :param target: 目标点\n", + " :return:\n", + " '''\n", + "\n", + " # 定义已访问的点的集合\n", + " S = set()\n", + " # 定义一个待访问点的列表\n", + " Q = []\n", + " # 把初始点放进列表中\n", + " Q.append(start)\n", + " while Q:\n", + " # 只要带访问的列表不为空，那么从列表中拿取最后一个元素，也就是一个点，记作 u\n", + " u = Q.pop()\n", + " # 如果当前点是目标点，则结束查找\n", + " if u == target:\n", + " break\n", + " # 如果该点已经被访问了，则跳过此点\n", + " if u in S:\n", + " continue\n", + " # 访问此点，将点加入已访问点的结合 S 中\n", + " S.add(u)\n", + " # 将点 u 相邻的点放入待访问的列表中\n", + " Q.extend(G[u])\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.1.4 启发式搜索" ] }, @@ -232,7 +308,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "能否在搜索过程中利用问题的定义以外**辅助信息**？" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "能否在搜索过程中利用问题的定义以外的辅助信息？" ] }, { @@ -266,6 +356,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "“贪婪”机制下找到的最佳路径是什么呢？它是最短路径吗？为什么会产生这样的搜索结果？\n" ] }, @@ -273,10 +370,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "如何克服贪婪算法的不足？\n", - " A\\*算法\n", - "\n", - " A\\*算法搜索过程：\n" + "另一种启发式搜索算法—— A* 算法克服了贪婪算法的不足。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A\\*算法搜索过程：" ] }, { @@ -300,6 +408,13 @@ "# 可以调整辅助信息的比重\n", "# 当只考虑额外信息时，即 origin_info_weight 设置为 0 的时候，A* 算法退化为贪婪算法。\n", "g.animation_search_tree('a_star',help_info_weight=1, origin_info_weight=0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" ] }, { diff --git a/02.01 基于搜索的问题求解.ipynb b/02.01 基于搜索的问题求解.ipynb index f68ebf1..9298c5b 100644 --- a/02.01 基于搜索的问题求解.ipynb +++ b/02.01 基于搜索的问题求解.ipynb @@ -11,6 +11,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "现实世界中许多问题都可以通过搜索的方法来求解，例如设计最佳出行路线或是制订合理的课程表。当给定一个待求解问题后，搜索算法会按照事先设定的逻辑来自动寻找符合求解问题的答案，因此一般可将搜索算法称为问题求解智能体。" ] }, @@ -25,7 +39,28 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.1.1 搜索算法基本概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -86,8 +121,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "观察上图，可以看到从起点 A 到目标点 G 距离最短的路径是 A -> B -> D -> G，其距离是 38，我们可以设计一个计算机程序，按照既定的规则，从起点 A 出发，不断尝试从一个节点移动到下一个节点，直到抵达目标点 G。\n", - "\n", + "观察上图，可以看到从起点 A 到目标点 G 距离最短的路径是 A -> B -> D -> G，其距离是 38，我们可以设计一个计算机程序，按照既定的规则，从起点 A 出发，不断尝试从一个节点移动到下一个节点，直到抵达目标点 G。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "在详细描述搜索算法之前，先看看下面四个重要的概念。" ] }, @@ -110,22 +157,6 @@ "metadata": {}, "source": [ "+ **动作**。动作指的是搜索算法从一个状态转变到另外一个状态所采取的行为。一般假设在每个状态下所能够采取的行为数量都是有限的。例如：在起点 A，只有 B 和 C 两个节点与之相连，所以只有转移到 B 或者转移到 C 这两种选择。一般情况从一个状态到另外一个状态的过程叫做**状态转移**。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "下图中，我们在初始状态采取了转移到 B 这个动作。" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "g.show_graph(this_path=\"AB\")" ] }, { @@ -137,6 +168,13 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "看下图，理解上面的四个概念。" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -149,6 +187,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.1.2 搜索算法" ] }, @@ -156,6 +201,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "搜索算法就是不断从某一状态转移到下一状态，直至到达终止状态为止。\n", "\n", "\n", @@ -169,6 +228,13 @@ "outputs": [], "source": [ "g.show_search_tree()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -195,6 +261,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.1.3 深度优先搜索和广度优先搜索" ] }, @@ -202,6 +275,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "" ] }, @@ -229,6 +316,13 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -242,6 +336,13 @@ "metadata": {}, "source": [ "需要强调的是，对于一个搜索问题，只要存在答案（即从初始节点到终止节点存在满足条件的一条路径），那么排除了回路的深度优先搜索和广度优先搜索均能找到一个答案，但是这个找到的答案不一定是最优的，例如距离最短。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -287,7 +388,28 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.1.4 启发式搜索" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -328,6 +450,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "但是在“贪婪”机制下找到的路径 A -> C -> D -> G 并非最短路径。产生这样的搜索结果，其原因是：最佳优先算法在当前节点时，每次均贪婪的从当前节点相邻的节点中选择**与目标节点直线距离最近的节点**，作为后续节点。这样就会造成贪婪最佳优先算法**过于重视当前的最优，而忽视了全局最优**。\n" ] }, @@ -335,8 +464,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "另一种启发式搜索算法—— A\\* 算法克服了这一不足。\n", - "\n", + "另一种启发式搜索算法—— A\\* 算法克服了这一不足。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "其算法思路是：将初始节点到目标节点的距离分成两部分，\n", "- 初始节点到当前节点的路径代价；\n", "- 当前节点到目标节点之间的直线距离。将两者之和作为评价函数的取值大小。\n", @@ -346,7 +494,7 @@ "+ 函数 h(n): 表示当前节点 n 到目标节点的直线距离。函数 h(n) 也称为**启发函数**。\n", "\n", "\n", - " A\\* 算法搜索过程：\n" + " A\\* 算法搜索过程：" ] }, { @@ -362,6 +510,13 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -379,6 +534,13 @@ "与贪婪最佳优先算法不一定能够找到最短路径不同，A\\* 算法找到的路径一定是最短路径；另一方面，由于A\\* 算法能够利用辅助信息，因此它比其他算法用更少的步骤。\n", "\n", "在实际中，A\\* 算法的性能表现取决于启发函数的设计，只要定义一个合适的启发函数，A\\* 算法就能够大幅缩减搜索所需的时间。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -448,6 +610,13 @@ "source": [ "# 查看 dfs 的搜索过程\n", "h_graph.animation_search_tree('dfs')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { diff --git a/02.02 决策树(学生版).ipynb b/02.02 决策树(学生版).ipynb index 0dd0f17..40599d3 100644 --- a/02.02 决策树(学生版).ipynb +++ b/02.02 决策树(学生版).ipynb @@ -11,16 +11,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "决策树是一种通过**树形结构**进行分类的方法，使用层层推理来实现最终的分类。决策树由下面几种元素构成：\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "决策树的组成元素有哪些？" + "决策树是一种通过**树形结构**进行分类的方法，使用层层推理来实现最终的分类。\n", + "\n", + "决策树由下面几种元素构成：\n", + "+ 根节点：最顶层的分类条件。\n", + "+ 决策节点（中间节点）：中间分类条件。\n", + "+ 叶子节点：代表标签类别。\n", + "\n", + "\n" ] }, { @@ -32,7 +30,8 @@ "贷款用户主要具备三个属性：**是否拥有房产**，**是否结婚**，**平均月收入**。\n", "\n", "每一个内部节点都表示一个属性条件判断，叶子节点表示贷款用户是否具有偿还能力。\n", - "\n" + "\n", + "\n" ] }, { @@ -46,11 +45,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "决策树的流程是什么？\n", "\n", "在有一个贷款用户A，其情况是月收入 3K、已经结婚、没有房产，那么他是否具有偿还贷款的能力呢？ \n", "\n", - "上图中我们为啥要用“是否拥有房产”作根节点呢？可不可以用“是否结婚”和“平均月收入”做根节点呢？" + "上图中我们为什么要用“是否拥有房产”作根节点呢？可不可以用“是否结婚”和“平均月收入”做根节点呢？" ] }, { @@ -58,6 +64,13 @@ "metadata": {}, "source": [ "## 2.2.1 决策树分类概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -95,16 +108,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "根节点是天气状况，具有雨、多云和晴三种属性取值。\n", - "+ 多云: 样本子集是 { 3, 7, 12, 13 } ，仅有“前往游乐场游玩”一个类别，即肯定去游乐场。 \n", + "根节点是天气状况，具有 **雨**、 **多云** 和 **晴** 三种属性取值。\n", + "+ **多云**: 样本子集是 { 3, 7, 12, 13 } ，仅有“前往游乐场游玩”一个类别，即肯定去游乐场。 \n", " \n", " \n", - "+ 晴: 样本子集是 { 1, 2, 8, 9, 11 }\n", + "+ **晴**: 样本子集是 { 1, 2, 8, 9, 11 }\n", " + 湿度大于 75：样本子集为 { 1, 2, 8 }，不前往游乐场。\n", " + 湿度不大于 75：样本子集 { 9, 11 }，前往游乐场。\n", " \n", " \n", - "+ 雨：样本子集为 { 4, 5, 6, 10, 14 }\n", + "+ **雨**：样本子集为 { 4, 5, 6, 10, 14 }\n", " + 有风：样本子集 { 6, 14 },不去游乐场。\n", " + 无风：样本子集 { 4, 5, 10 }，前往游乐场。\n", " " @@ -114,22 +127,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "由上面的例子可以看到，构建决策树的过程就是：\n", - "1. 选择一个属性值；\n", - "2. 基于该属性对样本集进行划分；\n", - "3. 重复步骤 1 和 2 直到最后所得划分结果中每个样本为同一类别。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "把数据导入 DataFrame 数据结构：" - ] - }, - { - "cell_type": "code", - "execution_count": 1, + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "通过上面的例子，你观察到构建决策树的过程是哪几步？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下面，我们创建数据并进行一些预处理" + ] + }, + { + "cell_type": "code", + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -146,199 +163,29 @@ }, { "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
天气温度湿度是否有风是否前往游乐场
0晴>26>75否0
1晴<=26>75是0
2多云>26>75否1
3雨<=26>75否1
4雨<=26>75否1
5雨<=26<=75是0
6多云<=26<=75是1
7晴<=26>75否0
8晴<=26<=75否1
9雨<=26>75否1
10晴<=26<=75是1
11多云<=26>75是1
12多云>26<=75否1
13雨<=26>75是0
\n", - "
" - ], - "text/plain": [ - " 天气 温度 湿度 是否有风 是否前往游乐场\n", - "0 晴 >26 >75 否 0\n", - "1 晴 <=26 >75 是 0\n", - "2 多云 >26 >75 否 1\n", - "3 雨 <=26 >75 否 1\n", - "4 雨 <=26 >75 否 1\n", - "5 雨 <=26 <=75 是 0\n", - "6 多云 <=26 <=75 是 1\n", - "7 晴 <=26 >75 否 0\n", - "8 晴 <=26 <=75 否 1\n", - "9 雨 <=26 >75 否 1\n", - "10 晴 <=26 <=75 是 1\n", - "11 多云 <=26 >75 是 1\n", - "12 多云 >26 <=75 否 1\n", - "13 雨 <=26 >75 是 0" - ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# 原始数据\n", "datasets = [\n", - " ['晴',29,85,'否','0'],\n", - " ['晴',26,88,'是','0'],\n", - " ['多云',28,78,'否','1'],\n", - " ['雨',21,96,'否','1'],\n", - " ['雨',20,80,'否','1'],\n", - " ['雨',18,70,'是','0'],\n", - " ['多云',18,65,'是','1'],\n", - " ['晴',22,90,'否','0'],\n", - " ['晴',21,68,'否','1'],\n", - " ['雨',24,80,'否','1'],\n", - " ['晴',24,63,'是','1'],\n", - " ['多云',22,90,'是','1'],\n", - " ['多云',27,75,'否','1'],\n", - " ['雨',21,80,'是','0']\n", + " ['晴', 29, 85, '否', '0'],\n", + " ['晴', 26, 88, '是', '0'],\n", + " ['多云', 28, 78, '否', '1'],\n", + " ['雨', 21, 96, '否', '1'],\n", + " ['雨', 20, 80, '否', '1'],\n", + " ['雨', 18, 70, '是', '0'],\n", + " ['多云', 18, 65, '是', '1'],\n", + " ['晴', 22, 90, '否', '0'],\n", + " ['晴', 21, 68, '否', '1'],\n", + " ['雨', 24, 80, '否', '1'],\n", + " ['晴', 24, 63, '是', '1'],\n", + " ['多云', 22, 90, '是', '1'],\n", + " ['多云', 27, 75, '否', '1'],\n", + " ['雨', 21, 80, '是', '0']\n", "]\n", - "\n", "# 数据的列名\n", - "labels = ['天气','温度','湿度','是否有风','是否前往游乐场']\n", - "\n", + "labels = ['天气', '温度', '湿度', '是否有风', '是否前往游乐场']\n", "# 将湿度大小分为大于 75 和小于等于 75 这两个属性值，\n", "# 将温度大小分为大于 26 和小于等于 26 这两个属性值\n", "for i in range(len(datasets)):\n", @@ -350,32 +197,62 @@ " datasets[i][1] = '>26'\n", " else:\n", " datasets[i][1] = '<=26'\n", - "\n", "# 构建 dataframe 并查看数据\n", "df = pd.DataFrame(datasets, columns=labels)\n", - "df\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.2.2 构建决策树 \n", - "\n", - "**信息增益**是什么？\n", - "\n", - "**信息熵**是什么？" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "假设有 $K$ 个信息，其组成了集合样本 $D$ ，记第 $k$ 个信息发生的概率为$P_k(1≤k≤K)$。 \n", - "这 $K$ 个信息的信息熵： \n", - "$$E(D)=-\\sum_{k=1}^{K}p_k log_{2} p_k$$\n", - "\n", - "需要指出：**所有 $p_k$ 累加起来的和为1**。" + "df" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.2.2 构建决策树 " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "信息熵和信息增益分别是什么？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "假设有 $K$ 个信息，其组成了集合样本 $D$ ，记第 $k$ 个信息发生的概率为 $p_k(1≤k≤K)$。 \n", + "\n", + "这 $K$ 个信息的信息熵该如何计算？ \n", + "\n", + "所有 $p_k$ 累加起来的和是多少？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根据公式编写代码计算信息熵" ] }, { @@ -391,11 +268,8 @@ " :param count_dict: 每类样本及其对应数目的字典\n", " :return: 信息熵\n", " \"\"\"\n", - " # 使用信息熵公式计算\n", - " ent = -sum([(p / total_num) * log(p / total_num, 2) for p in count_dict.values() if p != 0])\n", - " # 避免 print 显示异常\n", - " if ent == 0:\n", - " ent = 0\n", + " # todo 使用公式计算信息熵\n", + " ent = \n", " # 返回信息熵，精确到小数点后 4 位\n", " return round(ent, 4)\n" ] @@ -408,7 +282,7 @@ "\n", "现在用**信息熵**来构建决策树。数据中 14 个样本分为 “游客来游乐场 （9 个样本）” 和 “游客不来游乐场（ 5 个样本）” 两个类别，即 K = 2。\n", "\n", - "记 “游客来游乐场” 和 “游客不来游乐场” 的概率分别为 $p_1$ 和 $p_2$ ,显然 $p_1=\\frac{9}{14}$，$p_1=\\frac{5}{14}$，则这 14 个样本所蕴含的信息熵：\n", + "记 “游客来游乐场” 和 “游客不来游乐场” 的概率分别为 $p_1$ 和 $p_2$ ，显然 $p_1=\\frac{9}{14}$，$p_1=\\frac{5}{14}$，则这 14 个样本所蕴含的信息熵：\n", "\n", "$$E(D)=-\\sum_{k=1}^{2}p_{k}log_{2}{p_k}=-(\\frac{9}{14}×log_{2}{\\frac{9}{14}}+\\frac{5}{14}×log_{2}{\\frac{5}{14}})=0.940$$" ] @@ -446,7 +320,7 @@ "# 总样本数\n", "total_num = df.shape[0]\n", "# 每类样本及其对应数目的字典\n", - "count_dict = {'前往':df[df['是否前往游乐场']=='1'].shape[0], '不前往':df[df['是否前往游乐场']=='1'].shape[1]}\n", + "count_dict = {'前往': df[df['是否前往游乐场']=='1'].shape[0], '不前往': df[df['是否前往游乐场']=='1'].shape[1]}\n", "# 计算信息熵\n", "entropy = calc_entropy(total_num, count_dict)\n", "entropy\n" @@ -456,9 +330,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**计算天气状况所对应的信息熵**： \n", "天气状况的三个属性记为 $a_0=“晴”$ ，$a_1=“多云”$ ，$a_2=“雨”$ ， \n", - "属性取值为 $a_i$ 对应分支节点所包含子样本集记为 $D_i$ ,该子样本集包含样本数量记为 $|D_i|$ 。" + "属性取值为 $a_i$ 对应分支节点所包含子样本集记为 $D_i$ ，该子样本集包含样本数量记为 $|D_i|$ 。" ] }, { @@ -483,7 +364,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "我们可以使用下面的写法，对 dataframe 进行多个条件的筛选。" + "现在，我们来编写代码进行完成上面的计算。\n", + "\n", + "首先，我们可以使用下面的写法，对 Dataframe 进行多个条件的筛选。" ] }, { @@ -576,6 +459,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "使用上面的公式计算信息增益。" ] }, @@ -585,10 +475,8 @@ "metadata": {}, "outputs": [], "source": [ - "# 信息增益\n", - "gain = entropy - (total_num_sun/total_num*ent_sun +\n", - " total_num_cloud/total_num*ent_cloud +\n", - " total_num_rain/total_num*ent_rain)\n", + "# todo 计算按天气状况分割的信息增益\n", + "gain = \n", "gain\n" ] }, @@ -596,6 +484,53 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "### 扩展内容\n", + "\n", + "**基尼指数**\n", + "\n", + "除了使用信息增益以外，我们也可以使用基尼指数来构建决策树。\n", + "\n", + "分类问题中，假设有 $K$ 个类，样本点属于第 $k$ 类的概率为 $p_{k}$，则概率分布的基尼指数定义为：\n", + "\n", + "$$\\operatorname{Gini}(p)=\\sum_{k=1}^{K} p_{k}\\left(1-p_{k}\\right)=1-\\sum_{k=1}^{K} p_{k}^{2}$$\n", + "\n", + "\n", + "对于给定的样本集合 $D$，其基尼指数为\n", + "\n", + "$$\\operatorname{Gini}(D)=1-\\sum_{k=1}^{K}\\left(\\frac{\\left|C_{k}\\right|}{|D|}\\right)^{2}$$\n", + "\n", + "这里，$C_{k}$ 是 $D$ 中属于第 $k$ 类的样本子集，$K$ 是类的个数。\n", + "\n", + "如果样本集合 $D$ 根据特征 $A$ 是否取某一可能值 $a$ 被分割为 $D_{1}$ 和 $D_{2}$ 两部分，即\n", + "\n", + "$$D_{1}=\\{(x, y) \\in D | A(x)=a\\}, \\quad D_{2}=D-D_{1}$$\n", + "\n", + "则在特征 $A$ 的条件下，集合 $D$ 的基尼指数定义为\n", + "\n", + "$$\\operatorname{Gini}(D, A)=\\frac{\\left|D_{1}\\right|}{|D|}\n", + "\\operatorname{Gini}\\left(D_{1}\\right)+\\frac{\\left|D_{2}\\right|}{|D|} \\operatorname{Gini}\\left(D_{2}\\right)$$\n", + "\n", + "基尼指数 $Gini(D)$ 表示集合 $D$ 的不确定性，基尼指数 $Gini(D, A)$ 表示经过分割后集合 $D$ 的不确定性。基尼指数值越大，样本集合的不确定性也就越大，这一点与信息熵相似。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "对于二分类问题，若样本点属于第 1 个类的概率是 $p$，则概率分布的基尼指数是多少？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### 思考与练习 " ] }, @@ -603,12 +538,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "1. 分别将天气状况、温度高低、湿度大小、风力强弱作为分支点来构建决策树，查看信息增益\n" - ] - }, - { - "cell_type": "code", - "execution_count": 4, + "1. 分别将天气状况、温度高低、湿度大小、风力强弱作为分支点来构建决策树，查看信息增益。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -617,7 +552,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -626,7 +561,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -663,7 +598,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "实际上是否如此呢？" + "实际上是否如此呢？你能否想到其他的切分方式？" ] }, { @@ -735,7 +670,8 @@ "from sklearn import tree\n", "from sklearn.tree import DecisionTreeClassifier\n", "# 初始化模型，可以调整 max_depth 来观察模型的表现\n", - "clf = tree.DecisionTreeClassifier(random_state=42, max_depth=2)\n", + "# 也可以调整 criterion 为 gini 来使用 gini 指数构建决策树\n", + "clf = tree.DecisionTreeClassifier(criterion='entropy', max_depth=2)\n", "# 训练模型\n", "clf = clf.fit(X_train, y_train)\n" ] @@ -770,7 +706,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "我们看模型在测试集上的表现" + "我们看模型在测试集上的表现。" ] }, { @@ -850,7 +786,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ diff --git a/02.02 决策树.ipynb b/02.02 决策树.ipynb index 2d96567..7b06586 100644 --- a/02.02 决策树.ipynb +++ b/02.02 决策树.ipynb @@ -11,29 +11,40 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "决策树是一种通过**树形结构**进行分类的方法，使用层层推理来实现最终的分类。决策树由下面几种元素构成：\n", - "\n", + "决策树是一种通过**树形结构**进行分类的方法，使用层层推理来实现最终的分类。\n", + "\n", + "决策树由下面几种元素构成：\n", "+ 根节点：最顶层的分类条件。\n", "+ 决策节点（中间节点）：中间分类条件。\n", "+ 叶子节点：代表标签类别。\n", "\n", - "\n", - "\n", - "\n", - "\n", - "预测时，在树的内部节点处用某一属性值进行判断，根据判断结果决定进入哪个分支节点，直到到达叶节点处，得到分类结果。\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "上面的说法过于抽象，下面来看一个实际的例子。构建一棵结构简单的决策树，用于预测贷款用户是否具有偿还贷款的能力。\n", + "\n", + "\n", + "预测时，在树的内部节点处用某一属性值进行判断，根据判断结果决定进入哪个分支节点，\n", + "\n", + "直到到达叶节点处，得到分类结果。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "上面的说法过于抽象，下面来看一个实际的例子。\n", + "\n", + "构建一棵结构简单的决策树，用于预测贷款用户是否具有偿还贷款的能力。\n", "\n", "贷款用户主要具备三个属性：**是否拥有房产**，**是否结婚**，**平均月收入**。\n", "\n", "每一个内部节点都表示一个属性条件判断，叶子节点表示贷款用户是否具有偿还能力。\n", - "\n", + "\n", + "\n", "\n" ] }, @@ -41,11 +52,29 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "首先判断贷款用户是否拥有房产，如果用户拥有房产，则说明该用户具有偿还贷款的能力；否则需要判断该用户是否结婚，如果已经结婚则具有偿还贷款的能力；否则需要判断该用户的收入大小，如果该用户月收入小于 4K 元，则该用户不具有偿还贷款的能力，否则该用户是具有偿还能力的。\n", - "\n", - "现在有一个贷款用户A，其情况是月收入 3K、已经结婚、没有房产，那么他是否具有偿还贷款的能力呢？很显然，他是具有偿还贷款能力的。\n", - "\n", - "那么，上图中我们为啥要用“是否拥有房产”作根节点呢？可不可以用“是否结婚”和“平均月收入”做根节点呢？学完本章即可知道答案。" + "首先判断贷款用户是否拥有房产，\n", + "如果用户拥有房产，则说明该用户具有偿还贷款的能力；\n", + "否则需要判断该用户是否结婚，\n", + "如果已经结婚则具有偿还贷款的能力；\n", + "否则需要判断该用户的收入大小，\n", + "如果该用户月收入小于 4K 元，\n", + "则该用户不具有偿还贷款的能力，\n", + "否则该用户是具有偿还能力的。\n", + "\n", + "现在有一个贷款用户A，其情况是月收入 3K、已经结婚、没有房产，那么他是否具有偿还贷款的能力呢？\n", + "\n", + "很显然，他是具有偿还贷款能力的。\n", + "\n", + "那么，上图中我们为什么要用“是否拥有房产”作根节点呢？可不可以用“是否结婚”和“平均月收入”做根节点呢？\n", + "\n", + "学完本章即可知道答案。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -53,6 +82,20 @@ "metadata": {}, "source": [ "## 2.2.1 决策树分类概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -90,16 +133,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "根节点是天气状况，具有雨、多云和晴三种属性取值。\n", - "+ 多云: 样本子集是 { 3, 7, 12, 13 } ，仅有“前往游乐场游玩”一个类别，即肯定去游乐场。 \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根节点是天气状况，具有 **雨**、**多云** 和 **晴** 三种属性取值。\n", + "+ **多云**: 样本子集是 { 3, 7, 12, 13 } ，仅有“前往游乐场游玩”一个类别，即肯定去游乐场。 \n", " \n", " \n", - "+ 晴: 样本子集是 { 1, 2, 8, 9, 11 }\n", + "+ **晴**: 样本子集是 { 1, 2, 8, 9, 11 }\n", " + 湿度大于 75：样本子集为 { 1, 2, 8 }，不前往游乐场。\n", " + 湿度不大于 75：样本子集 { 9, 11 }，前往游乐场。\n", " \n", " \n", - "+ 雨：样本子集为 { 4, 5, 6, 10, 14 }\n", + "+ **雨**：样本子集为 { 4, 5, 6, 10, 14 }\n", " + 有风：样本子集 { 6, 14 },不去游乐场。\n", " + 无风：样本子集 { 4, 5, 10 }，前往游乐场。\n", " " @@ -119,7 +169,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "首先我们读取数据" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下面，我们创建数据并进行一些预处理" ] }, { @@ -132,11 +189,10 @@ "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", - "\n", "import math\n", "from math import log\n", "import warnings\n", - "warnings.filterwarnings(\"ignore\")\n" + "warnings.filterwarnings(\"ignore\")" ] }, { @@ -147,25 +203,23 @@ "source": [ "# 原始数据\n", "datasets = [\n", - " ['晴',29,85,'否','0'],\n", - " ['晴',26,88,'是','0'],\n", - " ['多云',28,78,'否','1'],\n", - " ['雨',21,96,'否','1'],\n", - " ['雨',20,80,'否','1'],\n", - " ['雨',18,70,'是','0'],\n", - " ['多云',18,65,'是','1'],\n", - " ['晴',22,90,'否','0'],\n", - " ['晴',21,68,'否','1'],\n", - " ['雨',24,80,'否','1'],\n", - " ['晴',24,63,'是','1'],\n", - " ['多云',22,90,'是','1'],\n", - " ['多云',27,75,'否','1'],\n", - " ['雨',21,80,'是','0']\n", + " ['晴', 29, 85, '否', '0'],\n", + " ['晴', 26, 88, '是', '0'],\n", + " ['多云', 28, 78, '否', '1'],\n", + " ['雨', 21, 96, '否', '1'],\n", + " ['雨', 20, 80, '否', '1'],\n", + " ['雨', 18, 70, '是', '0'],\n", + " ['多云', 18, 65, '是', '1'],\n", + " ['晴', 22, 90, '否', '0'],\n", + " ['晴', 21, 68, '否', '1'],\n", + " ['雨', 24, 80, '否', '1'],\n", + " ['晴', 24, 63, '是', '1'],\n", + " ['多云', 22, 90, '是', '1'],\n", + " ['多云', 27, 75, '否', '1'],\n", + " ['雨', 21, 80, '是', '0']\n", "]\n", - "\n", "# 数据的列名\n", - "labels = ['天气','温度','湿度','是否有风','是否前往游乐场']\n", - "\n", + "labels = ['天气', '温度', '湿度', '是否有风', '是否前往游乐场']\n", "# 将湿度大小分为大于 75 和小于等于 75 这两个属性值，\n", "# 将温度大小分为大于 26 和小于等于 26 这两个属性值\n", "for i in range(len(datasets)):\n", @@ -177,18 +231,36 @@ " datasets[i][1] = '>26'\n", " else:\n", " datasets[i][1] = '<=26'\n", - "\n", "# 构建 dataframe 并查看数据\n", "df = pd.DataFrame(datasets, columns=labels)\n", - "df\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.2.2 构建决策树 \n", - "\n", + "df" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.2.2 构建决策树 " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**信息增益**用来衡量样本集合复杂度（不确定性）所减少的程度。 \n", "\n", "**信息熵**用来度量信息量的大小。从信息论的角度来看，对信息的度量等于计算信息不确定性的多少。 " @@ -198,11 +270,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "假设有 $K$ 个信息，其组成了集合样本 $D$ ，记第 $k$ 个信息发生的概率为$P_k(1≤k≤K)$。 \n", + "假设有 $K$ 个信息，其组成了集合样本 $D$ ，记第 $k$ 个信息发生的概率为 $p_k(1≤k≤K)$。 \n", "这 $K$ 个信息的信息熵： \n", "$$E(D)=-\\sum_{k=1}^{K}p_k log_{2} p_k$$\n", "\n", "需要指出：**所有 $p_k$ 累加起来的和为1**。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -221,10 +300,10 @@ " # 使用信息熵公式计算\n", " ent = -sum([(p / total_num) * log(p / total_num, 2) for p in count_dict.values() if p != 0])\n", " # 避免 print 显示异常\n", - " if ent == 0:\n", + " if not ent:\n", " ent = 0\n", - " # 返回信息熵，精确到小数点后 4 位\n", - " return round(ent, 4)\n" + " # 返回信息熵，精确到小数点后 3 位\n", + " return round(ent, 3)\n" ] }, { @@ -235,7 +314,7 @@ "\n", "现在用**熵**来构建决策树。数据中 14 个样本分为 “游客来游乐场（ 9 个样本）” 和 “游客不来游乐场（ 5 个样本）” 两个类别，即 K = 2。\n", "\n", - "记 “游客来游乐场” 和 “游客不来游乐场” 的概率分别为 $p_1$ 和 $p_2$ ,显然 $p_1=\\frac{9}{14}$，$p_1=\\frac{5}{14}$ ，则这 14 个样本所蕴含的信息熵：\n", + "记 “游客来游乐场” 和 “游客不来游乐场” 的概率分别为 $p_1$ 和 $p_2$ ，显然 $p_1=\\frac{9}{14}$，$p_1=\\frac{5}{14}$ ，则这 14 个样本所蕴含的信息熵：\n", "\n", "$$E(D)=-\\sum_{k=1}^{2}p_{k}log_{2}{p_k}=-(\\frac{9}{14}×log_{2}{\\frac{9}{14}}+\\frac{5}{14}×log_{2}{\\frac{5}{14}})=0.940$$" ] @@ -274,7 +353,7 @@ "total_num = df.shape[0]\n", "\n", "# 每类样本及其对应数目的字典\n", - "count_dict = {'前往':df[df['是否前往游乐场']=='1'].shape[0], '不前往':df[df['是否前往游乐场']=='1'].shape[1]}\n", + "count_dict = {'前往': df[df['是否前往游乐场']=='1'].shape[0], '不前往': df[df['是否前往游乐场']=='1'].shape[1]}\n", "\n", "# 计算信息熵\n", "entropy = calc_entropy(total_num, count_dict)\n", @@ -285,9 +364,30 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**计算天气状况所对应的信息熵**： \n", "天气状况的三个属性记为 $a_0=“晴”$ ，$a_1=“多云”$ ，$a_2=“雨”$ ， \n", - "属性取值为 $a_i$ 对应分支节点所包含子样本集记为 $D_i$ ,该子样本集包含样本数量记为 $|D_i|$ 。" + "属性取值为 $a_i$ 对应分支节点所包含子样本集记为 $D_i$ ，该子样本集包含样本数量记为 $|D_i|$ 。" ] }, { @@ -312,7 +412,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "我们可以使用下面的写法，对 dataframe 进行多个条件的筛选。" + "现在，我们来编写代码进行完成上面的计算。\n", + "\n", + "首先，我们可以使用下面的写法，对 Dataframe 进行多个条件的筛选。" ] }, { @@ -323,6 +425,20 @@ "source": [ "# 筛选出 天气为晴并且去游乐场的样本数据\n", "df[(df['天气']=='晴') & (df['是否前往游乐场']=='1')]\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "然后，我们是使用上面的筛选方法，分别计算不同天气下的信息熵。" ] }, { @@ -386,7 +502,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "计算天气状况的信息增益: \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "得到不同天气下的信息熵后，我们可以计算天气状况的信息增益: \n", "$$Gain(D,A)=E(D)-\\sum_{i}^{n}\\frac{|D_i|}{D}E(D)$$" ] }, @@ -405,7 +528,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "使用上面的公式计算信息增益。" + "我们来编写代码，使用上面的公式计算信息增益。" ] }, { @@ -425,6 +548,69 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 扩展内容\n", + "\n", + "**基尼指数**\n", + "\n", + "除了使用信息增益以外，我们也可以使用基尼指数来构建决策树。\n", + "\n", + "分类问题中，假设有 $K$ 个类，样本点属于第 $k$ 类的概率为 $p_{k}$，则概率分布的基尼指数定义为：\n", + "\n", + "$$\\operatorname{Gini}(p)=\\sum_{k=1}^{K} p_{k}\\left(1-p_{k}\\right)=1-\\sum_{k=1}^{K} p_{k}^{2}$$\n", + "\n", + "\n", + "对于给定的样本集合 $D$，其基尼指数为\n", + "\n", + "$$\\operatorname{Gini}(D)=1-\\sum_{k=1}^{K}\\left(\\frac{\\left|C_{k}\\right|}{|D|}\\right)^{2}$$\n", + "\n", + "这里，$C_{k}$ 是 $D$ 中属于第 $k$ 类的样本子集，$K$ 是类的个数。\n", + "\n", + "如果样本集合 $D$ 根据特征 $A$ 是否取某一可能值 $a$ 被分割为 $D_{1}$ 和 $D_{2}$ 两部分，即\n", + "\n", + "$$D_{1}=\\{(x, y) \\in D | A(x)=a\\}, \\quad D_{2}=D-D_{1}$$\n", + "\n", + "则在特征 $A$ 的条件下，集合 $D$ 的基尼指数定义为\n", + "\n", + "$$\\operatorname{Gini}(D, A)=\\frac{\\left|D_{1}\\right|}{|D|}\n", + "\\operatorname{Gini}\\left(D_{1}\\right)+\\frac{\\left|D_{2}\\right|}{|D|} \\operatorname{Gini}\\left(D_{2}\\right)$$\n", + "\n", + "基尼指数 $Gini(D)$ 表示集合 $D$ 的不确定性，基尼指数 $Gini(D, A)$ 表示经过分割后集合 $D$ 的不确定性。基尼指数值越大，样本集合的不确定性也就越大，这一点与信息熵相似。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "对于二分类问题，若样本点属于第 1 个类的概率是 $p$，则概率分布的基尼指数是多少？\n", + "\n", + "$$\\text { Gini }(p)=2 p(1-p)$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### 思考与练习 " ] }, @@ -432,7 +618,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "1. 分别将天气状况、温度高低、湿度大小、风力强弱作为分支点来构建决策树，查看信息增益\n" + "1. 在上面的游客去往游乐场的例子中，分别将温度高低、湿度大小、风力强弱作为分支点来构建决策树，查看信息增益。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "计算按温度高低进行切分的信息增益。" ] }, { @@ -464,6 +657,20 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "计算按湿度高低进行切分的信息增益。" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -492,6 +699,20 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "计算按风力强弱进行切分的信息增益。" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -523,7 +744,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "2. 每朵鸢尾花有萼片长度、萼片宽度、花瓣长度、花瓣宽度四个特征。现在需要根据这四个特征将鸢尾花分为杂色鸢尾花、维吉尼亚鸢尾和山鸢尾三类，试构造决策树进行分类。\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "2. 每朵鸢尾花有**萼片长度**、**萼片宽度**、**花瓣长度**、**花瓣宽度**四个特征。现在需要根据这四个特征将鸢尾花分为**杂色鸢尾花**、**维吉尼亚鸢尾**和**山鸢尾**三类，试构造决策树进行分类。\n", "\n", "|序号|萼片长度|萼片宽度|花瓣长度|花瓣宽度|种类|\n", "|:--:|:--:|:--:|:--:|:--:|:--:|\n", @@ -538,25 +766,32 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "观察上表中的五笔数据，我们可以看到 杂色鸢尾 和 维吉尼亚鸢尾 的花瓣宽度明显大于山鸢尾，所以可以通过判断花瓣宽度是否大于 0.7，来将山鸢尾区从其他两种鸢尾中区分出来。\n", - "\n", - "同时，杂色鸢尾 和 维吉尼亚鸢尾 的花瓣长度明显大于山鸢尾，所以也可以通过判断花瓣长度是否大于 2.4，来将山鸢尾区从其他两种鸢尾中区分出来。\n", - "\n", - "然后我们观察到 维吉尼亚鸢尾的花瓣长度明显大于杂色鸢尾，所以可以通过判断花瓣长度是否大于 4.75，来将杂色鸢尾和维吉尼亚鸢尾区分出来。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "实际上是否如此呢？" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "上面的表格只是 Iris 数据集的一小部分，完整的数据集包含 150 个数据样本，分为 3 类，每类 50 个数据，每个数据包含 4 个属性。即花萼长度，花萼宽度，花瓣长度，花瓣宽度4个属性。\n", + "观察上表中的五笔数据，我们可以看到 **杂色鸢尾** 和 **维吉尼亚鸢尾** 的花瓣宽度明显大于 **山鸢尾**，所以可以通过判断花瓣宽度是否大于 0.7，来将 **山鸢尾** 从其他两种鸢尾中区分出来。\n", + "\n", + "同时，**杂色鸢尾** 和 **维吉尼亚鸢尾** 的花瓣长度明显大于 **山鸢尾**，所以也可以通过判断花瓣长度是否大于 2.4，来将 **山鸢尾 **从其他两种鸢尾中区分出来。\n", + "\n", + "然后我们观察到 **维吉尼亚鸢尾** 的花瓣长度明显大于 **杂色鸢尾**，所以可以通过判断花瓣长度是否大于 4.75，来将 **杂色鸢尾** 和 **维吉尼亚鸢尾**区分出来。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "实际上是否如此呢？你能否想到其他的切分方式？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "上面的表格只是 Iris 数据集的一小部分，完整的数据集包含 150 个数据样本，分为 3 类，每类 50 个数据，每个数据包含 4 个属性。即**花萼长度**，**花萼宽度**，**花瓣长度**，**花瓣宽度**4个属性。\n", "\n", "我们使用 sklearn 工具包来构建决策树模型，先导入数据集。" ] @@ -580,9 +815,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "setosa 是山鸢尾，versicolor是杂色鸢尾，virginica是维吉尼亚鸢尾。\n", - "\n", - "sepal length， sepal width，petal length，petal width 分别是萼片长度，萼片宽度，花瓣长度，花瓣宽度。" + "setosa 是**山鸢尾**，versicolor是**杂色鸢尾**，virginica是**维吉尼亚鸢尾**。\n", + "\n", + "sepal length， sepal width，petal length，petal width 分别是**萼片长度**，**萼片宽度**，**花瓣长度**，**花瓣宽度**。" ] }, { @@ -609,6 +844,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "接下来，我们在训练集数据上训练决策树模型。" ] }, @@ -620,8 +862,9 @@ "source": [ "from sklearn import tree\n", "from sklearn.tree import DecisionTreeClassifier\n", - "# 初始化模型，可以调整 max_depth 来观察模型的表现\n", - "clf = tree.DecisionTreeClassifier(random_state=42, max_depth=2)\n", + "# 初始化模型，可以调整 max_depth 来观察模型的表现， \n", + "# 也可以调整 criterion 为 gini 来使用 gini 指数构建决策树\n", + "clf = tree.DecisionTreeClassifier(criterion='entropy', max_depth=2)\n", "# 训练模型\n", "clf = clf.fit(X_train, y_train)\n" ] diff --git a/02.03 回归分析(学生版).ipynb b/02.03 回归分析(学生版).ipynb index 639c0ea..3e63083 100644 --- a/02.03 回归分析(学生版).ipynb +++ b/02.03 回归分析(学生版).ipynb @@ -4,8 +4,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 2.3 回归分析\n", - "\n", + "# 2.3 回归分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**回归分析**：分析不同变量之间存在关系的研究。 \n", "**回归模型**：刻画不同变量之间关系的模型。" ] @@ -14,8 +19,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.3.1 回归分析的基本概念\n", - "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.3.1 回归分析的基本概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**数据**：下表给出了莫纳罗亚山从 1970 年到 2005 年间每 5 年的二氧化碳浓度，单位是百万分比浓度（parts per million，简称ppm）\n", "\n", "\n", @@ -47,7 +71,7 @@ "
\n", "\n", "\n", - "**目标**：分析时间年份和二氧化碳浓度之间的关联关系，由此预测2010年二氧化碳浓度。\n" + "**目标**：分析时间年份和二氧化碳浓度之间的关联关系，由此预测2010年二氧化碳浓度。" ] }, { @@ -85,9 +109,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.3.2 回归分析中参数计算\n", - "\n", - "最简单的线性回归是**一元线性回归模型**，只包含一个自变量 $x$ 和一个因变量 $y$，并且假定自变量和因变量之间存在 $y=ax+b$ 的线性关系。求解参数 $a$ 和 $b$，需要给定若干组 $(x,y)$ 数据，然后从这些数据出发来计算参数 $a$ 和 $b$。\n" + "## 2.3.2 回归分析中参数计算" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "最简单的线性回归是**一元线性回归模型**，只包含一个自变量 $x$ 和一个因变量 $y$，并且假定自变量和因变量之间存在 $y=ax+b$ 的线性关系。\n", + "\n", + "求解参数 $a$ 和 $b$，需要给定若干组 $(x,y)$ 数据，然后从这些数据出发来计算参数 $a$ 和 $b$。" ] }, { @@ -96,20 +134,35 @@ "source": [ "在一元线性回归模型中，最关键的问题是如何计算参数 $a$ 和参数 $b$ 使误差最小化。\n", "\n", - "最拟合直线 $y=ax+b$ 应该与这 8 组样本数据点距离都很近，最好的情况是这些样本数据点都在该直线上（不现实），让所有样本数据点离直线尽可能的近（被定义为预测数值和实际数值之间的差）。\n", - "\n", - "**预测值**：\n", - "\n", - "**真实值**：\n", - "\n", - "**残差**：\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "我们根据公式编写如下的方法来求解 $a$ 和 $b$。" + "最拟合直线 $y=ax+b$ 应该与这 8 组样本数据点距离都很近，最好的情况是这些样本数据点都在该直线上（不现实），让所有样本数据点离直线尽可能的近（被定义为预测数值和实际数值之间的差）。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "预测值，真实值，残差分别是什么？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根据书中的计算公式编写代码来求解 $a$ 和 $b$。" ] }, { @@ -125,8 +178,9 @@ " :param y: np array 格式的因变量\n", " :return: 系数 a 和 b\n", " \"\"\"\n", - " pass\n", + " # todo 完成求解参数 a，b 的代码\n", " return a, b\n", + "\n", "a, b = cal_a_b(x, y)\n", "print(a, b)" ] @@ -135,8 +189,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "综上：得到的预测莫纳罗亚山地区二氧化碳浓度的一元线性回归模型为：。 \n", - "我们可以据此绘制出拟合直线。" + "综上：得到的预测莫纳罗亚山地区二氧化碳浓度的一元线性回归模型是什么？ " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根据求解结果绘制出拟合直线。" ] }, { @@ -162,7 +222,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "然后我们可以对该地区1970年之前和2005年之后的二氧化碳浓度进行估算。" + "然后对该地区1970年之前和2005年之后的二氧化碳浓度进行估算。" ] }, { @@ -179,7 +239,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "填写你的最终的预测结果在下表中： \n", + "将你的最终的预测结果填写在下表中： \n", "\n", "\n", "\n", @@ -221,18 +281,9 @@ }, { "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[[1.53438095]]\n", - "[-2698.87714286]\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# 导入工具包\n", "import numpy as np\n", @@ -278,7 +329,7 @@ "\n", "\n", " \n", - "其中 J 是代价函数，$\\theta_{0},\\theta_{1}$ 是待求参数， α 是学习率，它决定了我们沿着能让代价函数下降程度最大的方向向下迈出的步子有多大。 " + "其中 $J$ 是代价函数，$\\theta_{0},\\theta_{1}$ 是待求参数， $α$ 是学习率，它决定了我们沿着能让代价函数下降程度最大的方向向下迈出的步子有多大。 " ] }, { @@ -310,7 +361,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 探究莫纳罗亚山地区二氧化碳与温度之间的关系\n", + "#### 探究莫纳罗亚山地区二氧化碳与温度之间的关系\n", "\n", "该地区 1970 年到 2005 年间每 5 年的二氧化碳浓度以及全球温度（相对于 1961 - 1990 年经过平滑处理的平均温度增长量）\n", "\n", @@ -562,7 +613,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**问题 1**：观察上图，𝑧 与时间 𝑥 之间是否存在线性关系？如果是，我们用上面写好的方法来求解系数。" + "**问题 1**：观察上图，$z$ 与时间 $x$ 之间是否存在线性关系？如果是，我们用上面写好的方法来求解系数。" ] }, { diff --git a/02.03 回归分析.ipynb b/02.03 回归分析.ipynb index 35630da..6d44540 100644 --- a/02.03 回归分析.ipynb +++ b/02.03 回归分析.ipynb @@ -4,8 +4,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 2.3 回归分析\n", - "\n", + "# 2.3 回归分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**回归分析**：分析不同变量之间存在关系的研究。 \n", "**回归模型**：刻画不同变量之间关系的模型。" ] @@ -14,8 +19,48 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.3.1 回归分析的基本概念\n", - "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.3.1 回归分析的基本概念" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 探究莫纳罗亚山地区二氧化碳与温度之间的关系" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**数据**：下表给出了莫纳罗亚山从 1970 年到 2005 年间每 5 年的二氧化碳浓度，单位是百万分比浓度（parts per million，简称ppm）\n", "\n", "
\n", @@ -47,7 +92,7 @@ "
\n", "\n", "\n", - "**目标**：分析时间年份和二氧化碳浓度之间的关联关系，由此预测2010年二氧化碳浓度。\n" + "**目标**：分析时间年份和二氧化碳浓度之间的关联关系，由此预测2010年二氧化碳浓度。" ] }, { @@ -85,9 +130,37 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.3.2 回归分析中参数计算\n", - "\n", - "最简单的线性回归是**一元线性回归模型**，只包含一个自变量 $x$ 和一个因变量 $y$，并且假定自变量和因变量之间存在 $y=ax+b$ 的线性关系。求解参数 $a$ 和 $b$，需要给定若干组 $(x,y)$ 数据，然后从这些数据出发来计算参数 $a$ 和 $b$。\n" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.3.2 回归分析中参数计算" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "最简单的线性回归是**一元线性回归模型**，只包含一个自变量 $x$ 和一个因变量 $y$，并且假定自变量和因变量之间存在 $y=ax+b$ 的线性关系。\n", + "\n", + "求解参数 $a$ 和 $b$，需要给定若干组 $(x,y)$ 数据，然后从这些数据出发来计算参数 $a$ 和 $b$。" ] }, { @@ -113,6 +186,13 @@ "$a=\\frac{x_{1}y_{1}+x_2y_2+...+x_8y_8-8\\overline{x}\\overline{y}}{x_{1}^{2}+x_{2}^{2}+...+x_{8}^{2}-8\\overline{x}^2}=1.5344$\n", "\n", "$b = \\overline{y}-a\\overline{x}=-2698.9$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -150,6 +230,7 @@ " n - len(x) * x_avarage * x_avarage)\n", " b = y_avarage - a * x_avarage\n", " return a, b\n", + "\n", "a, b = cal_a_b(x, y)\n", "print(a, b)" ] @@ -158,7 +239,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "综上：预测莫纳罗亚山地区二氧化碳浓度的一元线性回归模型为：$y=1.5344x-2698.9$。 \n", + "综上：预测莫纳罗亚山地区二氧化碳浓度的一元线性回归模型为：$y=1.5344x-2698.9$。 " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "我们可以据此绘制出拟合直线。" ] }, @@ -231,6 +325,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### 扩展内容\n", "\n", "**1.使用 sklearn 工具包构建回归模型**" @@ -293,7 +394,7 @@ "\n", "\n", " \n", - "其中 J 是代价函数，$\\theta_{0},\\theta_{1}$ 是待求参数， α 是学习率，它决定了我们沿着能让代价函数下降程度最大的方向向下迈出的步子有多大。 " + "其中 $J$ 是代价函数，$\\theta_{0},\\theta_{1}$ 是待求参数， $α$ 是学习率，它决定了我们沿着能让代价函数下降程度最大的方向向下迈出的步子有多大。" ] }, { @@ -325,7 +426,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 探究莫纳罗亚山地区二氧化碳与温度之间的关系\n", + "#### 探究莫纳罗亚山地区二氧化碳与温度之间的关系\n", "\n", "该地区 1970 年到 2005 年间每 5 年的二氧化碳浓度以及全球温度（相对于 1961 - 1990 年经过平滑处理的平均温度增长量）\n", "\n", @@ -576,7 +677,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "我们看到 𝑧 与时间 𝑥 之间是否存在线性关系，我们用上面写好的方法来求解系数。" + "我们看到 $z$ 与时间 $x$ 之间是否存在线性关系，我们用上面写好的方法来求解系数。" ] }, { diff --git a/02.04 贝叶斯分析(学生版).ipynb b/02.04 贝叶斯分析(学生版).ipynb index 5af8fc2..88639d1 100644 --- a/02.04 贝叶斯分析(学生版).ipynb +++ b/02.04 贝叶斯分析(学生版).ipynb @@ -4,7 +4,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 2.4 贝叶斯分析\n", + "# 2.4 贝叶斯分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "贝叶斯分析是一种根据概率统计知识对数据进行分析的方法，属于统计学分类的范畴。" ] }, @@ -12,44 +18,54 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.4.1 贝叶斯公式\n", - "\n", - "- **频率学派**\n", - "- **贝叶斯学派**\n", - "\n", - "贝叶斯概率计算公式表达：\n", - "后验概率 = \n", - "\n", - "**条件概率**:\n", - "\n", - "$P(A|B)$：\n", - "\n", - "$P(A|B)$：\n", - "\n", - "$P(B|A)$：\n", - "\n", - "$P(B|A)$：\n", - "\n", - "由于：\n", - "\n", - "$$P(B|A){P(A)}=P(A|B){P(B)}={P(A ∩ B)}$$\n", - "\n", - "可得**贝叶斯公式**:\n", + "## 2.4.1 贝叶斯公式" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "频率学派 和 贝叶斯学派各自的主张是什么？\n", + "\n", + "**贝叶斯公式**:\n", "\n", "$$P(A|B) = \\frac{P(B|A)P(A)}{P(B)}$$\n", "\n", - "其中：\n", - "- $P(A)$ 是事件 $A$ 发生的先验概率，与事件 $B$ 是否发生无关；\n", - "- $P(B|A)$是事件 $A$ 发生前提下，事件 $B$ 发生的概率，也称为**似然概率**； \n", - "- $P(B)$ 是事件 $B$ 发生的先验概率，也称为**标准化常量**；\n", - "- $P(A|B)$是事件 $B$ 发生前提下，事件 $A$ 发生的概率，也是 $A$ 的后验概率。 " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.4.2 贝叶斯推断\n", + "中的各项分别代表什么意思？ " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.4.2 贝叶斯推断" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "贝叶斯推断是一种基于贝叶斯公式进行分析的统计学方法。" ] }, @@ -57,7 +73,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "小例子：根据邮件中的 “红包” 字样判别该邮件是不是垃圾邮件" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 根据邮件中的 “红包” 字样判别该邮件是不是垃圾邮件\n" ] }, { @@ -113,9 +136,9 @@ "source": [ "### 扩展内容\n", "\n", - "**化验结果为阳性就代表你真的患病了吗？**\n", - "\n", - "某同学 A 身体不舒服，去医院作了验血检查，看他是否得了 X 疾病，检查结果居然为阳性，他吓了一跳，赶紧上网查询。他看到网上有资料说，实验总是有误差的，这种实验有“百分之一的假阳性率和百分之一的假阴性率”。也就是说，在确实得了 X 疾病的人里面， 会有 1% 的人是假阴性，99％的人是真阳性， 也就是会有 。而没得病的人去做检查，有 1% 的人是假阳性，99％ 的人是真阴性。 于是，他认为，既然误检的概率这么低，那么他确实患病的概率应该是非常高的。\n", + "#### 化验结果为阳性就代表你真的患病了吗？\n", + "\n", + "某同学 A 身体不舒服，去医院作了验血检查，看他是否得了 X 疾病，检查结果居然为阳性，他吓了一跳，赶紧上网查询。他看到网上有资料说，实验总是有误差的，这种实验有“百分之一的假阳性率和百分之一的假阴性率”。也就是说，在确实得了 X 疾病的人里面， 会有 1% 的人是假阴性，99％的人是真阳性， 也就是会有 1% 的几率被误诊为没病。而没得病的人去做检查，有 1% 的人是假阳性，99％ 的人是真阴性，也就是会有 1% 的几率被误诊为有病。 于是，他认为，既然误检的概率这么低，那么他确实患病的概率应该是非常高的。\n", "\n", "可是，医生却告诉他，他被感染的概率只有 0.09 左右。这是怎么回事呢？\n", "\n", @@ -128,6 +151,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "$A$ ： 普通人患 X 病\n", "\n", "$B$ ： 化验结果为阳性\n", @@ -140,34 +170,48 @@ "\n", "$P(B|A)$：一个人患 X 病，其检测结果为阳性的概率， 99%\n", "\n", - "根据**贝叶斯公式**:\n", - "\n", - "$$\n", - "\\begin{align}\n", - "&P(A|B)=\\frac{P(B|A){P(A)}}{P(B)}\\\\\n", - "=&\\frac{99\\%*(1/1000)}{99\\%*(1/1000) + 1\\%*(999/1000)}\\\\\n", - "=&\\frac{99}{1098}\\\\\n", - "≈ & 9\\%\n", - "\\end{align}\n", - "$$\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.4.3 朴素贝叶斯分类器 \n", - "一种常用的分类算法，其基本假设为 。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "小例子：预测同学会不会在某店铺订餐。\n", - "\n", - "**目标**：根据某同学的订单记录，如果向他推荐一家“价位低、口味偏甜、距离远”的店铺，判断他会下单吗？\n", - "\n", + "根据**贝叶斯公式**，计算 $P(A|B)$\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.4.3 朴素贝叶斯分类器 " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**想一想**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "朴素贝叶斯分类器做为一种常用的分类算法，其基本假设是什么？" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 根据某同学的订单记录，判断其是否会对某店铺下单" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**数据**：该同学的下单记录如下\n", "\n", "|店铺价位|店铺口味|店铺距离|是否下单|\n", @@ -179,7 +223,9 @@ "|低|偏甜|近|是|\n", "|低|偏甜|近|是|\n", "|低|清淡|远|否|\n", - "|低|偏辣|远|是|\n" + "|低|偏辣|远|是|\n", + "\n", + "**目标**：根据某同学的订单记录，如果向他推荐一家“价位低、口味偏甜、距离远”的店铺，判断他会下单吗？" ] }, { @@ -260,6 +306,13 @@ "$$\n", "\n", "上述两个计算公式分母相同，对计算结果不影响，因此就从计算过程中略去了。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -361,6 +414,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "3.根据 **MNIST** 训练集训练朴素贝叶斯分类器" ] }, @@ -446,6 +506,13 @@ "\n", " print(\"数字 %s 的样本个数：%4s，预测正确的个数：%4s，准确率：%.4s%%\" % (\n", " i, class_num[i], predict_num[i][i], class_accuracy[i]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { diff --git a/02.04 贝叶斯分析.ipynb b/02.04 贝叶斯分析.ipynb index f75c292..eb12026 100644 --- a/02.04 贝叶斯分析.ipynb +++ b/02.04 贝叶斯分析.ipynb @@ -4,7 +4,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# 2.4 贝叶斯分析\n", + "# 2.4 贝叶斯分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "贝叶斯分析是一种根据概率统计知识对数据进行分析的方法，属于统计学分类的范畴。" ] }, @@ -12,8 +18,27 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.4.1 贝叶斯公式\n", - "\n", + "## 2.4.1 贝叶斯公式" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "- **频率学派**：从历史数据中计算某个事件的概率，认为只要采样足够多，则事件发生的频率就可以无限逼近真实概率。\n", "- **贝叶斯学派**：认为某个事件发生的概率不仅与先前这个事件发生的概率相关（称为**先验概率**），也与后期计算该事件概率时所观测的“新近”信息有关（称为**似然概率**）\n", "\n", @@ -49,7 +74,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.4.2 贝叶斯推断\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.4.2 贝叶斯推断" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "贝叶斯推断是一种基于贝叶斯公式进行分析的统计学方法。" ] }, @@ -57,7 +95,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "小例子：根据邮件中的 “红包” 字样判别该邮件是不是垃圾邮件" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "根据邮件中的 “红包” 字样判别该邮件是不是垃圾邮件" ] }, { @@ -111,11 +163,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### 扩展内容\n", "\n", - "**化验结果为阳性就代表你真的患病了吗？**\n", - "\n", - "某同学 A 身体不舒服，去医院作了验血检查，看他是否得了 X 疾病，检查结果居然为阳性，他吓了一跳，赶紧上网查询。他看到网上有资料说，实验总是有误差的，这种实验有“百分之一的假阳性率和百分之一的假阴性率”。也就是说，在确实得了 X 疾病的人里面， 会有 1% 的人是假阴性，99％的人是真阳性， 也就是会有 。而没得病的人去做检查，有 1% 的人是假阳性，99％ 的人是真阴性。 于是，他认为，既然误检的概率这么低，那么他确实患病的概率应该是非常高的。\n", + "#### 化验结果为阳性就代表你真的患病了吗？\n", + "\n", + "某同学 A 身体不舒服，去医院作了验血检查，看他是否得了 X 疾病，检查结果居然为阳性，他吓了一跳，赶紧上网查询。他看到网上有资料说，实验总是有误差的，这种实验有“百分之一的假阳性率和百分之一的假阴性率”。也就是说，在确实得了 X 疾病的人里面， 会有 1% 的人是假阴性，99％的人是真阳性， 也就是会有 1% 的几率被误诊为没病。而没得病的人去做检查，有 1% 的人是假阳性，99％ 的人是真阴性，也就是会有 1% 的几率被误诊为有病。 于是，他认为，既然误检的概率这么低，那么他确实患病的概率应该是非常高的。\n", "\n", "可是，医生却告诉他，他被感染的概率只有 0.09 左右。这是怎么回事呢？\n", "\n", @@ -156,7 +215,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 2.4.3 朴素贝叶斯分类器 \n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.4.3 朴素贝叶斯分类器 " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "一种常用的分类算法，其假设**样本各个特征之间相互独立、互不影响**。" ] }, @@ -164,10 +236,20 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "小例子：预测同学会不会在某店铺订餐。\n", - "\n", - "**目标**：根据某同学的订单记录，如果向他推荐一家“价位低、口味偏甜、距离远”的店铺，判断他会下单吗？\n", - "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "**数据**：该同学的下单记录如下\n", "\n", "|店铺价位|店铺口味|店铺距离|是否下单|\n", @@ -179,7 +261,9 @@ "|低|偏甜|近|是|\n", "|低|偏甜|近|是|\n", "|低|清淡|远|否|\n", - "|低|偏辣|远|是|\n" + "|低|偏辣|远|是|\n", + "\n", + "**目标**：根据某同学的订单记录，如果向他推荐一家“价位低、口味偏甜、距离远”的店铺，判断他会下单吗？" ] }, { @@ -260,6 +344,13 @@ "$$\n", "\n", "上述两个计算公式分母相同，对计算结果不影响，因此就从计算过程中略去了。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -361,6 +452,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "3.根据 **MNIST** 训练集训练朴素贝叶斯分类器" ] }, @@ -446,6 +544,13 @@ "\n", " print(\"数字 %s 的样本个数：%4s，预测正确的个数：%4s，准确率：%.4s%%\" % (\n", " i, class_num[i], predict_num[i][i], class_accuracy[i]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -521,7 +626,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 扩展阅读\n", + "## 扩展阅读" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "1. [一文读懂概率论学习：贝叶斯理论](https://www.jiqizhixin.com/articles/2019-11-21)\n", "2. [朴素贝叶斯法讲解](https://www.bilibili.com/video/av57126177?from=search&seid=1588787263892359481)\n", "3. [sklearn 贝叶斯方法](https://scikit-learn.org/stable/modules/naive_bayes.html)" diff --git a/02.05 神经网络学习(学生版).ipynb b/02.05 神经网络学习(学生版).ipynb index e3d4dca..f4ad448 100644 --- a/02.05 神经网络学习(学生版).ipynb +++ b/02.05 神经网络学习(学生版).ipynb @@ -19,6 +19,13 @@ "metadata": {}, "source": [ "## 2.5.1 人脑神经机制" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -45,60 +52,54 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "人工智能中神经网络正是体现“逐层抽象、渐进学习”机制的学习模型。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "人眼在辨识图片时，会先提取边缘特征，再识别部件，最后再得到最高层的模式。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.5.2 感知机模型" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**感知机模型**：\n", + "\n", "\n", " \n", - " \n", - " \n", - " \n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "人工智能中神经网络正是体现“逐层抽象、渐进学习”机制的学习模型。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "人眼在辨识图片时，会先提取边缘特征，再识别部件，最后再得到最高层的模式。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.5.2 感知机模型" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**感知机模型**：\n", - "\n", - "\n", - " \n", - " \n", - " \n", + " \n", + " \n", " \n", "
\n", - "
" ] @@ -223,7 +224,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "我们根据上面的定义可以编写一个简单的感知机模型。" + "**动手练**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "编写一个简单的感知机模型。" ] }, { @@ -263,19 +271,108 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "神经网络架构示意图如下：\n", + "\n", "\n", "\n", "与感知机的不同，神经网络：\n", "+ 输入层和输出层之间存在若干隐藏层。\n", "+ 每个隐藏层中包含若干神经元。\n", "\n", - "通过下面几个小视频，你会更好的理解神经网络：\n", - "
\n", - "
\n", - "
\n", - "
\n", - "
\n", - "
\n" + "通过下面几个小视频，你会更好的理解神经网络：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 扩展内容\n", + "\n", + "**卷积神经网络**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -367,25 +464,18 @@ " \"\"\"\n", " # 选择模型，选择序贯模型（Sequential())\n", " model = Sequential()\n", - " \n", " # 添加全连接层，共 512 个神经元\n", " model.add(Dense(512,input_shape=(784,),kernel_initializer='he_normal'))\n", - " \n", " # 添加激活层，激活函数选择 relu \n", " model.add(Activation('relu'))\n", - " \n", " # 添加全连接层，共 512 个神经元\n", " model.add(Dense(512,kernel_initializer='he_normal'))\n", - " \n", " # 添加激活层，激活函数选择 relu \n", " model.add(Activation('relu'))\n", - " \n", " # 添加全连接层，共 10 个神经元\n", " model.add(Dense(nb_classes))\n", - " \n", " # 添加激活层，激活函数选择 softmax\n", " model.add(Activation('softmax'))\n", - " \n", " return model\n", "\n", "model = create_model()" @@ -413,16 +503,12 @@ " \"\"\"\n", " # 编译模型\n", " model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n", - " \n", " # 模型训练\n", " model.fit(X_train, y_train, epochs=5, batch_size=64, verbose=1, validation_split=0.05)\n", - " \n", " # 保存模型\n", " model.save(model_path)\n", - " \n", " # 模型评估，获取测试集的损失值和准确率\n", " loss, accuracy = model.evaluate(X_test, y_test)\n", - "\n", " # 打印结果\n", " print('Test loss:', loss)\n", " print(\"Accuracy:\", accuracy)\n", @@ -556,13 +642,6 @@ "2. [利用 pix2pix 将你的草图变成图片](https://momodel.cn/explore/5c0cb5df1afd945819064752?type=app)\n", "3. [自动生成图片描述](https://momodel.cn/explore/5ba33f578fe30b412042ac08?&type=app&tab=1)" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { diff --git a/02.05 神经网络学习.ipynb b/02.05 神经网络学习.ipynb index bb5a5d9..e19e995 100644 --- a/02.05 神经网络学习.ipynb +++ b/02.05 神经网络学习.ipynb @@ -19,6 +19,20 @@ "metadata": {}, "source": [ "## 2.5.1 人脑神经机制" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -38,67 +52,82 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "高层的特征是低层特征的组合，从低层到高层的特征表示越来越抽象，越来越能表现语义。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "人工智能中神经网络正是体现“逐层抽象、渐进学习”机制的学习模型。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "人眼在辨识图片时，会先提取边缘特征，再识别部件，最后再得到最高层的模式。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2.5.2 感知机模型" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**感知机模型**：\n", + "\n", "\n", " \n", - " \n", - " \n", - " \n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "高层的特征是低层特征的组合，从低层到高层的特征表示越来越抽象，越来越能表现语义。\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "人工智能中神经网络正是体现“逐层抽象、渐进学习”机制的学习模型。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "人眼在辨识图片时，会先提取边缘特征，再识别部件，最后再得到最高层的模式。" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2.5.2 感知机模型" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**感知机模型**：\n", - "\n", - "\n", - " \n", - " \n", - " \n", + " \n", + " \n", " \n", "
\n", - "
" ] @@ -122,7 +151,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "在输出的判断上，其实不仅可以简单的按照阈值来判断，可以通过一个函数来进行计算，这个函数称为激活函数。常见的激活函数有： sigmoid，tanh，relu 等。下面我们看看这些激活函数的曲线图。" + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在输出的判断上，其实不仅可以简单的按照阈值来判断，可以通过一个函数来进行计算，这个函数称为**激活函数**。\n", + "\n", + "常见的激活函数有： sigmoid，tanh，relu 等。\n", + "\n", + "下面我们看看这些激活函数的曲线图。" ] }, { @@ -163,6 +203,13 @@ ] }, { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { "cell_type": "code", "execution_count": null, "metadata": {}, @@ -216,6 +263,13 @@ "\n", "# 绘制 relu 函数\n", "plot_activation_function(relu)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -258,6 +312,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 2.5.3 神经网络" ] }, @@ -265,19 +326,136 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "神经网络架构示意图如下：\n", + "\n", "\n", "\n", "与感知机的不同，神经网络：\n", "+ 输入层和输出层之间存在若干隐藏层。\n", "+ 每个隐藏层中包含若干神经元。\n", "\n", - "通过下面几个小视频，你会更好的理解神经网络：\n", - "
\n", - "
\n", - "
\n", - "
\n", - "
\n", - "
" + "通过下面几个小视频，你会更好的理解神经网络：" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 扩展内容\n", + "\n", + "**卷积神经网络**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -353,6 +531,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "3. 搭建神经网络模型" ] }, @@ -369,28 +554,28 @@ " \"\"\"\n", " # 选择模型，选择序贯模型（Sequential())\n", " model = Sequential()\n", - " \n", " # 添加全连接层，共 512 个神经元\n", " model.add(Dense(512,input_shape=(784,),kernel_initializer='he_normal'))\n", - " \n", " # 添加激活层，激活函数选择 relu \n", " model.add(Activation('relu'))\n", - " \n", " # 添加全连接层，共 512 个神经元\n", " model.add(Dense(512,kernel_initializer='he_normal'))\n", - " \n", " # 添加激活层，激活函数选择 relu \n", " model.add(Activation('relu'))\n", - " \n", " # 添加全连接层，共 10 个神经元\n", " model.add(Dense(nb_classes))\n", - " \n", " # 添加激活层，激活函数选择 softmax\n", " model.add(Activation('softmax'))\n", - " \n", " return model\n", "\n", "model = create_model()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -415,20 +600,16 @@ " \"\"\"\n", " # 编译模型\n", " model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n", - " \n", " # 模型训练\n", " model.fit(X_train, y_train, epochs=5, batch_size=64, verbose=1, validation_split=0.05)\n", - " \n", " # 保存模型\n", " model.save(model_path)\n", - " \n", " # 模型评估，获取测试集的损失值和准确率\n", " loss, accuracy = model.evaluate(X_test, y_test)\n", - "\n", " # 打印结果\n", " print('Test loss:', loss)\n", " print(\"Accuracy:\", accuracy)\n", - "\n", + " \n", "# 训练模型和评估模型\n", "fit_and_predict(model, model_path='./model.h5')" ] @@ -437,6 +618,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "### 实践与体验\n", "#### 调节神经网络结构和参数\n", "\n", @@ -456,24 +644,18 @@ " \"\"\"\n", " # 选择模型，选择序贯模型（Sequential())\n", " model = Sequential()\n", - "\n", " # 添加全连接层，共 512 个神经元\n", " model.add(Dense(512, input_shape=(784,), kernel_initializer='he_normal'))\n", - "\n", " # 添加激活层，激活函数选择 relu\n", " model.add(Activation('relu'))\n", - "\n", " # 添加全连接层，共 10 个神经元\n", " model.add(Dense(nb_classes))\n", - "\n", " # 添加激活层，激活函数选择 softmax\n", " model.add(Activation('softmax'))\n", - "\n", " return model\n", "\n", "# 搭建神经网络\n", "model1 = create_model1()\n", - "\n", "# 训练神经网络模型，保存模型和评估模型\n", "fit_and_predict(model1, model_path='./model1.h5')" ] @@ -482,6 +664,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "2. 修改两层隐藏层神经元的数量，然后训练模型得出准确率。" ] }, @@ -498,33 +687,31 @@ " \"\"\"\n", " # 选择模型，选择序贯模型（Sequential())\n", " model = Sequential()\n", - "\n", " # 添加全连接层，共 256 个神经元\n", " model.add(Dense(256, input_shape=(784,), kernel_initializer='he_normal'))\n", - "\n", " # 添加激活层，激活函数选择 relu\n", " model.add(Activation('relu'))\n", - "\n", " # 添加全连接层，共 256 个神经元\n", " model.add(Dense(256, kernel_initializer='he_normal'))\n", - "\n", " # 添加激活层，激活函数选择 relu\n", " model.add(Activation('relu'))\n", - "\n", " # 添加全连接层，共 10 个神经元\n", " model.add(Dense(nb_classes))\n", - "\n", " # 添加激活层，激活函数选择 softmax\n", " model.add(Activation('softmax'))\n", - "\n", " return model\n", "\n", "# 搭建神经网络模型\n", "model2 = create_model2()\n", - "\n", "# 训练神经网络模型，保存模型并评估模型\n", - "fit_and_predict(model2,model_path='./model2.h5')\n", - "\n" + "fit_and_predict(model2,model_path='./model2.h5')\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "
" ] }, { @@ -571,18 +758,18 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "## 扩展阅读\n", "1. [利用starGan算法改变人物的面部特征](https://momodel.cn/explore/5c0cc4591afd945c5177fb51?type=app)\n", "2. [利用 pix2pix 将你的草图变成图片](https://momodel.cn/explore/5c0cb5df1afd945819064752?type=app)\n", "3. [自动生成图片描述](https://momodel.cn/explore/5ba33f578fe30b412042ac08?&type=app&tab=1)" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { diff --git a/Untitled.ipynb b/Untitled.ipynb new file mode 100644 index 0000000..a9f96f3 --- /dev/null +++ b/Untitled.ipynb @@ -0,0 +1,603 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import networkx as nx" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "def bfs_search(G, max_depth, start_node, target_node):\n", + " # 待访问的路径\n", + " to_search = [(start_node, 0)]\n", + " # 存储所有的历史路径，及此路径的距离\n", + " bfs_path = []\n", + " # 正确的路径列表，及此路径的距离\n", + " bfs_correct_path = []\n", + " # 当还有待访问的路径时\n", + " while to_search:\n", + " # 从待访问的路径中取第一个待访问路径及其路径长度，例如 AC\n", + " this_path, this_path_dis = to_search.pop(0)\n", + " # 如果待访问的路径达到最大搜索深度，跳出循环\n", + " if len(this_path) > max_depth :\n", + " break\n", + " # 把刚取出的路径存入历史路径中\n", + " bfs_path.append((this_path, this_path_dis))\n", + " # 如果路径的最后一个节点是目标节点，路径 AC 的最后一个节点是 C\n", + " if this_path[-1] == target_node:\n", + " # 其为一条正确的路径，将存入正确的路径列表中，\n", + " # 并不再继续往其子节点进行探索\n", + " bfs_correct_path.append((this_path, this_path_dis))\n", + " continue\n", + " # 找到路径最后一个节点的相邻节点\n", + " for ne in sorted(G[this_path[-1]]):\n", + " # 如果相邻节点不在路径中，即不存在回路\n", + " if ne not in this_path:\n", + " # 则加入到待访问的路径中\n", + " to_search.append((this_path + ne,\n", + " this_path_dis + G[this_path[-1]][ne][\n", + " 'weight']))\n", + " return bfs_path, bfs_correct_path" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "# 定义节点列表\n", + "node_list = ['A', 'B', 'C', 'D', 'E', 'F', 'G']\n", + "\n", + "# 定义边及权重列表\n", + "weighted_edges_list = [('A', 'B', 8), ('A', 'C', 20),\n", + " ('B', 'F', 40), ('B', 'E', 30),\n", + " ('B', 'D', 20), ('C', 'D', 10), \n", + " ('D', 'G', 10), ('D', 'E', 10),\n", + " ('E', 'F', 30), ('F', 'G', 30)]\n", + "\n", + "# 定义绘图中各个节点的坐标\n", + "nodes_pos = {\"A\": (1, 1), \"B\": (3, 3), \"C\": (5, 0), \"D\": (9, 2),\n", + " \"E\": (7, 4), \"F\": (6,6),\"G\": (11,5)}\n", + "\n", + "G = nx.Graph()\n", + "G.add_nodes_from(node_list)\n", + "G.add_weighted_edges_from(weighted_edges_list)\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "bfs_path, bfs_correct_path = bfs_search(G, 3, 'A', 'G')" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "paths = [e[0] for e in bfs_path]" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "import collections\n", + "def get_search_tree_node_position(paths):\n", + " \"\"\"得到绘图时各个节点的坐标\n", + " \"\"\"\n", + " max_depth = 3 \n", + " # 得到每条路径的子路径\n", + " path_childern = {}\n", + " for path in paths:\n", + " father = path[:-1]\n", + " if father in paths:\n", + " if father in path_childern:\n", + " path_childern[father].append(path)\n", + " else:\n", + " path_childern[father] = [path]\n", + " # 对每条子路径排序\n", + " o_path_childern = collections.OrderedDict(\n", + " sorted(path_childern.items()))\n", + " # 计算每个树图中每个节点的位置\n", + " tree_node_position = {paths[0][0]: (1, 0, 2)}\n", + " for path, sub_paths in o_path_childern.items():\n", + " y_pos = -1.0 / max_depth * len(path)\n", + " dx = tree_node_position[path][2] / len(sub_paths)\n", + " sub_paths.sort()\n", + " for index, e_s in enumerate(sub_paths):\n", + " x_pos = tree_node_position[path][0] - tree_node_position[path][\n", + " 2] / 2 + dx / 2 + dx * index\n", + " tree_node_position[e_s] = (x_pos, y_pos, dx)\n", + " print(tree_node_position)" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'A': (1, 0, 2), 'ACD': (1.5, -0.6666666666666666, 1.0), 'AC': (1.5, -0.3333333333333333, 1.0), 'ABD': (0.16666666666666666, -0.6666666666666666, 0.3333333333333333), 'AB': (0.5, -0.3333333333333333, 1.0), 'ABF': (0.8333333333333333, -0.6666666666666666, 0.3333333333333333), 'ABE': (0.5, -0.6666666666666666, 0.3333333333333333)}\n" + ] + } + ], + "source": [ + "get_search_tree_node_position(paths)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import collections\n", + "from IPython import display\n", + "import networkx as nx\n", + "import numpy as np\n", + "import time\n", + "\n", + "\n", + "class SearchGraph():\n", + " def __init__(self,\n", + " node_list, \n", + " weighted_edges_list, \n", + " start_node,\n", + " target_node,\n", + " max_depth=1000,\n", + " nodes_pos=None,\n", + " help_info=None,):\n", + " # 图中的节点\n", + " self.node_list = node_list\n", + " self.weighted_edges_list = weighted_edges_list\n", + " self.start_node = start_node\n", + " self.target_node = target_node\n", + " self.nodes_pos = nodes_pos\n", + " self.max_depth = min(max_depth, len(node_list))\n", + " self.temp_best_path = None\n", + " \n", + " self.weighted_edges_dic = {frozenset([e[0],e[1]]):e[2] for e in weighted_edges_list}\n", + " self.help_info = help_info\n", + " self.path_score={self.start_node:0}\n", + " \n", + " self.animation_type = 'dfs'\n", + " \n", + " self.basic_node_color = '#6CB6FF'\n", + " self.start_node_color = 'y'\n", + " self.target_node_color = 'r'\n", + " self.visited_node_color = 'g'\n", + " \n", + " self.basic_edge_color = 'b'\n", + " self.visited_edge_color = 'g'\n", + " \n", + " self.success_color = 'r'\n", + " \n", + " self.correct_paths={}\n", + " self.show_correct_path = []\n", + " self.build_graph()\n", + " self.get_search_tree_node_position()\n", + " self.bfs_search()\n", + " \n", + " \n", + "\n", + " def build_graph(self):\n", + " self.G = nx.Graph()\n", + " self.G.add_nodes_from(self.node_list)\n", + " self.G.add_weighted_edges_from(self.weighted_edges_list)\n", + " \n", + " def get_search_tree_node_position(self):\n", + " \"\"\"得到绘图的点的坐标\n", + " \"\"\"\n", + " self.dfs_search()\n", + " # 得到 dfs 的搜索路径图\n", + " paths = self.dfs_path\n", + " # 得到每条路径的子路径\n", + " path_childern = {}\n", + " for path in paths:\n", + " father = path[:-1]\n", + " if father in paths:\n", + " if father in path_childern:\n", + " path_childern[father].append(path)\n", + " else:\n", + " path_childern[father] = [path]\n", + " # 对每条子路径排序\n", + " o_path_childern = collections.OrderedDict(sorted(path_childern.items()))\n", + " # 计算每个树图中每个节点的位置\n", + " tree_node_position = {self.start_node:(1, 0, 2)}\n", + " for path, sub_paths in o_path_childern.items():\n", + " y_pos = -1.0/self.max_depth * len(path)\n", + " dx = tree_node_position[path][2]/len(sub_paths) \n", + " sub_paths.sort()\n", + " for index, e_s in enumerate(sub_paths):\n", + " x_pos = tree_node_position[path][0] - tree_node_position[path][2]/2 + dx/2 + dx*index\n", + " tree_node_position[e_s]=(x_pos,y_pos, dx)\n", + " self.tree_node_position = tree_node_position\n", + " \n", + " def show_edge_labels(self, ax, pos1, pos2, label):\n", + " (x1, y1) = pos1\n", + " (x2, y2) = pos2\n", + " (x, y) = (x1*0.5 + x2*0.5, y1*0.5 + y2*0.5)\n", + "\n", + " angle = np.arctan2(y2 - y1, x2 - x1) / (2.0 * np.pi) * 360\n", + " if angle > 90:\n", + " angle -= 180\n", + " if angle < - 90:\n", + " angle += 180\n", + " xy = np.array((x, y))\n", + " trans_angle = ax.transData.transform_angles(np.array((angle,)),\n", + " xy.reshape((1, 2)))[0]\n", + " bbox = dict(boxstyle='round',\n", + " ec=(1.0, 1.0, 1.0),\n", + " fc=(1.0, 1.0, 1.0),\n", + " )\n", + " label = str(label) \n", + " ax.text(x, y,\n", + " label,\n", + " size=16,\n", + " color='k',\n", + " alpha=1,\n", + " horizontalalignment='center',\n", + " verticalalignment='center',\n", + " rotation=trans_angle,\n", + " transform=ax.transData,\n", + " bbox=bbox,\n", + " zorder=1,\n", + " clip_on=True,\n", + " )\n", + " \n", + " def show_search_tree(self, \n", + " this_path=None, \n", + " show_success_color=False,\n", + " best_path=None\n", + " ):\n", + " \"\"\"展示搜索树\n", + " \"\"\"\n", + " # 画出树图 \n", + " fig, ax = plt.subplots()\n", + " fig.set_figwidth(15)\n", + " fig.set_figheight(self.max_depth*1.5)\n", + " plt.axis('off')\n", + " \n", + " for path, pos in self.tree_node_position.items():\n", + " if path[-1] == self.start_node:\n", + " node_color = self.start_node_color\n", + " edge_color = self.basic_edge_color\n", + " elif this_path and path in this_path:\n", + " if show_success_color:\n", + " node_color = self.success_color\n", + " edge_color = self.success_color\n", + " else:\n", + " node_color = self.visited_node_color\n", + " edge_color = self.visited_edge_color\n", + " elif path[-1] == self.target_node:\n", + " node_color = self.target_node_color\n", + " edge_color = self.basic_edge_color\n", + " else:\n", + " node_color = self.basic_node_color\n", + " edge_color = self.basic_edge_color\n", + " ax.scatter(pos[0], pos[1], c=node_color, s=1000,zorder=1)\n", + " plt.annotate(\n", + " path[-1],\n", + " xy=(pos[0], pos[1]),\n", + " xytext=(0, 0),\n", + " textcoords='offset points',\n", + " ha='center',\n", + " va='center',\n", + " size=15,)\n", + " if len(path)>1:\n", + " plt.plot([self.tree_node_position[path[:-1]][0],pos[0]], \n", + " [self.tree_node_position[path[:-1]][1],pos[1]], \n", + " color=edge_color,\n", + " zorder=0)\n", + " if len(path)>1:\n", + " label = self.weighted_edges_dic[frozenset([path[-2],path[-1]])]\n", + " if self.animation_type in ['greedy','a_star']:\n", + " label = self.help_info_weight*self.help_info[path[-1]] + self.origin_info_weight*label\n", + " self.show_edge_labels(ax, self.tree_node_position[path[:-1]][0:2], pos[0:2], label)\n", + " display.clear_output(wait=True)\n", + " \n", + " show_res_text = \"\"\n", + " for e_c in self.show_correct_path:\n", + " show_res_text += '找到一条路径: %-7s' % e_c + '。距离为:' +str(self.correct_paths[e_c]) + '\\n'\n", + " plt.text(0, -1.1, show_res_text, fontsize=18,horizontalalignment='left', verticalalignment='top',)\n", + " \n", + " if best_path:\n", + " top_text = '最终最短路径为: %-7s' % this_path + '。距离为:' +str(self.correct_paths[this_path]) + '\\n'\n", + " elif this_path and self.animation_type in ['dfs','bfs']:\n", + " top_text = '当前路径: %-7s' % this_path + '。距离为:' +str(self.path_score[this_path]) + '\\n' \n", + " if self.temp_best_path:\n", + " top_text += '当前最短路径为: %-7s' % self.temp_best_path + '。距离为:' +str(self.correct_paths[self.temp_best_path]) + '\\n'\n", + " else:\n", + " top_text = ''\n", + "\n", + " plt.text(0, 0, \n", + " top_text, \n", + " fontsize=18,\n", + " horizontalalignment='left', \n", + " verticalalignment='top',)\n", + " \n", + " if self.animation_type in ['greedy','a_star']:\n", + " show_greedy_text = self.generate_greedy_help_text(this_path)\n", + " plt.text(0, 0, show_greedy_text, fontsize=18, horizontalalignment='left', verticalalignment='top',)\n", + " plt.show()\n", + " \n", + " def animation_search_tree(self,search_method='dfs', help_info_weight=1, origin_info_weight=1):\n", + " \"\"\"动画展示搜索过程\n", + " \"\"\"\n", + " self.animation_type = search_method\n", + " self.show_correct_path = []\n", + " self.temp_best_path = None\n", + " if search_method == 'bfs':\n", + " paths = self.bfs_path\n", + " elif search_method == 'dfs':\n", + " paths = self.dfs_path\n", + " elif search_method == 'greedy':\n", + " self.greedy_search()\n", + " paths = self.greedy_search_path\n", + " elif search_method == 'a_star':\n", + " self.a_star_search(help_info_weight=help_info_weight, origin_info_weight=origin_info_weight)\n", + " paths = self.greedy_search_path\n", + " else:\n", + " paths = []\n", + " for e_path in paths:\n", + " self.show_search_tree(e_path)\n", + " if e_path in self.correct_paths:\n", + " if not self.temp_best_path:\n", + " self.temp_best_path = e_path\n", + " elif self.path_score[e_path] < self.path_score[self.temp_best_path]:\n", + " self.temp_best_path = e_path\n", + " self.show_correct_path.append(e_path)\n", + " self.show_search_tree(e_path, True)\n", + " if search_method in ['greedy', 'a_star']:\n", + " time.sleep(5)\n", + " if search_method in ['bfs', 'dfs']:\n", + " if self.correct_paths:\n", + " best_path = min(self.correct_paths, key=self.correct_paths.get)\n", + " self.show_search_tree(best_path, True, True)\n", + " \n", + " def animation_graph(self, search_method='bfs', help_info_weight=1, origin_info_weight=1):\n", + " \n", + " \"\"\"\n", + " \"\"\"\n", + " self.animation_type = search_method\n", + " self.show_correct_path = []\n", + " if search_method == 'bfs':\n", + " paths = self.bfs_path\n", + " elif search_method == 'dfs':\n", + " paths = self.dfs_path\n", + " elif search_method == 'greedy':\n", + " self.greedy_search()\n", + " paths = self.greedy_search_path\n", + " elif search_method == 'a_star':\n", + " self.a_star_search(help_info_weight=help_info_weight, origin_info_weight=origin_info_weight)\n", + " paths = self.greedy_search_path\n", + " else:\n", + " paths = []\n", + " for e_path in paths:\n", + " self.show_graph(e_path)\n", + " if e_path in self.correct_paths:\n", + " self.show_correct_path.append(e_path)\n", + " self.show_graph(e_path, True)\n", + " time.sleep(5)\n", + " if search_method in ['bfs', 'dfs']:\n", + " best_path = min(self.correct_paths, key=self.correct_paths.get)\n", + " self.show_graph(best_path, True, True)\n", + " \n", + " def show_graph(self, this_path='', \n", + " show_success_color=False,\n", + " best_path=None):\n", + " \"\"\"\n", + " 绘制图\n", + " :return:\n", + " \"\"\"\n", + " fig, ax = plt.subplots()\n", + " fig.set_figwidth(6)\n", + " fig.set_figheight(8)\n", + " plt.axis('off')\n", + "\n", + " # 绘制节点与边颜色\n", + " visited_edges = []\n", + " if not this_path:\n", + " this_path = self.start_node\n", + " path_node_list = list(this_path)\n", + " for i in range(1,len(path_node_list)):\n", + " visited_edges.append(frozenset([path_node_list[i],path_node_list[i-1]]))\n", + " \n", + " # 节点与标识\n", + " nlabels = dict(zip(self.node_list, self.node_list))\n", + " edge_labels = dict([((u, v,), d['weight']) for u, v, d in self.G.edges(data=True)])\n", + " \n", + " # 节点颜色变化\n", + " val_map = {self.target_node: self.target_node_color}\n", + " if path_node_list:\n", + " for i in path_node_list:\n", + " if show_success_color:\n", + " val_map[i] = self.success_color\n", + " else:\n", + " val_map[i] = self.visited_node_color\n", + " val_map[self.start_node] = self.start_node_color \n", + " values = [val_map.get(node, self.basic_node_color) for node in self.G.nodes()]\n", + "\n", + " # 处理边的颜色\n", + " edge_colors = []\n", + " for edge in self.G.edges():\n", + " # 如果边在result_red_edges,分2种情况:\n", + " # 如果this_path[0]/this_path[-1] 对应起始点和终点，颜色为绿色，否则颜色为红色\n", + " # 如果边不在result_red_edges,则初始化边的颜色为黑色\n", + " if frozenset(edge) in visited_edges:\n", + " if show_success_color:\n", + " edge_colors.append(self.success_color)\n", + " else:\n", + " edge_colors.append(self.visited_edge_color)\n", + " else:\n", + " edge_colors.append(self.basic_edge_color)\n", + "\n", + " # 绘制节点及其标签\n", + " nx.draw_networkx_nodes(self.G, self.nodes_pos, node_size=800, node_color=values, width=6.0)\n", + " nx.draw_networkx_labels(self.G, self.nodes_pos, nlabels, font_size=20)\n", + " # 绘制边及其标签\n", + " nx.draw_networkx_edges(self.G, self.nodes_pos, edge_color=edge_colors, width=2.0, alpha=1.0)\n", + " nx.draw_networkx_edge_labels(self.G, self.nodes_pos, edge_labels=edge_labels, font_size=18)\n", + "\n", + " display.clear_output(wait=True)\n", + " # show_text = \"\"\n", + " # for e_c in self.show_correct_path:\n", + " # show_text += '找到一条路径: %-7s' % e_c + '。距离为:' +str(self.correct_paths[e_c]) + '\\n'\n", + " # plt.text(0, -2.6, show_text, fontsize=18, horizontalalignment='left', verticalalignment='top', )\n", + " \n", + "# if best_path:\n", + "# top_text = '最佳路径为: %-7s' % this_path + '。 距离为:' +str(self.correct_paths[this_path]) + '\\n'\n", + "# elif this_path and self.animation_type in ['dfs','bfs']:\n", + "# top_text = '当前路径: %-7s' % this_path + '。 距离为:' +str(self.cal_dis(this_path)) + '\\n'\n", + "# else:\n", + "# top_text = ''\n", + "# plt.text(0, 0, \n", + "# top_text, \n", + "# fontsize=18,\n", + "# horizontalalignment='left', \n", + "# verticalalignment='top',)\n", + " plt.show()\n", + " \n", + " def _dfs_helper(self, G, node, father, target_node,level, res, path):\n", + " path+=str(node)\n", + " if len(path)>1:\n", + " self.path_score[path] = self.path_score[path[:-1]] + self.weighted_edges_dic[frozenset([path[-2],path[-1]])]\n", + " res.append(path)\n", + " # 找到目标，停止搜索\n", + " if node==target_node:\n", + " return\n", + " if level< self.max_depth:\n", + " for neighbor in sorted(G[node]):\n", + " if str(neighbor) not in path:\n", + " self._dfs_helper(G, neighbor, node, target_node, level+1, res, path)\n", + " \n", + " def dfs_search(self):\n", + " dfs_path=[]\n", + " this_path=''\n", + " if self.start_node:\n", + " self._dfs_helper(self.G, self.start_node, None, self.target_node, 0, dfs_path, this_path)\n", + " self.dfs_path = dfs_path\n", + " for p in dfs_path:\n", + " if p[-1]==self.target_node and p not in self.correct_paths:\n", + " self.correct_paths[p] = self.cal_dis(p) \n", + " \n", + " def bfs_search(self):\n", + " to_search=[self.start_node]\n", + " bfs_path = []\n", + " bfs_correct_path = []\n", + " depth = 0\n", + " while to_search:\n", + " this_search = to_search.pop(0)\n", + " if len(this_search)>self.max_depth+1 :\n", + " break\n", + " bfs_path.append(this_search)\n", + " if this_search[-1]==self.target_node:\n", + " bfs_correct_path.append(this_search)\n", + " continue\n", + " for ne in sorted(self.G[this_search[-1]]):\n", + " if ne not in this_search:\n", + " to_search.append(this_search+ne)\n", + " self.bfs_path = bfs_path\n", + " for p in bfs_path:\n", + " if p[-1]==self.target_node and p not in self.correct_paths:\n", + " self.correct_paths[p] = self.cal_dis(p)\n", + " \n", + " def greedy_search(self, help_info_weight=1, origin_info_weight=0):\n", + " self.help_info_weight = help_info_weight\n", + " self.origin_info_weight = origin_info_weight\n", + " search_path = self.start_node\n", + " # 存储每一步的可选项及其分数，用来在动态演示时显示出来\n", + " search_scores = {}\n", + " while len(search_path) <= self.max_depth:\n", + " this_node = search_path[-1]\n", + " neighbour_nodes = [e_n for e_n in sorted(self.G[this_node]) if e_n not in search_path]\n", + " if len(neighbour_nodes) == 0:\n", + " search_scores[search_path]={}\n", + " break\n", + " if self.help_info:\n", + " scores = {e_n:help_info_weight*self.help_info[e_n]+origin_info_weight*self.weighted_edges_dic[frozenset([this_node,e_n])] for e_n in neighbour_nodes }\n", + " else:\n", + " scores = {e_n:self.weighted_edges_dic[frozenset([this_node,e_n])]\n", + " for e_n in neighbour_nodes }\n", + " search_scores[search_path]=scores\n", + " nearest_node = min(scores, key=scores.get)\n", + " search_path += nearest_node\n", + " if nearest_node == self.target_node:\n", + " break\n", + " self.greedy_search_path = [search_path[0:index+1] for index in range(len(search_path))]\n", + " self.search_scores = search_scores\n", + " \n", + " def a_star_search(self, help_info_weight=1, origin_info_weight=1):\n", + " self.greedy_search(help_info_weight, origin_info_weight)\n", + " \n", + "\n", + " def generate_greedy_help_text(self,path):\n", + " if path[-1] == self.target_node:\n", + " return '抵达目标节点' + str(self.target_node)\n", + " elif path not in self.search_scores:\n", + " return '抵达最大搜索深度，未找到目标节点'\n", + " \n", + " base_text = '当前可选的子节点及其信息值为 \\n'+ \\\n", + " str(self.search_scores[path]) + '\\n'\n", + " if self.target_node in self.search_scores[path]:\n", + " return base_text + '当前可选的子节点包含了目标节点，\\n所以选择目标节点'\n", + " elif len(self.search_scores[path]) == 1:\n", + " return base_text + '因为只有一个子节点，所以选择此节点'\n", + " else:\n", + " return base_text + '因为'+ \\\n", + " str(min(self.search_scores[path], key=self.search_scores[path].get)) + \\\n", + " '的值最小，所以选择此节点'\n", + " \n", + " def cal_dis(self,path):\n", + " dis = 0\n", + " if len(path) > 1:\n", + " for i in range(len(path)-1):\n", + " dis += self.weighted_edges_dic[frozenset([path[i],path[i+1]])]\n", + " return dis\n", + " " + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/decision_tree_1.mp4 b/decision_tree_1.mp4 new file mode 100644 index 0000000..fbe07d3 Binary files /dev/null and b/decision_tree_1.mp4 differ