{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 神经网络学习"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 人工智能是什么呢？"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"人工智能被分为强人工智能和弱人工智能：\n",
"+ 强人工智能：在各方面都能够和人类比肩的人工智能，“一种宽泛的心理能力，能够进行思考、计划、解决问题、抽象思理解复杂理念、快速学习和从经验中学习等”\n",
"+ 弱人工智能：擅长于单个方面的人工智能，比如AlphaGo。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"弱人工智能，本质上还是人类为计算机设计的算法，一系列定义良好的计算过程，能够将输入数据转化成输出数据。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经网络模拟人脑神经元的连接来达到学习功能，通过逐层抽象将输入数据逐层映射为概念等高等语义。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<div align=\"center\">\n",
"<img src=\"https://imgbed.momodel.cn/alg2.png\" width = 600>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1 人脑神经机制"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"首先，人脑神经元的活动和学习机制是怎样的呢？"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <iframe\n",
" width=\"800\"\n",
" height=\"600\"\n",
" src=\"https://view.officeapps.live.com/op/view.aspx?src=https://files.momodel.cn/nn_brain_3.pptx\"\n",
" frameborder=\"0\"\n",
" allowfullscreen\n",
" ></iframe>\n",
" "
],
"text/plain": [
"<IPython.lib.display.IFrame at 0x7fb8fceea7d0>"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import IFrame\n",
"src = 'https://files.momodel.cn/nn_brain_3.pptx'\n",
"IFrame('https://view.officeapps.live.com/op/view.aspx?src='+src, width=800, height=600)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"神经学假说认为："
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<table>\n",
" <tr>\n",
" <td ><center><img src=\"http://imgbed.momodel.cn/微信图片_20200114133721.png\"/></center></td>\n",
" <td><center><img src=\"http://imgbed.momodel.cn/微信图片_20200114133731.png\"/></center></td>\n",
" <td><center><img src=\"http://imgbed.momodel.cn/微信图片_20200114133746.png\"/></center></td>\n",
" </tr>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"高层的特征是低层特征的组合，从低层到高层的特征表示越来越抽象，越来越能表现语义。\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"人工智能中神经网络正是体现“逐层抽象、渐进学习”机制的学习模型。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<img src=\"http://imgbed.momodel.cn/微信图片_20200114133755.png\"/>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"人眼在辨识图片时，会先提取边缘特征，再识别部件，最后再得到最高层的模式。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<img src=\"http://imgbed.momodel.cn//20200103102429.png\" width=500>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经元的功能：\n",
"+ 物理反应：将前序神经元所传递过来的信息按联结权重累加\n",
"\n",
"$ 神经元i获得信息 = 神经元k信息 \\times 联结权重k + 神经元l信息 \\times 联结权重l + 神经元m信息 \\times 联结权重m $\n",
"\n",
"+ 化学反应：对神经元i获得的信息施以一个非线性变换（通过激活函数），激活若干信息、而非“来者不拒”\n",
"+ 信息流通：将神经元非线性变换后所得的信息继续向后传递"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2 感知机模型"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### 2.1 感知机模型：\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"<table>\n",
" <tr>\n",
" <td ><img src=\"http://imgbed.momodel.cn/感知器模型.png\" width=300/></td>\n",
" <td><img src=\"http://imgbed.momodel.cn//20200208141322.png\" width=400></td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"**输入项**：3个，$x_1,x_2,x_3$ \n",
"**神经元**：1个，用圆圈表示 \n",
"**权重**：每个输入项均通过权重与神经元相连（比如 $w_i$ 是 $x_i$ 与神经元相连的权重） \n",
"**输出**：1个\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"**工作方法**：\n",
"+ 计算输入项传递给神经元的信息加权总和，即：$y_{sum} = w_1x_1+w_2x_2+w_3x_3$\n",
"+ 如果 $y_{sum}$ 大于某个预定阀值（比如 0.5），则输出为 1，否则为 0 。\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 激活函数"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"在输出的判断上，其实不仅可以简单的按照阈值来判断，可以通过一个函数来进行计算，这个函数称为**激活函数**。\n",
"\n",
"常见的激活函数有： sigmoid，tanh，relu 等。\n",
"\n",
"下面我们看看这些激活函数的曲线图。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import warnings\n",
"warnings.filterwarnings(\"ignore\")\n",
"\n",
"\n",
"def plot_activation_function(activation_function):\n",
" \"\"\"\n",
" 绘制激活函数\n",
" :param activation_function: 激活函数名\n",
" :return:\n",
" \"\"\"\n",
" x = np.arange(-10, 10, 0.1)\n",
" y_activation_function = activation_function(x)\n",
"\n",
" # 绘制坐标轴\n",
" ax = plt.gca()\n",
" ax.spines['right'].set_color('none')\n",
" ax.spines['top'].set_color('none')\n",
" ax.xaxis.set_ticks_position('bottom')\n",
" ax.yaxis.set_ticks_position('left')\n",
" ax.spines['bottom'].set_position(('data', 0))\n",
" ax.spines['left'].set_position(('data', 0))\n",
"\n",
" # 绘制曲线图\n",
" plt.plot(x, y_activation_function)\n",
" \n",
" # 展示函数图像\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1.sigmoid函数\n",
"\n",
"Sigmoid函数是一个在生物学中常见的S型函数，也称为S型生长曲线。在信息科学中，由于其单增以及反函数单增等性质，Sigmoid函数常被用作神经网络的阈值函数，将变量映射到0,1之间。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$ f(x) = \\frac{1}{1+e^{-x}}$$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def sigmoid(x):\n",
" \"\"\"\n",
" sigmoid函数\n",
" :param x: np.array 格式数据\n",
" :return: sigmoid 函数\n",
" \"\"\"\n",
" y = \n",
" \n",
" return y\n",
"\n",
"# 绘制 sigmoid 函数图像\n",
"plot_activation_function(sigmoid)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADyCAYAAAB6U7/KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4VHWe7/H3yQ4hBBIIQhKWUBhCIIBUJC6NoLKIdkanEdHbKm37xAXbnsdutefONK32tPI8Xp32Nt0zFweX23qJS9sGpyERaHF63CBsQsISMIQkkJCEJWHJVvW7f1SICSSYQKpOpfJ5PU+eqnPOr+p8OXX45JdfncUyxiAiIvYLsrsAERHxUCCLiPgJBbKIiJ9QIIuI+AkFsoiIn1Agi4j4CQWyiIifUCCLiPgJBbKIiJ8I6WZ7ndYnfmvevHnk5ubaXYZIR6yuNFIPWQJGdXW13SWIXBYFsoiIn1Agi4j4CQWyiIifUCCLiPgJBbKIiJ9QIIvPPfDAA8TFxTFx4sQOlxtjePzxx3E4HKSlpbF161YfVyhiDwWy+NzixYsverzw2rVrKSoqoqioiBUrVvDII4/4sDoR+3T3xBCRyzZjxgwOHjzY6fKcnBzuu+8+LMsiIyODEydOcOTIEYYPH+67IqUdYwyNLjf1TW4amlzUN7mpb3ZR3/K82eWmyW1wud00uQwut6HJ5cblNjS7Dc2u85a53bhcBgO4jcEYz1lnpuW525iW6ZZ5gNv97Tx3y63nvn2twW3atG9zCptpcz5b+/l0Mr/jRst+kEZYiHf7sApk8Tvl5eUkJia2TickJFBeXt5hIK9YsYIVK1YAUFVV5bMae6v6JhdVdQ0crWugqq6eqroGjp9poq6+idqzzdTWN1Fb30RdfTO1Zz2PZ5s8wev2wXm6lgVBloXV8txqeR5kWZ7pludYF85r294z32r3vq3P262v4xPo2rVveW58cKKyAll6taysLLKysgBwOp02V2M/t9tw6NgZDlSdoqTmDIeOfftTWVtPXX1zh6/rFxrMwH4hDIwIJSoihJjIMEbHRjIgIoT+ocFEhAYTERpERGgw4aHB9Ds3HeJZFhpsERJsERIURHCQRWiw5zEk6Nv5nkeLkOAgQoIsgiyL4KD24dvXKZDF78THx1NaWto6XVZWRnx8vI0V+SeX27C3oo4tJccoPFLHnopa9lbUcabR1domMiyYkbGRJA2J5HrHEIZGhTN0QDhDB3oe46LCGRwZRmiwvk7yBwpk8TuZmZksX76cRYsW8dVXXxEdHa3xYzy9312HT/L5gRo2FR9j88FjrT3eQf1DGX9FFAudiUwYPhDHsAGMiulPTGSYep69iAJZfO7uu+9m48aNVFdXk5CQwLPPPktTUxMADz/8MPPnz2fNmjU4HA769+/P66+/bnPF9nG5DZ/trya3oIINuyuprG0AYOzQSG5LG8HVYwbjHBVDwuB+Ct4AYBnTrYFqXX5T/JbT6SQ/P9/uMnpEUWUd720p48Nt5RytayAyLJgbkodyc8owZlw5lCEDwu0uUbqnS78t1UMW8RPGGD7dV8XK/y7mb0XVhARZzBofxw+uimdmchwRocF2lyhepkAWsZkxhvW7j/K/8vayt7KOuKhwnpybzKL0RGLVE+5TFMgiNtp88BjL1u5hS8lxxgyJ5KU7J/P9ySO8fgKC+CcFsogNTp5p4jdrCnk3v4y4qHCev2MSdzoTdPhZH6dAFvGxNTuPsDSngONnGnnohiT+4aYr6Rem8WFRIIv4TH2Ti2c/KmDVplImxUfzxo/SmRgfbXdZ4kcUyCI+UFJzmkff3krB4VoemTmWn82+khANT8h5FMgiXvblNzVk/d98LMti5f1ObkoZZndJ4qcUyCJelLvrCI9nb2dkTH9eX5xOYkx/u0sSP6ZAFvGSt74sYWnOLqYkDuK1xekM6h9md0ni5xTIIl7wxy8O8sucAm4cH8fv77lKR1FIlyiQRXpYzvZylq4u4OaUOP7th9N0bLF0mfYUkR60YXclT7y7g+ljYlh+z1UKY+kW7S0iPWTboeM8+vZWUkcM5D/uT9fFgKTbFMgiPeBoXT2PvLWVuIHhvPGjqxkQrtFA6T7tNSKXqbHZzZK3t3LibCMfPHIdMZE6mkIujQJZ5DL95i+FbD54nFcWTWHCiIF2lyO9mIYsRC7Df359mDe/KOHB68fwd1N0I1a5PApkkUt0tLaef/5wF5MTB/GLW8bbXY4EAAWyyCUwxvCLD3ZyttHFS3dO1oWCpEdoLxK5BO/ml/LXPUd5et54HHED7C5HAoQCWaSbyo6f4bmPCrkmKZbF1462uxwJIApkkW567qNCDPDinWkEBXXp7u4iXaJAFumGjXuP8nFhJT+5cRwJg3UpTelZCmSRLmpsdvPcR4WMGRLJA9ePtrscCUAKZJEueu2zYr6pPs2vvj+B8BBdp0J6ngJZpAsqTtbzvzcUcXPKMGYmx9ldjgQoBbJIF/zrun00uwxLb5tgdykSwBTIIt+huPo0728t457pIxkZqy/yxHsUyCLf4bfr9xEWHMSjs8baXYoEOAWy2CI3N5fk5GQcDgfLli27YPmhQ4eYNWsWU6dOJS0tjTVr1thQJeytqGP1jsPcf+1o4qIibKlB+g4Fsvicy+ViyZIlrF27lsLCQlatWkVhYWG7Nv/yL//CwoUL2bZtG9nZ2Tz66KO21PrSx3sZEBbCwzck2bJ+6VsUyOJzmzZtwuFwkJSURFhYGIsWLSInJ6ddG8uyqK2tBeDkyZOMGDHC53V+XXaCjwsrefB7SQzqr4vOi/fpAvXic+Xl5SQmJrZOJyQk8NVXX7Vr88wzzzBnzhx+97vfcfr0adavX+/rMln+1/1E9wvVSSDiM+ohi19atWoVixcvpqysjDVr1nDvvffidrsvaLdixQqcTidOp5OqqqoeW/+BqlOs213JfdeMIioitMfeV+RiFMjic/Hx8ZSWlrZOl5WVER/f/m4bK1euZOHChQBcc8011NfXU11dfcF7ZWVlkZ+fT35+PkOHDu2xGv/jb98QGhzE/bqam/iQAll8Lj09naKiIoqLi2lsbCQ7O5vMzMx2bUaOHMmGDRsA2L17N/X19T0auBdztK6eP20p585pCQwZEO6TdYqAAllsEBISwvLly5k7dy4pKSksXLiQ1NRUli5dyurVqwF46aWXePXVV5k8eTJ33303b7zxBpblm0tdvvn5QZrcbh78no6sEN+yjDHdad+txiK+5HQ6yc/Pv6z3ONXQzLUvbOA6xxD+7YfTeqgyEbrUm1APWaSNdzeXUlvfTNYM9Y7F9xTIIi3cbsMfvyxh2qjBTB052O5ypA9SIIu0+OxANcXVp7k3Y5TdpUgfpUAWafHHL0qIiQzjlklX2F2K9FEKZBHgyMmzrN9dyV3pibobiNhGgSwCrPrqEAa45+qRdpcifZgCWfq8xmY3qzaXMis5jsQYXYBe7KNAlj7v48IKquoa9GWe2E6BLH1e9qZSEgb3Y8aVvjk1W6QzCmTp08pPnOWzA9UsmJZAcJBvTs0W6YwCWfq0D7aUYQz84KoEu0sRUSBL32WM4f2tZVyTFKsv88QvKJClz9p88DglNWdYME29Y/EPCmTps97fUkpkWLDOzBO/oUCWPulMYzN/+foIt6YNp3+Ybi0p/kGBLH3S2p0VnG50sWBa4nc3FvERBbL0SR9sK2NUbH/SR+sym+I/FMjS5xytreeLAzX83eQRPrstlEhXKJClz/nPr4/gNpA5ZYTdpYi0o0CWPmf1jsNMGD4QR1yU3aWItKNAlj6lpOY020tPqHcsfkmBLH3KRzsOA/D9yQpk8T8KZOkzjDHkbD9M+ujBxA/qZ3c5IhdQIEufsaeijqKjp8hU71j8lAJZ+ozVOw4THGQxf9Jwu0sR6ZACWfoEYwxrdh7h2rGxxA4It7sckQ4pkKVP2FtZR0nNGeZN1IWExH8pkKVPyNtViWXB7AnD7C5FpFMKZOkT8goqmJo4iLioCLtLEemUAlkCXumxMxQeqWVuqoYrxL8pkCXg5RVUACiQxe8pkMUWubm5JCcn43A4WLZsWYdt3n33XSZMmEBqair33HPPJa/r44JKkodFMXpI5CW/h4gv6FYJ4nMul4slS5awbt06EhISSE9PJzMzkwkTJrS2KSoq4oUXXuCzzz5j8ODBHD169JLWVX2qgc0lx/jJLEdPlS/iNeohi89t2rQJh8NBUlISYWFhLFq0iJycnHZtXn31VZYsWcLgwZ4LyMfFxV3SutYXVmIMzNFwhfQCCmTxufLychITv711UkJCAuXl5e3a7Nu3j3379nHdddeRkZFBbm5uh++1YsUKnE4nTqeTqqqqC5bnFVQQP6gfqSMG9uw/QsQLNGQhfqm5uZmioiI2btxIWVkZM2bMYOfOnQwaNKhdu6ysLLKysgBwOp3tltXVN/HZ/hp+mDFKdwaRXkE9ZPG5+Ph4SktLW6fLysqIj49v1yYhIYHMzExCQ0MZM2YMV155JUVFRd1az8a9VTS63MxN1ckg0jsokMXn0tPTKSoqori4mMbGRrKzs8nMzGzX5vbbb2fjxo0AVFdXs2/fPpKSkrq1nryCCmIjw3COjump0kW8SoEsPhcSEsLy5cuZO3cuKSkpLFy4kNTUVJYuXcrq1asBmDt3LrGxsUyYMIFZs2bx4osvEhsb2+V1NDS72Li3iptThhEcpOEK6R0sY0x32nersYgvOZ1O8vPzAfhk71F+9PpmXlvs5MbxGrIQ23WpV6AesgSkjwsqiAwL5tqxQ+wuRaTLFMgScFxuw7rCSmaOjyMiNNjuckS6TIEsAWfroeNUn2rUtSuk11EgS8DJ21VBWHAQs5KH2l2KSLcokCWgGGPIK6zgWkcsURGhdpcj0i0KZAkou4/UUXrsrIYrpFdSIEtAySuowLLg5hQd6ia9jwJZAkpeQQXOUYMZGqU7S0vvo0CWgNHY7GZPRZ2GK6TXUiBLwKitbwJ0qybpvRTIEjBOnm0iZfhAEmP6212KyCVRIEtAqKpr4EyjS5falF5NgSwBYV1hJaDhCundFMgSEPIKPGfnjb8iyu5SRC6ZAll6vdr6Jj4/UM3AfqG6VZP0agpk6fU+2XOUJpchup9uESm9m/Zg6fU+LqhkyIBwCNPuLL2besjSq9U3udi49yizJ+joCun9FMjSq322v5rTOtxNAoQCWXq1vIIKosJDdKsmCQgKZOm1ml1u1u8+yqzxcYSFaFeW3k97sfRa+SXHOXZat2qSwKFAll4rr6CCsJAgZupWTRIgFMjSKxlj+Ligku85hhAZrsPdJDAokKVXKjhcS/kJ3apJAosCWXqlvIIKgiy4KSXO7lJEeowCWXqlvIIK0kfHEDtAt2qSwKFAll6nuPo0+ypPabhCAo4CWXqdvIIKAObo7DwJMApk6XXW7qpgUnw0CYN1qyYJLApksUVubi7Jyck4HA6WLVvWabs//elPWJZFfn4+AOUnzrKj9AS3TNJwhQQeBbL4nMvlYsmSJaxdu5bCwkJWrVpFYWHhBe3q6up45ZVXmD59euu83F2e4YpbJg73Wb0ivqJAFp/btGkTDoeDpKQkwsLCWLRoETk5ORe0++Uvf8nTTz9NRERE67y1O48w/oooxgyJ9GXJIj6hQBafKy8vJzExsXU6ISGB8vLydm22bt1KaWkpt956a+u8o7X1bDl0XL1jCVg651T8jtvt5oknnuCNN95oNz+voAJjYH6b8eMVK1awYsUKAKqqqnxZpkiPUyCLz8XHx1NaWto6XVZWRnx8fOt0XV0du3btYubMmQBUVFSQmZnJVT97g7FDIxk37Ns7S2dlZZGVlQWA0+n0zT9AxEs0ZCE+l56eTlFREcXFxTQ2NpKdnU1mZmbr8ujoaKqrqzl48CAHDx4kIyODP773IYXVTcyfpOEKCVwKZPG5kJAQli9fzty5c0lJSWHhwoWkpqaydOlSVq9e3eFrNpXX4zYwb6IOd5PAZRljutO+W41Fesp9r22ipOY0G38+E8uyOmzjdDpbj1cW8TMd77TnUQ9Z/N7JM018vr+aeROv6DSMRQKBAln83rrdlTS7DfN1uJsEOAWy+L21O48QP6gfaQnRdpci4lUKZPFrtfVN/K2omrmpGq6QwKdAFr/2cUEljS43t03WcIUEPgWy+LWc7eUkxvRjauIgu0sR8ToFsvit6lMNfH6ghszJIzRcIX2CAln81pqdR3C5DZmT47+7sUgAUCCL38rZfpjkYVEkXxH13Y1FAoACWfxS2fEzbCk5TuaUEXaXIuIzCmTxSx/tOAJA5mQFsvQdCmTxSznby5k6chCJMbqRqfQdCmTxO3sr6thTUafesfQ5CmTxO+9vKSUkyFIgS5+jQBa/0uRy8+dt5dyUEkfsgHC7yxHxKQWy+JVP91ZRfaqRBdMSv7uxSIBRIItfeW9LKUMGhDEzeajdpYj4nAJZ/EbNqQY27D7K7VPiCQ3Wril9j/Z68Rs52w/T7DYscCbYXYqILRTI4jfe31LGpPhoxl8x0O5SRGyhQBa/sKv8JIVHalkwTb1j6bsUyOIX3vqyhIjQIG6foiu7Sd+lQBbbnTzbRM72w/zd5Hii+4faXY6IbRTIYrsPtpZxtsnFvdeMsrsUEVspkMVWxhje+rKEyYmDmBivu0pL36ZAFlt98U0NB6pOc2+GesciCmSx1VtfljCofyi3pemu0iIKZLFNZW09HxdUstCZSERosN3liNhOgSy2eePzg7iN4X9MH2l3KSJ+QYEstqirb+KtL0u4ZeJwRsVG2l2OiF9QIIstsjeVUlffTNaMJLtLEfEbCmTxucZmN79fvxuraj8/mOVk2bJlF7R5+eWXmTBhAmlpadx0002UlJTYUKmIbymQxedytpdxohGev3cWhYWFrFq1isLCwnZtpk6dSn5+Pl9//TULFizgqaeesqlaEd9RIItPGWN4Ja+QsLPVLLphEmFhYSxatIicnJx27WbNmkX//p47TmdkZFBWVmZHuSI+pUAWn9qw+yhldS7GNR/EsiwAEhISKC8v7/Q1K1eu5JZbbulw2YoVK3A6nTidTqqqqrxSs4ivhNhdgPQdbrfhpXX7iA13M6y6az3et956i/z8fD799NMOl2dlZZGVlQWA0+nssVpF7KAesvjM2l0V7D5Sy12pUZSXHmqdX1ZWRnz8hZfdXL9+Pb/5zW9YvXo14eG6A7UEPgWy+ITLbXh53V7GxQ3gp7dfR1FREcXFxTQ2NpKdnU1mZma79tu2beOhhx5i9erVxMXF2VS1iG8pkMUn/rytnANVp3li9pWEh4WyfPly5s6dS0pKCgsXLiQ1NZWlS5eyevVqAJ588klOnTrFnXfeyZQpUy4IbJFAZBljutO+W41FwHPc8U0vbyS6XygfPXZ965d5Pc3pdJKfn++V9xa5TF3a6dVDFq97+6sSSo+d5Wezk70WxiKBQIEsXlV9qoGX1+3je+OGMDN5qN3liPg1BbJ41Yu5eznb6OJX309V71jkOyiQxWu2l57gnfxSHrh+DI64AXaXI+L3FMjiFW634Vc5u4iLCucnNzrsLkekV1Agi1es2nyIHWUn+cf544mKCLW7HJFeQYEsPe5QzRl+85fdXOeI5fYpF56BJyIdUyBLj3K5DT9/bwfBQRYvLpisL/JEukEXF5Ie9dp/F7Pp4DFeunMyIwb1s7sckV5FPWTpMfsq63jx473MmTCMv79KQxUi3aVAlh5RV9/Ew29tYWBECM///SQNVYhcAg1ZyGVzuw1PvLuDkpozvP3gdIYM0KUyRS6Feshy2ZZ/sp91hZX8860pZCTF2l2OSK+lQJbLsq6wkn9dv487psaz+NrRdpcj0qspkOWSfflNDY/9v61Mio/m+Ts0bixyuRTIckl2lp3kwTfzSRjcjzd+dDX9woLtLkmk11MgS7ftP1rH/a9vIrpfKG89OJ2YyDC7SxIJCApk6ZYdpSdY+H++JMiyePvB6QyP1skfIj1FgSxd9l/7qrj71S+JDA/mvYevYfSQSLtLEgkoOg5ZuuT9LWX84k9fM25YFG/+KJ24gRF2lyQScBTIclH1TS6e/aiQVZsOce3YWP793mkM1OU0RbxCgSydKqk5zaNvb6XgcC2PzBzLz2ZfSUiwRrlEvEWBLBdwuQ2vf1bMSx/vIzTYYuX9Tm5KGWZ3WSIBT4Es7RQcPsn//GAnO8pOctP4OH59+0RdRlPERxTIAkDpsTO8vG4fH24vJzYyjN/dPZXb0obr7DsRH1Ig93HF1adZ+d/f8M7mUoKDLB6+YSwPzxhLdH99cSfiawrkPsjtNnzxTQ2vf3aQDXsqCQ0K4k5nIj+9aRzDdDibiG0UyH3I/qOn+HBbOX/eVk75ibPERIbxkxvH8cOMkcRFKYhF7KZADmD1TS62HTrBJ3uPsr6wkm+qTxNkwffGDeWpecnMTb2CiFBdFEjEXyiQA8iJM43sKDvJpuIaNhUfY0fpSRpdbkKDLTKSYll83Wjmpl6hYQkRP6VA7mWMMRw/00RJzWlKas6wp6KOPRW17K2o48jJegCCgywmxkez+LrRpI+OISMphig/O7suNzeXn/70p7hcLh588EF+8YtftFve0NDAfffdx5YtW4iNjeWdd95h9OjR9hQr4iMKZD/hchtO1TdTW99E9akGquoaOFrX/rH8xFlKj53hVENz6+tCgy3GDh1ARlIs46+IInVENFNHDiIy3H8/WpfLxZIlS1i3bh0JCQmkp6eTmZnJhAkTWtusXLmSwYMHs3//frKzs3n66ad55513bKxaxPv8938tnt6gMS3Pz023LgPDt8vbzvv2uec1377e86SzNm7jCUaXMbhchma3G5fb0Ow27R49z9ssa2nb0OymoclNfbOL+iZXm+duz3Sz5/Fso4u6lvCtPdtEbX1zu5Bty7IgNjKMIQPCGR4dwfQxMYyM6e/5ie3P6NhIwkJ61+nMmzZtwuFwkJSUBMCiRYvIyclpF8g5OTk888wzACxYsIDHHnsMY4yOi5aA5pNAvu13f6Oo8lRrmHJemLYN27YBGwiCgywiQoKICA0mIjSY8NAg+oUGMzAilFGx/YmKCGVgRChRESEM7Od5jI0MIy4qgriB4cREhhEaYNePKC8vJzExsXU6ISGBr776qtM2ISEhREdHU1NTw5AhQ3xaq4gvWaYbCThv3jxTXV3d7ZVUn2qg9tRp+vfr32bN5xVykddb55Z20qjT11oXf1+A06dOM2DAANquwmp9rYXV5j2slgXn5rVOt7QPsjzLgiyLIAssy/rO9XemqqqKoUOHXuKrvetyazt+/Di1tbWMGjUKgJqaGk6fPs3IkSNb2xQUFDBu3DjCwjx3I9m5cycpKSmEhLTvQ1RVVXFun2xoaGDKlCmXXJe3+Otn6a91gf/Wdql1bdmyJc8YM+87G3qGBbr8c8mmTZt2OS/3GtXVfZdb2+eff27mzJnTOv3888+b559/vl2bOXPmmM8//9wYY0xTU5OJjY01brf7ou/bv3//y6rLW/z1s/TXuozx39ouo64uZWxg/S0svUJ6ejpFRUUUFxfT2NhIdnY2mZmZ7dpkZmby5ptvAvD+++9z4403avxYAp5ff6kngSkkJITly5czd+5cXC4XDzzwAKmpqSxduhSn00lmZiY//vGPuffee3E4HMTExJCdnW132SJe57NAzsrK8tWqukV1dV9P1DZ//nzmz5/fbt5zzz3X+jwiIoL33nuvW+/pr1/4+etn6a91gf/W5u26uvWlHhBgx0BIIHE6neTn59tdhkhHujTepjFkERE/0aOB/N5775GamkpQUNAFPZUXXngBh8NBcnIyeXl5Hb6+uLiY6dOn43A4uOuuu2hsbOzJ8gC46667mDJlClOmTGH06NGdHiY1evRoJk2axJQpU3A6nT1ex/meeeYZ4uPjW2tbs2ZNh+1yc3NJTk7G4XCwbNkyr9cF8OSTTzJ+/HjS0tK44447OHHiRIftfLXNvmsbNDQ0cNddd+FwOJg+fToHDx70Wi3nlJaWMmvWLCZMmEBqaiqvvPLKBW02btxIdHR062fcdojGm77rczHG8Pjjj+NwOEhLS2Pr1q1er2nv3r2t22HKlCkMHDiQ3/72t+3a+HJ7PfDAA8TFxTFx4sTWeceOHWP27NmMGzeO2bNnc/z48Q5f++abbzJu3DjGjRvX+kX0Jevq4RimC4e9FRYWmj179pgbbrjBbN68uXV+QUGBSUtLM/X19eabb74xSUlJprm5+YLX33nnnWbVqlXGGGMeeugh84c//OGSji/pqieeeMI8++yzHS4bNWqUqaqq8ur62/rVr35lXnzxxYu2aW5uNklJSebAgQOmoaHBpKWlmYKCAq/XlpeXZ5qamowxxjz11FPmqaee6rCdL7bZxbbBuUOSfv/735uHHnrIGGPMqlWrzMKFC71akzHGHD582GzZssUYY0xtba0ZN27cBZ/NJ598Ym699Vav13K+7/pc/vKXv5h58+YZt9ttvvjiC3P11Vf7sDrPZzps2DBz8ODBdvN9ub0+/fRTs2XLFpOamto678knnzQvvPCCMcaYF154ocP9vqamxowZM8bU1NSYY8eOmTFjxphjx451tArfH/aWkpJCcnLyBfNzcnJYtGgR4eHhjBkzBofDwaZNmy74xfDXv/6VBQsWAHD//ffz4Ycf9mR5F6zv3Xff5e677/baOnpa21OOw8LCWk859rY5c+a0npCRkZFBWVmZ19fZma5sg5ycHO6//37Ac9r1hg0bWk+P95bhw4dz1VVXARAVFUVKSgrl5eVeXWdPycnJ4b777sOyLDIyMjhx4gRHjhzx2fo3bNjA2LFjW08UssOMGTOIiYlpN6/tftRZHuXl5TF79mxiYmIYPHgws2fPJjc399IL6Wpyd+cH2Ag420wvB37YZnolsOC81wwB9reZTgR2eaO+lvefAeRfZHkxsBXYAmR5q44263sGOAh8DbwGDO6gzQLgP9pM3wss93Zt59XwUdvP0tfb7GLbAMhtedwFJLRpcwAY4sNtNBo4BAw8b/5MoAbYAawFUn1Uz0U/F+A/gevbTG9o+//XB/W9BjzWwXxsfZikAAADKklEQVSfbq+Wz21Xm+kTbZ5bbafbzP858M9tpn8J/PxSa+j2YW+WZa0Hruhg0T8ZY7zfXeuCLtZ4N7DqIm9zvTGm3LKsOGCdZVl7jDH/5a26gH8Dfo3nSJZfAy8BD1zO+nqqtnPbzLKsfwKagbc7eZse32bdYbpyaqqXWZY1APgT8A/GmNrzFm8FRhljTlmWNR/4EBjng7Js/VwuxrKsMCAT+McOFtu1vS5gjDGWZXn9KLNuB7Ix5uZLWE85nh7vOQkt89qqAQZZlhVijGnupE2P1GhZVgjw98C0i7xHecvjUcuy/gxcDVzWTtzVbWdZ1qt4ei3n68p2vCRd2GaLgduAm0xLV6CD9+jxbdaBrmyDc23KWj7raDz7l1dZlhWKJ4zfNsZ8cP7ytgFtjFljWdYfLMsaYozp/gViuqELn4vX9qsuuAXYaoypPH+BXdurjUrLsoYbY45YljUcONpBm3I8PflzEvCMEFwSXx32thpYZFlWuGVZY/D8lms3iNzyn/wTPH+SAtwPeKvHfTOwxxjT4WCoZVmRlmVFnXsOzMHzZ7DXtHzg59zRyfo2A+MsyxrT0rNYhGfbepVlWfOAp4BMY8yZTtr4apt1ZRusxrP/gGd/+mtnv0R6iuU5r3slsNsY83Inba5oaYdlWVfj+f/n1V8UXfxcVgP3WR4ZwEljjK8GkTv9S9WO7XWetvtRZ3mUB8yxLGuwZVmD8Wzfjg8j64oeHoO5AygDGoBKIK/Nsn/CM5a3F7ilzfw1wIiW50l4gno/8B4Q7qWxojeAh8+bNwJY06aOHS0/BXj+bPf2ONofgZ14xpBXA8PPr6tlej6wr2Vber2ulnXuB0qB7S0//27nNutoGwDP4fmFARDRsv/sb9mfknywja7HM9z0dZvtNB94+Ny+BjzWsm12AF8C1/qgrg4/l/PqsoDft2zPnfho/BiIxBOw0W3m2bK98PxSOAI0tWTYj4FYPOPpRcB6IKalrZP232M80LKv7Qd+dDl1dPdMPRER8RKdqSci4icUyCIifkKBLCLiJxTIIiJ+QoEsIuInFMgiIn5CgSwi4icUyCIifuL/A+juwDSeki0XAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def sigmoid(x):\n",
" \"\"\"\n",
" sigmoid函数\n",
" :param x: np.array 格式数据\n",
" :return: sigmoid 函数\n",
" \"\"\"\n",
" y = 1/(1+np.exp(-x))\n",
" \n",
" return y\n",
"\n",
"# 绘制 sigmoid 函数图像\n",
"plot_activation_function(sigmoid)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"优点：\n",
"+ Sigmoid 函数的输出映射在(0,1)之间，单调连续，输出范围有限，优化稳定，可以用作输出层。它在物理意义上最为接近生物神经元。\n",
"+ 求导容易。\n",
"\n",
"缺点：\n",
"+ 由于其软饱和性，容易产生梯度消失，导致训练出现问题。\n",
"+ 其输出并不是以0为中心的。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 2.tanh函数\n",
"tanh是双曲函数中的一个，$Tanh()$为双曲正切。在数学中，双曲正切“Tanh”是由基本双曲函数双曲正弦和双曲余弦推导而来。\n",
"\n",
"公式如下:"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$ f(x) = \\frac{1-e^{-2x}}{1+e^{-2x}}$$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def tanh(x):\n",
" \"\"\"\n",
" tanh函数\n",
" :param x: np.array 格式数据\n",
" :return: tanh 函数\n",
" \"\"\"\n",
" y = \n",
" return y\n",
"\n",
"# 绘制 tanh 函数图像\n",
"plot_activation_function(tanh)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADuCAYAAAAOR30qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X9YU+fdP/B3IALiDwQUpIm/QiICEmkNFbc9TkGKtms6V1vx6yqbbbGte7rZPVaerbq675x07bpuZet3rLal1zZpa1vhai0qtbadtWKotJuiRQQKGJBAEFBJSHJ//7CmIkEDkp+8X9flBTnnPjmfnJy+e7jPfc6RCCFARESeF+DpAoiI6BIGMhGRl2AgExF5CQYyEZGXYCATEXkJBjIRkZdgIBMReQkGMhGRl2AgExF5Cekg2/OyPvJaS5YsQWlpqafLIHJE4kwjHiGT3zAYDJ4ugeiGMJCJiLwEA5mIyEswkImIvAQDmYjISzCQiYi8BAOZXGbNmjWIiorC7NmzHc4XQuDRRx+FUqmEWq3GZ599Zp9XWFgIlUoFlUqFwsJCd5VM5FGDHYdM5LQf/ehH+MlPfoLVq1c7nP/ee++huroa1dXVOHz4MB5++GEcPnwY7e3t2LJlC3Q6HSQSCebOnQutVovw8HA3fwL/541PDPLCkgAAEgkgkTg1nHjIGMjkMgsWLEBdXd2A84uLi7F69WpIJBKkpqaio6MDer0eBw4cQEZGBiIiIgAAGRkZKC0txcqVK91UuWuYLFac7TShubMHzed60NZtQrfJgi6TBd09FnSbLDhvsuC8yYpeqw29NoFeiw0Wmw0Wq0Dv5Z9WGyw2AZtNXLpSS1y6YksI8fVPQEB8/fPy/G9eCyFg89LQ82Yn/u8ShIwKdOk6GMjkMU1NTZgyZYr9tVwuR1NT04DTHSkoKEBBQQEAoLW11bUFO0kIgbq2CyivbUOVvgtftnThy5ZuGLpNDtsHSQMwLliKsSFSjAmSYkxwIIKkAQgNDEBQoATSgABIAyUYFRiAUYESSAMDIA2QIODrozWJBJBA8vXPb47kJADgYF6AfZ7EPs3bSJy7sM2tpAGur4mBTD4tJycHOTk5AACNRuOxOqw2gY+rW1Hy+Rl8cqoNzZ09AIDQoEDMjB6HtFmTIA8PxeTxIYgOC8Hk8SGYODYIY0OkCJa69qiLfAcDmTxGJpOhoaHB/rqxsREymQwymQwHDhzoM33hwoXuL9AJZzt7UHioDm9WNKG5swcTQkfh28qJ+FZsJFIVkZgROQYBbjiyIv/AQCaP0Wq1yM/PR1ZWFg4fPoywsDDExMQgMzMTv/jFL2A0GgEAe/fuxbZt2zxcbV/dJgv+9H41Cj+pQ6/Vhu/OnIRf3ZmAtPgoHvHSkDGQyWVWrlyJAwcOwGAwQC6XY8uWLejt7QUAPPTQQ7j99tuxe/duKJVKhIaG4uWXXwYAREREYNOmTUhJSQEAbN682X6Czxt8+GUrHt/5Oc52mfCDm+V4NF2JaZFjPF0W+QHJIIe98NwseS2NRgOdTuey97fZBJ7ZexJ/OVCDuOhxyLs7CTdP5VA8copT/VY8QiZygslixWOvf453v9Bj5a1T8Ks7E10+BIpGHgYy0XX0Wm1Y94+jKKtqwS9un4UH/0vh8gsEaGRiIBNdgxACG3d+gbKqFvz6rkSsnj/d0yWRH+O9LIiuYfu/avHW0SasXzyTYUwux0AmGkB5bTu2vXcCSxIn49F0pafLoRGAgUzkwAWzBf/zxueQh4/G0/eo2WdMbsE+ZCIHfr/3S3zVfgFFOakYFzLK0+XQCMEjZKKrnGzuwssHa7Fq3lSkKiI9XQ6NIAxkoqtse68KY4Ol2JAZ5+lSaIRhIBNd4eApAw6cbMV/p6kwITTI0+XQCMNAJrrCH8uqcVNYCFZ/a5qnS6ERiIFM9LWK+naU17XjwQUK3rGNPIKBTPS1Fw6cRnjoKKxImXL9xkQuwEAmAlBnOI+yqhbcN386QoM4GpQ8g4FMBGDHka8QGCDBqnlTPV0KjWAMZHKZ0tJSxMXFQalUIi8vr9/89evXIzk5GcnJyZg5cyYmTJhgnxcYGGifp9VqXVqn2WLDmxWNSJ8VhejxIS5dF9G18G8zcgmr1Yp169Zh3759kMvlSElJgVarRUJCgr3NH/7wB/vvzz//PI4ePWp/PXr0aFRWVrql1rKqFhi6zVjJo2PyMB4hk0uUl5dDqVRCoVAgKCgIWVlZKC4uHrD9jh07sHLlSjdW+I03KxoRExaCBapJHlk/0WUMZHKJpqYmTJnyzWgFuVyOpqYmh23r6+tRW1uLtLQ0+7Senh5oNBqkpqZi165dA66noKAAGo0GGo0Gra2tg66z44IZH1W34s45NyGQT4cmD2OXBXlcUVERli9fjsDAb8b+1tfXQyaT4fTp00hLS0NSUhJiY2P7LZuTk4OcnBwAl56pN1h7jjWj1yrwPXXM0D8A0TDhETK5hEwmQ0NDg/11Y2MjZDKZw7ZFRUX9uisut1UoFFi4cGGf/uXh9M4XekyLDEWSLMwl7080GAxkcomUlBRUV1ejtrYWZrMZRUVFDkdLnDhxAkajEfPnz7dPMxqNMJlMAACDwYCDBw/2ORk4XNq6TTh4yoDvqWN4v2PyCuyyIJeQSqXIz89HZmYmrFYr1qxZg8TERGzevBkajcYezkVFRcjKyuoTiFVVVVi7di0CAgJgs9mQm5vrkkA+cLIVNgEsSWR3BXkHiRBiMO0H1ZjInTQaDXQ6ndPt1/3jMxypa8en/5uOAJ7QI9dyagdjlwWNSL1WGz76shVps6IYxuQ1GMg0Ih2pa0eXyYK0WVGeLoXIjoFMI9L+qrMICgzAt5UTPV0KkR0DmUakf50yIGVGOMYE87w2eQ8GMo04hm4TTjR34VuxPDom78JAphHn09NtAIBvxfKJ0uRdGMg04hw81YZxwVJenUdeh4FMI86hGgPmKSIgDeTuT96FeySNKGc6LqKu7QL7j8krMZBpRDlS1w4AuHVGhIcrIeqPgUwjiq7OiDFBgZg1eZynSyHqh4FMI4qu3oibp4az/5i8EvdKGjG6enpxsrkTc6eFe7oUIocYyDRiHP2qAzYBaKYzkMk7MZBpxKioNyJAAtw8lYFM3omBTCPGZ18ZETd5PMby/hXkpRjI5DKlpaWIi4uDUqlEXl5ev/mvvPIKJk2ahOTkZCQnJ+PFF1+0zyssLIRKpYJKpUJhYeEN1yKEwBeN55A8hVfnkffioQK5hNVqxbp167Bv3z7I5XKkpKRAq9X2exTTihUrkJ+f32dae3s7tmzZAp1OB4lEgrlz50Kr1SI8fOhdDfVtF3DuYi/U8glDfg8iV+MRMrlEeXk5lEolFAoFgoKCkJWVheLiYqeW3bNnDzIyMhAREYHw8HBkZGSgtLT0hur5vLEDAKCW8wiZvBcDmVyiqakJU6ZMsb+Wy+Voamrq1+7NN9+EWq3G8uXL0dDQMKhlAaCgoAAajQYajQatra0D1vNF4zkESwMwM5oXhJD3YiCTx9x5552oq6vDF198gYyMDGRnZw/6PXJycqDT6aDT6TBp0qQB2/278RwSbxqPUbwghLwY905yCZlMZj/iBYDGxkbIZLI+bSIjIxEcHAwAeOCBB1BRUeH0soNhtQn858w59h+T12Mgk0ukpKSguroatbW1MJvNKCoqglar7dNGr9fbfy8pKUF8fDwAIDMzE3v37oXRaITRaMTevXuRmZk55FpOne3GBbOV/cfk9TjKglxCKpUiPz8fmZmZsFqtWLNmDRITE7F582ZoNBpotVr86U9/QklJCaRSKSIiIvDKK68AACIiIrBp0yakpKQAADZv3oyIiKHfne24/hwAYDZvSE9eTiKEGEz7QTUmcieNRgOdTtdv+rbdVXj5YB2O/zqTNxUiT5E404h7J/m94/pOqKLHMozJ63EPJb93orkL8THjPV0G0XUxkMmvGbpNaO0y8Yb05BMYyOTXTui7AIBHyOQTGMjk1040dwIAj5DJJzCQya8d13cialwwIscGe7oUoutiIJNfO6HnCT3yHQxk8lu9VhtOne3GrBh2V5BvYCCT3zrdeh5mqw0JPEImH8FAJr/1zQk9BjL5BgYy+a3j+k4EBQZAMWmMp0shcgoDmfzWCX0XlFFjeQ9k8hncU8lvnWju5Ak98ikMZPJL7efNaOk08YQe+RQGMvmlE/pLJ/TieIUe+RAGMvml6rPdAIA4PtSUfAgDmVymtLQUcXFxUCqVyMvL6zf/2WefRUJCAtRqNdLT01FfX2+fFxgYiOTkZCQnJ/d79JMzalq7MS5EiknjeMk0+Q4+wolcwmq1Yt26ddi3bx/kcjlSUlKg1WqRkJBgb3PzzTdDp9MhNDQUL7zwAh5//HG89tprAIDRo0ejsrJyyOs/dbYbsZPGQiJx6kENRF6BR8jkEuXl5VAqlVAoFAgKCkJWVhaKi4v7tFm0aBFCQ0MBAKmpqWhsbBy29de0dkMZNXbY3o/IHRjI5BJNTU2YMmWK/bVcLkdTU9OA7bdv346lS5faX/f09ECj0SA1NRW7du0acLmCggJoNBpoNBq0trYCADp7etHSaULsJAYy+RZ2WZDH/f3vf4dOp8OHH35on1ZfXw+ZTIbTp08jLS0NSUlJiI2N7bdsTk4OcnJyAFx6yClw6R4WAHiETD6HR8jkEjKZDA0NDfbXjY2NkMlk/dqVlZVh69atKCkpQXBwcJ/lAUChUGDhwoU4evSo0+s+9fUIi1heMk0+hoFMLpGSkoLq6mrU1tbCbDajqKio32iJo0ePYu3atSgpKUFUVJR9utFohMlkAgAYDAYcPHiwz8nA66lp7caoQAmmRoQOz4chchN2WZBLSKVS5OfnIzMzE1arFWvWrEFiYiI2b94MjUYDrVaLDRs2oLu7G/fccw8AYOrUqSgpKUFVVRXWrl2LgIAA2Gw25ObmDiqQT53txvTIMZDyHhbkYyRCiMG0H1RjInfSaDTQ6XRI+/0BxEWPwws/nOvpkoguc2r8JQ8hyK+YLTbUt13gCAvySQxk8iv1bedhtQmOsCCfxEAmv1LTenmEBQOZfA8DmfzK5SFvfEoI+SIGMvmVmtbzuCksBGOCOYCIfA8DmfzKqbPdiGX/MfkoBjL5lZrWbvYfk89iIJPf6LXacMFs5QgL8lkMZPIbJosNAEdYkO9iIJPf6Om1AuBd3sh3MZDJb5gsNowPkWLi2CBPl0I0JAxk8humXhuUUXxsE/kuBjL5DZPFyv5j8mkMZPIL5y72wsJ7WJCPYyCTX/jmKSEMZPJdDGRymdLSUsTFxUGpVCIvL6/ffJPJhBUrVkCpVGLevHmoq6uzz9u2bRuUSiXi4uKwZ8+e667r8k2FeIRMvoyBTC5htVqxbt06vPfeezh+/Dh27NiB48eP92mzfft2hIeH49SpU1i/fj02btwIADh+/DiKiopw7NgxlJaW4pFHHoHVar3m+mrOdkMCYAof20Q+jIFMLlFeXg6lUgmFQoGgoCBkZWWhuLi4T5vi4mJkZ2cDAJYvX473338fQggUFxcjKysLwcHBmDFjBpRKJcrLy6+5vprWbgRLAxEYwBEW5LsYyOQSTU1NmDJliv21XC5HU1PTgG2kUinCwsLQ1tbm1LJXO3W2G8GjuDuTbxvUM/WWLFkiDAbDkFbU2tqKSZMmDWlZV2Jdg+dMbUajEZ2dnZg2bRoAoK2tDefPn8fUqVPtbY4dOwaVSoWgoEsXcvz73/9GfHw8zpw5g7FjxyIiIgIAUFdXh7CwMISHhzusxWAwwDLuJlja6nHzHPVwfcxh463fpbfWBXhvbUOtq6KiYo8QYsl1GwohBvNvyObOnXsji7sM6xo8Z2r75JNPxG233WZ//dvf/lb89re/7dPmtttuE5988okQQoje3l4RGRkpbDZbv7ZXtnPkhL5TTNv4jggZHznYj+IW3vpdemtdQnhvbTdQl1MZy7/xyCVSUlJQXV2N2tpamM1mFBUVQavV9mmj1WpRWFgIANi5cyfS0tIgkUig1WpRVFQEk8mE2tpaVFdX49Zbbx1wXZdHWEhsFtd9ICI34GMVyCWkUiny8/ORmZkJq9WKNWvWIDExEZs3b4ZGo4FWq8X999+P++67D0qlEhERESgqKgIAJCYm4t5770VCQgKkUin+/Oc/IzAwcMB1XR6DzEAmn+fsobS4wS6Lv/71rzeyuMuwrsHzttoe3fGZ+Na298XUqVM9XYpD3ra9LvPWuoTw3tpuoC6nMnZQJ/UADKoxkTvc8aePMXFsMI795RHodDpPl0PkiFPjMdmHTD7NZhM43Xqel0yTXxjWQH7jjTeQmJiIgICAfkcqzlwKW1tbi3nz5kGpVGLFihUwm83DWR4AYMWKFUhOTkZycjKmT5+O5ORkh+2mT5+OpKQkJCcnQ6PRDHsdV3vyySchk8nste3evdthu+tdjuwKGzZswKxZs6BWq7Fs2TJ0dHQ4bOeubXblNtiU9ywu9vZ9bNO1Lsl2lYaGBixatAgJCQlITEzEH//4x35tDhw4gLCwMPt3/Otf/9rldQHX/16EEHj00UehVCqhVqvx2WefubymkydP2rdDcnIyxo8fj+eee65PG3durzVr1iAqKgqzZ8+2T2tvb0dGRgZUKhUyMjJgNBodLltYWAiVSgWVSmU/ST1kzvZtCCf6kI8fPy5OnDghvvvd74ojR47Ypx87dkyo1WrR09MjTp8+LRQKhbBYLP2Wv+eee8SOHTuEEEKsXbtW/OUvfxlSZ42zHnvsMbFlyxaH86ZNmyZaW1tduv4r/epXvxJPP/30NdtYLBahUChETU2NMJlMQq1Wi2PHjrm8tj179oje3l4hhBCPP/64ePzxxx22c8c2u3obxKfdLaZtfEd8WmOwD0n685//LNauXSuEEGLHjh3i3nvvdWlNQghx5swZUVFRIYQQorOzU6hUqn7fzQcffCDuuOMOl9dytet9L++++65YsmSJsNls4tChQ+LWW291Y3WXvtPo6GhRV1fXZ7o7t9eHH34oKioqRGJion3ahg0bxLZt24QQQmzbts3hft/W1iZmzJgh2traRHt7u5gxY4Zob293tAr3D3uLj49HXFxcv+nOXAorhMD+/fuxfPlyAEB2djZ27do1nOX1W9/rr7+OlStXumwdw82Zy5Fd4bbbboNUemlATmpqKhobG12+zoFcvQ3m/FcmgL43FRrokmxXiomJwS233AIAGDduHOLj4697daG3KC4uxurVqyGRSJCamoqOjg7o9Xq3rf/9999HbGys/SIiT1iwYIH9QqTLrtyPBsqjPXv2ICMjAxEREQgPD0dGRgZKS0uHXIdb+pCduRS2ra0NEyZMsP+H78zlsjfi448/RnR0NFQqlcP5EokEt912G+bOnYuCggKX1XGl/Px8qNVqrFmzxuGfR0O5pHi4vfTSS1i6dKnDee7YZldvg97RERglzIgYE+SwzZWXZLtLXV0djh49innz5vWbd+jQIcyZMwdLly7FsWPH3FLP9b4XT+9XRUVFAx4YeWJ7XdbS0oKYmBgAwOTJk9HS0tKvzXBvu0GPQ168eDGam5v7Td+6dSvuuuuuIRcynJypcceOHdc8Ov7Xv/4FmUyGs2fPIiMjA7NmzcKCBQtcVtfDDz+MTZs2QSKRYNOmTfj5z3+Ol1566YbWN1y1Xd5mW7duhVQqxapVqxy+hyu22fUYTIEYa+vymsc2dXd34+6778Zzzz2H8ePH95l3yy23oL6+HmPHjsXu3bvx/e9/H9XV1S6vyRPfi7PMZjNKSkqwbdu2fvM8tb0ckUgkbtnHBh3IZWVlg16JTCZDQ0OD/XVjYyNkMlmfNpGRkejo6IDFYoFUKnXYZrhqtFgseOutt1BRUXHNmgEgKioKy5YtQ3l5+Q3vxM5uuwcffBDf+973HNZ0ve3oqtpeeeUVvPPOO3j//fcH3DFdsc0crePKbdB8UYLpIY7byOVyWCwWnDt3DpGRkcNahyO9vb24++67sWrVKvzgBz/oN//KgL799tvxyCOPwGAwYOLEiS6t63rfiyv3q+t57733cMsttyA6OrrfPE9tr8uio6Oh1+sRExMDvV6PqKiofm1kMhkOHDhgf93Y2IiFCxcOeZ1u6bJw5lJYiUSCRYsWYefOnQAunbl01RF3WVkZZs2aBblc7nD++fPn0dXVZf997969fc6+usKVfXZvv/22w/U5czmyK5SWluJ3v/sdSkpKEBrq+H7D7tpmV26Dlo5u9Agpvq2O7dNmoEuyXUkIgfvvvx/x8fF47LHHHLZpbm6292WXl5fDZrO5/H8UznwvWq0Wr776KoQQ+PTTTxEWFmb/U93VrvWXqie215Wu3I8GyqPMzEzs3bsXRqMRRqMRe/fuRWZm5tBX6uzZP+HEKIu33npLyGQyERQUJKKiovrcXOY3v/mNUCgUYubMmWL37t326UuXLhVNTU1CCCFqampESkqKiI2NFcuXLxc9PT3XW+WQZGdnixdeeKHPtKamJrF06VJ7HWq1WqjVapGQkCB+85vfuKSOK/3whz8Us2fPFklJSeLOO+8UZ86c6VeXEJfOiKtUKqFQKNxSlxBCxMbGCrlcLubMmSPmzJljH8HgqW12eRvMSEkX0za+I96vahabNm0SsbGxQgghLl68KJYvXy5iY2NFSkqKqKmpcVktl3388ccCgEhKSrJvp3fffVe88MIL9n3t+eefFwkJCUKtVot58+aJgwcPuryugb6XK+uy2WzikUceEQqFQsyePbvPCClX6u7uFhEREaKjo8M+zVPbKysrS0yePFlIpVIhk8nEiy++KAwGg0hLSxNKpVKkp6eLtrY2IYQQR44cEffff7992e3bt4vY2FgRGxsrXnrppYFWwSv1yL+9duQrbHzz3/howyJMjQyFRqPhlXrkrXilHvm3U2e7ESQNgCx8tKdLIRoWDGTyWTWt56GYOIaPbSK/wUAmn3XqbDdi+ZRp8iMMZPJJF81WNBgvQMmbCpEfYSCTT6pp7YYQQNzkcZ4uhWjYMJBp2Dlzl6zKykrMnz8fiYmJUKvVeO211+zzfvSjH2HGjBn2u3xVVlb2W/5k86WxtTOjGcjkPxjINOzy8vKQnp6O6upqpKenO7xNaGhoKF599VUcO3YMpaWl+NnPftbntp5PP/00KisrUVlZ6fAWqV+e7UJQYACmRzq+UIXIFzGQadg5c5esmTNn2m/sdNNNNyEqKgqtra1Or+PL5i4oJo2BNJC7MPkP7s007Jy5S9aVysvLYTabERv7zSXQv/zlL6FWq7F+/XqYTKZ+y3zZ0o2Z0eNQUFAAjUYDjUYzqEAn8kYMZBqSxYsXY/bs2f3+XX1/5uvdJUuv1+O+++7Dyy+/jICAS7vjtm3bcOLECRw5cgTt7e146qmn+izTbbKgqeMiZkaPRU5ODnQ6HXQ6HSZNmjT8H5TIjQZ9tzci4Np3h3PmLlkA0NnZiTvuuANbt25Famqqffrlo+vg4GD8+Mc/xjPPPNNnueoWntAj/8QjZBp2ztwly2w2Y9myZVi9erX9KTGXXb7znRACu3bt6nd3suqWbgAMZPI/DGQadrm5udi3bx9UKhXKysqQm5sLANDpdHjggQcAAK+//jo++ugjvPLKK/2Gt61atQpJSUlISkqCwWDAE0880ef9T7Z0IWRUAKZEcIQF+Rfe7Y18zn3bD8N4wYx3/vu/+kzn3d7Ii/Fub+SfvmzpwswodleQ/2Egk085d7EXLZ0mzOQl0+SHGMjkU74ZYcGbCpH/YSCTTznJIW/kxxjI5FOqW7oxJigQsgl8Sgj5HwYy+ZSTzV1QRY9z+VOkiTyBgUw+pfpsF/uPyW8xkMlntHaZYOg2s/+Y/BYDmXzGcX0nACDxpjAPV0LkGgxk8hnHz1wK5ISY8R6uhMg1GMjkM47rOyGbMBphoaM8XQqRSzCQyWccO3MOCTfx6Jj8FwOZfMIFswW1hvNIZCCTH2Mgk0840dwFIdh/TP6NgUw+wX5Cj0fI5McYyOQTjp05h7DRo3jJNPk1BjINu/b2dmRkZEClUiEjIwNGo9Fhu8DAQPvTQrRarX16bW0t5s2bB6VSiRUrVsBsNuPzhnNQy8N4yTT5NQYyDbu8vDykp6ejuroa6enpyMvLc9hu9OjRqKysRGVlJUpKSuzTN27ciPXr1+PUqVMIDw/H//vbSzjZ0gW1nBeEkH9jINOwKy4uRnZ2NgAgOzsbu3btcnpZIQT2799vf/BpdnY2dpYdgtUmoJZPcEm9RN6CgUzDrqWlBTExMQCAyZMno6WlxWG7np4eaDQapKam2kO7ra0NEyZMgFQqBQDI5XI09wYDAOY4COSCggJoNBpoNBq0tra64uMQuY3U0wWQb1q8eDGam5v7Td+6dWuf1xKJZMB+3/r6eshkMpw+fRppaWlISkpCWFj/bgnbBDmixgVjclhIv3k5OTnIyckBcOkhp0S+jIFMQ1JWVjbgvOjoaOj1esTExECv1yMqKsphO5lMBgBQKBRYuHAhjh49irvvvhsdHR2wWCyQSqVobGxEwMQZ7D+mEYFdFjTstFotCgsLAQCFhYW46667+rUxGo0wmUwAAIPBgIMHDyIhIQESiQSLFi3Czp07AQAvFv4D5pBw9h/TiMBApmGXm5uLffv2QaVSoaysDLm5uQAAnU6HBx54AABQVVUFjUaDOXPmYNGiRcjNzUVCQgIA4KmnnsKzzz4LpVKJhguBACS4ZWq4pz4OkdtIhBCDaT+oxkQ36tm9J5H/wSl88WQmxgZfu4dNo9FAp9O5qTKiQXFqAD2PkMmr6eqNiI8Zf90wJvIHDGTyWharDZUNHdBMY3cFjQwMZPJaVfouXDBbMXd6hKdLIXILBjJ5rYr6dgDgETKNGAxk8lrlde24KSwEN/EObzRCMJDJK9lsAodq2jA/dqKnSyFyGwYyeaWq5k4YL/Ti28pIT5dC5DYMZPJKh2raAADzYxnINHIwkMkrfVLTBsXEMYgJY/8xjRwMZPI6vVYbDp9uw7fYXUEjDAOZvM5n9UacN1vxbZ7QoxGGgUxeZ//JsxgVKMF3VAxkGlkYyOR19ledxbwZkRgXMsqrF8o7AAAHUElEQVTTpRC5FQOZvMpXbRdQfbYbabMc39SeyJ8xkMmr7D9x6fl76fEMZBp5GMjkVfYeb0HspDGYFjnG06UQuR0DmYZde3s7MjIyoFKpkJGRAaPR2K/NBx98gOTkZPu/kJAQvPr6Lnx6ug2W2iOYMWOGfV5lZaUHPgWR+zGQadjl5eUhPT0d1dXVSE9PR15eXr82ixYtQmVlJSorK7F//36EhobCFJUAmwCiLn6Fp59+2j4/OTnZA5+CyP0YyDTsiouLkZ2dDQDIzs7Grl27rtl+586dWLp0KfaeMCAuehzGWLvcUSaR12Eg07BraWlBTEwMAGDy5MloaWm5ZvuioiJkLluJI3VG3Dnn0nK//OUvoVarsX79evvTqR0pKCiARqOBRqNBa2vr8H0IIg/gQ05pSBYvXozm5uZ+07du3Yrs7Gx0dHTYp4WHhzvsRwYAvV4PtVqN//37x3j+QA0+2rAIUtM5TJ48GWazGTk5OYiNjcXmzZuvWxMfckpezKmHnPLJkTQkZWVlA86Ljo6GXq9HTEwM9Ho9oqIGHsL2+uuv4/vLluGNz5rwHeVETIkIBRAKAAgODsaPf/xjPPPMM8NdPpFXYpcFDTutVovCwkIAQGFhIe66664B2+7YsQOJi++F/lwP/s+tUwFcOmoGACEEdu3ahdmzZ7u+aCIvwECmYZebm4t9+/ZBpVKhrKwMubm5AACdTocHHnjA3q6urg4NDQ34/Px4TBwbhMUJ0QCAVatWISkpCUlJSTAYDHjiiSc88jmI3I19yORRJ5u7kPncR1i/eCZ+ulh1Q+/FPmTyYk71IfMImTzqrx/VYPSoQKyeP83TpRB5HAOZPKbReAEllWew8tapCB8T5OlyiDyOgUwe8/u9XyIwQIIHF8zwdClEXoGBTB7xn6ZzePtoE9Z8Zwafm0f0NQYyuZ3NJvDrd44jYkwQHl4Y6+lyiLwGA5nc7p/lX6G8th0bl8RhPJ8KQmTHQCa3ami/gG27q/Ad5UTcq5ni6XKIvAoDmdymp9eKR/7xGQIkEmz7QRIkEqeGZhKNGLyXBbmFEAJP7PoP/t10Dn9brfn6nhVEdCUeIZPLCSGQ994J7KxoxE/TVcj4+hJpIuqLR8jkUjabwFOlJ/DXj07jvtRp+NkNXh5N5M8YyOQy3SYLct/8Au98ocd9qdOwRZvIfmOia2Agk0tU1BvxP298jvq289i4ZBYe+q6CYUx0HQxkGlaNxgv4Y1k13qhoRExYCP75YCpSFZGeLovIJzCQ6YZZbQIHTxnwRkUjSv+jhwQS5CxQ4KfpKowJ5i5G5CyOsqAh6bhgRul/9PhV8X/wnaf2Y/VL5fjoy1asmjcNGxLO4x8blmPc6KBr3p+4tLQUcXFxUCqVyMvLs0+vra3FvHnzoFQqsWLFCpjNZnd8JCKP4+ELDain1wpDtwlnOnpwpuMiag3n8WVLF062dKHWcB5CAKNHBWJ+bCSeuCMBixOiECwNRFVVABa89RbWrl074HtbrVasW7cO+/btg1wuR0pKCrRaLRISErBx40asX78eWVlZeOihh7B9+3Y8/PDDbvzkRJ7hlkA+cPIsuk0WAMDVDyi5+hEk13qCSf9lxcDzrrGeq9fRb43iyl+vanvN9x24vuste/XM/tvlyt8H0RaAxWpDr9UGs1XAbLn0e6/VBrPFBrPVhl6rwHmTBecu9uLcxV50fv3TZLH1ed8ACTA9cgxmRo/DsmQZ5sdGQi2fgCBp3z+04uPjcT3l5eVQKpVQKBQAgKysLBQXFyM+Ph779+/HP//5TwBAdnY2nnzySQYyjQhuCeSt71ah+my3O1ZF1xEUGIAgaQBGBUq+/hmAoMAAhAYHImz0KESPH4vxIaMQNnoUxo8ehcgxQZCFj0ZM2GjIw0cjZFTgsNTR1NSEKVO+uZeFXC7H4cOH0dbWhgkTJkAqldqnNzU1Dfg+BQUFKCgoAAC0trYOS21EnuKWQP7bag16rd8cbfUf/dR3wpXzr2569dApSZ95V7/rwO/br4Krl71iQv8aBl5P/3lXLzz0ZYdakzRQglGBl0J4uIaeLV68GM3Nzf2mb9269ZpPmR5uOTk5yMnJAXDpmXpEvswtgTx94hh3rIbcqKys7IaWl8lkaGhosL9ubGyETCZDZGQkOjo6YLFYIJVK7dOJRgKOsiCPSElJQXV1NWpra2E2m1FUVAStVguJRIJFixZh586dAIDCwkK3HnETeRIDmYbd22+/DblcjkOHDuGOO+5AZmYmAODMmTO4/fbbAQBSqRT5+fnIzMxEfHw87r33XiQmJgIAnnrqKTz77LNQKpVoa2vD/fff77HPQuROkmuNanBgUI2J3Emj0Vxz3DORBzl18oZHyEREXoKBTETkJRjIRERegoFMROQlBntSj8hrSSSSUiHEEk/XQTRUDGQiIi/BLgsiIi/BQCYi8hIMZCIiL8FAJiLyEgxkIiIvwUAmIvISDGQiIi/BQCYi8hIMZCIiL/H/AQGRDwgIKcptAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def tanh(x):\n",
" \"\"\"\n",
" tanh函数\n",
" :param x: np.array 格式数据\n",
" :return: tanh 函数\n",
" \"\"\"\n",
" y = (1-np.exp(-2*x))/(1+np.exp(-2*x))\n",
" return y\n",
"\n",
"# 绘制 tanh 函数图像\n",
"plot_activation_function(tanh)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"优点：\n",
"+ 比Sigmoid函数收敛速度更快。\n",
"+ 相比Sigmoid函数，其输出以0为中心。\n",
"\n",
"缺点：\n",
"+ 还是没有改变Sigmoid函数的最大问题——由于饱和性产生的梯度消失。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 3.ReLU函数\n",
"\n",
"Relu激活函数（The Rectified Linear Unit），用于隐层神经元输出。公式如下："
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$ f(x) = max(0, x)$$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def relu(x):\n",
" \"\"\"\n",
" relu 函数\n",
" :param x: np.array 格式数据\n",
" :return: relu 函数\n",
" \"\"\"\n",
" \n",
" y = \n",
"\n",
" return y\n",
"\n",
"# 绘制 relu 函数\n",
"plot_activation_function(relu)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADyCAYAAAB6U7/KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAH+lJREFUeJzt3XtYlGXCP/DvoxwUERQBRfGAgnISOYyivq7lGmpq9vMs2ptlvpimZa6W5VttvptadtIsW7a23ARPq0nrcfNQXpabDgcNCRUDFSTlpMh5DvfvD9CLBHRAZu6Hme/nurxyhnuYb88M37m555lbRQgBIiKSr5XsAEREVI2FTESkEixkIiKVYCETEakEC5mISCVYyEREKsFCJiJSCRYyEZFKsJCJiFTCrpHj+bE+Uq0xY8bgwIEDsmMQ1UcxZRBnyGQ18vPzZUcgeiAsZCIilWAhExGpBAuZiEglWMgk1Zw5c+Dp6Yng4OA71xUWFiIqKgp+fn6IiopCUVGRxIRElsNCJqmeeuqpOmdGrFmzBiNHjsSFCxcwcuRIrFmzRlI6IstiIZNUw4cPh5ub2++uS0hIwOzZswEAs2fPxu7du2VEI7K4xp6HTGR2165dg5eXFwCgS5cuuHbtWoNjY2NjERsbCwDIy8uzSD4ic+EMmVRNURQoSsPn1MfExECr1UKr1cLDw8OCychWJF8uwpJtKSit1Jv9vjhDJtXp3LkzcnNz4eXlhdzcXHh6esqORDaqqLQKz8UloVUrBTqD0ez3xxkyqc6ECROwadMmAMCmTZvw+OOPS05EtshoFFi8LQX5JVX4ZFY4Ojg5mP0+WcgkVXR0NIYMGYJz587B29sbn3/+OZYvX45vv/0Wfn5+OHToEJYvXy47Jtmgj49m4PvzeXj9sUCEeHewyH0qQjRqvyBuLkSqpdFooNVqZccgK3D8Qj7+++8/4fEBXfHB9NB7vo9hIm4uRETUWL/drMALW5Ph6+GMtyb2b44yNhkLmYiohs5gxML4JJTrDNj4RDjaOVr2vAeeZUFEVOPt/enQXirC+ugw+Hq2t/j9c4ZMRATgQGouPjueiSeH9MSEAV2lZGAhE5HNy8wvxbIdZzCgewesGBcgLQcLmYhsWoXOgPmbE9G6tYKPZ4bB0a61tCxcQyYim/Z6QirSf7uFL54eCO+OTlKzcIZMRDZr+6kr2K7NxqI/+mJEP/kf0WchE5FNSrtajNcSUjG0TycsfqSv7DgAWMhEZIOKK3RYEJeIDk72WB8dhtatLPfhj3vhGjIR2RQhBF7acQZXisqxNWYw3J0dZUe6gzNkIrIpnx/PxIGzv2H5GH8M7OV2/xtYEAuZiGyGNqsQa/anY1RgZ8z9g4/sOHWwkInIJuSXVOK5+CR069gWa6cOsOimQabiGjIRWT2DUeCFrcm4UabDrgUD4drWXnakerGQicjqrTt0Hj9kFOCdySEI6uoqO06DuGRBRFbt6LnrWH8kA1MjvDFtYHfZce6JhUxEVivnRjle3JYC/y7tsfLxYNlx7ouFTERWqUpvxIK4JOgNAhufiEBbB3mbBpmKa8hEZJVW7fsFp6/cwMZZ4fBxbyc7jkk4QyYiq/Ov01fx5Y9ZeGaYDx7t7yU7jslYyERkVTKul2D5zjOI6NkRyx/1lx2nUVjIRGQ1yqr0WBCXCEf71tgwMwz2rVtWxXENmYisghACK75OxYXrJfjHnEHwcm0rO1KjtayXDyKiBsSfvIyvk3OweGRf/MHPQ3acJmEhE1GL93P2Tbz5TRqG9/XAoj/6yo7TZCxkImrRbpbpMD8uEe7ODvhweihaqWSz+abgGjIRtVhGo8CS7Sm4VlyBbfOGwK2dg+xID4QzZCJqsf567FccTr+OFWMDEN6jo+w4D4yFTEQt0omLBVh7MB3jQrwwe2gv2XGaBQuZiFqc68UVWLQlGb3c2+HtySGq3Gy+KVjIpFoffPABgoKCEBwcjOjoaFRUVMiORCqgNxixaEsySip12DgrAs6O1vNWGAuZVCknJwfr16+HVqtFamoqDAYDtm7dKjsWqcB7357HT5mFWDWxP/p1aS87TrNiIZNq6fV6lJeXQ6/Xo6ysDF27dpUdiSQ7lHYNG7+7iOhBPTAp3Ft2nGbHQiZV6tatG5YuXYoePXrAy8sLrq6uGDVqVJ1xsbGx0Gg00Gg0yMvLk5CULOVyQRmWbE9BUFcXvPFYoOw4ZsFCJlUqKipCQkICMjMzcfXqVZSWlmLz5s11xsXExECr1UKr1cLDo2V+XJbur0JnwIL4RAgAG2dFoI29+jebbwoWMqnSoUOH4OPjAw8PD9jb22PSpEn48ccfZcciSVbuSUNqTjHenxaKHp2cZMcxGxYyqVKPHj3wn//8B2VlZRBC4PDhwwgICJAdiyTYlZSN+J8u49mH+iAqsLPsOGbFQiZVioyMxJQpUxAeHo7+/fvDaDQiJiZGdiyysHO/3cKKr1MxyMcNS0f1lR3H7BQhRGPGN2owkSVpNBpotVrZMaiZlFTqMWHDcRSX67Hv+WHwdGkjO9KDMOmTK5whE5HqCCHw8s4zyMovxUfRYS29jE3GQiYi1fnHiUvYeyYXS0f3w5A+nWTHsRgWMhGpSvLlIvxlbxpG+nvi2eF9ZMexKBYyEalGUWkVnotLQmeXNnhv2oAWvdl8U1jPrhxE1KIZjQKLt6Ugv6QK/5w/BB2cWvZm803BGTIRqcLHRzPw/fk8vP5YIEK8O8iOIwULmYikO34hH+8fOo//F9oVsyJ7yI4jDQuZiKTKvVmO57cmw9fDGasm9beazeabgoVMRNLoDEYsjE9Ghc6AjU9EwMnBtt/Wsu3/eyKS6u396Ui8VIT10WHw9XSWHUc6zpCJSIoDqbn47HgmZg/piQkD+I8PACxkIpIgM78Uy3acwYDuHfDqOO7idxsLmYgsqkJnwPzNiWjdWsHHM8PgaGedm803BdeQiciiXk9IRfpvt/DF0wPh3dF6N5tvCs6Qichitp+6gu3abCz6oy9G9POUHUd1WMhEZBFpV4vxWkIqhvbphMWPWP9m803BQiYisyuu0GFBXCI6ONljfXQYWtvYpkGm4hoyEZmVEAIv7TiDK0Xl2BozGO7OjrIjqRZnyERkVp8fz8SBs79h+Rh/DOzlJjuOqrGQichsTmUVYvX+dIwO6oy5f/CRHUf1WMhEZBb5JZVYGJ8E745tsXbqAJveNMhUXEMmomZnMAq8sDUZN8p02LVgIFza2MuO1CKwkImo2a07dB4/ZBTgnckhCOrqKjtOi8ElCyJqVkfPXcf6IxmYGuGNaQO7y47TorCQiajZ5Nwox4vbUuDfpT1WPh4sO06Lw0ImomZRpTdiQVwS9AaBjU9EoK0DNw1qLK4hE1GzWLXvF5y+cgMbZ4XDx72d7DgtEmfIRPTA/nX6Kr78MQvPDPPBo/29ZMdpsVjIRPRAMq6XYPnOM4jo2RHLH/WXHadFYyETUZOVVekxf3MiHO1bY8PMMNi3ZqU8CB49Uq0bN25gypQp8Pf3R0BAAE6cOCE7EtUihMCru35GRl4J1s8Ig5drW9mRWjy+qUeq9cILL2DMmDH45z//iaqqKpSVlcmORLXEn7yM3SlXsSSqL4b5ucuOYxVYyKRKN2/exLFjx/Dll18CABwcHODg4CA3FN3xc/ZNvPlNGob39cDCEb6y41gNLlmQKmVmZsLDwwNPP/00wsLCMHfuXJSWlsqORQBulukwPy4R7s4O+HB6KFpxs/lmw0ImVdLr9UhKSsL8+fORnJyMdu3aYc2aNXXGxcbGQqPRQKPRIC8vT0JS22I0CizZnoJrxRXYMCscbu34W0tzYiGTKnl7e8Pb2xuRkZEAgClTpiApKanOuJiYGGi1Wmi1Wnh4eFg6ps3567FfcTj9OlaMDUB4j46y41gdFjKpUpcuXdC9e3ecO3cOAHD48GEEBgZKTmXbTlwswNqD6RgX4oXZQ3vJjmOV+KYeqdZHH32EWbNmoaqqCr1798YXX3whO5LNul5cgUVbktHLvR3enhzCzebNhIVMqhUaGgqtVis7hs3TG4xYtCUZJZU6xM2NhLMja8NceGSJ6J7e/fd5/JRZiPenDUC/Lu1lx7FqXEMmogZ9m3YNn35/EdGDemBSuLfsOFaPhUxE9bpcUIY/bU9BcDcXvPEY31C1BBYyEdVRoTNgQXwiAGDjrAi0sedm85bANWQiqmPlnjSk5hTjb09q0N3NSXYcm8EZMhH9zq6kbMT/dBnPPtQHUYGdZcexKSxkIrrj3G+3sOLrVAzyccPSUX1lx7E5LGQiAgCUVOoxPy4R7RztsCE6DHbcbN7ieMSJCEIIvLzzDLLyS/FRdBg8XdrIjmSTWMhEhH+cuIS9Z3KxdHQ/DOnTSXYcm8VCJrJxyZeL8Je9aRjp74lnh/eRHcemsZCJbFhRaRWei0tCZ5c2eH8aN5uXjechE9koo1Fg8bYU5JdUYef8oXB1spcdyeZxhkxkozYczcD35/PwxoRA9Pd2lR2HwEImsknHL+Tjg0PnMTGsG2YO6iE7DtVgIRPZmNyb5Xh+azL8PJ3x1sRgbjavIixkIhuiMxixMD4ZFToDPpkVAScHvo2kJnw0iGzI2/vTkXipCOujw+Dr6Sw7Dt2FM2QiG3EgNRefHc/E7CE9MWFAV9lxqB4sZCIbkJlfimU7zmBA9w54dVyA7DjUABYykZWr0Bkwf3MiWrdW8PHMMDjacbN5teIaMpGVez0hFem/3cIXTw+Ed0duNq9mnCETWbHtp65guzYbi/7oixH9PGXHoftgIRNZqbNXb+K1hFT8l28nLH6Em823BCxkIitUXKHDgrgkdHCyx7oZYWjNTYNaBK4hE1kZIQSW7TiN7KJybIsZDHdnR9mRyEScIRNZmc+PZ+Lg2Wt45VF/aHq5yY5DjcBCJrIip7IKsXp/OkYHdcYzw3xkx6FGYiETWYn8kkosjE+Cd8e2WDt1ADcNaoG4hkxkBQxGgRe2JuNGmQ67FgyESxtuNt8SsZCJrMC6Q+fxQ0YB3pkcgqCu3Gy+peKSBamawWBAWFgYxo8fLzuKah09dx3rj2RgaoQ3pg3sLjsOPQAWMqnaunXrEBDAzXAaknOjHC9uS4F/l/ZY+Xiw7Dj0gFjIpFrZ2dnYu3cv5s6dKzuKKlXpjVgQlwSDQWDjExFo68BNg1o6FjKp1uLFi/HOO++gVSs+Tevz1t40nL5yA2unhsDHvZ3sONQM+EwnVdqzZw88PT0RERFxz3GxsbHQaDTQaDTIy8uzUDr5vjl9FZtOXMLcYT4YE+wlOw41E0UI0ZjxjRpM1FSvvPIKvvrqK9jZ2aGiogLFxcWYNGkSNm/e3OBtNBoNtFqtBVPKkXH9FiZs+AEBXi7YGjMY9q05r2oBTDopnIVMqvfdd9/h3XffxZ49e+45zhYKuaxKj8c3/IDC0irsff4P6OLaRnYkMo1JhczzkIlaCCEEXt31MzLySvDVnEiWsRViIZPqPfzww3j44Ydlx5Au/uRl7E65iiVRfTHMz112HDIDLj4RtQA/Z9/Em9+kYXhfDywc4Ss7DpkJC5lI5W6W6TA/LhHuzg74cHooWnGzeavFJQsiFTMaBZZsT8G14gpsmzcEbu0cZEciM+IMmUjF/nrsVxxOv44VYwMQ3qOj7DhkZixkIpU6cbEAaw+mY1yIF2YP7SU7DlkAC5lIha4XV2DRlmT0cm+HtyeHcLN5G8E1ZCKV0RuMWLglGaWVesT/TyScHfljaiv4SBOpzLv/Po+TmYV4f9oA9O3cXnYcsiAuWRCpyLdp1/Dp9xcxM7IHJoV7y45DFsZCJlKJywVl+NP2FAR3c8Hr4wNlxyEJWMhEKlChM2BBfCIAYOOsCLSx52bztohryEQqsHJPGlJzivG3JzXo7uYkOw5JwhkykWS7krIR/9NlPPtQH0QFdpYdhyRiIRNJdO63W1jxdSoG+bhh6ai+suOQZCxkIklKKvWYH5eIdo522BAdBjv+yx82j88AIgmEEHh55xlk5Zfio+gweLpws3liIRNJ8Y8Tl7D3TC6WjfbHkD6dZMchlWAhE1lY0uUi/GVvGh4J8MS84b1lxyEVYSETWVBhaRUWxiWhs0sbvDeVm83T7/E8ZCILMRoFFm9LQX5JFXbOHwpXJ3vZkUhlOEMmspANRzNw7Hwe3pgQiP7errLjkAqxkIks4PiFfHxw6DwmhnXDzEE9ZMchlWIhE5lZ7s1yPL81GX6eznhrYjA3m6cGsZCJzEhnMGJhfDIqdAZ8MisCTg5824YaxmcHkRm9vT8diZeKsD46DL6ezrLjkMpxhkxkJgdSc/HZ8UzMHtITEwZ0lR2HWgAWMpEZZOaXYtmOMxjQvQNeHRcgOw61ECxkomZWoTNg/uZEtG6t4JNZ4XC042bzZBquIRM1s9d2p+LctVv44qmB6Nahrew41IJwhkzUjLafuoIdidlYNMIXD/fzlB2HWhgWMlEzOXv1Jl5LSMV/+XbCC49ws3lqPBYyqdKVK1cwYsQIBAYGIigoCOvWrZMd6Z6KK3RYEJeEDk72WDcjDK25aRA1AdeQSZXs7Ozw3nvvITw8HLdu3UJERASioqIQGBgoO1odQggs23Ea2UXl2BYzGO7OjrIjUQvFGTKpkpeXF8LDwwEA7du3R0BAAHJyciSnqt/nxzNx8Ow1vPKoPzS93GTHoRaMhUyql5WVheTkZERGRsqOUseprEKs3p+O0UGd8cwwH9lxqIXjkgWpWklJCSZPnowPP/wQLi4udb4eGxuL2NhYAEBeXp5Fs+WXVGJhfBK8O7bF2qkDuGkQPTBFCNGY8Y0aTPQgdDodxo8fj9GjR2PJkiX3Ha/RaKDVai2QDDAYBZ78+0/QZhVh14KhCOrK/Y3pnkx6teYMmVRJCIFnnnkGAQEBJpWxpa07dB4/ZBTgnckhLGNqNlxDJlX64Ycf8NVXX+HIkSMIDQ1FaGgo9u3bJzsWAODouetYfyQD0zTemDawu+w4ZEU4QyZVGjZsGBq5nGYR2UVleHFbCvy7tMfKx4NlxyErwxkykYkq9QY8F58Mg0Fg4xMRaGPPTYOoeXGGTGSiVXt/wekrN/DpE+HwcW8nOw5ZIc6QiUzwzemr2HTiEuYO88GYYC/ZcchKsZCJ7iPj+i0s33kGET074uVH/WXHISvGQia6h7IqPeZvTkJb+9b4eGY47FvzR4bMh2vIRA0QQuDVXT8jI68EX82JRBfXNrIjkZXjyz1RA+JPXsbulKt48ZG+GObnLjsO2QAWMlE9fs6+iTe/ScPwvh5YOMJXdhyyESxkorvcLNNhflwi3J0d8OH0ULTiZvNkIVxDJqrFaBRYsj0F14orsH3eELi1c5AdiWwIZ8hEtfz12K84nH4d/zsuEGE9OsqOQzaGhUxU48TFAqw9mI7xIV54ckhP2XHIBrGQiQBcL67Aoi3J6OXeDmsmh3CzeZKCa8hk8/QGIxZuSUZppR7x/xMJZ0f+WJAcfOaRzXv33+dxMrMQ708bgL6d28uOQzaMSxZk075Nu4ZPv7+ImZE9MCncW3YcsnEsZLJZlwvK8KftKQju5oLXxwfKjkPEQibbVKEzYEF8IgBg4yxuNk/qwDVkskkr96QhNacYf3tSg+5uTrLjEAHgDJls0K6kbMT/dBnPPtQHUYGdZcchuoOFTDbl3G+3sOLrVET6uGHpqL6y4xD9DguZbEZJpR7z4xLh3MYOH80Mgx03myeV4TOSbIIQAi/vPIOs/FJ8FB0Gz/bcbJ7Uh4VMNmHTj1nYeyYXy0b7Y3DvTrLjENWLhUxWL+lyEd7a9wseCfDEvOG9ZcchahALmaxaYWkVFsYlobNLG7w3lZvNk7rxPGSyWkajwOJtKcgvqcLO+UPh6mQvOxLRPXGGTFZrw9EMHDufhzcmBKK/t6vsOET3xUImq3T8Qj4+OHQeE8O6YeagHrLjEJmEhUxWJ/dmOZ7fmgw/T2e8NTGYm81Ti8FCJquiMxixMD4ZlToDPpkVAScHvk1CLQcLmVTrwIED6NevH3x9fbFmzRqTbvP2/nQkXirCmskh8PV0NnNCoubFQiZVMhgMeO6557B//36kpaVhy5YtSEtLu+dtist1+Ox4Jp4a2guPDehqoaREzYeFTKp08uRJ+Pr6onfv3nBwcMCMGTOQkJDQ4PjM/FJcKSpHaPcOeHVsgAWTEjUfRQjRmPGNGnzbhA3HcfF6SVNuSjZKr9dDr9ejTZvqPSd0ej0MBgPaODrWO77KYETOl4tx4expdOvQ1pJRiUxh0jvLjSrkMWPGiPz8/EYnybtViVulpWjbVn0bgZeXlzFXI1kiW0VFBaqqKuHiUn3+cEV5OXQ6Hdq7uNTJUlZWDgWArjgfoaEDzJqrKfLy8uDh4SE7Rh1qzQWoN1tTcyUmJh4UQoy570AhRGP+NFlERMSD3NxsmKvxLJHtxx9/FKNGjbpzedWqVWLVqlX3vI2Tk5O5YzWJWh9LteYSQr3ZHiCXSR3LNWRSpYEDB+LChQvIzMxEVVUVtm7digkTJsiORWRWPEmTVMnOzg4bNmzA6NGjYTAYMGfOHAQFBcmORWRWFivkmJgYS91VozBX41kq29ixYzF27FiTx7u7u5sxTdOp9bFUay5AvdnMncsiZ1kQWYJGo4FWq5Udg6g+Jp1lwTVkIiKVaNZC3rFjB4KCgtCqVas6M5XVq1fD19cX/fr1w8GDB+u9fWZmJiIjI+Hr64vp06ejqqqqOeMBAKZPn47Q0FCEhoaiV69eCA0NrXdcr1690L9/f4SGhkKj0TR7jrv9+c9/Rrdu3e5k27dvX73jmvJx4ge1bNky+Pv7IyQkBBMnTsSNGzfqHWepY3a/Y1BZWYnp06fD19cXkZGRyMrKMluW265cuYIRI0YgMDAQQUFBWLduXZ0x3333HVxdXe88xitXrjR7LuD+j4sQAs8//zx8fX0REhKCpKQks2c6d+7cneMQGhoKFxcXfPjhh78bY8njNWfOHHh6eiI4OPjOdYWFhYiKioKfnx+ioqJQVFRU7203bdoEPz8/+Pn5YdOmTQ8WxNTTMYQJp72lpaWJ9PR08dBDD4lTp07duf7s2bMiJCREVFRUiF9//VX07t1b6PX6OrefOnWq2LJlixBCiHnz5olPPvmkSeeXmGrJkiXizTffrPdrPXv2FHl5eWa9/9reeOMNsXbt2nuO0ev1onfv3uLixYuisrJShISEiLNnz5o928GDB4VOpxNCCPHSSy+Jl156qd5xljhm9zoGt09J+vjjj8W8efOEEEJs2bJFTJs2zayZhBDi6tWrIjExUQghRHFxsfDz86vz2Bw9elSMGzfO7Fnudr/HZe/evWLMmDHCaDSKEydOiEGDBlkwXfVj2rlzZ5GVlfW76y15vL7//nuRmJgogoKC7ly3bNkysXr1aiGEEKtXr673eV9QUCB8fHxEQUGBKCwsFD4+PqKwsLC+u7D8aW8BAQHo169fnesTEhIwY8YMODo6wsfHB76+vjh58mSdF4YjR45gypQpAIDZs2dj9+7dzRmvzv1t374d0dHRZruP5tbYjxM3l1GjRsHOrvr938GDByM7O9vs99kQU45BQkICZs+eDQCYMmUKDh8+DNG490oazcvLC+Hh4QCA9u3bIyAgADk5OWa9z+aSkJCAJ598EoqiYPDgwbhx4wZyc3Mtdv+HDx9Gnz590LNnT4vd592GDx8ONze3311X+3nUUB8dPHgQUVFRcHNzQ8eOHREVFYUDBw40PYipzd2YPwC+A6CpdXkDgCdqXf4cwJS7buMOIKPW5e4AUs2Rr+b7DwegvcfXMwEkAUgEEGOuHLXu788AsgCcAfB3AB3rGTMFwGe1Lv83gA3mznZXhn/VfiwtfczudQwAHKj5byoA71pjLgJwt+Ax6gXgMgCXu65/GEABgNMA9gMIslCeez4uAPYAGFbr8uHaP78WyPd3AAvrud6ix6vmcUutdflGrb8rtS/Xun4pgP+tdfk1AEubmqHRp70pinIIQJd6vrRCCGH+6ZoJTMwYDWDLPb7NMCFEjqIongC+VRQlXQhxzFy5AGwE8H+oPpPl/wC8B2DOg9xfc2W7fcwURVkBQA8groFv0+zHrDGEKR9NNTNFUZwB7ASwWAhRfNeXkwD0FEKUKIoyFsBuAH4WiCX1cbkXRVEcAEwA8Eo9X5Z1vOoQQghFUcx+llmjC1kI8UgT7icH1TPe27xrrqutAEAHRVHshBD6BsY0S0ZFUewATAIQcY/vkVPz3+uKonwNYBCAB3oSm3rsFEX5G6pnLXcz5Tg2iQnH7CkA4wGMFDVTgXq+R7Mfs3qYcgxuj8mueaxdUf38MitFUexRXcZxQohdd3+9dkELIfYpivKJoijuQojGbxDTCCY8LmZ7XpngUQBJQohrd39B1vGq5ZqiKF5CiFxFUbwAXK9nTA6qZ/K3eaN6haBJLHXa2zcAZiiK4qgoig+qX+V+t4hc80N+FNW/kgLAbADmmnE/AiBdCFHvYqiiKO0URWl/++8ARqH612CzqXnAb5vYwP2dAuCnKIpPzcxiBqqPrVkpijIGwEsAJgghyhoYY6ljZsox+AbVzx+g+vl0pKEXkeaiVP87UZ8D+EUI8X4DY7rUjIOiKINQ/fNn1hcKEx+XbwA8qVQbDOCmEMJSi8gN/qYq43jdpfbzqKE+OghglKIoHRVF6Yjq41v/aWSmaOY1mIkAsgFUArgG4GCtr61A9VreOQCP1rp+H4CuNX/vjeqizgCwA4CjmdaKvgTw7F3XdQWwr1aO0zV/zqL613Zzr6N9BeBnVK8hfwPA6+5cNZfHAjhfcyzNnqvmPjMAXAGQUvPnU5nHrL5jAGAlql8wAKBNzfMno+b51NsCx2gYqpebztQ6TmMBPHv7uQZgYc2xOQ3gPwCGWiBXvY/LXbkUAB/XHM+fYaH1YwDtUF2wrrWuk3K8UP2ikAtAV9NhzwDohOr19AsADgFwqxmrwe/fx5hT81zLAPD0g+Ro7Cf1iIjITPhJPSIilWAhExGpBAuZiEglWMhERCrBQiYiUgkWMhGRSrCQiYhUgoVMRKQS/x9zHfHAR+1qFwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def relu(x):\n",
" \"\"\"\n",
" relu 函数\n",
" :param x: np.array 格式数据\n",
" :return: relu 函数\n",
" \"\"\"\n",
" \n",
" y = np.maximum(0,x)\n",
"\n",
" return y\n",
"\n",
"# 绘制 relu 函数\n",
"plot_activation_function(relu)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"优点：\n",
"+ 因为它线性、非饱和的形式，ReLU在SGD中能够快速收敛。\n",
"+ 有效缓解了梯度消失的问题。\n",
"+ 提供了神经网络的稀疏表达能力。\n",
"\n",
"缺点：\n",
"+ 随着训练的进行，可能会出现神经元死亡，权重无法更新的情况。如果发生这种情况，那么流经神经元的梯度从这一点开始将永远是0。也就是说，ReLU神经元在训练中不可逆地死亡了。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 单层感知机实例"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"我们根据上面的定义可以编写一个简单的感知机模型。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def perceptron(x, w, threshold):\n",
" \"\"\"\n",
" 感知机模型\n",
" :param x: 输入数据 np.array 格式\n",
" :param w: 权重 np.array 格式，需要与 x 一一对应\n",
" :param threshold: 阀值\n",
" :return: 0或者1\n",
" \"\"\"\n",
" x = np.array(x)\n",
" w = np.array(w)\n",
" \n",
" #计算信息加权总和\n",
" y_sum = \n",
" \n",
" # 计算输出大于阀值返回 1，否则返回 0\n",
" output = \n",
" return output\n",
"\n",
"\n",
"# 输入数据\n",
"x = np.array([1, 1, 4])\n",
"# 输入权重\n",
"w = np.array([0.5, 0.2, 0.3])\n",
"# 返回结果\n",
"perceptron(x, w, 0.8)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def perceptron(x, w, threshold):\n",
" \"\"\"\n",
" 感知机模型\n",
" :param x: 输入数据 np.array 格式\n",
" :param w: 权重 np.array 格式，需要与 x 一一对应\n",
" :param threshold: 阀值\n",
" :return: 0或者1\n",
" \"\"\"\n",
" x = np.array(x)\n",
" w = np.array(w)\n",
" \n",
" #计算信息加权总和\n",
" y_sum = np.sum(w * x)\n",
" \n",
" # 计算输出大于阀值返回 1，否则返回 0\n",
" output = 1 if y_sum > threshold else 0\n",
" return output\n",
"\n",
"\n",
"# 输入数据\n",
"x = np.array([1, 1, 4])\n",
"# 输入权重\n",
"w = np.array([0.5, 0.2, 0.3])\n",
"# 返回结果\n",
"perceptron(x, w, 0.8)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3 神经网络"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.1 神经网络的结构"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"神经网络：相互连接形成一个无环图的神经元集合。\n",
"\n",
"神经网络通常由不同的神经元层组织而成，常见的层类型有：\n",
"+ 全连接层\n",
"+ 卷积层\n",
"+ 池化层\n",
"+ Dropout层\n",
"+ RNN层\n",
"+ ......"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"首先，我们来看一个完全由全连接层联结起来的神经网络，也被称为多层感知器(Multilayer Perceptrons, MLP)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_fullconnect.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经网络架构示意图如下：\n",
"\n",
"<img src=\"http://imgbed.momodel.cn//20200103111837.png\" width=400>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"与感知机的不同，神经网络：\n",
"+ 输入层和输出层之间存在若干隐藏层。\n",
"+ 每个隐藏层中包含若干神经元。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 补充材料：梯度下降与误差反向传播"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 正向传播 与 反向传播"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"\n",
"\n",
"<img src=\"http://imgbed.momodel.cn//20200103111837.png\" width=400>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ 前馈神经网络接收一个输入 $x$ ，并产生一个输出 $\\hat{y}$ 信息正向流过网络，称为**正向传播(forward propagation)**"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ 训练过程中：正向传播可以一直继续下去直到产生一个标量代价\n",
"$J(\\theta)$ ."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ 反向传播：允许信息从代价出发，反向流过网络"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 误差反向传播"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ **核心问题**：给定某个函数 ，计算f在点x处的梯度$\\epsilon{f(x)}$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ 在神经网络中， 对应于**损失函数**，而输入x 则是由**训练数据**和神经网络**权重参数**构成"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"+ 反向传播：通过递归地应用链式法则来计算表达式梯度的一种方法"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"反向传播指的仅仅是计算梯度的方法，并不是多层神经网络的整个学习算法！"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"简单表达式与梯度解释：\n",
"\n",
"梯度：是一个向量，表示函数在该点处沿着该方向变化最快，变化率最大\n",
"\n",
"而对于单变量的情况下，梯度也就等同于它的导数。\n",
"\n",
"<img src=\"https://imgbed.momodel.cn/20200327040148.png\" width=800>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"复合表达式和链式法则：\n",
"\n",
"<img src=\"https://imgbed.momodel.cn/20200327040301.png\" width=800>\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"反向直播的直观理解：\n",
"$f(x,y,z) = (x+y)z， q=x+y， f=qz$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<img src=\"https://imgbed.momodel.cn/20200327040327.png\" width=600>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.3 神经网络实现MNIST手写数字分类"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"好，我们接下来，依旧使用MNIST手写数字数据集来实现一个神经网络的分类方法。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"采用 **keras** 框架搭建一个神经网络解决手写体数字识别问题。 \n",
"1. 导入相关包"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# 数据集 mnist\n",
"from tensorflow.keras.datasets import mnist\n",
"# 序列模型 Sequential\n",
"from tensorflow.keras.models import Sequential\n",
"# 神经网络层 Dense,Activation,Dropout\n",
"from tensorflow.keras.layers import Dense, Activation, Dropout\n",
"# 工具 np_utils\n",
"from tensorflow.python.keras.utils.np_utils import to_categorical\n",
"\n",
"import warnings\n",
"warnings.filterwarnings(\"ignore\")\n",
"\n",
"!mkdir -p ~/.keras/datasets\n",
"!cp ./mnist.npz ~/.keras/datasets/mnist.npz"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. 下载 **MNIST** 数据集并将它们转换为模型所能使用的格式。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# 获取数据\n",
"(X_train, y_train),(X_test,y_test) = mnist.load_data()\n",
"\n",
"# 将训练集数据形状从（60000,28,28）修改为（60000,784）\n",
"X_train = X_train.reshape(len(X_train), -1)\n",
"X_test = X_test.reshape(len(X_test), -1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"通过plot查看数据集情况"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAToAAAEyCAYAAABqERwxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xm0FMX1B/DvlQCiiAKaJ24sCkFUFNwQOUAUN1xwiagBBUPE4woeNeISl7iBJp7gLiqCygkhQQGNBAmCuCARDf4iiwIq8pRF3FhUCFq/P6bnvvseM8zW3dPd8/2c8867r2bpGq9TdHVVV4lzDkRESbZduStARBQ0NnRElHhs6Igo8djQEVHisaEjosRjQ0dEiceGjogSr6SGTkROFJEPRGSpiAzzq1JUOOYiGpiHaJJiJwyLSD0AHwI4DkA1gLcBnOecW+hf9SgfzEU0MA/R9bMSXnsEgKXOuY8AQETGA+gDIGtSRYS3YZRmrXNutwzlBeWCeSiZL3nwnsNclCZbLmoppeu6J4AV5u9qr6wWERksIvNEZF4Jx6KU5VnKc+aCefBV0XkAmAufZctFLaWc0eXFOTcKwCiA/3qVE/MQHcxF+Eo5o/sMwN7m7728MgofcxENzENEldLQvQ2grYi0FpEGAM4FMMWfalGBmItoYB4iquiuq3Nui4hcDmAagHoARjvnFvhWM8obcxENzEN0FT29pKiD8XpEqd5xzh1W6pswDyXzJQ8Ac+GDvHLBOyOIKPHY0BFR4rGhI6LEY0NHRIkX+IRholIceuihGl9++eUaX3DBBQCAp59+WsseeOABjd99990QakdxwTM6Iko8NnRElHgVOY+uXr16Gu+88845n2+7TDvssAMA4Be/+IWWXXbZZRr/8Y9/1Pi8887T+IcfftB4+PDhGt922235VhuokHl0hxxyiMavvPKKxk2aNNnm67799luNmzdv7n/FanAeXQGOPfZYjceNG6dxjx49NP7ggw+KfXvOoyMiAtjQEVEFSMyo6z777KNxgwYNNO7atSsAoFu3blq2yy67aHzWWWcVdbzq6mqN77//fo3POOMMjdevX6/xe++9p/Grr75a1DGT7IgjjtB44sSJGttLC/YyS/q/7ebNm7XMdle7dOmisR2Btc+Pm+7du2tsP+vzzz9fjurk7fDDD9f47bffLksdeEZHRInHho6IEi/WXddso3P5jKQW66effgIA3HTTTVq2YcMGje2o0sqVKzX++uuvNS5hhCn20qPWANC5c2eNn332WY1btGiR832WLFkCALjnnnu0bPz48Rq/8cYbGttc3X333QXWODp69uypcdu2bTWOYtd1u+1qzqFat26tccuWLTUWkfDqE9qRiIjKhA0dESVerLuun376qcZffvmlxsV2XefOnavxN998o/Evf/lLjdOjds8880xRx6h0jz32mMZ2QnWh0t3exo0ba5kdzbbdvI4dOxZ9nChJ398LAHPmzCljTXKzlx8uuugije0lisWLF4dWH57REVHisaEjosSLddf1q6++0vjaa6/V+JRTTtH4P//5D4Dak3qt+fPna3zcccdpvHHjRo0POOAAjYcMGVJCjStXermlk08+WcuyjbrZLugLL7ygsb2P+PPPPwdQk1+g9sj2Mccck/M4cWNHMqPuiSeeyFieHi0PW87/ciIyWkTWiMj7pqyZiEwXkSXe76bBVpM8rZiLSGAeYiaffyLGADixTtkwADOcc20BzPD+puCtBXMRBcxDzOTsujrnZotIqzrFfQD09OKxAGYBuM7HehVs0qRJGtvJw+l7Ig8++GAtGzRokMa2O2S7q9aCBTVbcw4ePLj0yhZvA4Cv6pRFLhdpdkL39OnTAdReasneuzp16lSN7WisXcrHTvxNd42++OILLbP3E6cndgO1u8t2knIJqxCHmof0qHFVVZUfbxeKbDMf0v8fhK3Ya3RVzrn0tP9VALJmQEQGAyhr65BweeWCeQgcvxMRVvJghHPObWvxQOfcKACjgPAWGVy3bt1WZXZRRsvO8fnrX/+qsT0jiItt5SKsPLRr105jO0CU/hd+7dq1WmZvkRs7dqzG9pa6f/zjHxnjQjRq1Ejjq6++WuN+/foV9X65+P2d6N27N4DanyOK7Bmnve3L+uyzz8KqTi3FDuOsFpEWAOD9XuNflahAzEU0MA8RVmxDNwXAAC8eAGCyP9WhIjAX0cA8RFjOrquI/AWpi6y7ikg1gFsADAcwQUQGAVgOoG+QlfTDrbfeqrHdQs9e7O7Vq5fGL7/8cij1KlBrAHMQsVw0bNhQYzu4k+5yATWDQvY2pnnz5mkcVrfMLtBaglDzYPcnSbMDZFFhc2+7sR9++KHGdjHaMOUz6prthsRjs5RTcD7OshEIcxEu5iFm4jPVmoioSLG+BawQdo6cHWm1c6kef/xxjWfOnKmx7WI99NBDAGrPAat0nTp10th2V60+ffoA4H4ZfinH3gt2DuSJJ9bMl+7fvz8A4Pjjj8/4uttvv11juypQmHhGR0SJx4aOiBKvYrqu1rJlyzQeOHCgxk899ZTG559/fsZ4xx13BAA8/fTTWmYnvlai++67T2O7UojtpobdZbUrfcRx8ncuzZo1K+j59hbIdI7sLIO99tpLY7tdqJ1Ubf+bfv/99xqnF6zdtGmTlv3sZzVNyzvvvFNQXYPAMzoiSjw2dESUeBXZdbXsVnF2UUDbHTv22JrpUXfddReA2tu23XnnnRqX616+sNnFTe0qJXY0esqUKaHWybLdVVsnu9BqXKS7ifZzPProoxrfcMMNOd/D7puR7rpu2bJFy7777juNFy5cqPHo0aM1trMP7KWI1atXAwCqq6u1zE4AD3NviGx4RkdEiceGjogSr+K7rtb77+vK2Ojbt+ZWxVNPPVXj9MjsxRdfrGV213S770SS2a6JHaVbs6Zm0Q677FVQ7H229n5myy7Eev311wddJd9deumlAIDly5drWdeuXQt6D7s1aHqR2kWLFmnZW2+9VXT90ovR7rbbblr20UcfFf1+QeAZHRElHhs6Iko8dl2zsPfkPfPMMxqn9yqwEyK7d++usd0hftasWcFVMKLspNEgJ1Knu6x2Hwm7orEdAfzTn/6ksV29OG5GjBhR7ipkZGclpE2cOLEMNcmOZ3RElHg8ozPsXKNf/epXGh9++OEa2zO5NDvvaPbs2QHVLh6CnDtn5+ulz97OOeccLZs8uWZR37POOiuwelBudn5qFPCMjogSjw0dESVeRXZd7Rr8l19+ucZnnnmmxrvvvvs23+PHH3/U2F50T+JKGZnYVUpsfPrpp2s8ZMiQko9z1VVXafz73/9e4/T2iePGjdMyux8FkZXzjE5E9haRmSKyUEQWiMgQr7yZiEwXkSXe76bBV7fitWMeIqE+vxPxkk/XdQuAq51zHQB0AXCZiHQAMAzADOdcWwAzvL8pWNXMQ2TwOxEj+ewCthLASi9eLyKLAOwJoA9S2yACwFgAswBcF0gtS5Dugp53Xs1mZra72qpVq4LeL72Cg12xJMRVOr4DopEHu5KGjW2X//7779c4vQrGl19+qWVdunTR2C5uaheJtAtC2tuYpk2bBgB4+OGHi/sApfmfc+5dIBq5iAp7CaNdu3Yal3J7mV8KGowQkVYAOgGYC6DKawQBYBWAqiwvI58xD9HBXMRD3oMRItIYwEQAQ51z62zr7ZxzIpJxWywRGQxgcKkVpRTmITqYi/jIq6ETkfpIJXScc+45r3i1iLRwzq0UkRYA1mR6rXNuFIBR3vsEtkeg3Rm8Q4cOGj/44IMAgPbt2xf0ful18AHg3nvv1Tg9KbVMo6uCiOehXr16GqdX3QBqJvCuW7dOy+yqL9m8+eabGtstKG+++eaS6lmqOHwnwmYvYdj9JaIgn1FXAfAkgEXOufvMQ1MADPDiAQAm130t+a4lmIeo4HciRvI5ozsawPkA/isi6XWobwAwHMAEERkEYDmAvlleT/5pDuAY5qHsGoPfiVjJZ9T1daS6TJlsvWxBwOw2b4899pjG9j7INm3a5P1+tmtkV7lIj+oBtbd2K7N3nHOHZSgPPQ9z5szR2O4ab+8LttKjsfYSg2VHY8ePH6+xH5OOA7DBOReZ70QUHXXUURqPGTOmfBXxRKsjTUQUADZ0RJR4kb3X9cgjj9TYLqh4xBFHaLznnnvm/X52Ozc7kTW9fSEAbNy4seB6Viq7sKW9R9jupWEXxcxk5MiRGj/yyCMaL1261I8qUsjs9Jqo4RkdESUeGzoiSrzIdl3POOOMjHE2dpXfF198UeP0buR2RNXuB0Gls8tU2S0Hs20/SMkydepUAMDZZ59d5ppkxzM6Iko8sbdtBH6wBN3uUibZ5tEVhHkomS95AJgLH+SVC57REVHisaEjosRjQ0dEiceGjogSjw0dESUeGzoiSjw2dESUeGzoiCjxwr4FbC2Ajd7vJNsVwXzGlj69z1qkVsANqp5REsRn9CsPAL8TpcorF6HeGQEAIjLPr1nlURWXzxiXepYiDp8xDnUsVbk/I7uuRJR4bOiIKPHK0dCNKsMxwxaXzxiXepYiDp8xDnUsVVk/Y+jX6IiIwsauKxElHhs6Ikq8UBs6ETlRRD4QkaUiMizMYwdFRPYWkZkislBEFojIEK+8mYhMF5El3u+m5a5rGvMQHcxFSJxzofwAqAdgGYA2ABoAeA9Ah7COH+DnagGgsxfvBOBDAB0A3ANgmFc+DMCIcteVeYhOHpiLcHMR5hndEQCWOuc+cs5tBjAeQJ8Qjx8I59xK59y7XrwewCIAeyL12cZ6TxsL4PTy1HArzEN0MBchCbOh2xPACvN3tVeWGCLSCkAnAHMBVDnn0ttjrQJQVaZq1cU8RAdzERIORvhERBoDmAhgqHNunX3Mpc7VOY8nBMxDdEQpF2E2dJ8B2Nv8vZdXFnsiUh+phI5zzj3nFa8WkRbe4y0ArClX/epgHqKDuQhJSQ1dgSNGbwNoKyKtRaQBgHMBTCnl+FEgIgLgSQCLnHP3mYemABjgxQMATA64HvnmgnkIth78TkQkF7WEOWIEoDdSIzDLANxY7tEhP34AdEPqFPz/AMz3fnoDaA5gBoAlAP4FoFmAdSgoF8xDNPLAXASXi63qVMKHOQrANPP39QCuz/Eax5+Sfr7wIxcR+Bxx//ElD8xFcLmo+1NK1zWvESMRGSwi80RkXgnHopTlWcpz5oJ58FXReQCYC59ly0Utga8w7JwbBW/lAhFxQR+PMmMeooO5CF8pZ3SJHTGKIeYiGpiHiCqloUvkiFFMMRfRwDxEVNFdV+fcFhG5HMA0pEabRjvnFvhWM8obcxENzEN0hbrwJq9HlOwd58MGI8xDyXzJA8Bc+CCvXPAWMCJKPDZ0RJR4bOiIKPHY0BFR4rGhI6LEY0NHRInHho6IEi/we12T6qabbtL4tttu03i77Wr+7ejZs6fGr776aij1IgrKTjvtpHHjxo01PvnkkwEAu+22m5bdd1/NMnSbNm0KoXbbxjM6Iko8NnRElHjsuhZo4MCBAIDrrrtOy3766aeMzw3z9joiv7Rq1Upj+//5UUcdpfGBBx64zfdo0aKFxldeeaV/lSsSz+iIKPHY0BFR4rHrWqCWLVsCALbffvsy1yT+jjzySI379++vcY8ePTQ+4IADMr72mmuuAQB8/vnnWtatWzeNn332WY3nzp1bemUTqH379hoPHTpU4379+mncqFEjjVObe6WsWFGzYvz69esBAPvvv7+W9e3bV+OHH35Y48WLF5da7aLwjI6IEo8NHRElHruueejVq5fGV1xxxVaP29PxU045RePVq1cHW7EYOuecczQeOXKkxrvuuqvGtos0a9Ysje2E1HvvvXer97avs88999xzi69wQuy8884ajxgxAkDtXNjJwNksWbJE4xNOOEHj+vXrA6j9PbD5tHG58IyOiBKPDR0RJR67rlnYEbynnnpKY9sFSLPdqOXL89pPtyL87Gc1/3sddlhqWf/HH39cy3bYYQeNZ8+erfHtt9+u8euvv65xw4YNNZ4wYQIA4Pjjj8947HnzuDe0dcYZZ2j829/+Nu/XLVu2TOPjjjtOYzvqut9++5VYu+DlPKMTkdEiskZE3jdlzURkuogs8X43Dbaa5GnFXEQC8xAz+XRdxwA4sU7ZMAAznHNtAczw/qbgrQVzEQXMQ8zk7Lo652aLSKs6xX0A9PTisQBmAbgOCTJgwACN99hjj60et6OBTz/9dBhVAoANAL6qUxbZXNhJwE888cRWj0+fPl1jOwK4bt26jO9nn5Opy1pdXa3x2LFjC6tsYWKVBwA4++yzt/n4J598ovHbb7+tsb3X1XZXLTtROKqKvUZX5Zxb6cWrAFRle6KIDAYwuMjjUG555YJ5CBy/ExFW8mCEc85taxNe59woAKOA6G/Wa+f7/OY3v9HYrk7yzTffAADuuOOO8CqWp23lIqw82IGEG264wR4fQO3bgezipdnO4qwbb7xxm4/bVTK++OKL3JUNSBS/ExdddJHGgwen2tiXX35Zy5YuXarxmjVrCnrvqqqsbXpkFDu9ZLWItAAA73dh/2XIT8xFNDAPEVZsQzcFQPoi1gAAk/2pDhWBuYgG5iHCcnZdReQvSF1k3VVEqgHcAmA4gAkiMgjAcgB9s79DtNlFBidOnJjz+Q888AAAYObMmUFVaVtaA5iDiOXi5ptv1th2Vzdv3qzxtGnTANS+uP39999nfD+7MowddNhnn300Tt/uZS8hTJ4cWtsSyTxsi13l5dZbb/X1ve2CnFGVz6jreVkeOtbnulBuHzvnDstQzlyEi3mIGd4CRkSJV/G3gJ14Ys28z44dO2Z8zowZMzS2K25Usl122UXjSy+9VGO7T0a6uwoAp59++jbfz95GNG7cOI0PPfTQjM//+9//DgC455578qwx5cuOXu+44445n3/QQQdtVfbmm29qPGfOHH8qVgKe0RFR4rGhI6LEq8iuq+1GDR8+PONz7KoZ9nawb7/9NriKxUiDBg00zrawou0C/fznPwcAXHjhhVp22mmnaWy3z7O7wNuusI3Te0Js3Lix4LpXuvSqMR06dNCyW265RePevXtnfN1229WcF2Xa4tOO7No8//jjj8VX1ic8oyOixGNDR0SJVzFd10InBn/00Ucac++HrdnJwPa+UrtXw8cff6yx7XZmYrs99r5Xu+P72rVrNX7hhRcKrHHlSe/lAACdOnXSOP3/v/1vaydv21zYEVM7Q8EumppmF1o988wzNbYzFez/N2HiGR0RJR4bOiJKvIrputp7LDONGNWVbTSWUtLLVQG1R7FffPFFjZs1a6Zxeu8Bez/qmDFjNP7qq5p1LMePH6+x7V7ZcsrMjobbruZzzz231XNvu+02jV955RWN33jjDY1tDu1z7Ch5mr1scffdd2v86aefajxp0iSNN23alOVT+I9ndESUeGzoiCjxEt91PeSQQwBk3xbPst2qDz74ILA6Jc3cuXM1tt2XQnTv3l3jHj16aGwvM9iRcKphR1dtd/Taa6/N+PypU6cCqFlyDKh9KcLm8KWXXtLY3tNqR0/T9xvb7myfPn00tvcu/+tf/9J4xIgRGn/99ddb1XP+/PkZ618MntERUeIl/owuvS5+06aZt9l86623NB44cGAYVaIMGjVqpLE9i7Pz7zgYUaNevXoa2306rrnmGo3t7XHDhtXsvpj+72jP4tIbjAPAgw8+qLGdf7dkyRKNL7nkEo3Ti9A2adJEy7p27apxv379NLa3/dld4Kz0bmOtW7fO+HgxeEZHRInHho6IEi/xXdfmzZsDyD53zm6/t2HDhlDqRFuzi3RSbuktC4Ha3dXvvvtO44svvlhju7Vhly5dANReYeSkk07S2F5G+MMf/qDxU089pXGmzaztrXv//Oc/M8bnnVezM8Ovf/3rrd4DAK666qqM5aXIeUYnInuLyEwRWSgiC0RkiFfeTESmi8gS73fmi2Dkp3bMQyTU53ciXvLpum4BcLVzrgOALgAuE5EOAIYBmOGcawtghvc3BauaeYgMfidiRHKtKrHVC0QmA3jQ++npnFvpbdg7yzn3ixyvDWVXcnuKnR5JzdZ1bdOmjcbLly8PtF4+eCe9+1Qc8lCIE044QWM7d8v+/2lvB7MrppTBO3YXsHLkYuXKlRrbeW/2tqrFixdrbPd+sPtzZGK3Q7S3ckVhAc0M3smyI1stBV2jE5FWADoBmAugyjmX/q+9CkBVltcMBjA402NUHOYhOpiLeMh71FVEGgOYCGCoc26dfcyl/tnN+C+Tc26Uc+6wfFpdyo15iA7mIj7yOqMTkfpIJXSccy69DMJqEWlhTtPXBFXJfKRv9QKAXr16aZzustpbVh566CGNY7aopiDieSiWvYQQB+X+TqxatUpj23Vt2LChxgcffHDG16YvDcyePVvL7Koin3zyicYR7a4WLJ9RVwHwJIBFzrn7zENTAKR3jRkAYHLd15LvWoJ5iAp+J2IknzO6owGcD+C/IpK+y/YGAMMBTBCRQQCWA+gbTBXJaA7gGOah7BqD34lYydnQOedeR6rLlMmx/laneHbn+N13332rxz/77DON7QTLmMk2whSZPBTrtdde0zjXtnoRsME5V9bvhF3txS582rlzZ43XrKnpOY8ePVrj9Eoh5dq/oRx4CxgRJR4bOiJKvMTf60rx8P7772tslwOyo7H77ruvxmWeMFx269ev1/iZZ57JGFMNntERUeKxoSOixEtM19Xe1/fmm29q3K1bt3JUh0pw1113afzEE09ofOedd2p8xRVXAAAWLlwYXsUotnhGR0SJV/DqJSUdLIKrZsRMXis15BL1PNi9ByZMmKCxvbUvvSGzXTzS7pEQMF/yAEQ/FzGQVy54RkdEiceGjogSj13XeKmIrqtlu7F2MCK93V7Hjh21LMSBCXZdo4NdVyIigA0dEVUAdl3jpeK6rhHFrmt0sOtKRASwoSOiChD2LWBrAWz0fifZrgjmM7b06X3WIrUCblD1jJIgPqNfeQD4nShVXrkI9RodAIjIvKTvfhSXzxiXepYiDp8xDnUsVbk/I7uuRJR4bOiIKPHK0dCNKsMxwxaXzxiXepYiDp8xDnUsVVk/Y+jX6IiIwsauKxElHhs6Ikq8UBs6ETlRRD4QkaUiMizMYwdFRPYWkZkislBEFojIEK+8mYhMF5El3u+m5a5rGvMQHcxFSJxzofwAqAdgGYA2ABoAeA9Ah7COH+DnagGgsxfvBOBDAB0A3ANgmFc+DMCIcteVeYhOHpiLcHMR5hndEQCWOuc+cs5tBjAeQJ8Qjx8I59xK59y7XrwewCIAeyL12cZ6TxsL4PTy1HArzEN0MBchCbOh2xPACvN3tVeWGCLSCkAnAHMBVDnnVnoPrQJQVaZq1cU8RAdzERIORvhERBoDmAhgqHNunX3Mpc7VOY8nBMxDdEQpF2E2dJ8B2Nv8vZdXFnsiUh+phI5zzj3nFa8WkRbe4y0ArClX/epgHqKDuQhJSQ1dgSNGbwNoKyKtRaQBgHMBTCnl+FEgIgLgSQCLnHP3mYemABjgxQMATA64HvnmgnkIth78TkQkF7WEOWIEoDdSIzDLANxY7tEhP34AdEPqFPz/AMz3fnoDaA5gBoAlAP4FoFmAdSgoF8xDNPLAXASXi63qVMKHOQrANPP39QCuz/Eax5+Sfr7wIxcR+Bxx//ElD8xFcLmo+1NK1zWvESMRGSwi80RkXgnHopTlWcpz5oJ58FXReQCYC59ly0Utga8w7JwbBW/lAm4EUj7MQ3QwF+Er5YwusSNGMcRcRAPzEFGlNHSJHDGKKeYiGpiHiCq66+qc2yIilwOYhtRo02jn3ALfakZ5Yy6igXmILm5gHS/cwDoauIF1dHADayIigA0dEVUANnRElHhs6Igo8QKfMBxFI0eO1PjKK6/U+P3339f4lFNO0Xj58rwmXxNRRPGMjogSjw0dESVexXRdW7VqpXH//v01/umnnzTef//9NW7fvr3G7Lr6p127dhrXr19f4+7du2v88MMPa2zzU4jJk2uWOjv33HM13rx5c1Hvl3Q2F127dtX4rrvu0vjoo48OtU5+4hkdESUeGzoiSryK6bp+8cUXGs+ePVvj0047rRzVqQgHHHCAxgMHDgQAnH322Vq23XY1/87uscceGtvuarG3KNq8PvrooxoPHTpU43Xrau3XUtF23nlnjWfOnKnxqlWrNN59990zlscBz+iIKPHY0BFR4lVM13Xjxo0acxQ1HHfffbfGvXv3Lls9LrjgAo2ffPJJjd94441yVCdWbHeVXVcioghjQ0dEiVcxXddddtlF44MPPriMNakc06dP1zhT13XNmpqN2m2X0o7GZpswnJ7U2qNHj5LrSdml9qKOP57REVHisaEjosSrmK7rDjvsoPE+++yT8/mHH364xosXL9aYI7b5e+SRRzSeNGnSVo//73//07jQUbwmTZoAqL20lp10bNljz5vHPaMLYSdsb7/99mWsSWlyntGJyGgRWSMi75uyZiIyXUSWeL+bBltN8rRiLiKBeYiZfLquYwCcWKdsGIAZzrm2AGZ4f1Pw1oK5iALmIWZydl2dc7NFpFWd4j4AenrxWACzAFznY7189/nnn2s8ZswYjW+99daMz7fl33zzjcYPPvig31UrxAYAX9Upi2wutmzZovGKFSt8fe8TTjgBANC0ae4Tp+rqao03bdrkx+FjlQe/HHZYza6Cb731VhlrUrhir9FVOedWevEqAFXZnigigwEMLvI4lFteuWAeAsfvRISVPBjhnHPb2oTXOTcKwCggOpv13n777RpnO6OLo23lIop5KJRdQPOiiy4CADRq1Cjn626++ebA6pRJHL8T9uz722+/1diuarLvvvuGWic/FTu9ZLWItAAA7/eaHM+n4DAX0cA8RFixDd0UAAO8eACAydt4LgWLuYgG5iHCcnZdReQvSF1k3VVEqgHcAmA4gAkiMgjAcgB9g6xkkPK53ShCWgOYg4TmIq1fv34aDxtWM3i53377aWz3OMhk/vz5Gtv5ej5JXB7sgNtrr72msd32M87yGXU9L8tDx/pcF8rtY+fcYRnKmYtwMQ9c6VTeAAAG9UlEQVQxw1vAiCjxKuYWsGz82J+AMrNbTJ5//vkAgF69euV8Xbdu3TTOlRO774Pt5r700ksaf//99zmPScnGMzoiSjw2dESUeBXfdSV/HXjggRpPmTJF43xWjCmGHSEcNWpUIMeglObNm5e7CkXjGR0RJR4bOiJKPHZdKTB2v4FC9h4oZBK3ndB60kknaTx16tS8j0f5Oe2008pdhaLxjI6IEo8NHRElXsV3XfPpJnXv3l3jMi+8GXl2D4eePXtq3L9/fwDAtGnTtOyHH34o6L0HDRqk8RVXXFFkDSmXmTNnapyUe115RkdEiceGjogST8K8vzMqq6laP/74o8b5/Lfo2LEjAGDhwoWB1Wkb3smyakZBopiHfNjVbr/88sutHj/11FM1DnjU1Zc8ANHMxVlnnaXx3/72N43tPcMdOnTQuMxbgOaVC57REVHiVfxgxKOPPqrxxRdfnPP5gwen9jQZOnRoYHWizNI7f1Gw7P4Rlp0L2bBhw7Cq4wue0RFR4rGhI6LEq/iu6+LFi8tdhViyezYcf/zxGr/yyisa+7Hg5YUXXqjxyJEjS34/ym3y5Jp9fez3o3379hrbSzeXXnppOBUrQc4zOhHZW0RmishCEVkgIkO88mYiMl1Elni/c2+ZTqVqxzxEQn1+J+Iln67rFgBXO+c6AOgC4DIR6QBgGIAZzrm2AGZ4f1OwqpmHyOB3IkYKnkcnIpMBPOj99HTOrfQ27J3lnPtFjtdGbs6Q9eGHH2qcbVfy9C1jduu9ZcuWBVuxGjpnqBx5sHs53HjjjRofd9xxGrdu3VrjFStW5P3ezZo107h3794aP/DAAxrvtNNOW73Odo/t6hr2NqYA1Jq7leTvxJ///GeN7WWEqqoqjQu9lc9nec2jK+ganYi0AtAJwFwAVc65ld5DqwBUZXnNYACDCzkObRvzEB3MRTzkPeoqIo0BTAQw1Dm3zj7mUqeFGf9lcs6Ncs4d5tdM8krHPEQHcxEfeZ3RiUh9pBI6zjn3nFe8WkRamNP0NUFVMiwLFizQuE2bNhmfk2shyIAJypgHu3KL3RvC+t3vfqfx+vXr835v2/3t3LmzxtkurcyaNQsA8Mgjj2hZwN3VWirlO2HZXGzevLmMNSlcPqOuAuBJAIucc/eZh6YAGODFAwBMrvta8l1LMA9Rwe9EjORzRnc0gPMB/FdE5ntlNwAYDmCCiAwCsBxA32CqSEZzAMcwD2XXGPxOxErOhs459zpSXaZMjvW3OuVlt8uzK2FESLYRpsjk4ZJLLvH1/dasqen9vfDCCxoPGTIEQNlG/DY45yriO2E1adJE4z59+mj8/PPPl6M6BeEtYESUeGzoiCjxKv5eV8suprlo0SKN999//3JUJ3IGDhyosd2zYcCAARmenZudaP3dd99p/Nprr2lsLyfY/SgoHH371lxm3LRpk8b2+xEHPKMjosRjQ0dEiceuq2HXvj/ooIPKWJNomj9/vsZ2aZ5///vfGt9xxx0aN21as3jHpEmTAADTp0/XMrsc0KpVq/ytLPli9uzZGttLOH4swRUmntERUeJV/C5gMVPRu4BFSKJ3AYsZ7gJGRASwoSOiCsCGjogSjw0dESUeGzoiSjw2dESUeGzoiCjx2NARUeKFfQvYWgAbvd9JtiuC+YwtfXqftUitgBtUPaMkiM/oVx4AfidKlVcuQr0zAgBEZF7Sdz+Ky2eMSz1LEYfPGIc6lqrcn5FdVyJKPDZ0RJR45WjoRuV+SuzF5TPGpZ6liMNnjEMdS1XWzxj6NToiorCx60pEiceGjogSL9SGTkROFJEPRGSpiAwL89hBEZG9RWSmiCwUkQUiMsQrbyYi00Vkife7aa73CgvzEB3MRUicc6H8AKgHYBmANgAaAHgPQIewjh/g52oBoLMX7wTgQwAdANwDYJhXPgzAiHLXlXmITh6Yi3BzEeYZ3REAljrnPnLObQYwHkCfEI8fCOfcSufcu168HsAiAHsi9dnGek8bC+D08tRwK8xDdDAXIQmzodsTwArzd7VXlhgi0gpAJwBzAVQ551Z6D60CUFWmatXFPEQHcxESDkb4REQaA5gIYKhzbp19zKXO1TmPJwTMQ3REKRdhNnSfAdjb/L2XVxZ7IlIfqYSOc8495xWvFpEW3uMtAKwpV/3qYB6ig7kISZgN3dsA2opIaxFpAOBcAFNCPH4gREQAPAlgkXPuPvPQFAADvHgAgMl1X1smzEN0MBdh1ckbAQnnYCK9AfwZqdGm0c65O0M7eEBEpBuA1wD8F8BPXvENSF2TmABgH6SWROrrnPuqLJWsg3mIRh4A5iKsXPAWMCJKPA5GEFHisaEjosRjQ0dEiceGjogSjw0dESUeGzoiSjw2dESUeP8PgT8jcWwmkVIAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 360x360 with 9 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"# 查看一些图片\n",
"def plot_images(imgs):\n",
" \"\"\"绘制几个样本图片\n",
" :param show: 是否显示绘图\n",
" :return:\n",
" \"\"\"\n",
" sample_num = min(9, len(imgs))\n",
" img_figure = plt.figure(1)\n",
" img_figure.set_figwidth(5)\n",
" img_figure.set_figheight(5)\n",
" for index in range(0, sample_num):\n",
" ax = plt.subplot(3, 3, index + 1)\n",
" ax.imshow(imgs[index].reshape(28, 28), cmap='gray')\n",
" ax.grid(False)\n",
" plt.margins(0, 0)\n",
" plt.show()\n",
"\n",
"\n",
"plot_images(X_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"对图像数据进行数据处理。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"# 将数据集图像像素点的数据类型从 uint8 修改为 float32\n",
"X_train = X_train.astype('float32')\n",
"X_test = X_test.astype('float32')\n",
"\n",
"# 把数据集图像的像素值从 0-255 放缩到[-1,1]之间\n",
"X_train = (X_train - 127) / 127\n",
"X_test = (X_test - 127) / 127\n",
"\n",
"# 数据集类别个数\n",
"nb_classes = 10\n",
"\n",
"# 把 y_train 和 y_test 变成了 one-hot 的形式，即之前是 0-9 的一个数值， \n",
"# 现在是一个大小为 10 的向量，它属于哪个数字，就在哪个位置为 1，其他位置都是 0。\n",
"y_train = to_categorical(y_train, nb_classes)\n",
"y_test = to_categorical(y_test, nb_classes)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3. 搭建神经网络模型"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def create_model():\n",
" \"\"\"\n",
" 采用 keras 搭建神经网络模型\n",
" :return: 神经网络模型\n",
" \"\"\"\n",
" # 选择模型，选择序贯模型（Sequential())\n",
" model = Sequential()\n",
" # 添加全连接层，共 512 个神经元\n",
" model.add(Dense(512, input_shape=(784,)))\n",
" # 添加激活层，激活函数选择 relu \n",
" model.add(Activation('relu'))\n",
" # 添加全连接层，共 512 个神经元\n",
" model.add(Dense(512))\n",
" # 添加激活层，激活函数选择 relu \n",
" model.add(Activation('relu'))\n",
" # 添加全连接层，共 10 个神经元\n",
" model.add(Dense(10))\n",
" # 添加激活层，激活函数选择 softmax\n",
" model.add(Activation('softmax'))\n",
" return model\n",
"\n",
"# 实例化模型\n",
"model = create_model()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4. 训练和测试神经网络模型"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"938/938 [==============================] - 42s 45ms/step - loss: 0.2597 - accuracy: 0.9200\n",
"Epoch 2/5\n",
"938/938 [==============================] - 41s 43ms/step - loss: 0.1255 - accuracy: 0.9614\n",
"Epoch 3/5\n",
"938/938 [==============================] - 42s 45ms/step - loss: 0.0952 - accuracy: 0.9702\n",
"Epoch 4/5\n",
"938/938 [==============================] - 41s 44ms/step - loss: 0.0805 - accuracy: 0.9748\n",
"Epoch 5/5\n",
"938/938 [==============================] - 41s 44ms/step - loss: 0.0691 - accuracy: 0.9777\n",
"313/313 [==============================] - 1s 2ms/step - loss: 0.0867 - accuracy: 0.9732\n",
"Test loss: 0.08665528893470764\n",
"Accuracy: 0.9732000231742859\n"
]
}
],
"source": [
"def fit_and_predict(model, model_path):\n",
" \"\"\"\n",
" 训练模型、模型评估、保存模型\n",
" :param model: 搭建好的模型\n",
" :param model_path:保存模型路径\n",
" :return:\n",
" \"\"\"\n",
" # 编译模型\n",
" model.compile(optimizer='Adam',\n",
" loss='categorical_crossentropy', \n",
" metrics=['accuracy'])\n",
" # 模型训练\n",
" model.fit(X_train, y_train, epochs=5, batch_size=64)\n",
" # 保存模型\n",
" model.save(model_path)\n",
" # 模型评估，获取测试集的损失值和准确率\n",
" loss, accuracy = model.evaluate(X_test, y_test)\n",
" # 打印结果\n",
" print('Test loss:', loss)\n",
" print(\"Accuracy:\", accuracy)\n",
" \n",
"# 训练模型和评估模型\n",
"fit_and_predict(model, model_path='./model.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"那么，神经网络模型，是如何识别这一张张图片的呢？以下，我们使用通过下面几个小视频，你会更好的理解神经网络："
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"神经网络的整体模型"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media1.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"通过激活函数，神经元将每一层的“化学状态”传递到下一层。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media2.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"对于MNIST的图像分类，这里需要传递的信息就是图形信息："
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media3.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"图像信息在神经网络中的传递："
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media4.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"具体表述成函数的形式，一个第二层的神经元的值就应该是这样获得的："
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media5.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"source": [
"<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"全连接网络中的参数设置。"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"<center><video src=\"http://files.momodel.cn/nn_media6.mp4\" controls=\"controls\" width=800px></center>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 实践与体验\n",
"#### 调节神经网络结构和参数\n",
"\n",
"1. 将两层隐藏层改为一层，训练模型并在测试集上测试，得出准确率。\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def create_model1():\n",
" \"\"\"\n",
" 搭建神经网络模型 model1，比 model 少一层隐藏层\n",
" :return: 模型 model1\n",
" \"\"\"\n",
" # 选择模型，选择序贯模型（Sequential())\n",
" model = \n",
" # 添加全连接层，共 512 个神经元\n",
" model.add()\n",
" # 添加激活层，激活函数选择 relu\n",
" model.add()\n",
" # 添加全连接层，共 10 个神经元\n",
" model.add()\n",
" # 添加激活层，激活函数选择 softmax\n",
" model.add()\n",
" return model\n",
"\n",
"# 搭建神经网络\n",
"model1 = create_model1()\n",
"# 训练神经网络模型，保存模型和评估模型\n",
"fit_and_predict(model1, model_path='./model1.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. 修改两层隐藏层神经元的数量，然后训练模型得出准确率。"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"def create_model2():\n",
" \"\"\"\n",
" 搭建神经网络模型 model2，隐藏层的神经元数目比 model 少一半\n",
" :return: 神经网络模型 model2\n",
" \"\"\"\n",
" # 选择模型，选择序贯模型（Sequential())\n",
" model = Sequential()\n",
" # 添加全连接层，共 256 个神经元\n",
" model.add()\n",
" # 添加激活层，激活函数选择 relu\n",
" model.add()\n",
" # 添加全连接层，共 256 个神经元\n",
" model.add()\n",
" # 添加激活层，激活函数选择 relu\n",
" model.add()\n",
" # 添加全连接层，共 10 个神经元\n",
" model.add()\n",
" # 添加激活层，激活函数选择 softmax\n",
" model.add()\n",
" return model\n",
"\n",
"# 搭建神经网络模型\n",
"model2 = create_model2()\n",
"# 训练神经网络模型，保存模型并评估模型\n",
"fit_and_predict(model2,model_path='./model2.h5')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3. 输入一个手写数字，比较三种模型输出结果的差异，对其差异进行分析解释。"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"np.set_printoptions(suppress=True)\n",
"from tensorflow.keras.models import load_model\n",
"\n",
"# 加载模型\n",
"model = load_model('./model.h5')\n",
"model1 = load_model('./model1.h5')\n",
"model2 = load_model('./model2.h5') "
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"原始模型\n",
"其各类别预测概率:[0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]，预测值: 7，真实值:7\n",
"\n"
]
}
],
"source": [
"# 预测结果\n",
"predict_results = np.round(model.predict(X_test)[0],3)\n",
"# predict_results1 = np.round(model1.predict(X_test)[0],3)\n",
"# predict_results2 = np.round(model2.predict(X_test)[0],3)\n",
"\n",
"# 打印预测结果\n",
"print('原始模型\\n其各类别预测概率:%s，预测值: %s，真实值:%s\\n' % (predict_results,np.argmax(predict_results),np.argmax(y_test[0])))\n",
"print('只有一个隐藏层的模型\\n其各类别各类别预测概率:%s，预测值: %s，真实值:%s\\n' % (predict_results1,np.argmax(predict_results1),np.argmax(y_test[0])))\n",
"print('隐藏神经元数量更改后的模型\\n其各类别预测概率:%s，预测值: %s，真实值:%s' % (predict_results2,np.argmax(predict_results2),np.argmax(y_test[0])))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.5"
},
"pycharm": {
"stem_cell": {
"cell_type": "raw",
"metadata": {
"collapsed": false
},
"source": []
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
