Learning Spark

[toc]

Intro Distributed Deep Learning

We’ll look at how the training of deep learning models can be significantly accelerated with distributed computing on GPUs, as well as discuss some of the challenges and examine current research on the topic. The original post is here, the reason I re-copy the post is that it’s and my equations both do not render properly in browser because…

Mxnet Seq2seq Rnn

source code

Seq2seq 基础知识

In this post we will learn the foundations behind sequence to sequence models and how neural networks can be used to build powerful models capable of analyzing data that varies over time.

Bptt反向传播算法 In Rnn

In this part we’ll give a brief overview of BPTT and explain how it differs from traditional backpropagation. We will then try to understand the vanishing gradient problem which has led to the development of LSTMs and GRUs, two of the currently most popular and powerful models used in NLP (and other areas). 我的参考／学习资料

Rnn Lstm 基础知识

[TOC]

Dcgan 内容总结

未开始。 目标：为DCGAN 写一个MXNet 版本的源码分析。

Deep Learning 总结

这篇文章的一些截图均来自“一天学会深度学习” – 一个301页的幻灯片，深入浅出的讲解了深度学习的主要内容(我个人非常推荐初学者进行学习)。另外，在知乎，也有非常多的经验分享。

Nnvm 源码阅读

未完成。

C C++ Learning Note

C/C++ 学习笔记。 自己因为不是科班出身，C／C++，Python的基础都不够好。只能遇到不懂，就查一查记录下来。这次在这里记录的，也都是想看tinyflow，nnvm里面的内存管理部分，而遇到的新知识。

Mxnet Memonger 内存优化

别人训练好的模型我们常常不知道有哪些层，这时候需要列出所有的层，以便于我们找到特征层：

internals=model.symbol.get_internals()   #list all symbol
internals.list_outputs()
1
2
1
2

列出网络中所有的层，像这样：

['data',
 'conv1_weight',
 'conv1_bias',
 'conv1_output',
 'slice1_output0',
 'slice1_output1',
 '_maximum0_output',
 ……
 ……
 'slice_fc1_output0',
 'slice_fc1_output1',
 '_maximum9_output',
 'drop1_output',
 'fc2_weight',
 'fc2_bias',
 'fc2_output',
 'softmax_label',
 'softmax_output']

在下面，我们将推断所有的需要作为输入数据的模型的参数

>>> net = mx.symbol.Variable('data')
>>> net = mx.symbol.FullyConnected(data=net, name='fc1', num_hidden=10)
>>> arg_shape, out_shape, aux_shape = net.infer_shape(data=(100, 100))
>>> dict(zip(net.list_arguments(), arg_shape))
{'data': (100, 100), 'fc1_weight': (10, 100), 'fc1_bias': (10,)}
>>> out_shape
[(100, 10)]

Cs231n Visualizing Convnets

Visualizing what ConvNets learn.

Cs231n Convolutional Networks

ConvNet architectures make the explicit assumption that the inputs are images, which allows us to encode certain properties into the architecture.

Cs231n Neural Networks

Mxnet Mnist Cnn

The code of this work is on github. About how we design the “hyper parameters” is from the CS231 class