复杂网络分类器

来自集智百科
跳转到: 导航搜索

目录

算法说明

要对复杂网络进行分类,首先要对网络有一个准确的描述,然后才能放入机器学习算法中进行训练。网络作为明显区分于图像和时间序列的一种数据,在很多研究中,都将网络转化成矢量嵌入到高维空间中。在这个算法中,采用了DeepWalk的算法将网络嵌入到高维空间中,然后再进行降维后得到二维图片,然后再用CNN进行训练和测试。

deepWalk:主要是利用word2vec技术将网络嵌入到向量空间中,所以关键在于如何利用网络生成语料库:给定一个网络,选择一个初始节点,然后从其邻居中随机选择下一步移动到的节点,重复这一选择过程,到达步长为l,则称完成一次随机游走。记录这一次游走的路线,它就是广义的“句子”,每一个经过的节点即广义的“单词”。重复若干次这种随机游走即可得到一系列“句子”,这就是我们需要的语料库。将它给word2vec即可完成嵌入。

基于流网络的deepWalk详见Deepwalk

PCA降维算法:PCA(Principal Component Analysis)是一种常用的数据分析方法。它通过线性变换将原始数据变换为一组各维度线性无关的表示,可用于提取数据的主要特征分量,常用于高维数据的降维。使用deepWalk将网络嵌入到高维空间后,使用PCA对其进行降维放在欧氏二维坐标系中,生成图片。

算法程序

network_dw.py:包含word2vector和pca方法的python文件,方便后边程序调用。

import random
import networkx as nx
import numpy as np
from numpy import *
import gensim
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
 
def randomWalk(g,num):
    corpus = []
    n = g.nodes()
    for i in range(num):
        seq = []
        s = random.choice(n)
# random.randint(len(n))
        for j in range(10):
            nei = g.neighbors(s)
            next_node = random.choice(nei)
            seq.append(str(next_node))
            s = next_node
        corpus.append(seq)
    return corpus
# word2vec
def word_2_vec(sequences, dim):
    model = gensim.models.Word2Vec(sequences, min_count=1, size=dim)
 
    nodes = model.vocab.keys()
    em_vec = []
    for node in nodes:
        em_vec.append(model[node].tolist())
#   print np.mat(em_vec)
    return (np.mat(em_vec),model,nodes)
#二维数据栅格化 
#还有一点问题,没有处理数据超过255的情况,待修改!
def img_max(lowdim,size):
 
    #确定二维数据的x,y范围,分为128个
    xmax = np.amax(lowdim, axis=0)[0]
    ymax = np.amax(lowdim, axis=0)[1]
    xmin = np.amin(lowdim, axis=0)[0]
    ymin = np.amin(lowdim, axis=0)[1]
 
    x_scale = np.linspace(xmin, xmax, size)
    y_scale = np.linspace(ymin, ymax, size)
 
    #生成一个空128*128矩阵
    new_mat = np.zeros((size, size))
 
    #根据索引填充128*128矩阵
    for each in lowdim:
        x = each[0]
        y = each[1]
        for idx, each in enumerate(x_scale):
            if idx < len(x_scale) - 1:
                if x >= each and x < x_scale[idx + 1]:
                    i = idx
            else:
                if x == each:
                    i = idx
        for idx, each in enumerate(y_scale):
            if idx < len(y_scale) - 1:
                if y > each and y < y_scale[idx + 1]:
                    j = idx
            else:
                if y == each:
                    j = idx
        new_mat[i][size - 1 - j] += 1
    return new_mat

实验

实验的思路是什么? Jake讨论

对上边的算法,打算从三个方面进行实验,看一下网络嵌入是否合理。

首先是无权无向的跆拳道网络,原始数据已经对网络中的节点进行了分类,对本算法进行实验后,得到网络各个节点的向量,降维后在二维图上看节点是否与原数据的分类情况一致。

然后是两个分离的网络,实验后观察网络嵌入是否体现了这个特征,也就是看在二维图上是否是分离的两堆节点。

最后是对复杂网络中经典的随机网络、无标度网和小世界网进行试验,观察它们在二维坐标上的形态。

1、跆拳道网络

使用跆拳道网络对算法进行试验,根据原始数据,对网络中的节点进行染色,然后观察节点在二维图上的情况来看算法的效果。跆拳道网络中包含了34个节点,用上边的算法,对跆拳道网络随机游走10000次,得到每一个节点的20维的向量,然后进行降维得到二维图像。

程序

from network_dw import *
 
#karate data
g = nx.Graph()
g.add_edges_from([(2,1),(3,1),(3,2),(4,1),(4,2),(4,3),(5,1),(6,1),(7,1),(7,5),(7,6),(8,1),(8,2),(8,3),(8,4),(9,1),(9,3),(10,3),(11,1),(11,5),(11,6),(12,1),(13,1),(13,4),(14,1),(14,2),(14,3),(14,4),(17,6),(17,7),(18,1),(18,2),(20,1),(20,2),(22,1),(22,2),(26,24),(26,25),(28,3),(28,24),(28,25),(29,3),(30,24),(30,27),(31,2),(31,9),(32,1),(32,25),(32,26),(32,29),(33,3),(33,9),(33,15),(33,16),(33,19),(33,21),(33,23),(33,24),(33,30),(33,31),(33,32),(34,9),(34,10),(34,14),(34,15),(34,16),(34,19),(34,20),(34,21),(34,23),(34,24),(34,27),(34,28),(34,29),(34,30),(34,31),(34,32),(34,33)])
 
plt.figure(1)
nx.draw(g,with_labels=True)
plt.savefig('karate_data')
 
#deepwalk
corp = randomWalk(g,10000)
vec,mod,nod = word_2_vec(corp,20)
 
#PCA
pca = PCA(n_components=2)
lowdim = pca.fit_transform(vec)
 
#plot
type_1 = ['5','6','7','11','17']
type_2 = ['1','2','4','8','12','13','14','18','20','22']
type_3 = ['3','10','25','26','28','29','32']
type_4 = ['9','15','16','19','21','23','24','27','30','31','33','34']
 
plt.figure(2)
type_list = [type_1,type_2,type_3,type_4]
color = ['blue','red','green','purple']
for each in type_list:
    m = type_list.index(each)
    for i in each:
        idx = nod.index(i)
        cor = lowdim[idx]
        plt.scatter(cor[0],cor[1],c=color[m],s=100)
plt.savefig('karate_w2v')
plt.show()

结果

(1)跆拳道网络图

Karate data.png

(2)二维图

Karate w2v.png

从二维图中可以看出来之前网络中染色的节点在二维图上基本可以看成一类,所以算法的效果还好。

2、两个不同的随机网络

在一个空间上随机生成两个独立的网络,然后用上边的算法生成二维图片之后观察效果。

程序

from network_dw import *
#1.create network
G = nx.random_graphs.erdos_renyi_graph(50, 0.2)
H = nx.random_graphs.erdos_renyi_graph(50, 0.2)
g = nx.disjoint_union(G,H)
plt.figure(1)
nx.draw(g,with_labels=False,s=200)
plt.savefig('two_network')
 
#2.deepWalk
corp = randomWalk(g,20000)
vec,mod,nod = word_2_vec(corp,20)
 
#3.pca
pca = PCA(n_components=2)
lowdim = pca.fit_transform(vec)
 
#4.plot
plt.figure(2)
par = zip(*lowdim)
x = par[0]
y = par[1]
plt.scatter(x,y,s=100)
plt.savefig('2n_w2v')
 
plt.show()

结果

(1)网络图

Two network.png

(2)二维图

2n w2v.png

从上图可以看出两个不用的网络在二维图上也是明显分开的,所以可以说算法实现了将网络转化成二维图片。

3、复杂网络

选用复杂网络中经典的随机网络、无标度网和小世界网进行试验,观察它们转化成二维图片后的效果。随机生成的三种网络都是有1000个节点,约有100000条连边数。

程序

from network_dw import *
 
#随机网
E = nx.random_graphs.erdos_renyi_graph(1000,0.2)
#print len(nx.edges(E))
#无标度网
B = nx.random_graphs.barabasi_albert_graph(1000,110)
#print len(nx.edges(B))
#小世界网
W = nx.random_graphs.watts_strogatz_graph(1000,200,0.4)
#print len(nx.edges(W))
 
#g = nx.disjoint_union(E,B)
g = E
#g = B
#g = W
 
plt.figure(1)
nx.draw(g)
plt.show()
 
#2.deepWalk
corp = randomWalk(g,10000)
vec,mod,nod = word_2_vec(corp,20)
 
#3.pca
pca = PCA(n_components=2)
lowdim = pca.fit_transform(vec)
 
#4.plot
plt.figure(2)
par = zip(*lowdim)
x = par[0]
y = par[1]
plt.scatter(x,y,s=100,color='w',edgecolors='b')
plt.savefig('2n_dif')
 
plt.show()

结果

(1)随机网

2n dif 1.png

(2)无标度网

2n dif 2.png

(3)小世界网

p值从0.1,0.2逐渐递增至1.0

可以看出的是小世界网在p值取0.1-0.3的时候图片特征比较明显,但是随机网和无标度网的二维图片的特征不是很明显,所以有待进一步训练看一下效果。

数据预处理

1.栅格化

对嵌入二维空间中的网络节点向量进行栅格化处理,转化成可以作为cnn输入的图片文件。方法是调用network_dw.py文件中的栅格化的方法处理成48*48的图片数据。初步的效果如下:

跆拳道网络

Karate 48.png

随机网络

Random 48.png

无标度网络

Ba 48.png

小世界网络

Ws 48.png

CNN训练及测试

首先生成训练和测试的数据集。

数据集现在一共有:

数据集.png

然后用tensorflow,将数据集放入cnn中训练和测试,后来加入了validation的数据,都是为了查看训练效果。训练结果良好,但是存在不稳定的现象(初始值依赖)。

1.生成训练及测试的数据集

# -*- coding: utf-8 -*-
"""
Created on Sat Jul 30 20:30:07 2016
 
@author: xinruyue
"""
 
from network_dw import *
import pickle
import multiprocessing
import datetime
 
def processing():
    #随机网
    E = nx.random_graphs.erdos_renyi_graph(1000,0.2)
#    print len(nx.edges(E))
    #无标度网
    B = nx.random_graphs.barabasi_albert_graph(1000,110)
#    print len(nx.edges(B))
    #小世界网
    W = nx.random_graphs.watts_strogatz_graph(1000,200,0.5)
#    print len(nx.edges(W))
 
    g = W
 
    #2.deepWalk
    corp = randomWalk(g,10000)
    vec,mod,nod = word_2_vec(corp,20)
 
    #3.pca
    pca = PCA(n_components=2)
    lowdim = pca.fit_transform(vec)
 
    #4.栅格化
    img_data = img_max(lowdim,48)
 
    return img_data
 
def subprocessing(lis):
    print(str(datetime.datetime.now()) + "\tPID = " + str(multiprocessing.current_process().pid) + " start...")
    img_batch = []
    for i in range(lis):
        i_matrix = processing()
        img_batch.append(i_matrix)
    print(str(datetime.datetime.now()) + "\tPID = " + str(multiprocessing.current_process().pid) + " finished.")
    return img_batch
 
if __name__ == "__main__":
    process = multiprocessing.cpu_count() - 2
    print process
    pool = multiprocessing.Pool(processes=process)
    result = []
    for i in xrange(5):
        result.append(pool.apply_async(subprocessing, (1000,)))
    pool.close()
    pool.join()
 
    img_batch = []
    for res in result:
        r = res.get()
        for i in r:
            img_batch.append(i)
    output_1 = open('train_batch_05.pkl', 'wb')
    pickle.dump(img_batch, output_1)
    print 'done'
    output_1.close()

2.使用tensorflow在CNN中训练

# coding: utf-8
import tensorflow as tf
import numpy
import cPickle as pickle
import random
 
# We'll bundle groups of examples during training for efficiency.
# This defines the size of the batch.
BATCH_SIZE = 100
# We have only one channel in our grayscale images.
NUM_CHANNELS = 1
# The random seed that defines initialization.
SEED = random.randint(0,9999)
 
IMAGE_SIZE = 48
 
NUM_LABELS = 2
 
VALIDATION_SIZE = 1000
 
# prepare train datas and labels
f1 = open("train_batch_3.pkl",'rb')
content1 = pickle.load(f1)
f2 = open("train_batch_4.pkl",'rb')
content2 = pickle.load(f2)
dummy_train_data = content1 + content2
 
dummy_train_labels = numpy.zeros((10000,2))
dummy_train_labels[:5000, 0 ] = 1
dummy_train_labels[5000:, 1 ] = 1
 
data_label_pair = zip(dummy_train_data, dummy_train_labels)
random.shuffle(data_label_pair)
 
train_data_temp = zip(*data_label_pair)[0]
train_labels_temp = zip(*data_label_pair)[1]
 
train_data = numpy.array(train_data_temp).reshape((10000,IMAGE_SIZE,IMAGE_SIZE,1)).astype(numpy.float32)
train_labels = numpy.array(train_labels_temp)
 
validation_data = train_data[:VALIDATION_SIZE, :, :, :]
validation_labels = train_labels[:VALIDATION_SIZE]
train_data = train_data[VALIDATION_SIZE:, :, :, :]
train_labels = train_labels[VALIDATION_SIZE:]
 
train_size = train_labels.shape[0]
 
# prepare test datas and labels
f3 = open("train_batch_1.pkl",'rb')
content3 = pickle.load(f3)
f4 = open("train_batch_2.pkl",'rb')
content4 = pickle.load(f4)
dummy_test_data = content3 + content4
 
dummy_test_labels = numpy.zeros((1200,2))
dummy_test_labels[:600, 0 ] = 1
dummy_test_labels[600:, 1 ] = 1
 
test_data_label_pair = zip(dummy_test_data, dummy_test_labels)
random.shuffle(test_data_label_pair)
 
test_data_temp = zip(*test_data_label_pair)[0]
test_labels_temp = zip(*test_data_label_pair)[1]
 
test_data = numpy.array(test_data_temp).reshape((1200,IMAGE_SIZE,IMAGE_SIZE,1)).astype(numpy.float32)
test_labels = numpy.array(test_labels_temp)
 
# This is where training samples and labels are fed to the graph.
# These placeholder nodes will be fed a batch of training data at each
# training step, which we'll write once we define the graph structure.
train_data_node = tf.placeholder(
  tf.float32,
  shape=(BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS))
train_labels_node = tf.placeholder(tf.float32,
                                   shape=(BATCH_SIZE, NUM_LABELS))
 
# For the validation and test data, we'll just hold the entire dataset in
# one constant node.
validation_data_node = tf.constant(validation_data)
test_data_node = tf.constant(test_data)
 
# The variables below hold all the trainable weights. For each, the
# parameter defines how the variables will be initialized.
conv1_weights = tf.Variable(
  tf.truncated_normal([5, 5, NUM_CHANNELS, 32],  # 5x5 filter, depth 32.
                      stddev=0.1,
                      seed=SEED))
conv1_biases = tf.Variable(tf.zeros([32]))
conv2_weights = tf.Variable(
  tf.truncated_normal([5, 5, 32, 64],
                      stddev=0.1,
                      seed=SEED))
conv2_biases = tf.Variable(tf.constant(0.1, shape=[64]))
fc1_weights = tf.Variable(  # fully connected, depth 512.
  tf.truncated_normal([IMAGE_SIZE / 4 * IMAGE_SIZE / 4 * 64, 512],
                      stddev=0.1,
                      seed=SEED))
fc1_biases = tf.Variable(tf.constant(0.1, shape=[512]))
fc2_weights = tf.Variable(
  tf.truncated_normal([512, NUM_LABELS],
                      stddev=0.1,
                      seed=SEED))
fc2_biases = tf.Variable(tf.constant(0.1, shape=[NUM_LABELS]))
 
 
def model(data, train=False):
    """The Model definition."""
    # 2D convolution, with 'SAME' padding (i.e. the output feature map has
    # the same size as the input). Note that {strides} is a 4D array whose
    # shape matches the data layout: [image index, y, x, depth].
    conv = tf.nn.conv2d(data,
                      conv1_weights,
                      strides=[1, 1, 1, 1],
                      padding='SAME')
 
    # Bias and rectified linear non-linearity.
    relu = tf.nn.relu(tf.nn.bias_add(conv, conv1_biases))
 
    # Max pooling. The kernel size spec ksize also follows the layout of
    # the data. Here we have a pooling window of 2, and a stride of 2.
    pool = tf.nn.max_pool(relu,
                        ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1],
                        padding='SAME')
    conv = tf.nn.conv2d(pool,
                      conv2_weights,
                      strides=[1, 1, 1, 1],
                      padding='SAME')
    relu = tf.nn.relu(tf.nn.bias_add(conv, conv2_biases))
    pool = tf.nn.max_pool(relu,
                        ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1],
                        padding='SAME')
 
    # Reshape the feature map cuboid into a 2D matrix to feed it to the
    # fully connected layers.
    pool_shape = pool.get_shape().as_list()
    reshape = tf.reshape(
      pool,
      [pool_shape[0], pool_shape[1] * pool_shape[2] * pool_shape[3]])
 
    # Fully connected layer. Note that the '+' operation automatically
    # broadcasts the biases.
    hidden = tf.nn.relu(tf.matmul(reshape, fc1_weights) + fc1_biases)
 
    # Add a 50% dropout during training only. Dropout also scales
    # activations such that no rescaling is needed at evaluation time.
    if train:
        hidden = tf.nn.dropout(hidden, 0.5, seed=SEED)
    return tf.matmul(hidden, fc2_weights) + fc2_biases
 
def error_rate(predictions, labels):
    """Return the error rate and confusions."""
    correct = numpy.sum(numpy.argmax(predictions, 1) == numpy.argmax(labels, 1))
    total = predictions.shape[0]
 
    error = 100.0 - (100 * float(correct) / float(total))
 
    return error
 
# Training computation: logits + cross-entropy loss.
logits = model(train_data_node, True)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
  logits, train_labels_node))
 
# L2 regularization for the fully connected parameters.
regularizers = (tf.nn.l2_loss(fc1_weights) + tf.nn.l2_loss(fc1_biases) +
                tf.nn.l2_loss(fc2_weights) + tf.nn.l2_loss(fc2_biases))
# Add the regularization term to the loss.
loss += 5e-4 * regularizers
 
# Optimizer: set up a variable that's incremented once per batch and
# controls the learning rate decay.
batch = tf.Variable(0)
# Decay once per epoch, using an exponential schedule starting at 0.01.
learning_rate = tf.train.exponential_decay(
  0.01,                # Base learning rate.
  batch * BATCH_SIZE,  # Current index into the dataset.
  train_size,          # Decay step.
  0.95,                # Decay rate.
  staircase=True)
# Use simple momentum for the optimization.
optimizer = tf.train.MomentumOptimizer(learning_rate,
                                       0.9).minimize(loss,
                                                     global_step=batch)
 
# Predictions for the minibatch, validation set and test set.
train_prediction = tf.nn.softmax(logits)
# We'll compute them only once in a while by calling their {eval()} method.
validation_prediction = tf.nn.softmax(model(validation_data_node))
test_prediction = tf.nn.softmax(model(test_data_node))
 
#train
# Create a new interactive session that we'll use in
# subsequent code cells.
s = tf.InteractiveSession()
#save model
saver = tf.train.Saver()
# Use our newly created session as the default for 
# subsequent operations.
s.as_default()
# Initialize all the variables we defined above.
tf.initialize_all_variables().run()
# Train over the first 1/4th of our training set.
steps = int(train_size / BATCH_SIZE)
for step in xrange(steps):
    # Compute the offset of the current minibatch in the data.
    # Note that we could use better randomization across epochs.
    print step
    offset = (step * BATCH_SIZE) % (train_size - BATCH_SIZE)
    print offset
    batch_data = train_data[offset:(offset + BATCH_SIZE), :, :, :]
    batch_labels = train_labels[offset:(offset + BATCH_SIZE)]
    # This dictionary maps the batch data (as a numpy array) to the
    # node in the graph it should be fed to.
    feed_dict = {train_data_node: batch_data,
                train_labels_node: batch_labels}
    # Run the graph and fetch some of the nodes.
    _, l, lr, predictions = s.run(
      [optimizer, loss, learning_rate, train_prediction],
      feed_dict=feed_dict)
 
    # Print out the loss periodically.
    error = error_rate(predictions, batch_labels)
    print 'Step %d of %d' % (step, steps)
    print 'Mini-batch loss: %.5f Error: %.5f Learning rate: %.5f' % (l, error, lr)
 
    print 'Validation error: %.1f%%' % error_rate(
        validation_prediction.eval(), validation_labels)
#save_path = saver.save(s, model_path) 
test_error = error_rate(test_prediction.eval(), test_labels)
print 'Test error: %.1f%%' % test_error

算法参考笔迹识别:https://www.tensorflow.org/versions/r0.10/tutorials/mnist/beginners/index.html

3.训练及测试的结果

(1)二分类:小世界网里的p值为0.1和1.0

对小世界网里的p=0.1和p=1进行分类。生成的训练数据集各为5000个,测试数据集各为600个。

以下的结果表示训练了90次,损失和错误率越来低,训练效果越来越好。最后测试的错误率为0,说明全部预测正确。

Mini-batch loss:最小损失

Error:错误率

Learning rate:学习率(一般constant)

Validation error:校验数据的错误率

0
0
Step 0 of 90
Mini-batch loss: 16.14303 Error: 45.00000 Learning rate: 0.01000
Validation error: 48.9%
1
100
Step 1 of 90
Mini-batch loss: 27.11566 Error: 54.00000 Learning rate: 0.01000
Validation error: 51.1%
2
200
Step 2 of 90
Mini-batch loss: 56.45219 Error: 48.00000 Learning rate: 0.01000
Validation error: 20.4%
3
300
Step 3 of 90
Mini-batch loss: 12.03109 Error: 28.00000 Learning rate: 0.01000
Validation error: 27.5%
4
400
Step 4 of 90
Mini-batch loss: 14.29832 Error: 33.00000 Learning rate: 0.01000
Validation error: 20.4%
5
500
Step 5 of 90
Mini-batch loss: 13.79097 Error: 29.00000 Learning rate: 0.01000
Validation error: 0.6%
6
600
Step 6 of 90
Mini-batch loss: 10.31240 Error: 3.00000 Learning rate: 0.01000
Validation error: 0.5%
7
700
Step 7 of 90
Mini-batch loss: 9.55898 Error: 1.00000 Learning rate: 0.01000
Validation error: 0.2%
8
800
Step 8 of 90
Mini-batch loss: 9.14136 Error: 0.00000 Learning rate: 0.01000
Validation error: 2.5%
9
900
Step 9 of 90
Mini-batch loss: 15.78014 Error: 12.00000 Learning rate: 0.01000
Validation error: 50.7%
10
1000
Step 10 of 90
Mini-batch loss: 335.58975 Error: 48.00000 Learning rate: 0.01000
Validation error: 48.9%
11
1100
Step 11 of 90
Mini-batch loss: 48.19222 Error: 54.00000 Learning rate: 0.01000
Validation error: 48.9%
12
1200
Step 12 of 90
Mini-batch loss: 15.42103 Error: 46.00000 Learning rate: 0.01000
Validation error: 48.9%
13
1300
Step 13 of 90
Mini-batch loss: 14.89036 Error: 57.00000 Learning rate: 0.01000
Validation error: 51.1%
14
1400
Step 14 of 90
Mini-batch loss: 15.67066 Error: 42.00000 Learning rate: 0.01000
Validation error: 48.9%
15
1500
Step 15 of 90
Mini-batch loss: 18.49513 Error: 42.00000 Learning rate: 0.01000
Validation error: 51.1%
...
83
8300
Step 83 of 90
Mini-batch loss: 11.22020 Error: 1.00000 Learning rate: 0.01000
Validation error: 0.2%
84
8400
Step 84 of 90
Mini-batch loss: 11.17491 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.2%
85
8500
Step 85 of 90
Mini-batch loss: 11.17399 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.2%
86
8600
Step 86 of 90
Mini-batch loss: 11.17306 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.1%
87
8700
Step 87 of 90
Mini-batch loss: 11.17212 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.1%
88
8800
Step 88 of 90
Mini-batch loss: 11.17117 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.1%
89
0
Step 89 of 90
Mini-batch loss: 11.17020 Error: 0.00000 Learning rate: 0.01000
Validation error: 0.1%
Test error: 0.0%

(2)三分类:小世界网里的p值为0.1、0.3和0.5

测试效果不是很好,虽然这里的测试错误率是0.8% 但是错误率极不稳定,有时候是60%以上。可能CNN网络不能很好的识别同类型的网络 或者是参数问题...

0
0
Step 0 of 225
Mini-batch loss: 9.57972 Error: 61.66667 Learning rate: 0.01000
Validation error: 57.8%
1
60
Step 1 of 225
Mini-batch loss: 8.29400 Error: 50.00000 Learning rate: 0.01000
Validation error: 47.5%
2
120
Step 2 of 225
Mini-batch loss: 8.18768 Error: 63.33333 Learning rate: 0.01000
Validation error: 36.5%
3
180
Step 3 of 225
Mini-batch loss: 7.41082 Error: 53.33333 Learning rate: 0.01000
Validation error: 31.5%
4
240
Step 4 of 225
Mini-batch loss: 6.55065 Error: 40.00000 Learning rate: 0.01000
Validation error: 34.5%
5
300
Step 5 of 225
Mini-batch loss: 6.92864 Error: 40.00000 Learning rate: 0.01000
Validation error: 33.7%
6
360
Step 6 of 225
Mini-batch loss: 6.70731 Error: 38.33333 Learning rate: 0.01000
Validation error: 33.0%
7
420
Step 7 of 225
Mini-batch loss: 6.53579 Error: 33.33333 Learning rate: 0.01000
Validation error: 33.3%
8
480
Step 8 of 225
Mini-batch loss: 6.34404 Error: 35.00000 Learning rate: 0.01000
Validation error: 31.0%
9
540
Step 9 of 225
Mini-batch loss: 6.37517 Error: 28.33333 Learning rate: 0.01000
Validation error: 31.6%
10
600
Step 10 of 225
Mini-batch loss: 6.32761 Error: 43.33333 Learning rate: 0.01000
Validation error: 33.4%
11
660
Step 11 of 225
Mini-batch loss: 6.60351 Error: 36.66667 Learning rate: 0.01000
Validation error: 33.5%
12
720
Step 12 of 225
Mini-batch loss: 6.29681 Error: 38.33333 Learning rate: 0.01000
Validation error: 31.0%
...
108
6480
Step 108 of 225
Mini-batch loss: 6.16156 Error: 26.66667 Learning rate: 0.01000
Validation error: 31.0%
109
6540
Step 109 of 225
Mini-batch loss: 6.12177 Error: 30.00000 Learning rate: 0.01000
Validation error: 31.1%
110
6600
Step 110 of 225
Mini-batch loss: 6.07313 Error: 23.33333 Learning rate: 0.01000
Validation error: 31.9%
111
6660
Step 111 of 225
Mini-batch loss: 6.14766 Error: 38.33333 Learning rate: 0.01000
Validation error: 32.9%
112
6720
Step 112 of 225
Mini-batch loss: 6.14581 Error: 30.00000 Learning rate: 0.01000
Validation error: 32.7%
113
6780
Step 113 of 225
Mini-batch loss: 6.16284 Error: 38.33333 Learning rate: 0.01000
Validation error: 33.1%
114
6840
Step 114 of 225
Mini-batch loss: 6.13162 Error: 43.33333 Learning rate: 0.01000
Validation error: 34.0%
115
6900
Step 115 of 225
Mini-batch loss: 6.07556 Error: 30.00000 Learning rate: 0.01000
Validation error: 33.7%
116
6960
Step 116 of 225
Mini-batch loss: 6.17700 Error: 38.33333 Learning rate: 0.01000
Validation error: 33.2%
117
7020
Step 117 of 225
Mini-batch loss: 6.12785 Error: 38.33333 Learning rate: 0.01000
Validation error: 34.7%
118
7080
Step 118 of 225
Mini-batch loss: 6.09914 Error: 33.33333 Learning rate: 0.01000
Validation error: 33.7%
119
7140
Step 119 of 225
Mini-batch loss: 6.11915 Error: 31.66667 Learning rate: 0.01000
Validation error: 33.8%
120
7200
Step 120 of 225
Mini-batch loss: 6.05630 Error: 21.66667 Learning rate: 0.01000
Validation error: 33.6%
...
217
13020
Step 217 of 225
Mini-batch loss: 6.10184 Error: 38.33333 Learning rate: 0.01000
Validation error: 31.0%
218
13080
Step 218 of 225
Mini-batch loss: 6.04546 Error: 25.00000 Learning rate: 0.01000
Validation error: 31.0%
219
13140
Step 219 of 225
Mini-batch loss: 6.07045 Error: 30.00000 Learning rate: 0.01000
Validation error: 31.0%
220
13200
Step 220 of 225
Mini-batch loss: 6.03162 Error: 28.33333 Learning rate: 0.01000
Validation error: 31.0%
221
13260
Step 221 of 225
Mini-batch loss: 6.13473 Error: 40.00000 Learning rate: 0.01000
Validation error: 31.1%
222
13320
Step 222 of 225
Mini-batch loss: 6.11932 Error: 41.66667 Learning rate: 0.01000
Validation error: 31.7%
223
13380
Step 223 of 225
Mini-batch loss: 6.05510 Error: 33.33333 Learning rate: 0.01000
Validation error: 33.8%
224
0
Step 224 of 225
Mini-batch loss: 6.00683 Error: 26.66667 Learning rate: 0.01000
Validation error: 33.3%
Test error: 0.8%

(3)区分随机网和无标度网

分类效果挺好,测试错误率只有1.8%,虽然随机网和无标度网肉眼上也不是很容易区分,但是说明通过算法应该可以区分不同的网络。

0
0
Step 0 of 100
Mini-batch loss: 15.63943 Error: 39.00000 Learning rate: 0.01000
1
100
Step 1 of 100
Mini-batch loss: 41.80207 Error: 49.00000 Learning rate: 0.01000
2
200
Step 2 of 100
Mini-batch loss: 18.96807 Error: 46.00000 Learning rate: 0.01000
3
300
Step 3 of 100
Mini-batch loss: 11.20298 Error: 35.00000 Learning rate: 0.01000
4
400
Step 4 of 100
Mini-batch loss: 12.84610 Error: 39.00000 Learning rate: 0.01000
5
500
Step 5 of 100
Mini-batch loss: 9.71335 Error: 14.00000 Learning rate: 0.01000
6
600
Step 6 of 100
Mini-batch loss: 12.77976 Error: 35.00000 Learning rate: 0.01000
7
700
Step 7 of 100
Mini-batch loss: 11.18729 Error: 16.00000 Learning rate: 0.01000
8
800
Step 8 of 100
Mini-batch loss: 11.67209 Error: 15.00000 Learning rate: 0.01000
9
900
Step 9 of 100
Mini-batch loss: 9.64156 Error: 8.00000 Learning rate: 0.01000
10
1000
Step 10 of 100
Mini-batch loss: 9.36895 Error: 3.00000 Learning rate: 0.01000
11
1100
Step 11 of 100
Mini-batch loss: 10.05438 Error: 9.00000 Learning rate: 0.01000
12
1200
Step 12 of 100
Mini-batch loss: 9.48890 Error: 5.00000 Learning rate: 0.01000
13
1300
Step 13 of 100
Mini-batch loss: 9.78176 Error: 4.00000 Learning rate: 0.01000
14
1400
Step 14 of 100
Mini-batch loss: 9.13174 Error: 0.00000 Learning rate: 0.01000
...
93
9300
Step 93 of 100
Mini-batch loss: 9.11507 Error: 1.00000 Learning rate: 0.01000
94
9400
Step 94 of 100
Mini-batch loss: 9.12368 Error: 2.00000 Learning rate: 0.01000
95
9500
Step 95 of 100
Mini-batch loss: 9.08066 Error: 0.00000 Learning rate: 0.01000
96
9600
Step 96 of 100
Mini-batch loss: 9.08101 Error: 1.00000 Learning rate: 0.01000
97
9700
Step 97 of 100
Mini-batch loss: 9.12844 Error: 1.00000 Learning rate: 0.01000
98
9800
Step 98 of 100
Mini-batch loss: 9.07648 Error: 0.00000 Learning rate: 0.01000
99
0
Step 99 of 100
Mini-batch loss: 9.10577 Error: 2.00000 Learning rate: 0.01000
Test error: 1.8%

(3)区分无标度网和p值为0.5的小世界网

分类效果挺好,测试错误率为0.2%,说明能很好的区分!

0
0
Step 0 of 100
Mini-batch loss: 15.95393 Error: 30.00000 Learning rate: 0.01000
1
100
Step 1 of 100
Mini-batch loss: 51.23495 Error: 49.00000 Learning rate: 0.01000
2
200
Step 2 of 100
Mini-batch loss: 13.23875 Error: 41.00000 Learning rate: 0.01000
3
300
Step 3 of 100
Mini-batch loss: 9.60072 Error: 15.00000 Learning rate: 0.01000
4
400
Step 4 of 100
Mini-batch loss: 9.63927 Error: 14.00000 Learning rate: 0.01000
5
500
Step 5 of 100
Mini-batch loss: 10.50591 Error: 14.00000 Learning rate: 0.01000
6
600
Step 6 of 100
Mini-batch loss: 9.12576 Error: 0.00000 Learning rate: 0.01000
7
700
Step 7 of 100
Mini-batch loss: 9.36399 Error: 5.00000 Learning rate: 0.01000
8
800
Step 8 of 100
Mini-batch loss: 9.37636 Error: 1.00000 Learning rate: 0.01000
9
900
Step 9 of 100
Mini-batch loss: 9.12568 Error: 0.00000 Learning rate: 0.01000
10
1000
Step 10 of 100
Mini-batch loss: 9.12585 Error: 0.00000 Learning rate: 0.01000
...
38
3800
Step 38 of 100
Mini-batch loss: 9.11499 Error: 0.00000 Learning rate: 0.01000
39
3900
Step 39 of 100
Mini-batch loss: 9.91485 Error: 1.00000 Learning rate: 0.01000
40
4000
Step 40 of 100
Mini-batch loss: 9.11353 Error: 0.00000 Learning rate: 0.01000
41
4100
Step 41 of 100
Mini-batch loss: 9.11278 Error: 0.00000 Learning rate: 0.01000
42
4200
Step 42 of 100
Mini-batch loss: 9.11202 Error: 0.00000 Learning rate: 0.01000
43
4300
Step 43 of 100
Mini-batch loss: 9.78888 Error: 1.00000 Learning rate: 0.01000
44
4400
Step 44 of 100
Mini-batch loss: 9.23015 Error: 1.00000 Learning rate: 0.01000
45
4500
Step 45 of 100
Mini-batch loss: 12.01475 Error: 9.00000 Learning rate: 0.01000
46
4600
Step 46 of 100
Mini-batch loss: 9.28377 Error: 1.00000 Learning rate: 0.01000
47
4700
Step 47 of 100
Mini-batch loss: 9.10805 Error: 0.00000 Learning rate: 0.01000
...
93
9300
Step 93 of 100
Mini-batch loss: 9.06923 Error: 0.00000 Learning rate: 0.01000
94
9400
Step 94 of 100
Mini-batch loss: 9.11931 Error: 1.00000 Learning rate: 0.01000
95
9500
Step 95 of 100
Mini-batch loss: 9.61286 Error: 2.00000 Learning rate: 0.01000
96
9600
Step 96 of 100
Mini-batch loss: 9.06642 Error: 0.00000 Learning rate: 0.01000
97
9700
Step 97 of 100
Mini-batch loss: 9.06567 Error: 0.00000 Learning rate: 0.01000
98
9800
Step 98 of 100
Mini-batch loss: 9.06464 Error: 0.00000 Learning rate: 0.01000
99
0
Step 99 of 100
Mini-batch loss: 9.06375 Error: 0.00000 Learning rate: 0.01000
Test error: 0.2%

可视化权重

第一层32个卷积核,提取出来后如下图所示(图太多,只放了前4个):

01.png

02.png

03.png

04.png

彩色.png

同样的权重的灰度图如下:

40.png

41.png

42.png

43.png

灰色.png

实验:查看对应的原图节点

查看第一个卷积层激活的神经元对应在网络上的节点

思路:查看filter与原图做卷积之后的feature map中数值比较大的神经元,对应到原图上的区域,再查看该区域中包含的节点,将节点在网络图上染色。

另外,本次实验中选取了较大的5个激活值。另外,包含一些最大值也为负的结果,被舍弃了。32个filter的时候负的结果比较多,于是调整第一层filter为15个重新进行了训练,训练结果还是很好,在这就不显示了。

实验结果如下:

结果分析:【待定】

另外也实验了测试集换成不同的节点和连边数量的网络,测试效果很不好!!

QUESTION&TODO

caffe安装教程【mac版】

https://shimo.im/doc/nt4Kec78W4AEuX7p

https://shimo.im/doc/GrTfLm7NXvY5tpbI

caffe使用简易版

https://shimo.im/doc/jdyTPyWRUa41rJkO

有以下脑洞:

1.是不是可以根据网络的大小转化成不同尺寸的图片,(因为转化成统一的大小,节点比较小的网络转化后会很稀疏),然后归一化成一样尺度的图片进行训练。

2.栅格化之后的图片和原来的二维图片有点像,不知道是不是可以直接用二维图。

3.word2vec训练后的向量就可以看成网络的特征,cnn卷积之后得到的也是特征,不造是不是可以直接把高维的向量转化成cnn输入所需的图片格式的矩阵(即Todo(4))。

4.能不能把网络的流距离矩阵当做图像矩阵输入训练。

Todo:

1.试试其它p值下小世界网和无标度网的情况。

2.加入其它类型的网络,比如ER随机网,改变参数,再测试结果。

3.用实际数据去测试模型的效果,比如之前关于网络分类的论文当中有一些实际的数据集,测试一下。

4.升维,目前这个方案当中每个节点都被降到了二维,那么接下来可以考虑不降到二维,直接在高维空间去做,也是栅格化处理,然后CNN做分类,当然这个需要找tensorflow的文档,看看该如何做高维的CNN。

To be continued...

个人工具
名字空间
操作
导航
工具箱