作者归档：xinlu

intel推出dpdk网络开发包

intel推出的dpdk网络开发包
http://dpdk.org/

DPDK is a set of libraries and drivers for fast packet processing.
It is designed to run on any processors. The first supported CPU was Intel x86 and it is now extended to IBM POWER and ARM.
It runs mostly in Linux userland. A FreeBSD port is available for a subset of DPDK features.
DPDK is an Open Source BSD licensed project. The most recent patches and enhancements, provided by the community, are available in master branch.
Main libraries

multicore framework
huge page memory
ring buffers
poll-mode drivers for networking , crypto and eventdev

vmware在Centos7共享文件夹

1.重装vmware-tools工具,把VMwareTools-10.0.1-3160059.tar.gz包拷到任意目录并解压。

执行./vmware-install.pl安装，后续过程不断按回车，使用默认值，即可。

2.在root权限下，执行

mount -t vmhgfs .host:/ /mnt/hgfs
如果报错 ERROR: can not mount filesystem: No such device，则改用如下命令：
vmhgfs-fuse .host:/ /mnt/hgfs，这个是当前测试通过的

3.配置启动项【vim /etc/rc.local】，使用随系统启动，默认开启。

vmhgfs-fuse .host:/ /mnt/hgfs
cd /mnt/hgfs目录下，检查是否OK.需要root权限

ELK的启动和停止脚本 

#!/bin/sh

path_base=`pwd`
path_elastic="$path_base/elasticsearch-5.5.1/bin/elasticsearch"
path_kibana="$path_base/kibana-5.5.1/bin/kibana"
path_logstash="$path_base/logstash-5.5.1/bin/logstash"

mode=$1

kibana_process=`ps -ef | grep "kibana"| grep -v grep`
elastic_process=`ps -ef | grep "elasticsearch"| grep -v grep|grep -v controller`

case "$mode" in
   'start')
	echo "it's ready to start op...."
	if test -n "$elastic_process"; then
		echo "---the elasticsearch had already started.."
	else
		`$path_elastic -d`
	fi

	if test -n "$kibana_process"; then
		echo "---the kibana had already started.."
	else
		`nohup $path_kibana > /dev/null 2>&1 &`
	fi
	echo 'success to start.'
	echo '---1--you can test logstash by way bellow----'
	echo 'cd logstash-5.5.1 && bin/logstash -f ./config/test.conf && cd ..'
	echo 'or cd logstash-5.5.1 && bin/logstash -f ./config/h5.api.vip.com.conf && cd ..'
	;;
   'stop')
	echo "it's ready to check process..."
	if test -n "$kibana_process"; then
		echo "had find kibana process.."
		`echo $kibana_process | awk '{print ($2)}' | xargs kill -9`
	fi
	if test -n "$elastic_process"; then
		echo "had find elasticsearch process.."
		`echo $elastic_process | awk '{print ($2)}' | xargs kill -9`
	fi
	echo 'success to kill.'
	;;
    *)
	basename=`basename "$0"`
	echo "Usage: $basename  {start|stop}  [ ELK server options ]"
	exit 1
	;;
esac
exit 1

数据库性能压测 

mysqltest]$ mysqlslap -S /tmp/mysqltest/mysql.sock -uroot --create=/tmp/mysqltest/user_cart.sql --create-schema=test --query=/tmp/mysqltest/cart1.sql --concurrency=1024 --iterations=3
Benchmark
Average number of seconds to run all queries: 59.774 seconds
Minimum number of seconds to run all queries: 59.570 seconds
Maximum number of seconds to run all queries: 60.040 seconds
Number of clients running queries: 1024
Average number of queries per client: 954
QPS=16343

机器学习的一些库

Gensim是一个相当专业的计算相似度的Python工具包。
在文本处理中，比如商品评论挖掘，有时需要了解每个评论分别和商品的描述之间的相似度，以此衡量评论的客观性。
评论和商品描述的相似度越高，说明评论的用语比较官方，不带太多感情色彩，比较注重描述商品的属性和特性，角度更客观。
http://radimrehurek.com/gensim/

————————————-
图像识别类库
https://github.com/tesseract-ocr/tesseract

原本由惠普开发的图像识别类库tesseract-ocr已经更新到2.04， 就是最近Google支持的那个OCR。原先是惠普写的，现在Open source了。

比ldd更好用的strace命令

strace -o whoami-strace.txt whoami
执行ldd会显相关依赖的最终结果，而strace则显示依赖的检查过程。

strace -o whoami-strace.txt whoami

open("/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/i686/mmx/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/lib/i686/mmx", 0xbffff190) = -1 ENOENT (No such file or directory) 
open("/lib/i686/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/lib/i686", 0xbffff190) = -1 ENOENT (No such file or directory)
open("/lib/mmx/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/lib/mmx", 0xbffff190) = -1 ENOENT (No such file or directory) 
open("/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/lib", {st_mode=S_IFDIR|0755, st_size=2352, ...}) = 0
open("/usr/lib/i686/mmx/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/i686/mmx", 0xbffff190) = -1 ENOENT (No such file or directory) 
open("/usr/lib/i686/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)

MySQL高可用主从分离

http://www.mycat.io/
一个彻底开源的，面向企业应用开发的大数据库集群
支持事务、ACID、可以替代MySQL的加强版数据库
一个可以视为MySQL集群的企业级数据库，用来替代昂贵的Oracle集群
一个融合内存缓存技术、NoSQL技术、HDFS大数据的新型SQL Server
结合传统数据库和新型分布式数据仓库的新一代企业级数据库产品
一个新颖的数据库中间件产品