月度归档:2017年08月

Install Ruby on Rails with Rbenv on CentOS 7

1.First, you need to install dependencies for rbenv and Ruby:

sudo yum install -y git-core zlib zlib-devel gcc-c++ patch readline readline-devel libyaml-devel libffi-devel openssl-devel make bzip2 autoconf automake libtool bison curl sqlite-devel

2.Install rbenv and ruby-build, be sure that you are in your non-root sudo user’s home directory:

cd ~
git clone git://github.com/sstephenson/rbenv.git .rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
git clone git://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bash_profile
source ~/.bash_profile

3.You need to determine the version of Ruby that you need. You can list available Ruby versions for installation with the following command:

rbenv install -l

4.Here, I will install the latest stable version, Ruby 2.2.3:

rbenv install -v 2.2.3
rbenv rehash

5.If you want to use another version, just install the version as above:

rbenv install -v 2.2.0
rbenv rehash

6.You can check all the versions you have installed with:

rbenv versions

The version with * is the active version
7.So, at the very least, you need to set your favorite version as global version for daily use:

rbenv global 2.2.3

8.Verify your choice with:

ruby -v

9.Also, you need to install the bundler gem to manage your application dependencies:

gem install bundler

10.Install the latest version of Rails:

gem install rails
rbenv rehash

gem install rails -v 版本号
gem uninstall rails -v 版本号

11.Check if Rails is installed properly:

rails -v

elasticsearch数据持久化

elasticSearch数据持久化,默认情况不启用数据保存,故数据一般会几分钟就消失,按以下步骤保存索引数据。

#不同的集群名字不能相同。
cluster.name: es_vm_test
node.name: vmmaster
network.host: 0.0.0.0
http.port: 9200
#数据索引保存
path.data: /home/abc/elk-5.5.1/elkdata/data
path.logs: /home/abc/elk-5.5.1/elkdata/log
#关闭登录验证
xpack.security.enabled: false

logstash抓取nginx日志

以下是基于elk+lnmp开源进行测试验证。
也可以参考官网的实现方法:https://kibana.logstash.es/content/logstash/plugins/codec/json.html
https://kibana.logstash.es/content/logstash/plugins/codec/multiline.html
在官网文档中,有较多应用场景:
https://kibana.logstash.es/content/
https://kibana.logstash.es/content/logstash/examples/

1.抓取nginx日志

input {
    file {
        # path => ["/home/wwwlogs/h5.vim.vim.com.log", "/home/wwwlogs/h5.vim.vim.com2.log"]
	path => "/home/wwwlogs/h5.vim.vim.com.log"
        exclude => "*.zip"
        type => "java"
        add_field => [ "domain", "h5.vim.vim.com" ]
        codec => multiline {
                      pattern => "^\s+"
                      what => previous
              }
    }
    file {
        # path => ["/home/wwwlogs/h5.api.vim.vim.com.log", "/home/wwwlogs/h5.api.vim.vim.com2.log"]
	path => "/home/wwwlogs/h5.api.vim.vim.com.log"
        exclude => ["*.zip", "*.gz"]
        type => "java"
        add_field => [ "domain", "h5.api.vim.vim.com" ]
        codec => multiline {
                        pattern => "^\s+"
                        what => previous
                 }
    }
}
filter {

}
output {
    stdout { 
		codec => rubydebug 
	}
    elasticsearch {
        hosts => ["0.0.0.0:9200"]
        index => "logstash-%{domain}-%{+YYYY.MM.dd}"
    }
}

2.定期清理索引

#!/bin/bash

# --------------------------------------------------------------
# This script is to delete ES indices older than specified days.
# Version: 1.0
# --------------------------------------------------------------

function usage() {
        echo "Usage: `basename $0` -s ES_SERVER -d KEEP_DAYS [-w INTERVAL]"
}


PREFIX='logstash-'
WAITTIME=2
NOW=`date  +%s.%3N`
LOGPATH=/apps/logs/elasticsearch


while getopts d:s:w: opt
do
        case $opt in
        s) SERVER="$OPTARG";;
        d) KEEPDAYS="$OPTARG";;
        w) WAITTIME="$OPTARG";;
        *) usage;;
        esac
done

if [ -z "$SERVER" -o -z "$KEEPDAYS" ]; then
        usage
fi

if [ ! -d $LOGPATH ]; then
        mkdir -p $LOGPATH
fi


INDICES=`curl -s $SERVER/_cat/indices?h=index | grep -P '^logstash-.*\d{4}.\d{2}.\d{2}' | sort`
for index in $INDICES
do
        date=`echo $index | awk -F '-' '{print $NF}' | sed 's/\./-/g' | xargs -I{} date -d {} +%s.%3N`
        delta=`echo "($NOW-$date)/86400" | bc`
        if [ $delta -gt $KEEPDAYS ]; then
                echo "deleting $index" | tee -a $LOGPATH/es_delete_indices.log
                curl -s -XDELETE $SERVER/$index | tee -a $LOGPATH/es_delete_indices.log
                echo | tee -a $LOGPATH/es_delete_indices.log
                sleep $WAITTIME
        fi
done

intel推出dpdk网络开发包

intel推出的dpdk网络开发包
http://dpdk.org/

DPDK is a set of libraries and drivers for fast packet processing.
It is designed to run on any processors. The first supported CPU was Intel x86 and it is now extended to IBM POWER and ARM.
It runs mostly in Linux userland. A FreeBSD port is available for a subset of DPDK features.
DPDK is an Open Source BSD licensed project. The most recent patches and enhancements, provided by the community, are available in master branch.
Main libraries

multicore framework
huge page memory
ring buffers
poll-mode drivers for networking , crypto and eventdev

vmware在Centos7共享文件夹

1.重装vmware-tools工具,把VMwareTools-10.0.1-3160059.tar.gz包拷到任意目录并解压。

执行./vmware-install.pl安装,后续过程不断按回车,使用默认值,即可。

2.在root权限下,执行

mount -t vmhgfs .host:/ /mnt/hgfs
如果报错 ERROR: can not mount filesystem: No such device,则改用如下命令:
vmhgfs-fuse .host:/ /mnt/hgfs,这个是当前测试通过的

3.配置启动项【vim /etc/rc.local】,使用随系统启动,默认开启。

vmhgfs-fuse .host:/ /mnt/hgfs
cd /mnt/hgfs目录下,检查是否OK.需要root权限