PinPoint分布式性能监控工具

avatar 2019年11月10日22:45:47 评论 4,325 次浏览

简介：pinpoint是由韩国人使用java写的一套全链路分析工具，是基于字节码注入技术，对代码无侵入的链路监控、方法执行详情查看、应用状态信息监控等功能。

环境配置

  • JDK1.8 ---- pinpoint依赖java
  • Tomcat8.0 ---- web服务
  • hbase1.0 ----- 数据库，用来存储监控信息
  • pinpoint-collector.war ---- pinpoint的控制器
  • pinpoint-web.war ------ pinpoint展示页面
  • pinpoint-collector.init ---- 用来快速启动pinpoint-collector，可以忽略
  • pinpoint-web.init ---- 用来启动pinpoint-web,可以忽略
    系统 IP 计算机名 角色
    CentOs7 10.211.55.32 centos1 client_wulaoer
    CentOs7 10.211.55.34 centos2 wulaoer

安装

因为我们打包的时候依赖的java1.6、java1.7、java1.8、java1.9环境的普通版本的包，所以这几个版本的java都需要安装

[root@centos2 ~]# yum -y install java-1.*

yum安装没有1.9版本，所以需要自己下载一个java1.9版本的，并指定一下路径。也可以把所有版本都下载了一份，然后解压使用自己设定的路径，yum安装的java路径在/usr/lib/jvm/目录下。这里不需要安装maven，有的说需要安装maven，我尝试了一下不安装maven也可以进行编译的。

[root@centos2 ~]# wget http://archive.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz
[root@centos2 ~]# tar -zxvf apache-maven-3.5.3-bin.tar.gz
[root@centos2 ~]# mv apache-maven-3.5.3 /usr/local/maven
[root@centos2 ~]# vim /etc/profiler
export M2_HOME=/usr/local/maven
export PATH=$PATH:$M2_HOME/bin

[root@centos2 ~]# vim /etc/profiler
.........
#自己安装使用1.8版本

export JAVA_HOME=/usr/local/jvm/jdk1.8
export JAVA_8_HOME=/usr/local/jvm/jdk1.8
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
＃ java 6
export JAVA_6_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
# java 7
export JAVA_7_HOME=/usr/lib/jvm/java-1.7.0-openjdk
# java 9
export JAVA_9_HOME=/usr/local/jvm/jdk1.9
[root@centos2 ~]# source /etc/profile

基础环境已经设置完成，下面下载pinpoint，并编译代码打包。

[root@centos2 ~]# git clone https://github.com/naver/pinpoint.git
Cloning into 'pinpoint'...
remote: Enumerating objects: 212, done.
remote: Counting objects: 100% (212/212), done.
remote: Compressing objects: 100% (180/180), done.
remote: Total 230657 (delta 47), reused 114 (delta 32), pack-reused 230445
Receiving objects: 100% (230657/230657), 216.88 MiB | 1.43 MiB/s, done.
Resolving deltas: 100% (120809/120809), done.
Checking out files: 100% (9098/9098), done.
[root@centos2 ~]# cd pinpoint/
[root@centos2 pinpoint]# ls
agent           bootstrap-java7              commons-buffer    flink     package-lock.json  quickstart            web
agent-it        bootstrap-java8              commons-hbase     grpc      plugins            README.md
agent-it8       bootstrap-java9              commons-profiler  hbase     plugins-loader     rpc
agent-plugins   checkstyle-suppressions.xml  commons-server    LICENSE   pom.xml            spotbugs-exclude.xml
annotations     checkstyle-UTF8(ver0.6).xml  CONTRIBUTING.md   mvnw      profiler           test
bootstrap       collector                    doc               mvnw.cmd  profiler-optional  thrift
bootstrap-core  commons                      filefilter.txt    NOTICE    profiler-test      tools
[root@centos2 pinpoint]# ./mvnw install -DskipTests=true
...................
[INFO] Reactor Summary:
[INFO] 
[INFO] pinpoint 1.9.0-SNAPSHOT ............................ SUCCESS [  1.605 s]
[INFO] pinpoint-annotations ............................... SUCCESS [  1.096 s]
[INFO] pinpoint-commons ................................... SUCCESS [  2.032 s]
[INFO] pinpoint-commons-buffer ............................ SUCCESS [  0.246 s]
[INFO] pinpoint-commons-profiler .......................... SUCCESS [  0.293 s]
[INFO] pinpoint-thrift .................................... SUCCESS [  0.620 s]
[INFO] pinpoint-grpc ...................................... SUCCESS [  7.545 s]
[INFO] pinpoint-bootstrap-core ............................ SUCCESS [  0.379 s]
[INFO] pinpoint-bootstrap ................................. SUCCESS [  0.248 s]
[INFO] pinpoint-rpc ....................................... SUCCESS [  0.484 s]
[INFO] pinpoint-plugins-loader ............................ SUCCESS [  0.223 s]
[INFO] pinpoint-profiler .................................. SUCCESS [  1.038 s]
[INFO] pinpoint-profiler-test ............................. SUCCESS [  0.718 s]
[INFO] pinpoint-profiler-optional-parent .................. SUCCESS [  0.569 s]
[INFO] pinpoint-profiler-optional-jdk6 .................... SUCCESS [  0.523 s]
[INFO] pinpoint-profiler-optional-jdk7 .................... SUCCESS [  0.576 s]
[INFO] pinpoint-test ...................................... SUCCESS [  0.396 s]
[INFO] pinpoint-profiler-optional-jdk8 .................... SUCCESS [  1.089 s]
[INFO] pinpoint-profiler-optional-jdk9 .................... SUCCESS [  0.118 s]
[INFO] pinpoint-profiler-optional ......................... SUCCESS [  0.016 s]
[INFO] pinpoint-plugins ................................... SUCCESS [  0.155 s]
[INFO] pinpoint-common-servlet ............................ SUCCESS [  0.226 s]
[INFO] pinpoint-external-plugins .......................... SUCCESS [  0.254 s]
[INFO] pinpoint-httpclient3-plugin ........................ SUCCESS [  0.321 s]
[INFO] pinpoint-httpclient4-plugin ........................ SUCCESS [  0.313 s]
[INFO] pinpoint-ning-asynchttpclient-plugin ............... SUCCESS [  1.216 s]
[INFO] pinpoint-resttemplate-plugin ....................... SUCCESS [  0.443 s]
[INFO] pinpoint-netty-plugin .............................. SUCCESS [  0.918 s]
[INFO] pinpoint-gson-plugin ............................... SUCCESS [  0.247 s]
[INFO] pinpoint-cubrid-jdbc-driver-plugin ................. SUCCESS [  0.236 s]
[INFO] pinpoint-mysql-jdbc-driver-plugin .................. SUCCESS [  0.251 s]
[INFO] pinpoint-mariadb-jdbc-driver-plugin ................ SUCCESS [  0.217 s]
[INFO] pinpoint-postgresql-jdbc-driver-plugin ............. SUCCESS [  0.236 s]
[INFO] pinpoint-jtds-plugin ............................... SUCCESS [  0.258 s]
[INFO] pinpoint-oracle-jdbc-driver-plugin ................. SUCCESS [  0.220 s]
[INFO] pinpoint-commons-dbcp-plugin ....................... SUCCESS [  0.298 s]
[INFO] pinpoint-commons-dbcp2-plugin ...................... SUCCESS [  0.706 s]
[INFO] pinpoint-hikaricp-plugin ........................... SUCCESS [  0.231 s]
[INFO] pinpoint-cassandra-driver-plugin ................... SUCCESS [  0.387 s]
[INFO] pinpoint-jdk-http-plugin ........................... SUCCESS [  0.212 s]
[INFO] pinpoint-commons-hbase ............................. SUCCESS [  1.657 s]
[INFO] pinpoint-commons-server ............................ SUCCESS [  2.099 s]
[INFO] pinpoint-redis-plugin .............................. SUCCESS [  0.388 s]
[INFO] pinpoint-redis-lettuce-plugin ...................... SUCCESS [  0.387 s]
[INFO] pinpoint-redis-redisson-plugin ..................... SUCCESS [  1.010 s]
[INFO] pinpoint-jboss-plugin .............................. SUCCESS [  6.070 s]
[INFO] pinpoint-tomcat-plugin ............................. SUCCESS [  0.560 s]
[INFO] pinpoint-json-lib-plugin ........................... SUCCESS [  0.205 s]
[INFO] pinpoint-jackson-plugin ............................ SUCCESS [  0.203 s]
[INFO] pinpoint-thrift-plugin ............................. SUCCESS [  0.272 s]
[INFO] pinpoint-user-plugin ............................... SUCCESS [  0.212 s]
[INFO] pinpoint-arcus-plugin .............................. SUCCESS [  0.386 s]
[INFO] pinpoint-google-httpclient-plugin .................. SUCCESS [  0.381 s]
[INFO] pinpoint-grpc-plugin ............................... SUCCESS [  0.303 s]
[INFO] pinpoint-jetty-plugin .............................. SUCCESS [  0.407 s]
[INFO] pinpoint-websphere-plugin .......................... SUCCESS [  0.335 s]
[INFO] pinpoint-spring-plugin ............................. SUCCESS [  0.303 s]
[INFO] pinpoint-spring-boot-plugin ........................ SUCCESS [  0.197 s]
[INFO] pinpoint-spring-webflux-plugin ..................... SUCCESS [  0.678 s]
[INFO] pinpoint-ibatis-plugin ............................. SUCCESS [  0.229 s]
[INFO] pinpoint-mybatis-plugin ............................ SUCCESS [  0.208 s]
[INFO] pinpoint-okhttp-plugin ............................. SUCCESS [  0.292 s]
[INFO] pinpoint-log4j-plugin .............................. SUCCESS [  0.224 s]
[INFO] pinpoint-logback-plugin ............................ SUCCESS [  0.356 s]
[INFO] pinpoint-log4j2-plugin ............................. SUCCESS [  0.412 s]
[INFO] pinpoint-dubbo-plugin .............................. SUCCESS [  0.509 s]
[INFO] pinpoint-activemq-client-plugin .................... SUCCESS [  0.341 s]
[INFO] pinpoint-cxf-plugin ................................ SUCCESS [  1.237 s]
[INFO] pinpoint-hystrix-plugin ............................ SUCCESS [  0.356 s]
[INFO] pinpoint-vertx-plugin .............................. SUCCESS [  0.832 s]
[INFO] pinpoint-resin-plugin .............................. SUCCESS [  0.442 s]
[INFO] pinpoint-jsp-plugin ................................ SUCCESS [  0.278 s]
[INFO] pinpoint-rxjava-plugin ............................. SUCCESS [  0.283 s]
[INFO] pinpoint-rabbitmq-plugin ........................... SUCCESS [  0.468 s]
[INFO] pinpoint-weblogic-plugin ........................... SUCCESS [  0.364 s]
[INFO] pinpoint-akka-http-plugin .......................... SUCCESS [  0.893 s]
[INFO] pinpoint-undertow-plugin ........................... SUCCESS [  0.430 s]
[INFO] pinpoint-undertow-servlet-plugin ................... SUCCESS [  0.414 s]
[INFO] pinpoint-kafka-plugin .............................. SUCCESS [  0.587 s]
[INFO] pinpoint-mongodb-driver-plugin ..................... SUCCESS [  0.303 s]
[INFO] pinpoint-fastjson-plugin ........................... SUCCESS [  0.200 s]
[INFO] pinpoint-druid-plugin .............................. SUCCESS [  0.290 s]
[INFO] pinpoint-hbase-plugin .............................. SUCCESS [  0.917 s]
[INFO] pinpoint-openwhisk-plugin .......................... SUCCESS [  1.215 s]
[INFO] pinpoint-elasticsearch-bboss-plugin ................ SUCCESS [  0.274 s]
[INFO] pinpoint-apache-dubbo-plugin ....................... SUCCESS [  0.499 s]
[INFO] pinpoint-elasticsearch-plugin ...................... SUCCESS [  1.196 s]
[INFO] pinpoint-plugins-assembly .......................... SUCCESS [  0.498 s]
[INFO] pinpoint-agent-proxy-common ........................ SUCCESS [  0.157 s]
[INFO] pinpoint-agent-proxy-apache-plugin ................. SUCCESS [  0.605 s]
[INFO] pinpoint-agent-proxy-app-plugin .................... SUCCESS [  0.569 s]
[INFO] pinpoint-agent-proxy-nginx-plugin .................. SUCCESS [  0.565 s]
[INFO] pinpoint-agent-plugins ............................. SUCCESS [  0.483 s]
[INFO] pinpoint-bootstrap-java7 ........................... SUCCESS [  0.198 s]
[INFO] pinpoint-bootstrap-java8 ........................... SUCCESS [  0.547 s]
[INFO] pinpoint-bootstrap-java9 ........................... SUCCESS [  0.035 s]
[INFO] pinpoint-tools ..................................... SUCCESS [  1.484 s]
[INFO] pinpoint-agent-distribution ........................ SUCCESS [ 50.261 s]
[INFO] pinpoint-agent-it .................................. SUCCESS [13:24 min]
[INFO] pinpoint-agent-it8 ................................. SUCCESS [04:55 min]
[INFO] pinpoint-collector ................................. SUCCESS [01:28 min]
[INFO] pinpoint-plugin-bom ................................ SUCCESS [  0.183 s]
[INFO] pinpoint-web ....................................... SUCCESS [06:35 min]
[INFO] pinpoint-hbase-distribution ........................ SUCCESS [  0.544 s]
[INFO] pinpoint-hbase-schema-definition ................... SUCCESS [02:40 min]
[INFO] pinpoint-hbase-schema .............................. SUCCESS [03:48 min]
[INFO] pinpoint-hbase-schema-manager ...................... SUCCESS [03:22 min]
[INFO] pinpoint-flink 1.9.0-SNAPSHOT ...................... SUCCESS [08:45 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:57 min
[INFO] Finished at: 2019-11-05T01:35:37+08:00
[INFO] ------------------------------------------------------------------------
[root@centos2 pinpoint]#

注：windows系统使用运行 ./mvnw.cmd clean install -Dmaven.test.skip=true 编译代码

配置一下存储服务HBase

HBase配置

初始化HBase，pinpoint的监控数据存储在HBase中，所以需要初始化一下，不过配置中默认是国外的网站，我们需要修改一下国内的网站。修改一下启动参数，默认是国外的网站修改成能够访问的网站，并设置版本和下载路径

[root@centos2 pinpoint]# vim quickstart/bin/start-hbase.sh 
#!/usr/bin/env bash
VERSION=2.2.2
HBASE_VERSION=hbase-$VERSION
HBASE_FILE=$HBASE_VERSION-bin.tar.gz
HBASE_DL_URL=http://mirror.bit.edu.cn/apache/hbase/$VERSION/$HBASE_FILE
HBASE_ARCHIVE_DL_URL=http://mirror.bit.edu.cn/apache/hbase/$VERSION/$HBASE_FILE

下载并初始化表

[root@CentOS2 pinpoint]# ./quickstart/bin/start-hbase.sh 
Hbase not detected.
Downloading hbase...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  221M  100  221M    0     0   305k      0  0:12:21  0:12:21 --:--:--  326k
running master, logging to /root/pinpoint/quickstart/hbase/hbase-2.2.2/bin/../logs/hbase-root-master-CentOS2.out
[root@CentOS2 pinpoint]# ./quickstart/bin/init-hbase.sh

在初始化时注意一下错误信息，如果出现下面的错误信息，重启一下机器，然后重新启动初始化

ERROR: Table already exists: ApplicationMapStatisticsSelf_Ver2!

验证一下HBase

[root@CentOS2 pinpoint]#  netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1304/master         
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1066/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1304/master         
tcp6       0      0 :::16030                :::*                    LISTEN      1699/java           
tcp6       0      0 10.211.55.34:16000      :::*                    LISTEN      1699/java           
tcp6       0      0 :::2181                 :::*                    LISTEN      1699/java           
tcp6       0      0 :::16010                :::*                    LISTEN      1699/java           
tcp6       0      0 10.211.55.34:16020      :::*                    LISTEN      1699/java           
tcp6       0      0 :::22                   :::*                    LISTEN      1066/sshd

启动HBase之后开启了HBase的端口16010，我们访问一下：http://10.211.55.34:16010/master-status

启动服务

[root@CentOS2 pinpoint]# ./quickstart/bin/start-collector.sh 
[root@CentOS2 pinpoint]# ./quickstart/bin/start-testapp.sh 
[root@CentOS2 pinpoint]# ./quickstart/bin/start-web.sh

验证启动状态：

[root@CentOS2 ~]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1243/master         
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1001/sshd           
tcp6       0      0 :::37049                :::*                    LISTEN      1607/java           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1243/master         
tcp6       0      0 :::2181                 :::*                    LISTEN      1607/java           
tcp6       0      0 10.211.55.34:34984      :::*                    LISTEN      1607/java           
tcp6       0      0 :::9994                 :::*                    LISTEN      5147/java           
tcp6       0      0 :::16010                :::*                    LISTEN      1607/java           
tcp6       0      0 10.211.55.34:35980      :::*                    LISTEN      1607/java           
tcp6       0      0 10.211.55.34:9997       :::*                    LISTEN      10970/java          
tcp6       0      0 :::28080                :::*                    LISTEN      10970/java          
tcp6       0      0 :::28081                :::*                    LISTEN      10545/java          
tcp6       0      0 :::28082                :::*                    LISTEN      5147/java           
tcp6       0      0 :::22                   :::*                    LISTEN      1001/sshd

查看一下端口28080，28081，28082，端口正常，下面我们访问一下28080，28081，这是testApp将自己作为注册TESTAPP测试代理

添加客户端

首先把pinpoint的客户端软件同步到客户端服务器上，客户端文件路径在agent/target目录下。

[root@CentOS2 pinpoint]# cd agent/target/
[root@CentOS2 target]# ls
archive-tmp  classes  maven-archiver  pinpoint-agent-1.7.3  pinpoint-agent-1.7.3.jar  pinpoint-agent-1.7.3.tar.gz  pinpoint-agent-1.7.3.zip
[root@CentOS2 target]# scp pinpoint-agent-1.7.3.tar.gz root@10.211.55.32:/root
The authenticity of host '10.211.55.32 (10.211.55.32)' can't be established.
ECDSA key fingerprint is SHA256:qUPArr8TTP+UrVqg+9aCL5EbnyY1XS8QeXj6DWCkEHQ.
ECDSA key fingerprint is MD5:3f:c4:8c:27:70:02:81:e6:ff:2b:82:8c:76:b7:d1:81.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.211.55.32' (ECDSA) to the list of known hosts.
root@10.211.55.32's password: 
pinpoint-agent-1.7.3.tar.gz

在客户端上，解压一下并移到指定的目录下，然后配置一下。

[root@CentOS1 ~]# mdkir -p /opt/pinpoint-agent/
[root@CentOS1 ~]# ls
anaconda-ks.cfg  apache-tomcat-8.5.47.tar.gz  jdk-8u201-linux-x64.tar.gz
[root@CentOS1 ~]# mv pinpoint-agent-1.7.3.tar.gz /opt/pinpoint-agent/
[root@CentOS1 ~]# cd /opt/pinpoint-agent/
[root@CentOS1 pinpoint-agent]# tar -zxf pinpoint-agent-1.7.3.tar.gz

把pinpoint客户端指定自己的服务器，以便客户端发送数据到服务器，并存储展示。

[root@CentOS1 pinpoint-agent]# pwd
/opt/pinpoint-agent
[root@CentOS1 pinpoint-agent]# vim pinpoint.config 
#
# Pinpoint agent configuration
#

###########################################################
# Collector server                                        #
###########################################################
profiler.collector.ip=10.211.55.34
.......................

把tomcat加入到pinpoint中，需要在catalina.sh中加入配置的pinpoint，如果一个机器多个tomcat，只需要设置agentId区别即可。

[root@CentOS1 ~]# cd /usr/local/tomcat/bin/
[root@CentOS1 bin]# vim catalina.sh 
.............
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/opt/pinpoint-agent/pinpoint-bootstrap-1.7.3.jar"
CATALINA_OPTS="$CATALINA_OPTS -Dpinpoint.agentId=wulaoer-agent" ＃必须没有重复
CATALINA_OPTS="$CATALINA_OPTS -Dpinpoint.applicationName=wulaoer" ＃必须没有重复

添加之后启动一下tomcat，启动正常之后我们多次访问，方面后面获取一下tomcat的访问数据。

root@CentOS1 bin]# ps -ef | grep tomcat
root     17436     1  1 23:14 pts/1    00:00:18 /usr/local/jdk/jre/bin/java -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -javaagent:/opt/pinpoint-agent/pinpoint-bootstrap-1.7.3.jar -Dpinpoint.agentId=wulaoer-agent -Dpinpoint.applicationName=wulaoer -Dignore.endorsed.dirs= -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/usr/local/tomcat/temp org.apache.catalina.startup.Bootstrap start
root     17736 16074  0 23:43 pts/1    00:00:00 grep --color=auto tomcat

pinpoint客户端已经在tomcat启动的时候加载了，在进程中可以看到指定的路径已经正常加载。下面看一下pinpoint服务器是否收到自己定义的wulaoer的数据

看一下访问的数据追踪的链路信息，

看一下tomcat的详细信息

安装过程出现的问题以及解决方法：

git报错

[root@CentOS2 ~]# git clone https://github.com/naver/pinpoint.git
Cloning into 'pinpoint'...
remote: Enumerating objects: 278, done.
remote: Counting objects: 100% (278/278), done.
remote: Compressing objects: 100% (170/170), done.
error: RPC failed; result=56, HTTP code = 200 MiB | 628.00 KiB/s   
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

解决方法：

主要是git clone的文件过大，第一次需要配置git缓存，如果还有错误增加缓存大小。

git config --global http.postBuffer 524288000

至此，pinpoint的编译安装已经完成，下一章介绍pinpoint的源码安装和docker的安装方式，没有了看一下其他的吧。。。

avatar

发表评论

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: