Tomcat信任https网站

在使用微信发送告警的时候出现无法使用https网址的问题,解决方式如下

导出微信的证书

openssl s_client -connect qyapi.weixin.qq.com:443 2>&1 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > weixin.crt

使用SSLPoke.class确认是否还有上层证书,如果存在也导入

java -Djavax.net.debug=ssl SSLPoke qyapi.weixin.qq.com 443

看到有类似下面的则表明有上层证书:

1
2
3
4
5
6
7
8
9
AuthorityInfoAccess [
[
accessMethod: ocsp
accessLocation: URIName: http://gn.symcd.com
,
accessMethod: caIssuers
accessLocation: URIName: http://gn.symcb.com/gn.crt
]
]

http://gn.symcb.com/gn.crt下载下来,并导入
sudo keytool -import -v -trustcacerts -alias gnsymcb -keystore $JAVA_HOME/jre/lib/security/jssecacerts -file gn.crt

导入到jdk中

sudo keytool -import -v -trustcacerts -alias qyapiweixin -keystore $JAVA_HOME/jre/lib/security/jssecacerts -file weixin.crt

更新证书

需要先删除再导入新的

1
2
sudo keytool -delete -keystore $JAVA_HOME/jre/lib/security/cacerts -alias qyapiweixin    
sudo keytool -import -v -trustcacerts -alias qyapiweixin -keystore $JAVA_HOME/jre/lib/security/cacerts -file weixin.crt

指定tomcat使用证书(非必须)

1
2
vim /usr/local/apache-tomcat-8.5.6/bin/setenv.sh
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms2048m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -XX:MaxPermSize=512m -XX:+DisableExplicitGC -Djavax.net.ssl.trustStore=/usr/local/java/jdk8/jre/lib/security/jssecacerts"

或者在 /usr/local/apache-tomcat-8.5.6/conf/server.xml的ssl配置部分指定keystore,如下

<Connector port="443" protocol="org.apache.coyote.http11.Http11Protocol"
              maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
              clientAuth="false" sslProtocol="TLS"
              URIEncoding="utf-8"
              keystoreFile="/usr/local/java/jdk7/jre/lib/security/jssecacerts"
              keystorePass="yourpassword"
              />

opensips安装使用整理

部分内容来自 https://github.com/Ci-Jie/OpenSIPS ,其他的很多教程都是坑!!!

安装opensips

  • CentOS release 6.9 (Final) (内核 4.10.4)
  • MySQL 5.6
  • Opensips 2.1

安装依赖库等

sudo yum -y install mysql mysql-server mysql-devel git make bison libdbi-dbd-mysql (具体还有什么忘记了,不能重现当时安装的了,后面缺什么再装吧)

检出代码

cd ~/ && git clone https://github.com/OpenSIPS/opensips.git -b 2.1 opensips_2_1

修改配置开启mysql支持

vim ~/opensips_2_1/Makefile.conf.tmplate ,移除exclude_modules中db_mysql

安装opensips

1
2
3
cd ~/opensips_2_1
sudo make all
sudo make install

安裝完后,修改部分opensipsctlrc文件,如下:

sudo vim /usr/local/etc/opensips/opensipsctlrc

将DB开头的修改为自己的mysql连接情况(事先创建好opensips用户,或者偷懒用root,但是一定要注意安全),SIP_DOMAIN修改为主机ip

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
## your SIP domain
SIP_DOMAIN=yourip
## chrooted directory
# $CHROOT_DIR="/path/to/chrooted/directory"
## database type: MYSQL, PGSQL, ORACLE, DB_BERKELEY, or DBTEXT,
## by default none is loaded
# If you want to setup a database with opensipsdbctl, you must at least specify
# this parameter.
DBENGINE=MYSQL
## database host
DBHOST=localhost
## database name (for ORACLE this is TNS name)
DBNAME=opensips
# database path used by dbtext or db_berkeley
DB_PATH="/usr/local/etc/opensips/dbtext"
## database read/write user
DBRWUSER=opensips
## password for database read/write user
DBRWPW="opensipsrw"
## database super user (for ORACLE this is 'scheme-creator' user)
DBROOTUSER="root"
# user name column
USERCOL="username"

初始化mysql数据库表

sudo /usr/local/sbin/opensipsdbctl create,会提示输入root密码,然后会出现下面的提示

1
2
3
4
5
6
7
8
9
10
MySQL password for root: 
INFO: test server charset
INFO: creating database opensips ...
INFO: Core OpenSIPS tables succesfully created.
Install presence related tables? (y/n): y
INFO: creating presence tables into opensips ...
INFO: Presence tables succesfully created.
Install tables for imc cpl siptrace domainpolicy carrierroute userblacklist? (y/n): y
INFO: creating extra tables into opensips ...
INFO: Extra tables succesfully created.

如果出现 ERROR: database engine not specified, please setup one in the config script,看看是否有执行权限

自定义监听端口及协议

可以通过修改/usr/local/etc/opensips/opensips.cfg文件,修改listen=,注意这里的ip要用外网ip,udp改为tcp可以使用keepalive,让手机长时间在线。我的配置如下:

1
2
3
4
5
6
7
8
9
10
advertised_address=myip
alias=myip
...
listen=tcp:myip:5060 # CUSTOMIZE ME
tcp_connection_lifetime=3600
tcp_connect_timeout=5000
tcp_keepalive=1
tcp_keepcount=30
tcp_keepidle=60
tcp_keepinterval = 60

新增domain到数据库

这个domain就是上面填写的ip地址,创建用户的时候也要用到。

用root连接mysql,选择opensips库,插入一条记录INSERT INTO opensips.domain(domain) VALUES('your ip');(注意修改为自己的ip)

opensips操作

启动 /usr/local/sbin/opensipsctl start

停止 /usr/local/sbin/opensipsctl stop

重启 /usr/local/sbin/opensipsctl restart

创建用户

/usr/local/sbin/opensipsctl add [email protected] password

这里的user就是号码,尽量用数字(键盘上没有字母啊)

客户端

  • PC
    • Zoiper
    • Linphone
  • Ios
    • Zoiper

其他未测试,可以到 http://www.voip-info.org/wiki/view/Open+Source+VOIP+Software 查找

ios端配置(尽量后台运行)

如图所示:

微信图片_20180129170327.jpg
微信图片_20180129170352.jpg
微信图片_20180129170357.jpg

自动清理k8s中的容器、卷、镜像

镜像源码 https://github.com/meltwater/docker-cleanup

注意:这个镜像会将所有已经退出的容器、未使用的镜像和data-only的容器,除非你将他们加到保存的变量中。注意正确配置docker api的版本,以免删除所有的镜像。小心挂载 /var/lib/docker,因为如果挂载后没有使用的话,也会被当作未使用的卷删掉。

支持的变量

  • CLEAN_PERIOD=1800 - Interval in seconds to sleep after completing a cleaning run. Defaults to 1800 seconds = 30 minutes.
  • DELAY_TIME=1800 - Seconds to wait before removing exited containers and unused images. Defaults to 1800 seconds = 30 minutes.
  • KEEP_IMAGES - List of images to avoid cleaning, e.g. “ubuntu:trusty, ubuntu:latest”. Defaults to clean all unused images.
  • KEEP_CONTAINERS - List of images for exited or dead containers to avoid cleaning, e.g. “ubuntu:trusty, ubuntu:latest”.
  • KEEP_CONTAINERS_NAMED - List of names for exited or dead containers to avoid cleaning, e.g. “my-container1, persistent-data”.
  • LOOP - Add the ability to do non-looped cleanups, run it once and exit. Options are true, false. Defaults to true to run it forever in loops.
  • DEBUG - Set to 1 to enable more debugging output on pattern matches
  • DOCKER_API_VERSION - The docker API version to use. This defaults to 1.20, but you can override it here in case the docker version on your host differs from the one that is installed in this container. You can find - this on your host system by running docker version --format '{{.Client.APIVersion}}'.

对于即使已经不运行了也不想清理的镜像,使用KEEP_IMAGES变量处理,此处我们添写:

vmware/harbor-*:*,*calico:*,*registry:*,*kubernetes-dashboard-amd64:*,*nginx-ingress-controller:*,*cvallance/mongo-k8s-sidecar:*

docker-cleanup-daemonset.yaml 配置如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
name: clean-up
name: clean-up
namespace: kube-system
spec:
updateStrategy:
type: "RollingUpdate"
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
app: clean-up
spec:
tolerations:
- key: "LB"
operator: "Exists"
effect: "NoExecute"
volumes:
- name: docker-sock
hostPath:
path: /var/run/docker.sock
- name: docker-directory
hostPath:
path: /data/kubernetes/docker
containers:
- image: meltwater/docker-cleanup:latest
name: clean-up
env:
- name: CLEAN_PERIOD
value: "1800"
- name: DELAY_TIME
value: "60"
- name: DOCKER_API_VERSION
value: "1.29"
- name: KEEP_IMAGES
value: "vmware/harbor-*:*,*calico:*,*registry:*,*kubernetes-dashboard-amd64:*,*nginx-ingress-controller:*,*cvallance/mongo-k8s-sidecar:*"
volumeMounts:
- mountPath: /var/run/docker.sock
name: docker-sock
readOnly: false
- mountPath: /var/lib/docker
name: docker-directory
readOnly: false

使用DaemonSet+Taint/Tolerations+NodeSelector部署Nginx ingress controller

使用DaemonSet+NodeSelector+Tolerations的方式定义Nginx Ingress Controller,给专门节点打上Label+Taint,使得这些专门节点只运行Nginx Ingress Controller,而不会调度和运行其他业务容器,只用来做代理节点。

  • 在Kuberntes Cluster中准备N个节点,我们称之为代理节点。在这N个节点上只部署Nginx Ingress Controller(简称NIC)实例,不会跑其他业务容器。

  • 给代理节点打上NoExecute Taint,防止业务容器调度或运行在这些节点。

    kubectl taint nodes 10.8.8.234 LB=NIC:NoExecute

  • 给代理节点打上Label,让NIC只部署在打了对应Lable的节点上。
    kubectl label nodes 10.8.8.234 LB=NIC

  • 修改calico-node配置,让calico可以在NoExecute节点上运行

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
        spec:
    ...
    spec:
    tolerations:
    - key: "LB"
    operator: "Exists"
    effect: "NoExecute"
    ```

    - 定义DaemonSet Yaml文件,注意加上Tolerations和Node Selector。(注意先创建serviceAccount、role等)

    apiVersion: extensions/v1beta1
    kind: DaemonSet
    metadata:
    annotations:

    deployment.kubernetes.io/revision: "4"
    

    labels:

    k8s-app: nginx-ingress-controller
    

    name: nginx-ingress-controller
    namespace: kube-system
    spec:
    selector:

    matchLabels:
    k8s-app: nginx-ingress-controller
    

    template:

    metadata:
    annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    creationTimestamp: null
    labels:
        k8s-app: nginx-ingress-controller
    spec:
    # 加上对应的Node Selector
    nodeSelector:
        LB: NIC
    # 加上对应的Tolerations
    tolerations:
    - key: "LB"
        operator: "Equal"
        value: "NIC"
        effect: "NoExecute"
    containers:
    - args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
        - --tcp-services-configmap=$(POD_NAMESPACE)/nginx-tcp-ingress-configmap
        - --configmap=$(POD_NAMESPACE)/nginx-configuration
        env:
        - name: POD_NAME
        valueFrom:
            fieldRef:
            apiVersion: v1
            fieldPath: metadata.name
        - name: POD_NAMESPACE
        valueFrom:
            fieldRef:
            apiVersion: v1
            fieldPath: metadata.namespace
        image: dceph02.rmz.flamingo-inc.com:8888/mynginx/nginx-ingress-controller:0.9.0-beta.11
        imagePullPolicy: IfNotPresent
        livenessProbe:
        failureThreshold: 3
        httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        initialDelaySeconds: 10
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
        name: nginx-ingress-controller
        ports:
        - containerPort: 80
        hostPort: 80
        protocol: TCP
        - containerPort: 443
        hostPort: 443
        protocol: TCP
        readinessProbe:
        failureThreshold: 3
        httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
        resources: {}
    hostNetwork: true
    serviceAccount: ingress
    serviceAccountName: ingress
    
    1
    2
        
    - 创建default backend服务

    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: default-http-backend
    labels:

    k8s-app: default-http-backend
    

    namespace: kube-system
    spec:
    replicas: 1
    template:

    metadata:
    labels:
        k8s-app: default-http-backend
    spec:
    terminationGracePeriodSeconds: 60
    containers:
    - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
        httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
        initialDelaySeconds: 30
        timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
        limits:
            cpu: 10m
            memory: 20Mi
        requests:
            cpu: 10m
            memory: 20Mi
    

    apiVersion: v1
    kind: Service
    metadata:
    name: default-http-backend
    namespace: kube-system
    labels:

    k8s-app: default-http-backend
    

    spec:
    ports:

    • port: 80
      targetPort: 8080
      selector:
      k8s-app: default-http-backend
      1
      2
      3
      4
      5
      6
      7
      8
      9
      根据default-backend.yaml创建对应的Deployment和Service。 `kubectl create -f default-backend.yaml`

      - 根据DaemonSet Yaml创建NIC DaemonSet,启动NIC。

      `kubectl create -f nginx-ingress-daemonset.yaml`

      至此,NIC已经运行在代理节点上了,下面为测试内容。

      - (选择性)确认NIC启动成功后,创建测试用的服务。
      kubectl run echoheaders –image=gcr.io/google_containers/echoserver:1.8 –replicas=1 –port=8080
      kubectl expose deployment echoheaders –port=80 –target-port=8080 –name=echoheaders-x
      kubectl expose deployment echoheaders –port=80 –target-port=8080 –name=echoheaders-y
      1
      创建测试用的Ingress Object
      apiVersion: extensions/v1beta1
      kind: Ingress
      metadata:
      name: echomap
      namespace: default
      spec:
      rules:
    • host: foo.bar.com
      http:
      paths:
      • backend:
        serviceName: echoheaders-x
        servicePort: 80
        path: /foo
    • host: bar.baz.com
      http:
      paths:
      • backend:
        serviceName: echoheaders-y
        servicePort: 80
        path: /bar
      • backend:
        serviceName: echoheaders-x
        servicePort: 80
        path: /foo
        1
        2

        - (选择性)查看ingress的代理地址
        [[email protected] ~]# kubectl describe ing echomap
        Name: echomap
        Namespace: default
        Address: 10.8.8.234
        Default backend: default-http-backend:80 (172.254.109.193:8080)
        Rules:
        Host Path Backends

    foo.bar.com

    /foo    echoheaders-x:80 (<none>)
    

    bar.baz.com

    /bar    echoheaders-y:80 (<none>)
    /foo    echoheaders-x:80 (<none>)
    

    Annotations:
    Events:
    FirstSeen LastSeen Count From SubObjectPath Type Reason Message


    35m 35m 1 ingress-controller Normal CREATE Ingress default/echomap
    35m 35m 1 ingress-controller Normal UPDATE Ingress default/echomap

    1
    2

    - 测试

    [[email protected] ~]# curl 10.8.8.234/foo -H ‘Host: foo.bar.com’

    Hostname: echoheaders-1076692255-p1ndv
    Pod Information:

    -no pod information available-
    

    Server values:

    server_version=nginx: 1.13.3 - lua: 10008
    

    Request Information:

    client_address=172.254.246.192
    method=GET
    real path=/foo
    query=
    request_version=1.1
    request_uri=http://foo.bar.com:8080/foo
    

    Request Headers:

    accept=*/*
    connection=close
    host=foo.bar.com
    user-agent=curl/7.29.0
    x-forwarded-for=10.8.8.234
    x-forwarded-host=foo.bar.com
    x-forwarded-port=80
    x-forwarded-proto=http
    x-original-uri=/foo
    x-real-ip=10.8.8.234
    x-scheme=http
    

    Request Body:

    -no body in request-
    

    [[email protected] ~]# curl 10.8.8.234/foo -H ‘Host: bar.baz.com’

    Hostname: echoheaders-1076692255-p1ndv
    Pod Information:

    -no pod information available-
    

    Server values:

    server_version=nginx: 1.13.3 - lua: 10008
    

    Request Information:

    client_address=172.254.246.192
    method=GET
    real path=/foo
    query=
    request_version=1.1
    request_uri=http://bar.baz.com:8080/foo
    

    Request Headers:

    accept=*/*
    connection=close
    host=bar.baz.com
    user-agent=curl/7.29.0
    x-forwarded-for=10.8.8.234
    x-forwarded-host=bar.baz.com
    x-forwarded-port=80
    x-forwarded-proto=http
    x-original-uri=/foo
    x-real-ip=10.8.8.234
    x-scheme=http
    

    Request Body:

    -no body in request-
    

    `

参考

https://my.oschina.net/jxcdwangtao/blog/1523812                

Java与Python安全转码链接

Java

1
2
3
4
5
6
7
8
String OriginUrl = "https://meijumao.cn/yunparse/index.php?url=//美剧/周一/GOT权游/S07/Online/02按时发达的十分.mp4~bdyun";
URL repoUrl = new URL(OriginUrl);
URI uri;
try {
uri = new URI(repoUrl.getProtocol(), repoUrl.getUserInfo(), repoUrl.getHost(), repoUrl.getPort(), repoUrl.getPath(), repoUrl.getQuery(), repoUrl.getRef());
System.out.println(uri.toASCIIString());
} catch (URISyntaxException e) {
}

Python

python3:

1
2
3
4
5
import string
import urllib.parse

url = "https://meijumao.cn/yunparse/index.php?url=//美剧/周一/GOT权游/S07/Online/02按时发达的十分.mp4~bdyun"
print(urllib.parse.quote(url,safe=string.printable))

python2:

1
2
3
4
5
import string
from urllib import quote

url = "https://meijumao.cn/yunparse/index.php?url=//美剧/周一/GOT权游/S07/Online/02按时发达的十分.mp4~bdyun"
print urllib.parse.quote(url,safe=string.printable)

结果 'https://meijumao.cn/yunparse/index.php?url=//%E7%BE%8E%E5%89%A7/%E5%91%A8%E4%B8%80/GOT%E6%9D%83%E6%B8%B8/S07/Online/02%E6%8C%89%E6%97%B6%E5%8F%91%E8%BE%BE%E7%9A%84%E5%8D%81%E5%88%86.mp4~bdyun

Java 解析json数组字符串为Java对象

套用罗锤子的话:这可能是最方便快捷的方式了

json字符串:

1
2
3
4
5
6
[{ "number" : "3",
"title" : "hello_world"
},
{ "number" : "2",
"title" : "hello_world"
}]

Java代码:

1
2
3
4
5
6
7
class Wrapper{
int number;
String title;
}

Gson gson = new Gson();
Wrapper[] data = gson.fromJson(idcstring, Wrapper[].class);

awk 计算一列的和

awk '{sum += $10};END {print sum}'

例如想计算nginx日志中206下载总数据和,则用
cat xxx.access.log|grep " 206 " |awk '{sum += $10};END {print sum}'

Python一些模块安装方法记录

首先需要安装pip模块,如果yum中没有,则需要用 get-pip.py安装

sudo python get-pip.py 安装

web.py

sudo pip install web.py

mysql.connector

1
2
3
4
wget http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-1.0.11.zip 
unzip mysql-connector-python-1.0.11.zip
cd mysql-connector-python-1.0.11
sudo python setup.py install

pytds(连接mssql)

1
2
sudo pip install python-tds
#sudo pip install bitarray

anjuke(中文转拼音)

sudo pip install pinyin4py

Python2.7安装

这样安装之后不会与之前的python2.6版本有冲突,3.x版本同理

1
2
3
4
5
wget https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz
tar xf Python-2.7.8.tgz
cd Python-2.7.8
./configure --prefix=/usr/local
make && make install

安装RemixOS到PC

!一定要用UEFI方式启动,前提是64位的

下载镜像

https://www.fosshub.com/Remix-OS.html

刻录到U盘

解压之后有一个iso文件一个exe文件,推荐用ultraiso安装(用那个exe会把U盘分成三个区。。。有病吗?)

修改参数

U盘开机启动之后,选择”Resident/Guest mode”,按TAB键或e键,在命令最后添加”INSTALL=2”,删掉”USB_DATA_PARTITION=1”

接下来系统会安装到硬盘上。