使用docker编译打包sailfishos
开坑
- ubuntu HA_BUILD
用官方的ubuntu镜像即可，16.04或18.04都可以，不要用最新的20.04。一般来说启动之后的镜像除了手动指定的目录是持久化的，其他的会重启后失效，所以最好自己做一个镜像，把安卓编译环境安装上。
启动时映射本地目录，当作ANDROID_ROOT目录。
mer MER_BUILD
OBS
gitlab ci
开坑
用官方的ubuntu镜像即可，16.04或18.04都可以，不要用最新的20.04。一般来说启动之后的镜像除了手动指定的目录是持久化的，其他的会重启后失效，所以最好自己做一个镜像，把安卓编译环境安装上。
启动时映射本地目录，当作ANDROID_ROOT目录。
mer MER_BUILD
OBS
gitlab ci
注意：必须是用域名的方式(也就是有内网的dns)，如果以前用ip，则本方法无效！
Harbor1.2之前的版本不能直接升级到新版本，想要升级到最新版并且业务不中断，可以采用如下方式。
大体流程如下：
B机器搭一个新harbor -> 手动将旧harbor的镜像push到新harbor -> 更改A域名指向到B主机ip ->
测试B的harbor服务是否正常 -> 铲掉A上的旧harbor -> 在A上重新搭建harbor -> B机器上的harbor同步到A上的harbor
测试A的harbor服务是否正常 -> 改回A域名指向A主机 -> 删掉B上的同步。
手动push旧harbor镜像到新harbor所用到的脚本：
pip install python_harborclient
get_all.py:
1 | #!/usr/bin/python |
python get_all.py > all_repos.txt
allimages=$(cat all_repos.txt)
ORIGIN_HOST=”pk8snode01.rmz.flamingo-inc.com:8888” #旧harbor
BACK_HOST=”pk8stemp02.rmz.flamingo-inc.com:8888” #新harbor
#提前登录一下
#docker login $BACK_HOST
for image in ${allimages}; do
docker pull ${ORIGIN_HOST}/$image
docker tag ${ORIGIN_HOST}/$image ${BACK_HOST}/$image
docker push ${BACK_HOST}/$image
sleep 1
echo $image “done”
done`
This post will show you how to install Docker on SailfishOS, and some hacks need to do.
这篇文章将介绍如何将Docker安装到SailfishOS上，和需要做的一些hack。
ps executable, usually provided by procps or a similar package.A properly mounted cgroupfs hierarchy; a single, all-encompassing cgroup mount point is not sufficient. See Github issues #2683, #3485, #4568).
64位系统
psUse this script check-config.sh
使用这个脚本 check-config.sh
1 | [nemo@Sailfish ~]$ ./check-config.sh |
Generally Necessary must be all enabled, if not enabled, you must enable it in your kernel defconfig, and rebuild kernel.
Generally Necessary 部分必须全部是enabled, 如果没有启用，必须启用然后重启编译内核。
https://download.docker.com/linux/static/stable/aarch64/
Extract the archive and put them to /usr/bin/, 18.06 is a working version.
1 | groupadd docker |
Start docker daemon/ 启动docker守护进程devel-su /usr/bin/dockerd
Or use systemd/ 或者使用systemd
1 | [Unit] |
Check version/检查版本1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20[root@Sailfish nemo]# docker version
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:20:38 2018
OS/Arch: linux/arm64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: e68fc7a
Built: Tue Aug 21 17:27:20 2018
OS/Arch: linux/arm64
Experimental: false
Test/测试devel-su docker run hello-world
This command downloads a test image and runs it in a container. When the container runs, it prints an informational message and exits. / 这个命令会下载一个测试镜像，如果执行成功会打印如下信息
1 | [root@Sailfish nemo]# docker run hello-world |
On one terminal/在一个终端中执行1
2
3[root@Sailfish nemo]# docker run -it --rm -p 6080:80 nginx:latest
172.17.0.1 - - [05/Sep/2018:08:54:52 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.58.0-DEV" "-"
172.17.0.1 - - [05/Sep/2018:08:55:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.58.0-DEV" "-"
Vist on another terminal/在另一个终端中访问1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27[nemo@Sailfish ~]$ curl -s 127.0.0.1:6080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[nemo@Sailfish ~]$
Wayland forward /wayland转发
Reference/参考:
Have fun ;)
长时间运行的k8s节点可能会存在某些pod不自动退出，一直处于Terminating的状态
于是我们可以用这个脚本定时进行清理
1 | #!/bin/bash |
支持的变量
docker version --format '{{.Client.APIVersion}}'.对于即使已经不运行了也不想清理的镜像，使用KEEP_IMAGES变量处理，此处我们添写：
vmware/harbor-*:*,*calico:*,*registry:*,*kubernetes-dashboard-amd64:*,*nginx-ingress-controller:*,*cvallance/mongo-k8s-sidecar:*
docker-cleanup-daemonset.yaml 配置如下：1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
name: clean-up
name: clean-up
namespace: kube-system
spec:
updateStrategy:
type: "RollingUpdate"
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
app: clean-up
spec:
tolerations:
- key: "LB"
operator: "Exists"
effect: "NoExecute"
volumes:
- name: docker-sock
hostPath:
path: /var/run/docker.sock
- name: docker-directory
hostPath:
path: /data/kubernetes/docker
containers:
- image: meltwater/docker-cleanup:latest
name: clean-up
env:
- name: CLEAN_PERIOD
value: "1800"
- name: DELAY_TIME
value: "60"
- name: DOCKER_API_VERSION
value: "1.29"
- name: KEEP_IMAGES
value: "vmware/harbor-*:*,*calico:*,*registry:*,*kubernetes-dashboard-amd64:*,*nginx-ingress-controller:*,*cvallance/mongo-k8s-sidecar:*"
volumeMounts:
- mountPath: /var/run/docker.sock
name: docker-sock
readOnly: false
- mountPath: /var/lib/docker
name: docker-directory
readOnly: false
使用DaemonSet+NodeSelector+Tolerations的方式定义Nginx Ingress Controller，给专门节点打上Label+Taint，使得这些专门节点只运行Nginx Ingress Controller，而不会调度和运行其他业务容器，只用来做代理节点。
在Kuberntes Cluster中准备N个节点，我们称之为代理节点。在这N个节点上只部署Nginx Ingress Controller（简称NIC）实例，不会跑其他业务容器。
给代理节点打上NoExecute Taint，防止业务容器调度或运行在这些节点。
kubectl taint nodes 10.8.8.234 LB=NIC:NoExecute
给代理节点打上Label，让NIC只部署在打了对应Lable的节点上。
kubectl label nodes 10.8.8.234 LB=NIC
修改calico-node配置，让calico可以在NoExecute节点上运行
1 | spec: |
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
annotations:
deployment.kubernetes.io/revision: "4"
labels:
k8s-app: nginx-ingress-controller
name: nginx-ingress-controller
namespace: kube-system
spec:
selector:
matchLabels:
k8s-app: nginx-ingress-controller
template:
metadata:
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
k8s-app: nginx-ingress-controller
spec:
# 加上对应的Node Selector
nodeSelector:
LB: NIC
# 加上对应的Tolerations
tolerations:
- key: "LB"
operator: "Equal"
value: "NIC"
effect: "NoExecute"
containers:
- args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
- --tcp-services-configmap=$(POD_NAMESPACE)/nginx-tcp-ingress-configmap
- --configmap=$(POD_NAMESPACE)/nginx-configuration
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
image: dceph02.rmz.flamingo-inc.com:8888/mynginx/nginx-ingress-controller:0.9.0-beta.11
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: nginx-ingress-controller
ports:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
hostNetwork: true
serviceAccount: ingress
serviceAccountName: ingress
1
2
3
- 创建default backend服务
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: default-http-backend
labels:
k8s-app: default-http-backend
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: default-http-backend
spec:
terminationGracePeriodSeconds: 60
containers:
- name: default-http-backend
# Any image is permissable as long as:
# 1. It serves a 404 page at /
# 2. It serves 200 on a /healthz endpoint
image: gcr.io/google_containers/defaultbackend:1.0
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
apiVersion: v1
kind: Service
metadata:
name: default-http-backend
namespace: kube-system
labels:
k8s-app: default-http-backend
spec:
ports:
1 | 根据default-backend.yaml创建对应的Deployment和Service。 `kubectl create -f default-backend.yaml` |
1 | 创建测试用的Ingress Object |
1 |
|
foo.bar.com
/foo echoheaders-x:80 (<none>)
bar.baz.com
/bar echoheaders-y:80 (<none>)
/foo echoheaders-x:80 (<none>)
Annotations:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
35m 35m 1 ingress-controller Normal CREATE Ingress default/echomap
35m 35m 1 ingress-controller Normal UPDATE Ingress default/echomap
1 |
|
[root@host ~]# curl 10.8.8.234/foo -H ‘Host: foo.bar.com’
Hostname: echoheaders-1076692255-p1ndv
Pod Information:
-no pod information available-
Server values:
server_version=nginx: 1.13.3 - lua: 10008
Request Information:
client_address=172.254.246.192
method=GET
real path=/foo
query=
request_version=1.1
request_uri=http://foo.bar.com:8080/foo
Request Headers:
accept=*/*
connection=close
host=foo.bar.com
user-agent=curl/7.29.0
x-forwarded-for=10.8.8.234
x-forwarded-host=foo.bar.com
x-forwarded-port=80
x-forwarded-proto=http
x-original-uri=/foo
x-real-ip=10.8.8.234
x-scheme=http
Request Body:
-no body in request-
[root@dceph04 ~]# curl 10.8.8.234/foo -H ‘Host: bar.baz.com’
Hostname: echoheaders-1076692255-p1ndv
Pod Information:
-no pod information available-
Server values:
server_version=nginx: 1.13.3 - lua: 10008
Request Information:
client_address=172.254.246.192
method=GET
real path=/foo
query=
request_version=1.1
request_uri=http://bar.baz.com:8080/foo
Request Headers:
accept=*/*
connection=close
host=bar.baz.com
user-agent=curl/7.29.0
x-forwarded-for=10.8.8.234
x-forwarded-host=bar.baz.com
x-forwarded-port=80
x-forwarded-proto=http
x-original-uri=/foo
x-real-ip=10.8.8.234
x-scheme=http
Request Body:
-no body in request-
`
https://my.oschina.net/jxcdwangtao/blog/1523812