There are multiple ways to use gitlab executor on macOS hosts, but two main are:
Shell executorSSH executor
Despite some serious disadvantages of ssh executor it keeps all configs centralized (e.g. in kubernetes configmap), opposite to Shell executor that holds config on runner (local filesystem).
My Landscape
- k8s cluster with installed gitlab runners
- Static macOS hosts
General troubleshooting
I suggest to consider using some tweeks
FF_ENABLE_JOB_CLEANUP useful for not dockerized static runnersGET_SOURCES_ATTEMPTS retries for git-clone/fetchGIT_TRACE_PERFORMANCE for git debuggingFF_USE_FASTZIP more performant compressingTRANSFER_METER_FREQUENCY shows artefact transfer speedFF_PRINT_POD_EVENTS kubectl eventsFF_TIMESTAMPS job logs with timestamps per line
1
2
3
4
5
6
7
8
9
10
11
12
13
|
#...
[[runners]]
...
environment = [
"FF_ENABLE_JOB_CLEANUP=1",
"GET_SOURCES_ATTEMPTS=10",
"GIT_TRACE_PERFORMANCE=0",
"FF_USE_FASTZIP=1",
"TRANSFER_METER_FREQUENCY=2s",
"FF_PRINT_POD_EVENTS=true",
"FF_TIMESTAMPS=true"
]
#...
|
SSH executor and its troubleshooting
The SSH executor supports only scripts generated in Bash. See more
Typical ssh config
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
#...
[[runners]]
name = "my-runner description"
limit = 4
request_concurrency = 10
url = "https://my-site.com/"
tls-ca-file = "/home/gitlab-runner/.gitlab-runner/my-site.com.crt"
token = "my_group_runner_token"
output_limit = 50000
# By defult CI_PROJECT_DIR returns relative path for ssh runner,
# below provides absolute
builds_dir = "/Users/my-user/builds"
executor = "ssh"
[runners.ssh]
host = "my-host.my-site.com"
port = "22"
user = "my-user"
password = "my-password"
disable_strict_host_key_checking = true
# Job failed: prearing AbstractExecutor: setting GIT_CLONE_PATH is not allowed,
# enable custom_build_dir feature
[runners.custom_build_dir]
enabled = true
#...
|
Mac ssh-runners unable to upload artifacts
According to the official doc, if you want to upload job artifacts, install gitlab-runner on the host you are connecting to via SSH.
Install respective gitlab-runner
1
2
3
4
5
6
7
8
9
|
# ARM
sudo curl --output /usr/local/bin/gitlab-runner "https://s3.dualstack.us-east-1.amazonaws.com/gitlab-runner-downloads/latest/binaries/gitlab-runner-darwin-arm64"
# X86
sudo curl --output /usr/local/bin/gitlab-runner "https://s3.dualstack.us-east-1.amazonaws.com/gitlab-runner-downloads/latest/binaries/gitlab-runner-darwin-amd64"
sudo chmod +x /usr/local/bin/gitlab-runner
gitlab-runner install
gitlab-runner start
|
GIT_CLONE_PATH is not allowed
Job failed: prearing AbstractExecutor: setting GIT_CLONE_PATH is not allowed, enable custom_build_dir feature
Add [runners.custom_build_dir] feature to gitlab toml config, as in the example below
1
2
3
4
5
6
|
[[runners]]
...
executor = "ssh"
...
[runners.custom_build_dir]
enabled = true
|
Shell executor and its troubleshooting
gitlab-runner acts as typical MacOS service.
Unable to start service on MacOS on shell runner
FATAL: Failed to start gitlab-runner: exit status 134
1
2
3
4
|
gitlab-runner uninstall
gitlab-runner install --user your_user
gitlab-runner stop
gitlab-runner start
|
Unable to create Keychain on MacOS
security: SecKeychainCreate signdmg87130.keychain: UNIX[Permission denied]
Follow this document to add <key>SessionCreate</key> in /Library/LaunchDaemons/gitlab-runner.plist, whole file is below
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Disabled</key>
<false/>
<key>KeepAlive</key>
<dict>
<key>SuccessfulExit</key>
<false/>
</dict>
<key>Label</key>
<string>gitlab-runner</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/gitlab-runner</string>
<string>run</string>
<string>--config</string>
<string>/etc/gitlab-runner/config.toml</string>
<string>--working-directory</string>
<string>/Users/your_user</string>
<string>--service</string>
<string>gitlab-runner</string>
<string>--syslog</string>
<string>--user</string>
<string>your_user</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>SessionCreate</key>
<true/>
<key>StandardErrorPath</key>
<string>/Users/your_user/.gitlab-runner/gitlab-runner.err.log</string>
<key>StandardOutPath</key>
<string>/Users/your_user/.gitlab-runner/gitlab-runner.out.log</string>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
</dict>
</dict>
</plist>
|
Also, an example of /etc/gitlab-runner/config.toml is below:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
concurrent = 1
check_interval = 0
connection_max_age = "10m0s"
shutdown_timeout = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "host.name.com"
url = "https://gitlab.mycompany.com/"
id = 1494
token = "<TOKEN>"
token_obtained_at = 2023-09-09T11:57:04Z
token_expires_at = 0001-01-01T00:00:00Z
executor = "shell"
[runners.feature_flags]
FF_RESOLVE_FULL_TLS_CHAIN = false
[runners.custom_build_dir]
[runners.cache]
MaxUploadedArchiveSize = 0
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
|
Gitlab-runner service hangs or not responding on MacOS
Gitlab jobs remain in pending state. When you log on the host then it starts processing the jobs.
Spotlight cache grows huge
It can commonly happen when you build a lot dmg packages as a part of CI/CD process.
Investigate the problem
1
2
3
4
5
|
# Check what disks indexed
mdutil -s -a
# Check folders size
sudo brew install ncdu
sudo ncdu /
|
Disable and remove indexing
1
2
3
4
5
|
# disables Spotlight indexing for all volumes (i.e., hard drives) on the system.
sudo mdutil -a -i off
# Clean cache directory
sudo rm -rf /.Spotlight*
#sudo rm -rf /System/Volumes/Data/.Spotlight*
|
