Earlier this week I was running KICS on my M1 Mac using Podman, playing with the --exclude-paths parameter, when suddenly KICS was hanging for what felt like forever at the Preparing Scan Assets: stage.
Command was something like
podman run --rm -ti -v "$(pwd)":/path docker.io/checkmarx/kics:latest scan -p /path -o /path --report-formats html,json --exclude-paths "/path/**/*example*,/path/.resources/*,/path/**/*.json"
Using the -v flag I was able to see that files were being excluded at a very slow pace (I could watch those "Excluded" lines being printed one by one): (I am trying to reproduce the situation here; if I remember correctly, at that day I had more than a 1000 files in that project, the majority of them inside .terraform/.)
1:16PM INF Total files in the project: 858
1:16PM INF .gitignore file was found in '/path' and it will be used to automatically exclude paths
Preparing Scan Assets: - 1:16PM INF Excluded file /path/.terraform/modules/bastion/.gitlab-ci.yml from analyzer
Preparing Scan Assets: \ 1:16PM INF Excluded file /path/.terraform/modules/bastion/context.tf from analyzer
Preparing Scan Assets: - 1:16PM INF Excluded file /path/.terraform/modules/bastion/examples/01-private-bastion-example/private-bastion-example.tf from analyzer
Preparing Scan Assets: |
…
Digging deeper I found that lots of time was spent finding files in the filesystem.
The issue seems to be that isExcludedFile is called for every file the project, which in turn performs one (or more) walks of the entire file tree when exclude patterns with ** are used – that's about a thousand walks through the filesystem when one would suffice.
That I even noticed must be due to the relative slowness of the filesystem access within the Podman container (running in a VM, accessing a project directory outside of the VM), I guess.
Expected Behavior
I expect the "Preparing Scan Assets" stage to be fast, not repeating work unnecessarily.
Actual Behavior
KICS felt stuck. See steps below to get an impression.
Steps to Reproduce the Problem
I cobbled together https://github.com/buehmann/kics/tree/globtest while doing my research.
I added some logging to GetExcludePaths in order to double-check how often this was called. Later I added a delay of 50 ms to it to simulate a slow filesystem outside of the container environment. I selected that value so that the durations of my two test runs inside and outside of Podman were comparable:
./globtest.run.podman 0,22s user 0,45s system 0% cpu 2:22,68 total
./globtest.run 46,69s user 29,57s system 53% cpu 2:23,06 total
Here is how to use it:
- Check out that branch and build it.
- Run
./globtest.prepare to generate a tree of 1000 empty files.
- Run
./globtest.run to see the (artifically) slow scanning in action.
- Run
./globtest.run.podman to run the same test with the released v2.1.20 inside Podman.
Output looks something like this in step 3:
3:48PM INF Total files in the project: 1000
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
Preparing Scan Assets: \ 3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
Preparing Scan Assets: - 3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
Preparing Scan Assets: | 3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
Preparing Scan Assets: / 3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
Preparing Scan Assets: - 3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
Preparing Scan Assets: | 3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
Preparing Scan Assets: \ 3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
Preparing Scan Assets: - 3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
3:48PM TRC filepathx.Glob(globtest/**/5*)
3:48PM TRC filepathx.Glob(globtest/**/3*)
I also looked briefly at system calls with strace earlier today like this, in order to confirm that the filesystem walk was actually repeated (you need to scroll to the right to see everything):
➜ kics (globtest) ✗ podman run --rm -ti -v "$(pwd)":/path --entrypoint bash docker.io/checkmarx/kics:debian
root@b2c563abfd50:/app/bin# apt update
…
root@b2c563abfd50:/app/bin# apt install strace
…
root@b2c563abfd50:/app/bin# cd /tmp
root@b2c563abfd50:/tmp# cp -r /path/globtest .
root@b2c563abfd50:/tmp# strace -f -P "globtest/1/2/3.tf" -e signal=none kics scan -p globtest --exclude-paths "globtest/**/5*"
strace: Requested path "globtest/1/2/3.tf" resolved into "/tmp/globtest/1/2/3.tf"
…
Scanning with Keeping Infrastructure as Code Secure v2.1.20
strace: Process 89 attached
[pid 84] newfstatat(AT_FDCWD, "/tmp/globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: \ [pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: - [pid 80] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: | [pid 85] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: / [pid 85] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: - [pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: | [pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: \ [pid 84] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: - [pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: | [pid 89] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: / [pid 89] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: - [pid 85] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: | [pid 84] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: \ [pid 84] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: - [pid 84] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: | [pid 87] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: / [pid 89] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
Preparing Scan Assets: - [pid 85] newfstatat(AT_FDCWD, "globtest/1/2/3.tf", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
Preparing Scan Assets: |
…
Specifications
- Version: v2.1.20 + master (600c046)
- Platform: macOS + Podman
➜ kics (globtest) ✗ uname -a
Darwin AM-D4FLDLHV67 25.5.0 Darwin Kernel Version 25.5.0: Mon Apr 27 20:38:56 PDT 2026; root:xnu-12377.121.6~2/RELEASE_ARM64_T6000 arm64 arm Darwin
➜ kics (globtest) ✗ podman machine list
NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE
podman-machine-default libkrun 7 months ago Currently running 5 3.725GiB 93GiB
Earlier this week I was running KICS on my M1 Mac using Podman, playing with the
--exclude-pathsparameter, when suddenly KICS was hanging for what felt like forever at thePreparing Scan Assets:stage.Command was something like
Using the
-vflag I was able to see that files were being excluded at a very slow pace (I could watch those "Excluded" lines being printed one by one): (I am trying to reproduce the situation here; if I remember correctly, at that day I had more than a 1000 files in that project, the majority of them inside.terraform/.)Digging deeper I found that lots of time was spent finding files in the filesystem.
The issue seems to be that isExcludedFile is called for every file the project, which in turn performs one (or more) walks of the entire file tree when exclude patterns with
**are used – that's about a thousand walks through the filesystem when one would suffice.That I even noticed must be due to the relative slowness of the filesystem access within the Podman container (running in a VM, accessing a project directory outside of the VM), I guess.
Expected Behavior
I expect the "Preparing Scan Assets" stage to be fast, not repeating work unnecessarily.
Actual Behavior
KICS felt stuck. See steps below to get an impression.
Steps to Reproduce the Problem
I cobbled together https://github.com/buehmann/kics/tree/globtest while doing my research.
I added some logging to
GetExcludePathsin order to double-check how often this was called. Later I added a delay of 50 ms to it to simulate a slow filesystem outside of the container environment. I selected that value so that the durations of my two test runs inside and outside of Podman were comparable:Here is how to use it:
./globtest.prepareto generate a tree of 1000 empty files../globtest.runto see the (artifically) slow scanning in action../globtest.run.podmanto run the same test with the releasedv2.1.20inside Podman.Output looks something like this in step 3:
I also looked briefly at system calls with
straceearlier today like this, in order to confirm that the filesystem walk was actually repeated (you need to scroll to the right to see everything):Specifications