Expected Behavior
KICS should complete the scan without crashing, even on repositories with a large number of files
Actual Behavior
KICS crashes with a fatal error during the "Preparing Scan Assets" phase before any queries are executed:
runtime: program exceeds 10000 thread limit
fatal error: thread exhaustion
In pkg/analyzer/analyzer.go:377, Analyze spawns one goroutine per discovered file with no upper bound:
for _, file := range files {
wg.Add(1)
// analyze the files concurrently
a := &analyzerInfo{
// ...
}
go a.worker(results, unwanted, locCount, fileInfo, &wg)
}
For a repository with tens of thousands of files this immediately creates tens of thousands of goroutines. If individual workers block on I/O, the Go scheduler is forced to spin up additional OS threads to keep progress — eventually hitting the hard 10000 thread limit and crashing the process.
Relevant part of stacktrace:
runtime stack:
runtime.throw(...)
/usr/lib/go/src/runtime/panic.go:1229
runtime.checkmcount()
/usr/lib/go/src/runtime/proc.go:977
runtime.mReserveID()
/usr/lib/go/src/runtime/proc.go:993
runtime.startm(...)
/usr/lib/go/src/runtime/proc.go:3088
runtime.handoffp(...)
/usr/lib/go/src/runtime/proc.go:3137
runtime.retake(...)
/usr/lib/go/src/runtime/proc.go:6715
runtime.sysmon()
/usr/lib/go/src/runtime/proc.go:6597
runtime.mstart1()
/usr/lib/go/src/runtime/proc.go:1935
runtime.mstart0()
/usr/lib/go/src/runtime/proc.go:1888
runtime.mstart()
/usr/lib/go/src/runtime/asm_arm64.s:266
goroutine 1 gp=0x6545bb5841e0 m=nil [runnable]:
sync.(*WaitGroup).Add(...)
/usr/lib/go/src/sync/waitgroup.go:77
github.com/Checkmarx/kics/v2/pkg/analyzer.Analyze(...)
/app/pkg/analyzer/analyzer.go:378
github.com/Checkmarx/kics/v2/pkg/scan.analyzePaths(...)
/app/pkg/scan/utils.go:177
github.com/Checkmarx/kics/v2/pkg/scan.(*Client).prepareAndAnalyzePaths(...)
/app/pkg/scan/utils.go:59
github.com/Checkmarx/kics/v2/pkg/scan.(*Client).initScan(...)
/app/pkg/scan/scan.go:50
github.com/Checkmarx/kics/v2/pkg/scan.(*Client).executeScan(...)
/app/pkg/scan/scan.go:132
github.com/Checkmarx/kics/v2/pkg/scan.(*Client).PerformScan(...)
/app/pkg/scan/client.go:94
Steps to Reproduce the Problem
- Run KICS scan against a large repository (tens of thousands of files):
docker run --user 0:0 \
-v /path/to/repo:/builds/repo \
-v /path/to/custom-queries:/rules/kics \
-v /path/to/output:/uploads \
-w /builds/repo \
--entrypoint kics checkmarx/kics scan \
--queries-path /rules/kics/queries \
--path /builds/repo \
--include-queries <query-id-1>,<query-id-2>,... \
--disable-secrets \
--disable-full-descriptions \
--report-formats sarif \
--output-path /uploads \
--output-name kics.result.sarif
- Observe the process crash during "Preparing Scan Assets" with
fatal error: thread exhaustion.
Fix suggestion
Replace unbounded goroutine per file execution in with a bounded pool of workers
Specifications
- Version: 2.1.20
- Platform: Linux (arm64, Docker)
- Subsystem: pkg/analyzer
Expected Behavior
KICS should complete the scan without crashing, even on repositories with a large number of files
Actual Behavior
KICS crashes with a fatal error during the "Preparing Scan Assets" phase before any queries are executed:
In
pkg/analyzer/analyzer.go:377,Analyzespawns one goroutine per discovered file with no upper bound:For a repository with tens of thousands of files this immediately creates tens of thousands of goroutines. If individual workers block on I/O, the Go scheduler is forced to spin up additional OS threads to keep progress — eventually hitting the hard 10000 thread limit and crashing the process.
Relevant part of stacktrace:
Steps to Reproduce the Problem
fatal error: thread exhaustion.Fix suggestion
Replace unbounded goroutine per file execution in with a bounded pool of workers
Specifications