testing of mcl

[miekg]

One of our issues is that we could write tests, but it is really cumbersome [in cfengine]. I would very much see a good story here. Right now what I came up with is for testing /examples/lang/hello0.mcl, is:

import "os"

print "" {
        msg => if os.file_exists("/tmp/mgmt-hello-world") {
                        "exists"
        } else {
                        "does not exist"
        },
}

but running this is not as easy as 'go test', so could not make mgmt exit with status 1, but I haven't looked really hard if that is possible.

[purpleidea]

Internally we have TestAstFunc2 (look at the .txtar tests)

Of course we need to add something for: #868

The bigger long-term problem is how to describe what the test of a dynamic, distributed system should look like. How would you do that? I have some ideas but it's a bigger project.

If you think there's a smaller subset of what we should allow today such that we could add an mgmt test command, I'm all ears! I did play around with a test resource to start some ideas flowing, but structurally we might need a smarter approach. What is it exactly that we want to test?

[miekg]

I hear you; testing a package install is already trouble, but having stuff (this is more like unit tests) https://cfengine.com/blog/2023/how-can-i-test-cfengine-policy/ can be helpful, esp when writing function.

Bigger tests in our case would involve installing a vm from scratch and a possibly running tests after we canary a rollout.

[purpleidea]

I support making progress/improvements in this area, but I'm kind of at the stage where I'd need a more concrete design proposal. Also happy to do a little hacking session with you and we can discuss back and forth a bit.