Move PASS/FAIL to the end of file name

[atomczak]

This is rather cosmetic, but it would be better to read and work with the test files if the PASS/FAIL/INVALID information was at the end of the file name. This way, we would have a pass/fail files next to each other when sorting alphabetically.

Example:
NOW: pass-nesting_may_be_indirect.ifc
PROPOSAL: nesting_may_be_indirect-pass.ifc

[CBenghi]

Hello @atomczak,
if we are going to restructure the testing logic, I think we can do better than just renaming them.
E.g.

• Most of the IFC files are replicated many times, and this could be reduced (1 IFC - many IDS)
• Single IDS files could be targeting multiple IFCs with different outcomes for schema and content (1 IDS, many IFCs)

This would just require a list of tests pointing to the files with expected outcomes, which can be easily generated in json or something similar. What do you reckon?

[andyward]

• Single IDS files could be targeting multiple IFCs with different outcomes for schema and content (1 IDS, many IFCs)

This would just require a list of tests pointing to the files with expected outcomes, which can be easily generated in json or something similar. What do you reckon?

This feels almost more trouble than it's worth? It's not like the IFC files are taking up much space, and they are essentially static (and auto-gen'd)? Having a 1-1 mapping between IDS and IFC test files makes it simple for implementor test suites to build the tests conventionally, and it's easier for debugging to have the file explicitly linked rather than via some indirection in a Json file.

I'd support the suffixing of 'outcome' rather than prefix.

One other option might be to group the tests further under folders - based on the concept being tested.

[atomczak]

In general I agree with @andyward that it doesn't hurt to have many small files even if some are duplicates with different names. However, the tests we have now are small unit tests, testing one aspect at a time, but @CBenghi, I think you suggested at some point having a more sophisticated test(s) with a bigger IFC file to test for example applicability on larger sets and a combination of multiple aspects of IDS. In that case, it would no longer be a 1-to-1 file, and we would need some mapping anyway.

Afaik, the IFC files are not auto-gen'd anymore. Am I right?

[CBenghi]

Yes,
that's right, they are not generated.

Ideally a new generation routine would have:

• the ability to produce multiple schema versions from one spec, and
• be more complete and valid (many of the current files would not pass IFC validation)

I did think a little about a scripting tool, but I could not find an elegant solution yet.
Admittedly, though, since the IFC schema is stable, IFC generation is less critical than IDS.

[atomczak]

The solution that I can think of is IfcOpenShell (in Python) to generate the IFC and a Docker image with the Validation Service to check them after creation. Would that work, or is that overkill?

Still, for this bigger model check, I would use an actual model like the ones from PCERT: https://github.com/buildingSMART/Sample-Test-Files

[TLiebich]

Jon @jmirtsch had created a small scripting app to generate sample ifc files a while ago (we used them for unit test cases in IFC4 / early IFC4.3 time frame). Maybe this is something to have in mind?

[andyward]

Ideally a new generation routine would have:

• the ability to produce multiple schema versions from one spec, and
• be more complete and valid (many of the current files would not pass IFC validation)

I did think a little about a scripting tool, but I could not find an elegant solution yet.

Hi @CBenghi, given the IDS build / automation system is .net would it make sense to use xbim Toolkit to generate these IFC files as part of the existing tool? I did a bit of work this weekend to provide a simple 'fluent' syntax for IFC generation for this kind of test case generation, which I think addresses the points above (multi-schema and valid output). A quick example:

var builder = new FluentModelBuilder();
var editor = GetEditor(); // Used by OwnerHistory and FILE_NAME metadata

builder
    .AssignEditor(editor)
    .UseStableGuids(BaselineGuid)      // So Guids remain stable across runs
    .UseStableDateTime(new DateTime(2025, 1, 1));
builder
    .CreateModel(XbimSchemaVersion.Ifc4)   // Easily switch between 2x3, 4 and 4x3
    .SetHeaders()
    .SetOwnerHistory()
    .CreateEntities(cfg =>
    {
        // an IfcWall of Userdefined ObjectType 'X', defined by an  IfcWallType with undefined PredefinedType 
        var type = cfg.WallType(o => o.WithDefaults(t => t with { Name = "n/a"}));
        cfg.Wall(o => o.WithDefaults(t => t with { PredefinedType = "X" }))
            .AddDefiningType(type);
    })
    .AssertValid()    // Validates properties meet schema constraints and Where rules adhere  
    .SaveAsIfc("pass-overridden_predefined_types_should_pass.ifc");

This creates a valid IFC4 test case like:

ISO-10303-21;
HEADER;
FILE_DESCRIPTION (('ViewDefinition [CoordinationView]'), '2;1');
FILE_NAME ('pass-overridden_predefined_types_should_pass.ifc', '2025-01-01T00:00:00', ('Claudio Benghi'), ('buildingSMART'), 'xbim Toolkit v6.0.0.0', 'BuildingSMART IDS tests 1.2.3-alpha', '');
FILE_SCHEMA (('IFC4'));
ENDSEC;
DATA;
#1=IFCAPPLICATION(#2,'1.2.3-alpha','BuildingSMART IDS tests','BSi-Ids-gen');
#2=IFCORGANIZATION($,'BSi',$,$,$);
#3=IFCPERSON('cbenghi@redacted.com','Benghi','Claudio',$,$,$,$,$);
#4=IFCORGANIZATION('org.buildingsmart','buildingSMART',$,$,$);
#5=IFCPERSONANDORGANIZATION(#3,#4,$);
#6=IFCOWNERHISTORY(#5,#1,.READWRITE.,.ADDED.,1735689600,$,$,1735689600);
#7=IFCWALLTYPE('1VTKQX0EX4wG0000000007',#6,'n/a',$,$,$,$,$,$,.NOTDEFINED.);
#8=IFCWALL('1VTKQX0EX4wG0000000008',#6,$,$,'X',$,$,$,.USERDEFINED.);
#9=IFCRELDEFINESBYTYPE('1VTKQX0EX4wG0000000009',#6,$,$,(#8),#7);
ENDSEC;
END-ISO-10303-21;

...which is the equivalent of the original test case: pass-overridden_predefined_types_should_pass.ifc - but with (optional) IfcOwnerHistory so the file becomes valid.

This also supports using stable Guids and timestamps so you can regenerate the IFCs and easily spot the differences in git.

The code is in xbim Essentials develop branch if you want to take a look.

[CBenghi]

Thanks @andyward.
This looks very promising, I'm gonna discuss with @atomczak soon, and see if we can use it for v.1.1