Merge changes to system model generator to SF tip.
.. _validation-overview:
Validation overview
===================
.. note::
This page gives an overview of ConfML and ImplML validation with ConE.
For the actual CLI action see :ref:`cli-action-validate`
ConE supports validation of the ConfML and ImplML files in configurations
projects. This basically means that you can pass a configuration for validation
and ConE will spit out a list for *problems*, which can be errors, warnings or
informational messages associated with a file and a line number.
For example, a list of validation problems might look something like the
following:
=================================== ====== ====================================== =========== ===========================================================================
File Line Type Severity Message
=================================== ====== ====================================== =========== ===========================================================================
Layer1/confml/broken.confml 1 xml.confml error no element found: line 1, column 0
Layer1/confml/test.confml 11 model.confml.invalid_value.maxlength error Setting ValidationTest.Foo: Maximum number of characters is 3 (value has 6)
Layer1/confml/test.confml 12 model.confml.missing_feature_for_data error Feature 'ValidationTest.Bar' not found
Layer1/implml/00000002_foo.crml 6 model.implml.crml.invalid_ref error Setting 'Foo.Bar' not found in configuration
Layer1/implml/00000003_bar.crml 10 model.implml.crml.duplicate_uid error Duplicate key UID 0x00000001 (duplicate with keys on lines 6, 7 and 8)
Layer1/root.confml 7 schema.confml error Element 'foo': This element is not expected.
=================================== ====== ====================================== =========== ===========================================================================
From a slightly lower-level perspective validation happens on three levels:
#. XML level - Files that contain invalid XML data are caught here
#. XML Schema level - Things like missing attributes in elements are caught here
#. Model level - The rest of possible problems that the other validation levels
cannot handle are caught here
Problem types and filtering
---------------------------
You might have noticed the *type* column in the example above. This is a
hierarchical problem type ID that can be used to filter a list of problems to
narrow it down to only problems of interest. *Hierarchical* here means that
the parts separated by commas are basically sub-IDs, the highest level sub-ID
being the leftmost one. For example, suppose that we have a problem whose type
is ``model.implml.crml.invalid_ref``. Reading the sub-IDs from left to right
we can see that:
#. It is a problem caught during model-level validation
#. It is an ImplML problem
#. The specific ImplML in this case is CRML
#. The problem is that a CRML key references a ConfML setting that does not exist
Filtering of problems by type happens by specifying a list of *includes* and
a list of *excludes*. The output will contain only problems that match any of
the includes, but do not match any of the excludes.
If we wanted to set up a filter that shows only problems of the type in the
example above, we could simply use a single include: ``model.implml.crml.invalid_ref``.
However, if we wanted to see all model-level CRML problems *except* that one,
we could include ``model.implml.crml`` and exclude ``model.implml.crml.invalid_ref``.
Now invalid reference errors would not show up, but duplicate UID errors
(type ``model.implml.crml.duplicate_uid``) would. Wildcards are also possible,
so including ``*.confml`` would include all ConfML problems: XML-level,
schema-level and model-level, or ``model.implml.*.invalid_ref`` would include
all ImplML errors for references to non-existent settings.
The two first sub-IDs for problem types come from the three validation
levels and the fact that ConfML and ImplML can be validated. The following
table shows the problem types for all cases:
+-------------------+--------------------+-------------------+
| | **ConfML** | **ImplML** |
+-------------------+--------------------+-------------------+
| **XML-level** | ``xml.confml`` | ``xml.implml`` |
+-------------------+--------------------+-------------------+
| **Schema-level** | ``schema.confml`` | ``schema.implml`` |
+-------------------+--------------------+-------------------+
| **Model-level** | ``model.confml`` | ``model.implml`` |
+-------------------+--------------------+-------------------+
The sub-IDs after that depend on the context. For example, ImplML validation
typically has the implementation language as the next sub-ID: ``model.implml.crml``
or ``schema.implml.genconfml``.