.. _validation-overview:

Validation overview

===================

.. note::

    This page gives an overview of ConfML and ImplML validation with ConE.

    For the actual CLI action see :ref:`cli-action-validate`

ConE supports validation of the ConfML and ImplML files in configurations

projects. This basically means that you can pass a configuration for validation

and ConE will spit out a list for *problems*, which can be errors, warnings or

informational messages associated with a file and a line number.

For example, a list of validation problems might look something like the

following:

=================================== ======  ======================================  =========== ===========================================================================

File                                Line    Type                                    Severity    Message

=================================== ======  ======================================  =========== ===========================================================================

Layer1/confml/broken.confml         1       xml.confml                              error       no element found: line 1, column 0

Layer1/confml/test.confml           11      model.confml.invalid_value.maxlength    error       Setting ValidationTest.Foo: Maximum number of characters is 3 (value has 6)

Layer1/confml/test.confml           12      model.confml.missing_feature_for_data   error       Feature 'ValidationTest.Bar' not found

Layer1/implml/00000002_foo.crml     6       model.implml.crml.invalid_ref           error       Setting 'Foo.Bar' not found in configuration

Layer1/implml/00000003_bar.crml     10      model.implml.crml.duplicate_uid         error       Duplicate key UID 0x00000001 (duplicate with keys on lines 6, 7 and 8)

Layer1/root.confml                  7       schema.confml                           error       Element 'foo': This element is not expected.

=================================== ======  ======================================  =========== ===========================================================================

From a slightly lower-level perspective validation happens on three levels:

#. XML level - Files that contain invalid XML data are caught here

#. XML Schema level - Things like missing attributes in elements are caught here

#. Model level - The rest of possible problems that the other validation levels

   cannot handle are caught here

Problem types and filtering

---------------------------

You might have noticed the *type* column in the example above. This is a

hierarchical problem type ID that can be used to filter a list of problems to

narrow it down to only problems of interest. *Hierarchical* here means that

the parts separated by commas are basically sub-IDs, the highest level sub-ID 

being the leftmost one. For example, suppose that we have a problem whose type

is ``model.implml.crml.invalid_ref``. Reading the sub-IDs from left to right

we can see that:

#. It is a problem caught during model-level validation

#. It is an ImplML problem

#. The specific ImplML in this case is CRML

#. The problem is that a CRML key references a ConfML setting that does not exist

Filtering of problems by type happens by specifying a list of *includes* and

a list of *excludes*. The output will contain only problems that match any of

the includes, but do not match any of the excludes.

If we wanted to set up a filter that shows only problems of the type in the 

example above, we could simply use a single include: ``model.implml.crml.invalid_ref``.

However, if we wanted to see all model-level CRML problems *except* that one,

we could include ``model.implml.crml`` and exclude ``model.implml.crml.invalid_ref``.

Now invalid reference errors would not show up, but duplicate UID errors

(type ``model.implml.crml.duplicate_uid``) would. Wildcards are also possible,

so including ``*.confml`` would include all ConfML problems: XML-level,

schema-level and model-level, or ``model.implml.*.invalid_ref`` would include

all ImplML errors for references to non-existent settings.

The two first sub-IDs for problem types come from the three validation

levels and the fact that ConfML and ImplML can be validated. The following

table shows the problem types for all cases:

+-------------------+--------------------+-------------------+

|                   | **ConfML**         | **ImplML**        |

+-------------------+--------------------+-------------------+

| **XML-level**     | ``xml.confml``     | ``xml.implml``    |

+-------------------+--------------------+-------------------+

| **Schema-level**  | ``schema.confml``  | ``schema.implml`` |

+-------------------+--------------------+-------------------+

| **Model-level**   | ``model.confml``   | ``model.implml``  |

+-------------------+--------------------+-------------------+

The sub-IDs after that depend on the context. For example, ImplML validation

typically has the implementation language as the next sub-ID: ``model.implml.crml``

or ``schema.implml.genconfml``.