Testing

An outline of testing and what test instances to use are provided here.

Testing Environment

Tests for this package are run primarily through pytest, and this is sufficient for unit tests. However, integration testing is very important for testing the interaction between an online repository and NekUpload. This requires a suitable test instance of the online repository to test interactions.

InvenioRDM Demo Environment

This is currently the test environment used to test interactions with the online repository. Currently, https://inveniordm.web.cern.ch hosts InvenioRDM V12, and this is used for integration testing.

AE Datastore Dev Environment

AE Datastore is built upon InvenioRDM V12, so basic functionality is already tested in the InvenioRDM Demo instance. However, as AE Datastore continues to develop, some features will only be present on AE Datastore. A dedicated testing environment of AE Datastore will be hosted at https://data-dev.ae.ic.ac.uk to facilitate testing.

Nektar Repository

The GitLab API for the Nektar++ repository will also be utilised during tests. This involves checking whether NekUpload can correctly identify whether a specified GitSHA is present in the public repository.

CI/CD

Automated testing is achieved on pushing to the repository on GitLab. Unit tests will be run first, followed by integration tests if unit tests succeed. This prevents unnecessary use of external resources during tests. Currently tests are performed on: Debian Python versions 3.11,3.12,3.13; MacOs Python versions 3.11,3.12. No Windows tests are triggered in the online CI/CD at present.

Testing Guidelines

All tests should be marked with a tag to allow identification of tests. Tests should either be marked with decorators as either unit or integration tests.

Note

Tests that are not marked will not be run in automated testing.

@pytest.mark.unit #marks as unit
def test_unit():
    ...

@pytest.mark.integration #marks as integration
def test_integration():
    ...

You may choose to run unit and/or integration tests with

pytest #runs every test, marked and unmarked
pytest -m unit #just unit
pytest -m integration #just integration
pytest -m "unit and integration" #both
pytest -m "unit and not integration" #only unit, no integration

NekUpload.testutils

NekUpload contains a set of test utilities to streamline the management of tests. Integration tests of NekUpload involve testing whether certain Nektar++ datasets are correctly validated. This requires use of real datasets. Under tests/datasets contains a large number of folders, each with the name of a particular solver, such as ADRSolver. Within each group, there is a file register_dataset.py. You can add your new ADRSolver datasets here, and add them to the testing suite through the use of decorators provided by testutils:

#tests/datasets/ADRSolver/register_dataset.py

from NekUpload.testutils import NekTestDataset,NekTestGeometryComposite,NekTestSessionExpansion
import NekUpload.testutils
from NekUpload.NekData.data_type import SolverType,Elements

@NekUpload.testutils.dataset #adds this to the testing suite
def ADR_3D_stacked_cylinders_curved():
return NekTestDataset(
    solver_type=SolverType.ADR_SOLVER,
    session=abs_path("ADR_3D_stacked_cylinders_curved.xml"),
    geometry=abs_path("ADR_3D_stacked_cylinders_curved.nekg"),
    output=abs_path("ADR_3D_stacked_cylinders_curved.fld"),
    checkpoints=[abs_path("ADR_3D_stacked_cylinders_curved_0.chk")],
    filters=[],
    boundary_conditions=[],
)

Your test is now added to the testing suite and will be run wherever all datasets are used in the integration tests (in particular tests/test_manager.py). To ensure correct functionality, the register_dataset.py file must be added to tests/conf.py as imports:

#tests/conf.py

#trigger registering of datasets
import datasets.AcousticSolver.register_datasets
import datasets.ADRSolver.register_datasets #this one imports the register_datasets.py from above
import datasets.CompressibleFlow.register_datasets
import datasets.IncNavierStokes.register_datasets
import datasets.ShallowWaterSolver.register_datasets

When pytest is called, the datasets are all registered on import time (therefore, it important to import them in test/conf.py). Then pytest will collect all tests, and then run them.

NekUpload.testutils also provides other test decorators to allow datasets to be used to test other aspects of NekUpload. One decorator, @NekUpload.testutils.geometry_composite_info (see below), is used to test whether NekUpload can correctly extract composite information from the geometry file. Meanwhile, @NekUpload.testutils.session_expansion_info (see below), is used to test whether expansion elements are correctly extraced from session files and processed accordingly.

@NekUpload.testutils.dataset
@NekUpload.testutils.geometry_composite_info(
    geom_info=NekTestGeometryComposite(
        composite_id_to_definition_map= {0:"Q[22-47]", 1:"T[0-21]", 2:"E[0-1]", 3:"E[2-5]",
            4:"E[45]", 5:"E[56]", 6:"E[67]", 7:"E[79]", 8:"E[88]",9:"E[94-95]",
            10:"E[84,75,69,62,51,40,30,20,6]"
        },
        dim=2,
        description="Only consists of 2D elements"
    )
)
@NekUpload.testutils.session_expansion_info(
    exp_info=NekTestSessionExpansion(
        composite_id_to_expansion_definitions_map={
            0: [_EXPANSION_FACTORY.get_expansion(Elements.QUAD,6,("u","v"))],
            1: [_EXPANSION_FACTORY.get_expansion(Elements.TRI,7,("u","v"))],
        },
        description="Basic test of MODIFIED expansions"
    )
)
def ADR_2D_TriQuad():
    return NekTestDataset(
        solver_type=SolverType.ADR_SOLVER,
        session=abs_path("ADR_2D_TriQuad.xml"),
        geometry=abs_path("ADR_2D_TriQuad.nekg"),
        output=abs_path("ADR_2D_TriQuad.fld"),
        checkpoints=[abs_path("ADR_2D_TriQuad_0.chk")],
        filters=[],
        boundary_conditions=[]
    )

Note

Why is this even necessary? Mocking test files would be the obvious setup, but this requires the entire file to be generated, a tiresome process when you only want to test a small portion of the extraction and parsing process. This method is brittle and can break in future changes to NekUpload.

Therefore, real Nektar++ datasets should be used. In fact, tests on real Nektar++ datasets revealed undocumented behaviour in Nektar++, which has been submitted to the Nektar++ maintainers for evaluation.