Semanticizing Key Digital Functionalities, Functional Requirements, and their testing artifacts

[basicavisual]

For some time @nicolk and myself have been working on revamping the testing experience on GovStack and so we have been working on making Machine-readable-specs which we are piloting here were you can see the Wallet one. While understanding how to streamline tests and have them been automated we noticed Key Digital Functionalities looked redundant to requirements. We were also concerned about how to clarify the CFR section to specifications.

On our current testing experience, we ask users to check yes/no to whether they comply with functionalities, which do not have requirement levels, and then we ask them about requirements. The compliance is however graded on complying to required requirements, so on the current experience KDFs seem redundant. However, in trying to make sense of what Functionalities should be, we noticed on Wallet BB there is a direct relation between KDFs, FR and Service API endpoints, where KDFs are containers of requirements (one functionality has one or many requirements) and each requirement can be fulfilled either by complying its corresponding operation on the Service API section or some other way.

To redefine the testing experience, I propose changes that can be explained by the following diagram:

erDiagram
    GOVSTACK_BB_SPECIFICATION ||--|{ KEY_DIGITAL_FUNCTIONALITY : "has one or more"
    KEY_DIGITAL_FUNCTIONALITY ||--o{ REQUIREMENT : "has one or more"
    REQUIREMENT ||--|{ TEST_CASE : "proved by"
    CORE_CROSS_FUNCTIONAL_REQUIREMENTS ||--|{ TEST_CASE : "proved by"
    TEST_CASE ||--|{ REST_API_TEST : "can be fulfilled by"
    TEST_CASE ||--|{ FUNCTIONAL_TEST : "can be fulfilled by"
    TEST_CASE ||--|{ AUDITABLE_TEST : "can be fulfilled by"
    GOVSTACK_BB_SPECIFICATION {
        string URN "urn:govstack:bb:wallet:spec:1.0"
    }
    KEY_DIGITAL_FUNCTIONALITY {
        int    id
        string urn "urn:govstack:bb:wallet:kdf:1/1.0"
        string name
        string description
    }
    REQUIREMENT {
        int    id
        string urn "urn:govstack:bb:wallet:req:1;1/1.0"
        string name
        string description
        string level
        string mutability
        string verificability
    }
    TEST_CASE {
        int id
        int requirement_id
        string requirement_url
    }
    FUNCTIONAL_TEST {
        int     id
        string  URI
        int     functional_requirement_id
    }
    REST_API_TEST {
        int    id
        string URI
        string open_api_url
        string open_api_operation_id
        int    functional_requirement_id
    }
    AUDITABLE_TEST {
        int    id
        string url
        int    cfr_uri
    }
    CORE_CROSS_FUNCTIONAL_REQUIREMENTS {
        int    id
        string urn "urn:govstack:cfr:deployment:1/2.0"
        string name
        string description
        string level
        string mutability
        string verificability
    }
    

Loading

To explain the kinds of tests that can fulfill a test case I am proposing 3 types:

• Web API tests, are tests that are relevant to platform interoperability. So these pertain to requirements that are related to web services. We typically represent them on specifications using HTTP Rest endpoints via an OpenAPI file, however other web protocols can be used as long as operations (verb, endpoint, headers or payload) can be transpiled. API tests are the bases for adaptor building, and a software solution shall submit API tests Test cases to this type of requirements fulfill GovStack's promise to build an interoperable ecosystem.
• Functional tests describe any test that is not proven by a web service operation but are directly tied to the business requirements of a solution. In the case of wallet, some examples can be found on section 5, such as Data Minimisation or 5.1.4 Consent. These types of tests are typically crafted by software projects during their development phase if they implement Test-Driven Development, so software solutions are probably already prepared to submit a test of this nature by "translating" an existing test from their repository. Functional test may also cover UI related features (as most testing suites do). Functional tests that are not Web API related fulfill GovStack's promise of certifying compliant product will fulfill a minimum set of functions that we recognize as valuable to perform a function. In this sense, Web API tests are also a type of functional test.
• Audit tests describe aspects that may not be machine-encodeable either because they pertain to implementation aspects, or because their fulfillment cannot be discretely encoded without sacrificing quality observance. Audit tests are declarative in nature, meaning that a statement is presented in the test case, and the party that is submitting the test can provide an open ended response. An example of this is found in cfr-development#req-2: EOL SHOULD be at Least 5 Years. This requirement is non-verifiable or poorly verifiable for a software solution project, but it is very relevant for a procurement contract.

All cases can be described using the cucumber suite, which would make us compatible with existing test suites of software solutions and our own testing suites (i.e. the ITB)

I still think there is space for improvement upon this proposal but hopefully it provides a clearer model to testing specifications. In summary this issue proposes the following changes:

1. That requirements from a specification are organized by Key Digital Functionalities
2. That requirement levels, their mutability and their verificability are used to guide test cases
3. That test cases are provided with each spec and each test type shall clearly enable different facets of the GovStack initiative value proposition: ecosystem interoperability by API testing, functional interoperability via API testing, and project sustainability via audit testing