Reporting Process¶
The RABET-V administrator creates a report for the RTP containing scores from the architecture assessment, organizational assessment, and product verification, a verification status, and recommendations for improvement. The administrator will send the RTP two versions of this report: a full report with detailed appendices and a roadmap for ways to improve and a short report verifying that the baseline requirements were met. Election officials can request the short report during procurement processes, as part of contract management, or during annual security reviews.
The RABET-V public listing site contains a list of verified products containing the tech provider name, the product version, some configuration details, and verified status. RTPs will have the option to opt out of publicly listing their product if they choose.
Inputs¶
Results from all relevant activities
Outputs¶
RABET-V product report and appendices
Status of verified or not verified
Workflow¶
Since each RABET-V activity generates artifacts with product or organization recommendations and scores, the RABET-V Administrators create a summary report of all the findings and assign the product a Verified or Not Verified status. The report can also be in one of three states: Draft, Conditional, and Final. The combination of the product status and the state of the report determines the overall RABET-V assessment status.
After the RABET-V Administrators finish a report, its status is Draft. The report is then shared with the RTP to allow the RTP to raise potential content disagreements. The RTP has ten days to raise any issues with the report. If the RTP disputes any findings or communicates that it plans to resubmit the product for testing in the current iteration, its status changes to Conditional. The RTP then has 30 days to resubmit the product. If an RTP submits a product for retesting in the current iteration—and the RABET-V Administrators accept it as a part of the current iteration—then the report generated from the newly tested product changes back to Draft, and the RABET-V cycle starts one additional time. If an RTP accepts the report as-is or the acceptance or resubmission period lapses, the report’s status changes to Final. When the report enters this stage, the report is signed by CIS and sent to the RTP.
If the product itself is given a Not Verified status, the RTP can assess the changes required to remediate the deviations. If they re-submit within the allowed time, they may be able to incorporate it into the same submission. This is at the RABET-V Administrator’s discretion. A cursory architecture analysis can evaluate the submission delta and consider whether it is scoped only to address the deviations. If it is appropriately scoped, it may fall under this provision. Otherwise, it must be regarded as a separate iteration.
Once the remediation submission is evaluated and through product verification, the verification outputs will be assessed to determine if the deviations have been resolved. If so, the product may move to Verified. Otherwise, the product is Not Verified, and the iteration is ended.
Report Statuses¶
- Draft
After the RABET-V Administrators finish an assessment report, its status is Draft. The report is then shared with the RTP to discuss potential content issues. The RTP has 10 days to communicate any disagreement with the findings or indicate that they will submit a product for the current iteration. At that point, the Draft status changes to Conditional. If the RTP signals that they accept the report or the 10-day period lapses with no communication from the RTP, the report status changes to Final, and the assessment is complete.
- Conditional
If the RTP disputes any findings or communicates that it plans to resubmit the product for testing in the current iteration, its status changes to Conditional. The RTP must suggest changes or resubmit the product within 30 days. If the RTP provides changes to the report, the RABET-V Administrators adjudicate the changes within 10 days, and the report will be changed to Final. If the RTP submits a product for testing within the current iteration, the reassessment results are folded into the report, and its status is reverted to Draft one final time. If the 30-day period lapses without the RTP resubmitting the product or the RTP indicates that they accept the report in its current form, the report status changes to Final, and the assessment is complete.
- Final
If an RTP accepts the report or the acceptance or resubmission period lapses, its status changes to Final.
RABET-V Product Statuses¶
- Not Verified
Not Verified means:
While the product is likely to perform as described, the RABET-V process identified at least one significant deficiency.
The RTP is expected to remediate the deficiency(s) and re-submit within 30 days.
Suppose no other changes are made to the product. In that case, the re-submission is considered part of the same submission and, upon review, can change the RABET-V product status to Verified. Resubmission may still undergo an expedited RABET-V process. For example, if organizational processes meet or exceed the baseline and remain the same, no organizational assessment needs to be performed. Additionally, depending on the change, the product may be able to undergo an expedited architectural assessment and product verification.
- Verified
A Verified status means the product will likely perform as described in the expected usage operating environment. To achieve a verified status, the organizational assessment, architecture assessment, and product verification results must meet or exceed the baseline in each area.
Product Report Generation¶
Report Template¶
The RABET-V results summary provides scores for organizational maturity, architecture maturity, and product implementation. For revision submissions, it will include any change from the previous submission.
Organizational maturity: quality of the RTP’s product development practices. The organizational assessment maturity result reflects the extent to which this is achieved for each of these areas:
Governance
Design
Implementation
Verification
Operations
Human factors
Architecture maturity: the reliability of the product’s such that changes to one product feature or service will not have unintended implications for other aspects of the product. The architecture assessment maturity result reflects the extent to which this is achieved for each of the control families.
Product implementation maturity: the quality of the product’s capabilities to meet the claims the RTP made about it. The product verification result reflects the extent to which this is achieved for each of the control families.
Product (Revision) Summary: details about the product that were submitted including its description, expected usage (i.e., use cases), version number(s), etc. This includes the change list for product revision submissions.
Verification Methods: a description of how the system was tested to include verification methods used in the testing.
Maturity Trends: a description of what caused a change for any product or process maturity level that changed.
Appendices: as needed.