+
+
+
+
+
+
+
+
+Template resources
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+No matching items
+
+This guidance provides a set of questions to help analytical and statistical teams evaluate the quality of their analysis throughout the production cycle. The guidance is here to support teams in meeting the Office for National Statistics’s (ONS) strategic objectives for improving statistical quality. You can find more information about our strategic objectives on statistical quality in the ONS Statistical Quality Improvement Strategy. ONS manages quality through a strategic risk approach.
+The guidance has five main aims:
+The AQuA book sets out a standard framework for managing analytical quality in government. The AQuA framework is there to make sure that our work can be trusted to inform good decision making, while the Code of Practice for Statistics sets out the principles and practices that producers of official statistics should commit to. Two other pieces of guidance have motivated us to produce this template. One is the Analysis Function guidance on Quality Questions and Red Flags. The other is the Office for Statistics Regulation (OSR) guidance on Thinking about quality when producing statistics. Both of these provide sets of questions that analysts can use to interrogate their work and assure its quality.
+Building on these cross-government resources, this guidance sets out quality questions that are relevant for each stage of analytical cycle. The quality questions are at their most effective if they are asked at the right stage. Once that stage is passed, experience suggests that it is normally difficult to go back and address the points the questions ask through retrofitting at a later stage of the analysis.
+Each question is linked with the Code of Practice for Statistics pillars of Trustworthiness, Quality and Value. We explain the importance and relevance of each question in light of the three pillars so teams can better understand and apply these principles through out the project life cycle.
+Quality questions are also categorised by which responsible role from the AQuA book would usually answer them. The idea is to highlight the clear line of accountability set out in the Aqua book in an easy-to-understand manner. We want to make it easier for teams to decide how the three key Aqua roles of commissioner, senior responsible owner (and their analysis team) and analytical assurer are covered in their own workflows.
+There are 60 quality questions in total. They cover all the stages of the analytical cycle.
+Answering 60 questions may seem a daunting task at first. The questions are designed to help you as you work your way through the analytical process, rather than to be answered all at once. Creating a log of answers as you move through the stages of the analysis workflow will help you to check that your work meets analytical standards, follows good practice and (if relevant) complies with the Code of Practice for Statistics.
+Answering the questions will also help you make sure that everybody working on the analysis has a clear understanding of how and why it works as it does, and to support your users when writing your outputs. Moreover, most of the answers will help you to produce the critical documents that mitigate risk like an assumptions log, decisions log, issues log, and technical guides for your team and your users.
+The Aqua book divides the stages of the analytical cycle into:
+The stages are covered by the following tabs on this website:
+This tab lists the quality questions that are relevant for each stage of analytical cycle. It provides space in the form of editable text boxes (with no word limit) to write the answers. We encourage commissioner, analyst team and analytical assurer to answer all the questions given their responsibilities.
+User needs or project aims often change over time, so the template allows you to edit answers by simply changing the text in the editable text boxes. To track these changes, the person writing the answer should include their full-name, the date they have added the answer, and the date the next review is due.
+The Senior responsible owner (SRO) is the lead analyst responsible for the workflow (or relevant part of it). The SRO needs to sign-off all the answers. It is best to complete and save the log of answers for each stage of analytical cycle. Answers can be saved by pressing Ctrl+S. To undo a action press Ctrl+Z. Pressing Ctrl+S will automatically download a copy of the HTML page on your machine.
+This tab explains why understanding each quality question matters and explains the potential risks and benefits around it. It tries to address the valid question of “why should I care about this?”.
+This tab shows the pillars and principles of Code of Practice for Statistics each quality question relates to. The text in this column is copied from Code of Practice for Statistics. Even if your work does not directly feed into the production of official statistics, compliance with the principles and practices of the Code is a good way to strengthen the resilience of your work, increase transparency and clarity and reduce risk.
+This tab explains the responsibilities required to deliver analysis that is fit-for-purpose. The roles and responsibilities in this tab are reproduced from AQuA book. The tab links the quality questions with the relevant AQuA Book responsibilities at each stage of analytical cycle.
+The AQuA book recommends the creation of four identifiable roles to cover different areas of responsibility: * Commissioner * Senior Responsible Owner * Analyst * Analytical assurer
+Responsibilities of the Senior Responsible Owner
+The Senior Responsible Owner (SRO) is accountable for the analytical workflow throughout its lifecycle. The Senior Responsible Owner is usually a senior member of the analytical team, and works closely with (or manages) the Analyst role. There is no minimum grade for the SRO role, but they should have the expertise, resources and accountability to ensure that the analysis is well designed, complies with relevant standards, works as intended and is fit for purpose. - Signs off all important decisions made about the analysis to ensure that it is fit-for-purpose, prior to use.
Responsibilities of Commissioner The Commissioner
+- Ensures that context is understood so quality assurance is appropriate and proportionate.
+- Ensures that there is enough time and resource for required assurance and account for risk.
+- Must understand strengths and limitations including uncertainty so results are interpreted correctly.
+- Delivers QUALITY OF OUTCOME (“The analysis meets user needs and we understand its limitations”).
Responsibilities of Analyst
+- Assist the Commissioner in framing the question to ensure the right analysis is done.
+- Manage external specialists.
+- Design, build, document and run the analysis.
+- Delivers QUALITY OF CONTENT (“The analysis is well-designed and users the right tools and methods to meet user needs”).
Responsibilities of Analytical Assurer
+- Ensures that the evidence is there to demonstrate that appropriate quality assurance has happened and that uncertainty is communicated appropriately.
+- Advises the Commissioner on whether appropriate QA has happened and about any outstanding risks.
+- Involved throughout from design through to use.
+- Delivers QUALITY OF PROCESS (“The analysis does what it is supposed to do and we can prove it”).
| - | Quality Questions | +Quality Question | Why do I need to know the answer to this? |
|---|
| - | Quality Questions | +Quality Question | Which pillar and principle of Code of Practice are relevant here? *Trustworthiness (T), Quality (Q), Value (V) | @@ -871,9 +871,9 @@ table, th, td {
|---|
| - | Quality Questions | -Who is responsible for answering this in light of AQUA book? | -Which AQUA book responsibilities have a role? | +Quality Question | +Which AQuA book role(s) would normally answer this? | +Why are these AQuA roles involved here? |
|---|---|---|---|---|---|---|