2011 September: Audit Report from Centre for Public Management Inc.

Audit of the Complaints Resolution and Compliance Branch of the Office of the Information Commissioner of Canada

Centre for Public Management Inc.

Final Audit Report, September 6, 2011

Table of Contents

• Executive Summary.
• Entity Information.
• Office of the Information Commissioner of Canada (OIC)
• Complaints Resolution and Compliance Branch (CRC)
• About the Audit
• Audit Objectives.
• Audit Scope and Criteria.
• Approach and Methodology.
• Documentation Review.
• Interviews.
• File Review.
• Risk Assessment
• Overall Conclusion.
• Detailed Observations and Recommendations.
• Policy and Procedures.
• Training.
• Roles and Responsibilities.
• Reporting, monitoring and information for management decision making.
• Completeness and consistency of file content
• Appendix A: Audit Criteria.
• Appendix B: List of Interviewees.

List of Tables and Figures

• Table 1: Risk Ranking.
• Table 2: Audit Criteria.

Executive Summary

The Office of the Information Commissioner (OIC) is dedicated to safeguarding individuals' rights to timely, accurate, and reliable public sector information under the Access to Information Act. OIC is primarily responsible for investigating complaints from individuals and organizations about federal institutions' processing of access requests made to their Access to Information and Privacy Offices (ATIP). The Complaints Resolution and Compliance Branch within the OIC is responsible for investigating such complaints. In addition to its investigatory responsibilities, OIC staff also conducts performance reviews of federal institutions' compliance under the Act, pursues judicial enforcement, where required, and advises Parliament on access to information issues.

Maintaining government accountability and transparency requires vigilant attention to the processes related to information access. The OIC revamped its business model in 2008-2009 to improve its processing regime and bolster compliance. As a result of these service delivery improvements, the OIC substantially reduced its historical complaint inventory, closed over 2,000 cases in each of the following years, and reduced its average complaint turnaround time.

The new business model had three objectives:

1. Promote investigations' efficiency and timeliness by streamlining the complaints process, thereby permanently reducing the backlog of historical complaints.
2. Proactively address systemic issues and non-compliance.
3. Maximize compliance with the Act and disclosure of public sector information by using the full range of tools, activities and powers at the Commissioner's disposal.

To further these goals, the OIC adopted a Three-Year Plan for report cards and systemic investigations to review federal institutions' compliance with the Act. As a consequence of these improvements, the OIC has either reached or has made considerable strides toward reaching the targets outlined in recent Reports on Plans and Priorities.

The OIC is committed to building on these innovations and efficiency gains to further improve its case management processes. This commitment led the OIC to task the Centre for Public Management to conduct an Audit of the Complaints Resolution and Compliance Branch.

The objectives of this audit are:

1. To assess the business processes, performance metrics, and information to support senior management decision making in the Complaints Resolution and Compliance (CRC) Branch to ensure that they support efficient and timely case management.
2. To determine the extent to which OIC staff is adhering to legislation and policy directives in completing files.

Using criteria outlined in Appendix A, our team reviewed all pertinent documents, executed a risk-based file review, and conducted interviews with individuals from each group within CRC, including:

• Complaints Resolution ;
• Strategic Case Management;
• Early Resolution
• Intake

We also interviewed the Assistant Commissioner to gain a better understanding of risk management and management reporting to senior staff.

CPM has assembled the following key findings using multiple lines of evidence.

Policy and Procedures

• Documentation procedure standardization can improve : Complaints are highly variable. Investigative styles and approaches vary depending on the complexity, nature and priority of a particular case. Adopting a "one-size fits all" approach to investigations would limit investigator flexibility and ultimately be counterproductive. However, file documentation and case file structures can become more consistent across investigations. Specifically, recorded details related to "documented events" vary from case-to-case, and "Investigation Plans" remain optional.

Training

• CRC conducts formal and informal training : A triaging system for complaints separates simple, often administrative-related complaints from complex, often refusal-based complaints. New recruits generally start in the Intake and Early Resolution Unit (IERU) or investigate administrative cases to gain experience. Refusal cases are assigned to the Complaints Resolution Team (CRT) members based on their experience and skills. Training instruments include review of completed cases, mentorship, review of exemption grids, review of feedback process, specific guidelines for completing cases, and training on relevant sections of the Act.

Roles and Responsibilities

• Responsibilities and accountabilities of CRC staff are well defined : Interviews with CRC staff indicate all roles and responsibilities are clearly defined, understood and documented.

Reporting, Monitoring, and Information for Management Decision Making

• A process exists to track and report performance : Performance metrics for reporting to senior management include average turnaround time in days compared to similar cases from the previous year and breakdowns by six process steps. Management receives a weekly progress report detailing new and closed cases.

• External factors can abruptly affect cases under investigation : The Intake and Early Resolution Director and the Assistant Commissioner read all complaints that come into the OIC, creating dual risk assessment that is useful for flagging priority cases. There is, however, no formal environmental scanning process integrated into the current case inventory. If a legislative or court action affects a case under investigation, it might not be identified to senior management.

• InTrac can help improve strategic management decision making : The new case management system allows for ad-hoc environmental scans via more sophisticated keyword searches.

Completeness and Consistency of File Content

• Files are complete and in compliance with relevant guidelines : Despite some file structure inconsistency-a factor that affects the core compliance object of this criterion-the files sampled included standard collections of documents.

Conclusions and Recommendations

In general, the audit found that business processes, performance metrics and information to support senior management decision making exist in the Complaints Resolution and Compliance (CRC) Branch. In addition, the audit found OIC staff adheres to legislation and policy directives in completing files. Opportunities for improvement were identified in the areas of consistency and environmental scanning. Such improvements would further improve OIC's performance in meeting its objectives.

Conclusion 1: Despite improved CRC policies and procedures, opportunities exist for efficiency gains.

Recommendation 1 : Guidance in defining the level of detail that should be included in "documented events" should be provided to investigators to help ensure the depth of information included in the file is consistent. This guidance will help improve the effectiveness of quality control during management review.

Recommendation 2 : Case file structure should be consistent, with a consistent use of tabbing and ordering of documents to allow reviewers or others who may have to consult the file to do so in the most efficient manner.

Recommendation 3 : An Investigation Plan should be consistently used for refusal files, but not necessarily for standard administrative cases. Case complexity is highly variable, but investigators should include relevant case background, investigative issues and a chronology in the Investigation Report. These details should be drafted at the beginning of an investigation. These practices would serve to document important issues in the refusal cases at the front end and potentially improve case structure inconsistencies. There is a benefit to having a plan for all cases, while allowing for varying levels of detail. Doing so will allow timelines for completion to be established, and required resourcing can be anticipated more clearly. Benchmarking priority cases at six months-with an adjustment option in between to account for especially complicated cases-would improve attention to timeliness and performance measurement.

Conclusion 2: Training and information relating to CRC business processes are provided to staff at all levels in both formal, scheduled settings and on more informal, on-the-job basis. However, there is not a consistent approach to documenting training exercises across all CRC units.

Conclusion 3: Roles and responsibilities are defined, understood and documented.

Conclusion 4: There is a process established to track and report performance. This includes performance metrics, issue resolution, monitoring and compliance, continuous improvement assessments, and information for management decision making. There is opportunity for improvement in the area of environmental scanning.

Recommendation 4 : Periodic environmental scanning should be undertaken in conjunction with the key word search capabilities of the new InTrac case management system in order to identify cases that may change in priority due to changing environmental factors that did not exist at the time of intake.

Conclusion 5: Files are complete and are prepared in compliance with relevant guidelines.

Entity Information

Office of the Information Commissioner of Canada (OIC)

The Office of the Information Commissioner of Canada (OIC) is an independent body established in 1983 under the Access to Information Act. The OIC assists the Information Commissioner in her role as Agent of Parliament and Ombudsperson to, "ensure that the rights conferred to information requestors by the Access to Information Act are respected, which ultimately enhances transparency and accountability across the federal government."[1]

The OIC's complaints process is complex. Under the Access to Information Act, anyone who makes a request for information to a federal institution and is not satisfied with the response or the way the request was handled has the right to file a complaint with the Information Commissioner. As a result, the OIC is heavily influenced by external forces. For example, caseload volumes are a function of how federal institutions process information requests, legislative and judicial actions, and the relative number of complaints filed in turn by individuals or entities. Given these external forces, the OIC faces considerable challenges in controlling and forecasting its workload.

The OIC recognises that its processes, systems and controls must operate effectively and efficiently in the context of varying volumes of complaints and requests. The OIC has also set ambitious targets in many aspects of its service delivery to Canadians. A key priority is to reduce its year-end inventory to a manageable size of about 500 cases by 2013-2014 and to maintain it at that level over the long term.

External forces create tremendous variation in expected cases. In response to the goal of reducing year-end complaint inventories and increasing intake process efficiency for new complaints, the OIC has developed a new complaints intake procedure through recommendations and the supporting management action plan, documented as part of the 2009-2010 Audit of the Intake and Early Resolution Unit (IERU).

As part of the business model renewal initiatives introduced since 2008, the Information Management and Information Technology (IM/IT) Strategy provided for the transition to a new case management system for investigations. The InTrac system has now been developed and fully tested for deployment in April 2011. It is expected to contribute important benefits in terms of process standardization and reporting.

Complaints Resolution and Compliance Branch (CRC)

The Complaints Resolution and Compliance Branch (CRC) conducts investigations and dispute resolution efforts to resolve complaints related to access to information requests. The Branch consists of three units, each with the mandate to provide thorough, unbiased and private investigations as described below.[2]

1. The Intake and Early Resolution Unit (IERU) was introduced as part of the OIC's efforts to strengthen and streamline its complaints-handling process. An important goal of the OIC is to eliminate its historical backlog of cases and to improve service to Canadians. IERU carries out the initial assessment of complaints, establishes the order of priority, and prepares files for investigation. It investigates all straightforward complaints, which are usually administrative in nature and pertain to delays, extensions, fees and similar issues, as well as some refusal cases that have been earmarked for early resolution. In doing so, it tries to reach a solution that satisfies both the complainant and the institution.

2. The Complaints Resolution Team (CRT) investigates more complex refusal cases, which are generally broader in scope and more resource intensive. These cases often result from institutions applying various exemptions and exclusions under the Act, or from changes in legislative or judicial policy.

3. The Strategic Case Management Team (SCMT) was created in November 2008 to address the inventory of complaints predating April 1, 2008. This team investigates the oldest and most complex cases with a view to preventing any substantial backlog.

About the Audit

Audit Objectives

The objectives of this audit are:

1. To assess the business processes, performance metrics, and information to support senior management decision making in the Complaints Resolution and Compliance (CRC) Branch to ensure that they support efficient and timely case management.

2. To determine the extent to which OIC staff is adhering to legislation and policy directives in completing files.

Audit Scope and Criteria

Using criteria outlined in Appendix A, this audit looked at business processes, performance metrics and information to support senior management decision making in CRC to ensure that they support efficient and timely case management. The scope of the audit includes files in process or completed in fiscal year 2010-2011 and business processes, as they existed at March 31, 2011. Areas that were examined include assessing the extent to which:

• business processes are formalized and documented;
• these business processes incorporate the concept of risk;
• business process formalization and documentation facilitate retention of corporate memory;
• operational, performance and risk information is drawn from the business processes and used in management decision making and corporate reporting;
• decisions are documented (quality assurance/quality of decisions made); and
• the internal and external environment, relevant to the CRC Branch, influences decision making.

Approach and Methodology

Documentation Review

Our team performed a detailed review of documents and materials. Templates, procedures and standards, training material, former audit results, along with pertinent information regarding the CRC investigations, were reviewed and assessed.

Interviews

During the conduct phase, our team conducted interviews with individuals from each group within CRC:

• Complaints Resolution
• Strategic Case Management
• Early Resolution
• Intake

In addition to the groups mentioned above, an interview with the Assistant Commissioner was conducted to gain a better understanding of risk management and management reporting to senior staff.

These interviews assisted in clarifying processes, risks and opportunities related to the operations of CRC. These interviews identified the process of conducting an investigation and identifying potential risks with the current process. Based on the findings of these interviews, CPM was able to gain a high-level perspective of the investigative process, which ensured a strong understanding of the investigative files before conducting the review of files.

File Review

Based upon our preliminary risk assessment, which follows, a judgemental exploratory sample was used to select files for review. The objective of this file review was to identify a broad cross section of files, and validate our preliminary risk assessment. If errors were found, the sample size would be extended. Files were selected in each of the following areas:

• Strategic Case Management
• Complaints Resolution
• Early Resolution

Based on the results of the file testing (see completeness and consistency of file content) and the low risk ranking attributed to file compliance, it was deemed not necessary to extend our file testing.

Risk Assessment

Ranked in order of inherent risk (i.e. the risk that exists prior to considering the mitigating controls the OIC may have in place), this assessment was the basis for our audit criteria and program.

Table 1: Risk Ranking

Risk	Rank
There is a risk that CRC business processes do not provide optimal, consistent and timely resolution of complaints due to lack of standardization, guidance, and clear roles and responsibilities.	1
There is a risk that the current CRC business processes do not provide adequate, timely and relevant reporting to Senior Management.	2
There is a risk that inappropriate performance measures result in non-optimal practices.	3
There is a risk that the current process for initiating contact and following up with federal institutions is inefficient and delays the investigative process.	4
There is a risk that staff turnover impacts organizational momentum and the ability to resolve complaints on a timely basis.	5
There is a risk that the OIC uses a "one size fits all" assessment approach using outdated tools and timeframes resulting in an inappropriate case assessment.	6
There is a risk that case files are not completed according to relevant policies and legislation, adversely impacting the execution of the investigation.	7

Overall Conclusion

In general, the audit found that business processes, performance metrics and information to support senior management decision making exist in the Complaints Resolution and Compliance (CRC) Branch. In addition, the audit found that OIC staff adheres to legislation and policy directives in completing files. Opportunities for improvement were identified in the areas of consistency and environmental scanning. Such improvements would further improve the OIC's performance in meeting its objectives.

Detailed Observations and Recommendations

Criterion: CRC policies, procedures and business processes are clear, consistent and understood by those who need to apply them.

Policy and Procedures

Observations

The investigative process within CRC covers case files that are inherently diverse. The pace of investigations is heavily influenced by the cooperation of the individual or group making the complaint and the institution being examined. The process differs with each investigation, based on the nature of the file and the style and preference of the individual responsible for conducting the investigation.

Given the multitude of changing variables, implementing a standard approach to conducting an investigation would be counterproductive to achieving operational goals. Professional judgement based on investigative experience was observed and is crucial to the effective handling of investigative files.

Although standardization of the investigative process should not be expected given the variability of files, the way in which a file is documented should be consistent across investigators and business lines. It was observed that the level of detail concerning "documented events" varied based on the investigator. In some instances, the investigator documented internal and external communication, whereas for the most part, investigators use "documented events" to record action items of interest to the file. In addition, case file structure varied from file to file, which increases the time and effort required to perform quality reviews. Finally, the completion of the Investigation Plan is currently optional, usually prepared only for complex cases.

Conclusion

While CRC policies, procedures and business processes are generally clear and understood by those who need to apply them, there are opportunities for improvement in the area of consistency.

Recommendations

Recommendation 1:

Guidance in defining the level of detail that should be included in "documented events" should be provided to investigators to help ensure the depth of information included in the file is consistent. This guidance will help improve the effectiveness of quality control during management review.

Recommendation 2:

Case file structure should be consistent, with a consistent use of tabbing and ordering of documents to allow reviewers or others who may have to consult the file to do so in the most efficient manner.

Recommendation 3:

An Investigation Plan should be consistently used for refusal files, but not necessarily for standard administrative cases. Case complexity is highly variable, but investigators should include relevant case background, investigative issues and a chronology in the Investigation Report. These details should be drafted at the beginning of an investigation. These practices would serve to document important issues in the refusal cases at the front end and potentially improve case structure inconsistencies. There is a benefit to having a plan for all cases, while allowing for varying levels of detail. Doing so will allow timelines for completion to be established, and required resourcing can be anticipated more clearly. Benchmarking priority cases at six months-with an adjustment option in between to account for especially complicated cases-would improve attention to timeliness and performance measurement.

Criterion: Training and information relating to CRC business processes are provided to staff at all levels.

Training

Observations

Once a complaint is received, acknowledgement of the complaint is sent to the complainant, the priority of the complaint is established, and the complaint is assigned to the appropriate investigation team. Straightforward complaints, as well as some refusal cases, are directed to Early Resolution with the bulk of incoming refusal cases going to Compliance. The division of work is applied by complexity. With this in mind, when new staff is brought into CRC, they are typically assigned to the Intake and Early Resolution Unit (IERU) to gain experience. Those not assigned to IERU begin by investigating administrative cases to gain experience. As part of the training plan, investigations are strategically assigned to an investigator with the corresponding skill set and experience level.

Formal training sessions are mandatory and take place on a monthly basis on topics directly related to investigations. The OIC also runs all-staff meetings that host high-level speakers and discussions of newsworthy topics, such as national security and WikiLeaks. Individualized instruction and training material for CRT and SCMT are sparse because many of their investigators have gained field experience working at IERU or investigating administrative cases. Development activities exist for these groups, but are done informally and as needed. Weekly meetings to discuss interesting or complex cases are held and investigators receive guidance through the feedback process. As part of their orientation, IERU runs training sessions that cover all investigative elements, including process checklists and provide trainees with a guide to conducting an investigation. This unit also has the lead for the Career Development Program for PM-2 to PM-5 level employees.

Training activities for new staff are conducted as deemed appropriate by the chiefs. Various methods are used, including: review of completed cases, mentorship, review of exemption grids, the review of feedback process, specific guidelines for completing cases, and training on sections of the Act.

Conclusion

Training and information relating to CRC business processes are provided to staff at all levels in both formal, scheduled settings and on more informal, on-the-job basis. However, there is not a consistent approach to documenting training exercises across all CRC units.

Criterion: Roles and responsibilities are defined, understood and documented.

Roles and Responsibilities

Observations

The responsibilities and accountabilities of the CRC staff are outlined in delegation of authority instruments. The delegation is done on an individual basis and describes the signing authority that an individual possesses. Interviews indicated that staff is clearly aware of their roles and responsibilities.

Conclusion

Roles and responsibilities are defined, understood and documented.

Criterion: There is a process established to track and report performance. This includes performance metrics, issue resolution, monitoring and compliance, continuous improvement assessments, and information for management decision making.

Reporting, monitoring and information for management decision making

Observations

Reporting to senior management occurs on a periodic basis. It includes weekly reports of new and closed complaints as well as regular inventory reviews to group similar cases and assign them to the same investigative teams. Statistics track average turnaround, file status, breakdown of administrative and refusal complaints received, and complaints registered per month for a three-year period.

The average turnaround time for the investigative process compares the total days that a case is open based on the type of file and compares it to the previous year. The report breaks each case into six process steps:

1. Registration
2. Notification
3. Obtainment of records
4. Assignment
5. Evidence gathering
6. Management approval

The SCMT case inventory is monitored based on the year the file was received and outlines the number of files remaining from previous years. Initially, the scope of SCMT included anything older than 2008, but as the inventory has reduced and SCMT capacity is available, the scope has recently been extended to include investigations through 2010. Each group also reports based on the number of cases opened and closed, differentiated based on normal investigations and priority investigations.

From a metrics perspective, there is sufficient reporting to keep senior management informed on progress and of any issues that may arise. We also noted that a dual risk assessment occurs at the beginning of the process, with both the IERU Director and the Assistant Commissioner reviewing every complaint received. This independent review of complaints ensures that senior management is able to identify priority cases and to monitor them throughout the investigative process.

We noted that there is no formal environmental scanning process integrated with the current case inventory, such that if something changes in the external environment that impacts a case that was not otherwise flagged at intake, it is possible the case may not be identified to senior management. The new InTrac system, with more sophisticated key word searching than its predecessor, will allow regular environmental scanning and searching of the database to identify and bring these cases to the attention of senior management.

Conclusion

There is a process established to track and report performance. This includes performance metrics, issue resolution, monitoring and compliance, continuous improvement assessments, and information for management decision making. There is opportunity for improvement in the area of environmental scanning.

Recommendation

Recommendation 4:

Periodic environmental scanning should be undertaken in conjunction with the key word search capabilities of the new InTrac case management system in order to identify cases that may change in priority due to changing environmental factors that did not exist at the time of intake. Environmental scanning of issues pertinent to priority cases should be discussed at weekly management meetings.

Criterion: Files are complete and are prepared in compliance with relevant guidelines.

Completeness and consistency of file content

Observations

File testing was conducted by the audit team for each of the three investigative groups. The standard documents that were included in the file were:

• Complaint letter
• Acknowledgement letter
• Correspondence with external stakeholders
• Background research
• Pertinent documentation from the institution
• Legal Opinion (if necessary)
• Documented events
• Investigation Report (IR)
• Report of Findings (RoF)
• File closure

The file testing approach verified that all required documentation was in the file, and sign-offs were verified back to the delegation of authority instrument to ensure that proper approval was obtained. As noted in the section "Policy and Procedures," there were opportunities for improvement in file consistency. However, this does not impact the core compliance objective of this criterion.

Conclusion

Criterion: Files are complete and are prepared in compliance with relevant guidelines.

Appendices

Appendix A: Audit Criteria

Appendix B: List of Interviewees

Appendix A: Audit Criteria

Table 2: Audit Criteria

Objective 1: To assess the business processes, performance metrics and information to support senior management decision making in the Complaints Resolution and Compliance (CRC) Branch to ensure that they support efficient and timely case management.
Control Elements	Criteria
1. Policy and Programs	1.1 Policies and procedures CRC policies, procedures and business processes are clear, consistent and understood by those who need to apply them.
2. People, Accountability	1.2 Training Training and information relating to CRC business processes are provided to staff at all levels.
	1.3 Roles and responsibilities Roles and responsibilities are defined, understood and documented.
3. Results and Performance	1.4 Reporting, monitoring and information for management decision making There is a process established to track and report performance. This includes performance metrics, issue resolution, monitoring and compliance, continuous improvement assessments, and information for management decision making.
Objective 2: To determine the extent to which OIC staff are adhering to legislation and policy directives in completing files.
Control Elements	Criteria
4. Compliance	2.1 Completeness and consistency of file content Files are complete and are prepared in compliance with relevant guidelines.

Appendix B: List of Interviewees

Group	Date
Complaints Resolution	April 11, 2011
Strategic Case Management	April 11, 2011
Early Resolution	April 19, 2011
Intake	May 9, 2011
Assistant Commissioner	May 9, 2011

---