The Performance Audit Report: Recent Episodes
Yusuf Moolla
The podcast for Performance Auditors. Hosted by Conor McGarrity and Yusuf Moolla. Produced by PA Reports - Performance Audit Research.
Maura Hayes-Chaffe is the Deputy Comptroller for Audit in New York City.
Maura joined us todiscuss the Bureau of Audit, including:
- the challenges with recruiting and attracting performance auditors
- new approaches to conducting PAs
- the changing environment (e.g., economics)
- equity in PA, citizen powered audits, and a recent housing related audit
- the use of data, including open data, by the audit team.
Links
- NYC Office of the Comptroller
- This episode's web page
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Our guest is Cathy O'Neil, author of Weapons of Math Destruction: how big data increases inequality and threatens democracy.Cathy joined us todiscuss critical aspects for auditors to consider, including:
- The human element: why understanding 'for whom an algorithm might fail' is more important than the detailed technical design of an algorithm.
- Algorithm testing: the importance of post-deployment testing to ensure there are no unintended adverse outcomes.
- Setting standards: the need for industry-wide standards so that regulators can ensure there is no bias or discrimination in how algorithms operate.
- Laggers and leaders: the four big industries/domains using algorithms that need to step up audit in their space and how law-makers and regulators are catching up.
- Emerging AI, e.g. language models/chatbots - why trust by users is key, and the pros and cons of using these models.
Links
- Cathy's company, ORCAA
- Weapons of Math Destruction (book)
- This episode's web page: https://pareports.com/podcast/24
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This episode featured guests from the North Dakota State Audit Office:
Allison Bader, Performance Audit Manager and Claudia Kuball, Performance Auditor.
In this episode we discuss:
- Team diversity – the value of recruiting performance auditors from different backgrounds, incl. how specialist expertise can help focus on major risks
- Impact – their communications team is instrumental in engaging citizens and maximising audit impact
- Significant audit – their audit of Storage and Handling of COVID Vaccines including its design, execution and findings
- Citizen engagement – how North Dakotans can get involved, and even speak to the State Auditor directly on radio every Wed morning
Links:
- Office of the State Auditor, North Dakota: https://www.nd.gov/auditor
- Submit ideas for audit topics here
- What’s On Your Mind – the radio program that Josh Gallion (State Auditor) guest appears on from 10-11 CST every Wednesday: https://kfyr.iheart.com/
- Sign up for the office's quarterly e-newsletter here
- The office on Facebook, LinkedIn and YouTube
- This episode's webpage: https://pareports.com/podcast/23
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
In this episode, we discuss how audit offices are engaging citizens in performance audits.
The show notes here and the full article here contain further details and examples.
We delve into how audit offices are engaging citizens:
- Forward work plans – consulting broadly for focus areas
- Refining the broad focus areas for specific citizen concerns
- Audit planning – citizen input on audit scope
- Audit execution – seeking agency staff views during fieldwork
- Reporting – tailoring reports for citizens and civil society organisations
Links:
- This episode's web page with further details and examples - https://pareports.com/podcast/22
- The full accompanying article - https://riskinsights.com.au/engaging-citizens-in-pa/
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as episode 50 of The Assurance Show.
In this episode, we discuss the benefits of adopting a data focused approach during audit planning.
Link: This episode's web page - https://pareports.com/podcast/21
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as episode 49 of The Assurance Show.
In this episode, we discuss the importance of balance in your work program - covering each of the four objectives:
- Effectiveness
- Efficiency
- Economy
- Compliance
Link: This episode's web page - https://pareports.com/podcast/20
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as episode 48 of The Assurance Show.
In this episode, we discuss strategic audit planning (a.k.a. forward work planning).
- Understanding your community / public expectations
- Understanding your elected body / parliamentarians
- Understanding the entities within your audit office's jurisdiction
Link: This episode's web page - https://pareports.com/podcast/19
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Michael Pickup is the Auditor General of British Columbia in Canada.
In this episode we discuss:
- the changing role of performance audit
- recruitment and retention
- workplace flexibility
- people and product focused innovation
- the use of video and other media to increase reach
Links:
- Michael Pickup on LinkedIn - https://www.linkedin.com/in/michael-pickup-fcpa-fca-a1ab91168/
- Auditor General of British Columbia - https://www.bcauditor.com/
- This episode's web page - https://pareports.com/podcast/18
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as episode 39 of The Assurance Show.
In using data for audits, quality assurance of our work is important.
In this episode, we discuss:
1. The four main QA steps
- Technical QA
- Functional QA
- Business QA (validation)
-
Audit QA
-
Why we need a QA Plan
Link: This episode's web page - https://pareports.com/podcast/17
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This is an edited version of episode 13 of The Assurance Show.
In this episode we discuss the use of Open Data.
Links
- The Open Data Handbook - https://opendatahandbook.org/
- Google dataset search - https://datasetsearch.research.google.com/
- This episode's web page - https://pareports.com/podcast/16
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Patrick Barr is Director, Performance Audit at the Northern Ireland Audit Office.
In this episode we discuss:
- Recruitment challenges and staff attraction strategies
- The audit office's approach to topic selection and responsive auditing
- Upskilling performance auditors in the use of data
- Recent significant reports and their impact
- Outreach to other offices to share experiences and collaborate.
Links:
- Patrick Barr on LinkedIn - https://www.linkedin.com/in/patrick-barr-22a5233
- Northern Ireland Audit Office - https://www.niauditoffice.gov.uk/
- This episode's web page - https://pareports.com/podcast/15
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as Episode 32 of The Assurance Show.
Alli is an information design consultant and host of the Data Viz Today podcast.
We talk about a range of data visualization matters, including 3 key mistakes that people make when starting out in dataviz (and solutions to those).
You can reach out to Alli via her website - https://www.allitorban.com
Links
- Financial Times Visual Vocabulary
- Stephanie Evergreen's What, So What, Now What
- Template for all those charts done in excel
- Alli's brainstorming worksheet for developing visual metaphors
- Alli's brainstorming template for focusing the story in your data viz
- Alli's Tessellated Unit Chart for Viz for Social Good
- Viz for Social Good
- Reuter's "Drowning in Plastic"
- This episode's web page - https://pareports.com/podcast/14
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
In this episode, we talk about how audit offices monitor (and report on) the status of implementation of performance audit recommendations.
Links:
- The associated article - https://riskinsights.com.au/monitoring-the-status-of-performance-audit-recommendations
- This episode's webpage - https://pareports.com/podcast/13
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This originally aired as Episode 37 of The Assurance Show.
Morag (Senior Manager) and Gemma (Director) are Performance Auditors with Audit Scotland.
In this episode we discuss how they are using data in their audits, their experience in developing Audit Scotland's data capabilities, collaboration with other audit offices and what the future roadmap looks like.
We also discuss their focus on ensuring their reports meet their audience’s needs, and how data visualisations and charts are being used to communicate key messages.
Link: This episode's web page - https://pareports.com/podcast/12
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Kelsey Thompson is an Assistant City Auditor with the Office of the City Auditor in Austin, Texas.
In this episode we discuss:
- What Kelsey enjoys about performance audit in local government
- The role of the Office of the City Auditor
- Why performance auditors need a data ‘mindset’
- Topical reports on issues such as climate change, homelessness and policing
We also talk about Audit ATX - the Office's podcast that Kelsey hosts - and its many benefits, including:
- how auditees are listening to it as a ‘starting point’ to understand the PA process
- its use as a recruitment tool to inform potential new hires
- how it maximises the reach and impact of their audits for community members
- how podcast feedback can inform future audit topic selection and focus
Audit ATX is available on Apple Podcasts and Spotify.
In the episode, you will hear Kelsey saying that she is a supervising senior auditor - she was at the time of recording, and was promoted to Assistant City Auditor in the short window between recording this episode and publishing it. Congrats Kelsey.
Links:
- Kelsey Thompson on LinkedIn - https://www.linkedin.com/in/kelslau/
- Office of the City Auditor at the City of Austin - https://www.austintexas.gov/department/auditor
- Audit Reports - https://www.austintexas.gov/page/audit-reports
- Audit ATX - https://podcasts.apple.com/us/podcast/audit-atx/id1541056396
- This episode's webpage - https://pareports.com/podcast/11/
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This episode originally aired as Episode 30 of The Assurance Show.
John talks to us about the role of the CTO in an audit office.
He shares his views on a range of issues including:
- How he supports the audit office
- Cyber and data governance frameworks
- Evolving IT practices that auditors need to keep a close eye on
Links
- John Moore on LinkedIn (www.linkedin.com/in/john-moore-01986a7a)
- Where John works (Queensland Audit Office) (www.qao.qld.gov.au)
- Where John studied (Loughborough University) (www.lboro.ac.uk)
- The Essential Eight (strategies to mitigate cyber security incidents) (www.cyber.gov.au/acsc/view-all-content/essential-eight/essential-eight-explained)
- ISO27001 (www.iso.org/isoiec-27001-information-security.html)
- Microsoft password policy recommendations (docs.microsoft.com/en-us/microsoft-365/admin/misc/password-policy-recommendations)
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Performance audits are usually conducted as reasonable assurance engagements.
Many audit offices also provide other types of assurance or other forms of reporting.
These are often produced by performance audit functions, or are closely related to PA.
Sometimes these alternate forms of delivery can be faster and cheaper, without diminishing the intended impact.
In this episode, we briefly introduce 15 types of alternate reporting/assurance.
The accompanying blog article has further detail and examples.
Links:
- This episode's webpage - https://pareports.com/podcast/9/
- The accompanying blog article - https://riskinsights.com.au/alternate-forms-of-pa-assurance-and-reporting
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This episode originally aired as Episode 35 of The Assurance Show.
Stephen McAteer leads the data science team within Performance Audit at VAGO, the Victorian Auditor General's Office.
In this episode, we discuss:
- What VAGO does
- Stephen's approach to helping auditors use data
- Why it's important for all performance auditors to have data skills
- How VAGO has used open data to improve transparency
You can reach out to Stephen via LinkedIn (www.linkedin.com/in/smcateer).
Links
- VAGO Audit Report – Measuring and reporting on service delivery with interactive dashboard for exploring Victorian department performance (www.audit.vic.gov.au/report/measuring-and-reporting-service-delivery)
- VAGO Audit Report – Accessibility of Tram Services with interactive map on the accessibility of Melbourne’s tram network (www.audit.vic.gov.au/report/accessibility-tram-services)
- VAGO Audit Report – Council Libraries which uses Data Envelopment Analysis (DEA) to evaluate efficiency (www.audit.vic.gov.au/report/council-libraries)
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
Mike Scott is Lead Auditor and Manager Assurance and Audit at the New Zealand Department of Internal Affairs. He has over 30 years of audit experience, including 15 years in performance audit with the New Zealand Office of the Auditor-General and 17 years with the UK National Audit Office.
In this episode, we discuss:
- How the discipline of performance audit/assessment is similar, whether you're auditing performance from outside the organization or improving it from the inside
- Maximising the impact of PA
- What has, and what hasn't, changed within the world of PA
- Some ideas on recruitment and retention of audit professionals
Links:
- Mike Scott on LinkedIn
- This episode's webpage - https://pareports.com/podcast/7/
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
This episode originally aired as Episode 34 of The Assurance Show.
Taka Ariga is the inaugural Chief Data Scientist at the US Government Accountability Office. He also leads the GAO's Innovation Lab.
In this episode, we discussed:
- What the GAO does
- How data science is helping GAO
- Innovation and experimentation at GAO
- Machine learning assurance
You can reach out to Taka via the GAO website.
About this podcast
The podcast for performance auditors.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by PA Reports - Performance Audit Research.
Get updates
New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
https://pareports.com/#cta-subscribe
In this episode, we explore how performance auditors improve equity.
With further detail and examples in this new companion article:
https://riskinsights.com.au/equity-in-performance-auditing
This is an episode that aired on The Rundown [Kansas Legislative Division of Post Audit], discussing the report "Evaluating the Kansas Department of Labor’s Response to COVID-19 Unemployment Claims (Part 2)"
Mark Funkhouser is President, Funkhouser & Associates. He is one of the founders of the International Center for Performance Auditing (ICPA) and previously spent 18 years with the Office of the City Auditor of Kansas and 10 years with the Tennessee Division of State Audit. He's been the Mayor of Kansas City, had spells in academia and publishing and has written a book on performance auditing.
In this episode, we discuss:
- Why performance auditors need information, ideas and allies
- PA is global - look to colleagues, peers and consider work from outside your jurisdiction/country
- The importance of equity, engaging citizens in your work and the overall ‘user experience’
- Why it’s ‘baloney’ to think that your report will speak for itself
Links:
- Mark Funkhouser on LinkedIn - https://www.linkedin.com/in/mark-funkhouser/
- Funkhouser & Associates - https://mayorfunk.com/
-
This episode's webpage - https://pareports.com/podcast/3/
-
More links available on this episode's webpage - https://pareports.com/podcast/3/
In this episode, we discuss our key takeaways from performance audit related sessions at the 2022 NSAA conference in Atlanta, Georgia.
Links:
- NASACT
- National State Audit Association Conference 2022
- North Dakota: Office of the State Auditor
- Kansas: Legislative Division of Post Audit
- Washington: Office of the Washington State Auditor
- Georgia Cyber Centre
- 20-20 Services
- NSAA Excellence in Accountability Awards
- California State Auditor
- Utah: Office of the State Auditor
- Connecticut: Auditors of Public Accounts
- This episode's webpage
Scott Frank is the Director of Performance and IT Audit at the Washington State Auditor’s Office.
This inaugural episode was recorded at the 2022 NSAA conference in Atlanta, Georgia.
In this episode, we discuss:
- How performance auditors are using data in their work and future opportunities to use data
- Washington’s unique funding mechanism for performance audits
- Recent performance audit reports on the topics of culture, cybersecurity and voting in the State of Washington
Links:
- https://sao.wa.gov/ - Office of the Washington State Auditor
- https://www.nasact.org/2022_nsaa - National State Audit Association Conference 2022
- https://pareports.com/ - PA Reports (performance audit research)
- https://pareports.com/podcast/1/ - this episode's page