Initiative: Improve the Code Coverage of all Planet Argon Client Applications

Background

Why it Matters

Improving code coverage is not just a quantitative exercise; it directly impacts the quality and reliability of the products we build for our clients. Increased code coverage means we are thoroughly testing the code that we write, catching issues before they reach production, and creating a safety net that allows for quicker, more confident development and deployment.

Alignment with Goals

This initiative is an essential part of our annual Deployment Safeguards goal. A robust test suite that adequately covers our codebase will act as an automated safeguard, reducing the risks associated with deployments.

Developer Experience (DX)

Recent DX snapshots have indicated that low code coverage is a significant concern for developers. Improving this metric will bolster developer confidence and experience, leading to faster, more efficient development cycles.

Audits and Decisions

Regular audits of our code coverage metrics will help us understand the current state of our projects and identify areas that require attention. It will allow us to make informed decisions about where to allocate resources to maximize impact.

Tools We’ll Use

SimpleCov

This is a Ruby gem that provides a straightforward way to track code coverage statistics in your Ruby applications. It will be used primarily for auditing existing coverage and identifying areas that need improvement.

Codecov

Codecov integrates well with our continuous integration (CI) pipeline and will be used to monitor code coverage over time. It offers graphical insights into how coverage is evolving and can flag pull requests that would decrease the overall coverage.

Ideal Outcomes

1. Audit of Existing Code Coverage

   The first step in this initiative is to conduct an audit to gauge the current code coverage of all our client apps. This will act as our baseline.

2. Percentage Goals

   Based on the initial audit, we will set a reasonable yet challenging percentage goal for code and test coverage that all client apps should strive to achieve.

3. Documentation

   A comprehensive document and Jira task templates will be created to guide teams on how to install, run, and interpret test and code coverage tools. These artifacts will include steps for generating and reading reports.

4. CI Checks

   Our CI pipeline will be configured to include a check that flags if a pull request or a direct code push results in the dropping of code coverage below our set threshold.

5. Incremental Improvements

   We aim to achieve incremental improvements towards our ideal code coverage percentage with each sprint. Progress will be reviewed and discussed in ETMs.

6. Client Communication

   We’ll express the value of increased code coverage to our clients, linking it to higher software quality, reliability, and faster feature rollout, helping them understand its tangible benefits.

By working collectively on this initiative, we will not only improve the quality of our client applications but also build a stronger, more confident development team!

more resources:

The Github Code Coverage Summary Action sets the goal threshold for coverage at 75%: https://github.com/marketplace/actions/code-coverage-summary

This was discussed on 12/5/22: https://planetargon.atlassian.net/wiki/spaces/PADEV/pages/2266759169/2022-12-05+Planning+Team+Improvements+for+2023+contd THE TOPIC: “Test coverage” received a low sentiment score and hire priority score in the DX snapshot on 12/19/22. Let’s continue with our conversation on test coverage started 12/5/22 to determine appropriate amount of test coverage, when and where to add coverage to existing apps, our preferred suite of tools, and checks in workflows to catch coverage reduction and prevent merges.

HOW WILL THIS IMPROVE THE CLIENT EXPERIENCE? This will make client apps safer and more stable and will make it easier to onboard future developers.

Next steps as Subtasks below:

Test coverage Tools for checking coverage Code Climate - general “Test Coverage” rating and Files list with percentage of coverage per file simplecov - generate report everytime test suite is run; needs some config in rails_helper to only run files changed since master wm - still a big fan, tells you what has been covered and what hasn’t it’s very clear and what isn’t jt - agree, it’s like rake or bundler, it’s a given in ror ecosystem

pronto-simplecov - after running full test suite, shows lines changed not covered by tests other tools: rcov? something that shows ratio of coverage any tools to show coverage of e2e tests? cypress plugin? if we want to keep going with cypress we should probably discuss this and add it to other instances of use

Minimum coverage goal? jt - on a previous team where we used simplecov tightly coupled with prs; we shot for 80-90%; simplecov tells you what lines of code were executed by your test, not which lines of code were tested by your test; suggestion: difficult to carve out time to take coverage from 20% - 90%, but we can add a step that runs simplecov on every pr and it doesn’t allow us to merge code that has REDUCED coverage; st - this is an ideal world in which we completely understand how an app works; if we don’t know how a full application works it’s hard to talk about coverage and the value of it; the way to implement tests is to implement the most important first. ww - TDD, or behavior driven dev, you write failing tests first, and then you write the code to get your tests to pass; it ensures you have working code and working tests at the same time;

Full coverage check frequency? if we’re writing new code, test coverage should increase over time

Audit existing app test coverage Decide on company-wide tool for automating test coverage checks Decide on test coverage % floor and goal Create PA Testing Philosophy (part of EH) Host workshop on topic Assign issues to tech leads to implement coverage checks Schedule meeting/presentation on test coverage improvement trends/work

Updates

Aug 2023

Codecov/Simplecov install updates:
Applegate (assigned to BM)
Simplecov: set-up; asked for update on issue
CodeCov: 5.75 hrs logged on task; asked for update on issue
GCO (assigned to LP)
Simplecov: 0 hrs logged; asked for update on issue
CodeCov: 0 hrs logged; asked for update on issue
NEWS (assigned to BM)
SimpleCov: 1 hr logged; asked for update on issue
Codecov: 3.25 hrs logged; asked for update on issue
GNOEDU (assigned to LP)
SimpleCov: 0 hrs logged; asked for update on issue
Codecov: 0 hrs logged; asked for update on issue
GNOWOR (assigned to LP)
SimpleCov: 0 hrs logged; asked for update on issue
Codecov: 0 hrs logged; asked for update on issue
PG (assigned to WW → JC)
SimpleCov: WW set-up on deployment and main web app; re-assigned to JC to tackle inventory app
Codecov: pacglobal-app is marked as done in the issue, but I don't see a codecov job in the circleci yml config file and no github action file at all; asked for update on issue
WALK (assigned to JC)
SimpleCov: done; requested a report on the issue
CodeCov: in progress; JC working with client to get the necessary repo token for their repos
RYNO (assigned to MK)
SimpleCov: in backlog; deprioritized by client/PM
CodeCov:  in backlog; deprioritized by client/PM
Sikich (assigned to WM)
SimpleCov: done; WM ran report on Portal and updated README with instructions on how to generate report
Codecov: Not started; this would require setting up Sikich with Codecov and them paying for it, so it's low priority currently