User Tools

Site Tools


wiki:projects:nz-covid-tracing-app

Testing the NZ Covid Tracing App

Lets try and test what we can about the NZ Covid Tracing App and document and record our progress here.

Bugs & Issues Found

With device: iPhone 6+ OS 12.4.6
  1. Unite your friends and whanau (ie. sharing the app with them) the copy option did not work when I first started with the app, it now seems to work more as I would expect
  2. email verification code entry screen has 6 boxes for entry of 6 digits. I tried to copy & paste the number from email into them but it pasted all 6 digits into the first box
  3. On airplane mode, screens for certain menu options under My Profile appear blank.
With device: iPhone 8 iOS 13.3.1
  1. App crashes at launch when attempting to scan QR code instore without wifi and data enabled (needs to be confirmed by another tester)
Device: Oppo F11 v9 & Samsung S9 android v10

Points of Interest: on entering a 10 digit password containing Uppercase, Lowercase and number was not accepted (NO error was displayed) at this stage until another character in this case (special character) was added. Now I had used the same 10 digit password before for another account and had worked in the past. -giving me the impression that the same password can't be used by any other user, this assumption can be completely wrong but atm I can't think of any other reason for this phenomenon. I navigated out of tracing app to the Gmail app to get the code, then came back to tracing app > Login with credentials > entered the code > error wrong code>It looks like that the app sends a new code when I logged in, causing the 1 I just got to be invalid - which as a user is annoying because I didn't click the 'Resend' button. In my case after the 3rd failed attempt of entering the invalid code, My login credentials won't let me in - Because it gave me a generic error, I can't be sure if this was really the case.(Video for this issue sent to Vincent)

Accessibility Bugs

All devices
  1. [High Severity] There is no auditory or haptic feedback after successfully scanning a QR Code for visually impaired users
With device: iPhone 8 iOS 13.3.1
  1. [High Severity] Users that require screen readers do not have any alternate means to check in their location.
  2. Using VoiceOver, on Privacy Policy screen, Close button is announced as ‘button’. Non sighted screen reader users would not know the purpose of this button if the button name is not announced. (also occurs on several other screens such as How it works, My details)
  3. Using VoiceOver, on Create your Account screen, error messages are not announced on display. Non sighted screen reader users won’t be aware of the error messages until they navigate to them (also occurs on several other screens such as My Details)
  4. Using VoiceOver, on Create your Account screen, text input field labels are not announced when the input fields receive focus. This may create ambiguity for non sighted screen reader users.
  5. Using VoiceOver, on Create your Account screen, password requirements located at the bottom of the screen are not announced when the Password field is focused. Screen reader users will not be aware of the requirements until they navigate to them.
  6. Using VoiceOver, on Confirm your email is correct screen, notifications displayed when 'Resend Code' is activated ('resending code', 'code sent') are not announced. Screen reader users will not get feedback on whether the code has been sent.
  7. Using VoiceOver, on My details screen, visually hidden content is focused and announced e.g. content under 'Why ask for my date of birth?'. This causes unnecessary announcements for screen reader users and requires additional swipes to navigate the screen. (also occurs on My address screen - visually hidden auto suggestions for Find address field are focused and announced)
  8. Using VoiceOver, on My address screen, Find address auto suggestions are not announced on display. Screen reader users will not be aware of the auto suggestions.
  9. Using VoiceOver, on My Profile, button/link role for My Profile options (e.g. Login & security) is not announced. Screen reader users may not realise that the options are interactive elements.
  10. On Create your Account screen, ‘Required’ grey text on white background fails colour contrast ratio requirements (contrast is < 4.5:1). This may affect low vision users such as the elderly. (also occurs for placeholder text on My details screen).
  11. On Dashboard, 'Thanks for registering' white text on green background fails colour contrast ratio requirements (contrast is < 4.5:1). This may affect low vision users. In addition, non text content on the Dashboard - white tick on green background contrast is < 3:1).

Accessibility is also referred to as a11y

Website Claims Bugs

  1. Only organisations with a New Zealand Business Number (NZBN) can obtain a QR code. However, organisations such as meetup groups, (eg. Ministry of Testing, and even events such as funerals and tangi should also be able to obtain a QR code

Resources

Test Ideas

Please add any new ideas you can think of
  • Install, register & first time usage
  • Deeper testing of the email verification code entry screen
  • Scan a QR code at a location
  • Scan other QR codes
  • Find specs for QR Code for this app
  • Capture & explore the API calls made by the app
  • Explore app after installation & registration
  • Uninstall & re-install the app
  • Attempt to test the API with a test tool (eg. Postman)
  • App performance of regular use cases - how fast can you scan a QR code at location
  • Try a range of devices
  • Explore security
  • Explore privacy
  • Explore accessibility
  • Ask for access to a test system
  • Can we see what happens when someone tests positive and their contacts need to be traced
  • Can we see what happens when someone has been in contact with a positive case and need to be contacted
  • How responsive are they to feedback/suggestions/bug reports
  • 'Android devices need to have Android 7.0 or later while Apple devices need iOS 12 or later' ⇒ how many people might have devices outside this range? Are there other OS's that people use? eg Windows Mobile OS
  • 'NZ COVID Tracer is currently available in English only. Future updates will include support for te reo Māori, Chinese and Pacific languages' ⇒ let's look out for these updates
  • Testing the claims published on the Govt's Covid tracer app pages

Communications

Vincent - 21-May-2020 - Offering to help test
wiki/projects/nz-covid-tracing-app.txt · Last modified: 2020/05/26 21:38 by Vincent Dirks