Commit Graph

48 Commits

Author SHA1 Message Date
Michael Telatynski
e6d9eccf1b
Add test reporter to prevent stale screenshots (#12743)
* Split up slow Playwright tests

To optimise parallelism

Deals with:

```
Slow test file: read-receipts/redactions.spec.ts (5.4m)
Slow test file: read-receipts/new-messages.spec.ts (3.9m)
Slow test file: read-receipts/high-level.spec.ts (3.6m)
Slow test file: read-receipts/editing-messages.spec.ts (3.1m)
Slow test file: read-receipts/reactions.spec.ts (2.2m)
Slow test file: crypto/crypto.spec.ts (2.4m)
Slow test file: settings/appearance-user-settings-tab/appearance-user-settings-tab.spec.ts (1.2m)
Slow test file: composer/composer.spec.ts (1.1m)
Slow test file: crypto/verification.spec.ts (1.1m)
```

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Move around snapshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add test reporter to prevent stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove darwin screenshots which should not have been checked in

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix absolute vs relative path mismatch

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Remove darwin screenshots which should not have been checked in"

This reverts commit 1e189977fa9ec873339fc02b2b231a314809b2d5.

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Revert "Remove darwin screenshots which should not have been checked in""

This reverts commit 5144b9b28e31ca543b2c5d02820c3f957dbd8c04.

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Remove stale screenshots"

This reverts commit 9beae9974557c1ffa99c2372da280bb0da407bd1.

* Apply same sanitization as Playwright for file name consistency

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* add dev dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Discard changes to playwright/flaky-reporter.ts

* Update end-to-end-tests.yaml

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-07-15 18:23:20 +00:00
Richard van der Hoff
1fbc97296c
Use LTS nodejs in github workflows (#12726)
Currently our GH workflows all use nodejs 18, but that's not compatible with the
stated support range of matrix-js-sdk, which requires the latest LTS nodejs;
so, when we attermpt to `yarn install` on an older nodejs, we get an error.

Switch to LTS nodejs for all the places we set up nodejs.
2024-07-04 09:24:02 +00:00
Michael Telatynski
353b6c0de9
Disable xvfb as it is not necessary for headless mode (#12720)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-07-03 17:01:51 +00:00
Michael Telatynski
eb28898754
Reduce number of parallel runners for Playwright (#12695) 2024-06-25 15:21:47 +01:00
Michael Telatynski
ccf751d6b8
Update end-to-end-tests.yaml 2024-06-13 10:43:21 +01:00
Michael Telatynski
8a0dcdd4de
Update end-to-end-tests.yaml 2024-06-07 18:07:12 +01:00
Michael Telatynski
9680a36b14
Update end-to-end-tests.yaml 2024-06-04 17:06:03 +01:00
Michael Telatynski
f2d9f13f1b
Update end-to-end-tests.yaml 2024-06-03 21:56:33 +01:00
Michael Telatynski
30850709ef
Build the playwright html report in the main workflow (#12570)
* Build the playwright html report in the main workflow

so we can get the report for MQ

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* fix

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-06-03 10:03:26 +00:00
Michael Telatynski
e2310e637c
Update end-to-end-tests.yaml 2024-05-07 11:47:35 +01:00
Michael Telatynski
f648f00727
Update end-to-end-tests.yaml 2024-04-30 11:03:44 +01:00
Michael Telatynski
3634e665b1
Update end-to-end-tests.yaml 2024-04-30 10:35:12 +01:00
Michael Telatynski
16a01d834f
Update end-to-end-tests.yaml 2024-04-30 10:28:29 +01:00
Michael Telatynski
02e7fb340e
Simplify Playwright CI (#12457)
* Simplify Playwright CI

and apply the least privilege principle

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix playwright.needs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove spurious report.needs

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add a completion stage to simplify branch protections

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-04-26 15:25:07 +00:00
Michael Telatynski
86469bdd32
Add custom reporter to auto-report flaky Playwright tests (#12290) 2024-02-28 13:02:09 +00:00
Michael Telatynski
b5048bf63e
Update setup-xvfb action to fix node16 deprecation warning (#12256)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-02-16 14:13:52 +00:00
renovate[bot]
2a2fe65983
Update actions/cache action to v4 (#12174)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2024-02-02 17:33:13 +00:00
Michael Telatynski
0d2253e7d0
Fix issues caused by the artifacts v4 upgrade
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2024-01-19 09:30:15 +00:00
Michael Telatynski
1dbcdfb6b2
Use Github Artifacts v4 (#12152) 2024-01-19 08:54:49 +00:00
renovate[bot]
b64d2e734a
Update dependency @playwright/test to v1.41.0 (#12150)
* Update dependency @playwright/test to v1.41.0

* Update end-to-end-tests.yaml

* Update end-to-end-tests.yaml

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2024-01-17 10:11:04 +00:00
renovate[bot]
0e06f38c1b
Update dawidd6/action-download-artifact digest to f29d1b6 (#12037)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-13 08:55:38 +00:00
Michael Telatynski
99ca613818
Move 2 runners from Cypress to Playwright (#12034)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-12-12 16:51:54 +00:00
David Baker
f628591227
Fix playwright result reporting for js-sdk upstream tests (#12010)
* Fix playwright result reporting for js-sdk upstream tests

Make the reporter job check out the react sdk explictly so it gets
the right thing when used from other workflows like the js-sdk.

* Remove c+ped lines from above that would have broken it
2023-12-07 13:45:44 +00:00
Michael Telatynski
d06fc7d623
Move two runners from Cypress to Playwright to rebalance runtime (#12006) 2023-12-06 15:13:03 +00:00
Michael Telatynski
d5abde0e6e
Improve usage of Playwright and documentation (#11999)
Co-authored-by: Andy Balaam <andy.balaam@matrix.org>
2023-12-06 10:13:48 +00:00
Michael Telatynski
09fddc0676
Update Sibz/github-status-action to use node16 to silence warning (#11955) 2023-12-04 15:42:54 +00:00
Michael Telatynski
d7c682d05e
Update end-to-end-tests.yaml (#11991) 2023-12-04 11:32:39 +00:00
Michael Telatynski
e207798a8f
Allow for running playwright tests upstream (#11959)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-29 11:42:00 +00:00
renovate[bot]
11695d3227
Update actions/setup-node action to v4 (#11969)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 09:07:26 +00:00
renovate[bot]
e2978ebb98
Update actions/checkout action to v4 (#11968)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-28 19:22:17 +00:00
Michael Telatynski
d47f856ba0
Fix end-to-end-tests.yaml netlify prefix
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-28 10:55:54 +00:00
Michael Telatynski
1f6287e2b5
Update end-to-end-tests.yaml 2023-11-28 10:42:23 +00:00
Michael Telatynski
e49a866114
Update end-to-end-tests.yaml 2023-11-28 10:29:29 +00:00
Michael Telatynski
253ba55951
Deploy Playwright html report to Netlify (#11923)
* Deploy Playwright html report to Netlify

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-28 09:18:28 +00:00
Michael Telatynski
eb5516b7b1
Update end-to-end-tests.yaml 2023-11-22 12:54:31 +00:00
Michael Telatynski
f3432ca838
Update end-to-end-tests.yaml 2023-11-22 08:52:59 +00:00
Michael Telatynski
04d095b871
Update end-to-end-tests.yaml 2023-11-22 08:46:33 +00:00
Michael Telatynski
b6521ba86e
Update end-to-end-tests.yaml 2023-11-22 08:38:26 +00:00
Michael Telatynski
4a0ffefae7
Update end-to-end-tests.yaml 2023-11-21 22:08:06 +00:00
Michael Telatynski
1529099143
Update end-to-end-tests.yaml 2023-11-21 21:47:11 +00:00
Michael Telatynski
0f1f056503
Add Playwright end to end testing (#11912)
* Install playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add foundations for writing tests under Playwright

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* .gitignore juggling

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add tsconfig and fix eslint rules

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-21 17:33:32 +00:00
Michael Telatynski
f566c600e2
Port remaining Puppeteer tests over to Cypress (#9104)
* Port remaining Puppeteer tests over to Cypress

* Remove puppeteer support files

* Fix lifecycle matrixclientpeg setup race condition

* Alternative solution to the lifecycle problem

* Dismiss the notifications toast
2022-07-29 14:03:25 +00:00
Michael Telatynski
79195dea8b
Skip running jobs on fork develop where they lack secrets (#8837)
* Skip running jobs on fork `develop` where they lack secrets

* Fix contexts
2022-06-14 11:37:07 +01:00
Michael Telatynski
f53207a4fe
Fix cypress checkout and use github actions environments (#8790)
* Fix cypress checkout and use github actions environments

* Harden github token for cypress.yaml

* Allow writing status checks
2022-06-08 16:00:47 +01:00
J. Ryan Stinnett
c122c5cd3b
Add basic performance testing via Cypress (#8586) 2022-05-17 15:38:45 +01:00
Michael Telatynski
92f440d9de
Consolidate deployment management into more maintained action (#8430) 2022-04-29 11:07:42 +01:00
Michael Telatynski
146bcdd6a6
Move more stuff from BK to GHA (#8372) 2022-04-21 11:55:32 +00:00
David Baker
1eb67047c9
Add some doc to the github actions / CI scripts (#8034)
* Add some doc to the githuib actions / CI scripts

Type: task

* alalytics-events is more name matching
2022-03-11 13:00:56 +00:00