CI: test with --with-snapshot on the slower platforms #14222

Activity

1. refack

   added

   buildIssues and PRs related to Node.js builds or CI infrastructure.Issues and PRs related to Node.js builds or CI infrastructure.

   flaky-testIssues and PRs involving tests that fail intermittently in CI.Issues and PRs involving tests that fail intermittently in CI.

   lts-agenda

   performanceIssues and PRs related to the performance of Node.js.Issues and PRs related to the performance of Node.js.

   testIssues and PRs related to Node.js core tests and test infrastructure.Issues and PRs related to Node.js core tests and test infrastructure.

   discussIssues opened for discussion and feedback.Issues opened for discussion and feedback.

   on Jul 13, 2017
2. evanlucas commented on Jul 13, 2017

   

   evanlucas

   on Jul 13, 2017

   Contributor

   More actions

   I don't think that it is a good idea to run the tests with snapshots, but release builds without snapshots.
3. cjihrig commented on Jul 13, 2017

   

   cjihrig

   on Jul 13, 2017

   Contributor

   More actions

   I agree with @evanlucas. And, as painful as the CI may currently be, I think we should try to work through the snapshot issue before taking any drastic (maybe too strong of a word) measures.
4. refack commented on Jul 13, 2017

   

   refack

   on Jul 13, 2017

   ContributorAuthor

   More actions

   Random sample Δt numbers:
   #14214 (comment)
   without - https://ci.nodejs.org/job/node-test-commit/11091/
   with - https://ci.nodejs.org/job/node-test-commit/11045/

   platform	with(minutes)	without(minutes)	Δt	good enough
   node-test-commit-aix	38	42	10%	✔️
   node-test-commit-arm	29	98	237%	🔴
   node-test-commit-arm-fanned	35	70	100%	🔴
   node-test-commit-freebsd	19	53	178%	🔴
   node-test-commit-linux	27	32	18%	✔️
   node-test-commit-linux-fips	30	67	123%	🔴
   node-test-commit-linuxone	3	6.5	116%	✔️
   node-test-commit-osx	19	25	30%	✔️
   node-test-commit-plinux	8.5	24	182%	🔴
   node-test-commit-smartos	16	21	131%	🔴
   node-test-commit-windows-fanned	39	40	2%	✔️
5. refack commented on Jul 13, 2017

   

   refack

   on Jul 13, 2017

   ContributorAuthor

   More actions

   Compromise: a check box in the submit job form?
6. mhdawson commented on Jul 13, 2017

   

   mhdawson

   on Jul 13, 2017

   Member

   More actions

   I'm in agreement that we need to test in the same way that we ship.
7. refack commented on Jul 13, 2017

   

   refack

   on Jul 13, 2017

   ContributorAuthor

   More actions

   I want to make an argument for leniency.

   1. We already make some trade offs in our test suite (skipping /pummel/ and /internet/).
   2. At some point we'll want to restore the snapshot and we don't want that to rot.
   3. Building with --with-snapshot is still a valid use case for those who build from source.

   tl;dr adding a checkbox (enable / disable snapshot) in the job submission form might be a good thing, by it increasing coverage.
8. gibfahn commented on Sep 16, 2017

   

   gibfahn

   on Sep 16, 2017

   Member

   More actions

   @refack snapshots have been re-enabled so this can be closed right?

   Reopen if I'm wrong.

   👍React with 👍1Reacted by Refael Ackermann
9. gibfahn

   closed this as completedon Sep 16, 2017