Incomplete Angular Karma Jasmine specs no longer fail on fit/fdescribe

My build system didn't reject a recent PR committed with a focused test fdescribe. In the past jasmine would return a non-zero exit code when run with fdescribe()/fit() tests. Recently (?) however that seems to have stopped working.

Since I can reproduce in a new Angular project I am guessing that it is not a problem with my setup. But I guess it could be a NodeJS v20.10 issue, or a MacOS problem.

Any advice on how to make this work properly again?

Example Output:

Create a new app: npx -p @angular/cli ng new --strict --standalone --routing --style=scss demo-v17

Mark one of the default tests as fit.

~/.../demo-v17 (main *)$ ng test --no-watch
✔ Browser application bundle generation complete.
12 12 2023 09:49:33.067:INFO [karma-server]: Karma v6.4.2 server started at http://localhost:9877/
12 12 2023 09:49:33.069:INFO [launcher]: Launching browsers Chrome with concurrency unlimited
12 12 2023 09:49:33.072:INFO [launcher]: Starting browser Chrome
12 12 2023 09:49:34.328:INFO [Chrome 120.0.0.0 (Mac OS 10.15.7)]: Connected on socket bTNnn02qMyLlZwf8AAAB with id 66356302
Chrome 120.0.0.0 (Mac OS 10.15.7): Executed 1 of 3 (skipped 2) SUCCESS (0.066 secs / 0.058 secs)
Chrome 120.0.0.0 (Mac OS 10.15.7) ERROR
Some of your tests did a full page reload!
Chrome 120.0.0.0 (Mac OS 10.15.7): Executed 1 of 3 (skipped 2) ERROR (0.066 secs / 0.058 secs)
Chrome 120.0.0.0 (Mac OS 10.15.7) ERROR
Chrome 120.0.0.0 (Mac OS 10.15.7): Executed 1 of 3 (skipped 2) ERROR (0.076 secs / 0.058 secs)
~/.../demo-v17 (main *)$ echo $?
0


It is weird that the "ERROR Some of your tests did a full page reload!" also didn't cause a bad exit code. But that is just how the default Angular v17 does, so I assume it isn't part of this.

$ ng version

_ _ ____ _ ___
/ \ _ __ __ _ _ _| | __ _ _ __ / ___| | |_ _|
/ △ \ | '_ \ / _` | | | | |/ _` | '__| | | | | | |
/ ___ \| | | | (_| | |_| | | (_| | | | |___| |___ | |
/_/ \_\_| |_|\__, |\__,_|_|\__,_|_| \____|_____|___|
|___/


Angular CLI: 17.0.6
Node: 20.10.0
Package Manager: npm 10.2.3
OS: darwin x64

Angular: 17.0.6
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package Version
---------------------------------------------------------
@angular-devkit/architect 0.1700.6
@angular-devkit/build-angular 17.0.6
@angular-devkit/core 17.0.6
@angular-devkit/schematics 17.0.6
@schematics/angular 17.0.6
rxjs 7.8.1
typescript 5.2.2
zone.js 0.14.2

Continue reading...