A shorter human oriented report with references to the source code in emacs

[hww]

Is there a way to make the test report less verbose (less empty lines, less unnecessary lines) but more human readable (with color markup) and references to the source code (For example failed test line M-. jump to test function definition)?

[hww]

Ok. Let me give few examples.

EXAMPLE 1

The test file has undefined function at the same time those two values open debugger when something going wrong

(setf 5am:*on-error* :debug)
(setf 5am:*on-failure* :debug)

What will be in the debugger?

The function TYPE-SYSTEM.TESTS::RUN-TESTS-FOR-CIA is undefined.
   [Condition of type UNDEFINED-FUNCTION]

Restarts:
 0: [CONTINUE] Retry calling RUN-TESTS-FOR-CIA.
 1: [USE-VALUE] Call specified function.
 2: [RETURN-VALUE] Return specified values.
 3: [RETURN-NOTHING] Return zero values.
 4: [ABORT] Abort compilation.
 5: [*ABORT] Return to SLY's top level.
 --more--

Backtrace:
 0: ("undefined function")
      [No Locals]
 1: (SB-FASL::LOAD-FASL-GROUP #S(SB-FASL::FASL-INPUT :STREAM #<SB-SYS:FD-STREAM for "file /tmp/slimebZRrit.fasl" {10049E5423}> :TABLE #(2 #<PACKAGE "TYPE-SYSTEM.TESTS"> RUN-TESTS-FOR-CIA 0 0 0 ...) :STACK..
 2: (SB-FASL::LOAD-AS-FASL #<SB-SYS:FD-STREAM for "file /tmp/slimebZRrit.fasl" {10049E5423}> NIL NIL)
 3: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<SB-SYS:FD-STREAM for "file /tmp/slimebZRrit.fasl" {10049E5423}> T)
 4: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {7F07C86ADD4B}> #<SB-SYS:FD-STREAM for "file /tmp/slimebZRrit.fasl" {10049E5423}> T #<SB-SYS:FD-STREAM for "file /..
 5: (LOAD #P"/tmp/slimebZRrit.fasl" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
 6: ((FLET SLYNK-BACKEND:CALL-WITH-COMPILATION-HOOKS :IN "/home/valery/.emacs.d.crafter/sly/slynk/backend/sbcl.lisp") #<FUNCTION (LAMBDA NIL :IN SLYNK-BACKEND:SLYNK-COMPILE-STRING) {10049E52DB}>)
 7: ((FLET SLYNK-BACKEND:SLYNK-COMPILE-STRING :IN "/home/valery/.emacs.d.crafter/sly/slynk/backend/sbcl.lisp") "(run-tests-for-cia) ..)
 8: ((LAMBDA NIL :IN SLYNK:COMPILE-STRING-FOR-EMACS))
 9: ((LAMBDA NIL :IN SLYNK::COLLECT-NOTES))
10: (SLYNK::MEASURE-TIME-INTERVAL #<FUNCTION (LAMBDA NIL :IN SLYNK::COLLECT-NOTES) {10049E0A7B}>)
11: (SLYNK::COLLECT-NOTES #<FUNCTION (LAMBDA NIL :IN SLYNK:COMPILE-STRING-FOR-EMACS) {10049E0A2B}>)
12: (SLYNK::CALL-WITH-BUFFER-SYNTAX NIL NIL #<FUNCTION (LAMBDA NIL :IN SLYNK:COMPILE-STRING-FOR-EMACS) {10049E09DB}>)
13: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SLYNK:COMPILE-STRING-FOR-EMACS "(run-tests-for-cia) ..)
14: (EVAL (SLYNK:COMPILE-STRING-FOR-EMACS "(run-tests-for-cia) ..)
15: (SLYNK:EVAL-FOR-EMACS (SLYNK:COMPILE-STRING-FOR-EMACS "(run-tests-for-cia) ..)
16: ((LAMBDA NIL :IN SLYNK::SPAWN-WORKER-THREAD))
17: (SLYNK-SBCL::CALL-WITH-BREAK-HOOK #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<FUNCTION (LAMBDA NIL :IN SLYNK::SPAWN-WORKER-THREAD) {10049E000B}>)
18: ((FLET SLYNK-BACKEND:CALL-WITH-DEBUGGER-HOOK :IN "/home/valery/.emacs.d.crafter/sly/slynk/backend/sbcl.lisp") #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<FUNCTION (LAMBDA NIL :IN SLYNK::SPAWN-WORKER-THREA..
19: ((LAMBDA NIL :IN SLYNK::CALL-WITH-LISTENER))
20: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<PACKAGE "TYPE-SYSTEM.TESTS">) (*DEFAULT-PATHNAME-DEFAULTS* . #P"/mnt/d/projects/type-system/tests/") (*) (** . :DEBUG) (*** :TYPE-SYSTEM.TESTS) (/ NIL (#<IT...
21: ((LAMBDA NIL :IN SLYNK::SPAWN-WORKER-THREAD))
22: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
23: ((FLET "WITHOUT-INTERRUPTS-BODY-10" :IN SB-THREAD::RUN))
24: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
25: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-THREAD::RUN))
26: (SB-THREAD::RUN)
27: ("foreign function: call_into_lisp")
28: ("foreign function: funcall1")

CONCLUSIONS

1 There is no any reference to source code
2 Stack trace full of data but most of about libraries, now I have to find a helpful info in this screen then find the error line in source - manually!.

The result screen is too verbose and does not have source code reference

[hww]

The example 2

The source code is

(def-suite* main-test)

(test dumy-test 
  (is (= 2 1)  "2 equal 2")
  )

(defun run-tests-for-ci ()
  (setf 5am:*on-error* :debug)
  (setf 5am:*on-failure* :debug)
  (run! 'main-test)
  (typespec-test))

(run-tests-for-ci)

The result screen

The following check failed: (EQUALP (TYPESPEC-ARGS-COUNT TS) 3)
  Expected equal
   given: 2
  expect: 3.
   [Condition of type IT.BESE.FIVEAM::CHECK-FAILURE]

Restarts:
 0: [IGNORE-FAILURE] Continue the test run.
 1: [RETEST] Rerun the test #<IT.BESE.FIVEAM::TEST-CASE TYPESPEC-ARGS-TEST {10070C85F3}>
 2: [IGNORE] Signal an exceptional test failure and abort the test #<IT.BESE.FIVEAM::TEST-CASE TYPESPEC-ARGS-TEST {10070C85F3}>.
 3: [EXPLAIN] Ignore the rest of the tests and explain current results
 4: [ABORT] Abort compilation.
 5: [*ABORT] Return to SLY's top level.
 --more--

Backtrace:
 0: (IT.BESE.FIVEAM::PROCESS-FAILURE (EQUALP (TYPESPEC-ARGS-COUNT TS) 3) "  Expected equal ..)
      Locals:
        REASON = "  Expected equal\n   given: 2\n  expect: 3"
        REASON-FORMAT = "  Expected equal\n   given: 2\n  expect: 3"
        TEST-EXPR = (EQUALP (TYPESPEC-ARGS-COUNT TS) 3)
        SB-DEBUG::MORE = NIL
 1: ((LABELS IT.BESE.FIVEAM::RUN-IT :IN IT.BESE.FIVEAM::RUN-TEST-LAMBDA))
 2: ((:METHOD IT.BESE.FIVEAM::RUN-TEST-LAMBDA (IT.BESE.FIVEAM::TEST-CASE)) #<IT.BESE.FIVEAM::TEST-CASE TYPESPEC-ARGS-TEST {10070C85F3}>) [fast-method]
 3: ((:METHOD IT.BESE.FIVEAM::RUN-RESOLVING-DEPENDENCIES (IT.BESE.FIVEAM::TEST-CASE)) #<IT.BESE.FIVEAM::TEST-CASE TYPESPEC-ARGS-TEST {10070C85F3}>) [fast-method]
 4: ((:METHOD IT.BESE.FIVEAM::%RUN (IT.BESE.FIVEAM::TEST-SUITE)) #<IT.BESE.FIVEAM::TEST-SUITE TYPESPEC-SUITE {10070C85C3}>) [fast-method]
 5: ((LAMBDA NIL :IN RUN))
 6: (RUN TYPESPEC-SUITE :PRINT-NAMES NIL)
 7: (RUN! TYPESPEC-SUITE :PRINT-NAMES NIL)
 8: (SB-FASL::LOAD-FASL-GROUP #S(SB-FASL::FASL-INPUT :STREAM #<SB-SYS:FD-STREAM for "file /tmp/slimecZqBOR.fasl" {1004B46DB3}> :TABLE #(2 #<PACKAGE "TYPE-SYSTEM.TESTS"> RUN-TESTS-FOR-CI 0 0 0 ...) :STACK ..
 9: (SB-FASL::LOAD-AS-FASL #<SB-SYS:FD-STREAM for "file /tmp/slimecZqBOR.fasl" {1004B46DB3}> NIL NIL)
10: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<SB-SYS:FD-STREAM for "file /tmp/slimecZqBOR.fasl" {1004B46DB3}> T)
11: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {7F07C824DD4B}> #<SB-SYS:FD-STREAM for "file /tmp/slimecZqBOR.fasl" {1004B46DB3}> T #<SB-SYS:FD-STREAM for "file /..
12: (LOAD #P"/tmp/slimecZqBOR.fasl" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
13: ((FLET SLYNK-BACKEND:CALL-WITH-COMPILATION-HOOKS :IN "/home/valery/.emacs.d.crafter/sly/slynk/backend/sbcl.lisp") #<FUNCTION (LAMBDA NIL :IN SLYNK-BACKEND:SLYNK-COMPILE-STRING) {1004B46C6B}>)
14: ((FLET SLYNK-BACKEND:SLYNK-COMPILE-STRING :IN "/home/valery/.emacs.d.crafter/sly/slynk/backend/sbcl.lisp") "(run-tests-for-ci) ..)
15: ((LAMBDA NIL :IN SLYNK:COMPILE-STRING-FOR-EMACS))
16: ((LAMBDA NIL :IN SLYNK::COLLECT-NOTES))
 --more--

The first line is actual test fail but 'v' shorcut open the source code of process-failure from file check.lisp. Browsing this line and down to the tree -- nothing refers to the failed test code

CONCLUSIONS

1 There is no any reference to source code
2 Stack trace full of data but most of about libraries, now I have to find a helpful info in this screen then find the error line in source - manually!.

[hww]

I believe, I can be wrong. I just do not know how to use FiveAM correctly. So please let me know