Speedup scrublet

[Intron7]

No description provided.

[scverse-benchmark]

Benchmark changes

Change	Before [c26480e]	After [57732d0]	Ratio	Benchmark (Parameter)
-	8.01±0.2ms	7.05±0.07ms	0.88	preprocessing_counts.time_calculate_qc_metrics('pbmc68k_reduced')
+	526M	611M	1.16	preprocessing_log.peakmem_pca('pbmc3k')

Comparison: https://github.com/scverse/scanpy/compare/c26480ed0dc2f7d27b796e0e355b29a8305886c6..57732d095a98202237b0a0882dc072aba9cb6a64
Last changed: Tue, 14 May 2024 11:28:07 +0000

More details: https://github.com/scverse/scanpy/pull/3044/checks?check_run_id=24945403112

[codecov]

Codecov Report

Attention: Patch coverage is 89.47368% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 76.28%. Comparing base (c26480e) to head (57732d0).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3044   +/-   ##
=======================================
  Coverage   76.27%   76.28%           
=======================================
  Files         117      117           
  Lines       12803    12802    -1     
=======================================
  Hits         9766     9766           
+ Misses       3037     3036    -1     

Files	Coverage Δ
scanpy/preprocessing/_scrublet/__init__.py	96.80% <100.00%> (+0.10%)	⬆️
scanpy/preprocessing/_scrublet/pipeline.py	94.59% <100.00%> (+0.30%)	⬆️
scanpy/preprocessing/_scrublet/sparse_utils.py	89.28% <71.42%> (-1.90%)	⬇️

... and 1 file with indirect coverage changes

[Zethson]

scrublet doesn't seem to be part of the benchmarks at the moment

[Intron7]

@Zethson I think that this is a bit better. But not as much as I hoped

[flying-sheep]

@Zethson I’m adding some in #3031, but it needs more thought: 30 minutes for a benchmark run is too long …

[flying-sheep]

With #3056 merged, this now says

before	after	ratio	benchmark
448±100ms	381±100ms	~0.85	preprocessing_counts.time_scrublet('pbmc68k_reduced')

but ASV seems to think that’s not enough to report.

Also unclear why it’s reported as taking 26 minutes by github:

I see in the server logs

• May 14 10:15:57 scvbench benchmark[1462905]: 2024-05-14T10:15:57.547945Z DEBUG handle_event:HTTP{http.method=PATCH http.url=https://api.github.com/repos/scverse/scanpy/check-runs/24942322156 otel.name="HTTP" otel.kind="client"}: octocrab: requesting
• […running benchmarks]
• May 14 10:27:18 scvbench benchmark[1462905]: 2024-05-14T10:27:18.793352Z DEBUG handle_event:HTTP{http.method=PATCH http.url=https://api.github.com/repos/scverse/scanpy/check-runs/24942322156 otel.name="HTTP" otel.kind="client"}: octocrab: requesting

which means that between setting the check run to “running” and to “done”, 11m21s passed.

Maybe GitHub counts the queue time?