Use proxy tester for better verification, and fix timeout support #129

mcg1969 · 2025-02-01T20:48:05Z

[EDIT: More improvements listed below]

For another project I worked with Claude.ai to develop a single-file proxy tester script. This script launches a man-in-the-middle proxy to capture and log requests that the child script is generating, making it easier to do live testing. One particular feature of the tester is that it can be run in a mode that intercepts all requests and returns a fixed status code, so you can actually do live testing without disturbing the upstream server.

I thought I'd bring it over here and use it for test_config and test_heartbeat. This is better for test_heartbeat in particular because I was running it in "dry run" mode during the tests here—no actual requests are being generated. This is confirming that in fact, yes, the script is indeed making the intended request and using the intended user agent.

And wouldn't you know it? Doing this uncovered an issue with how timeouts are being handled. In short, the thread that is making the request was being shut down as soon as the rest of the activation work was complete, and that is quite often not long enough for the connection to be made. Forcing the process to block until the connection attempt is complete is therefore necessary, but then you have to make sure that the connect timeout is reasonably short so that the user does not suffer unreasonable activation delays if the internet connection is down or is slow.

I added a delay capability into the proxy tester to emulate a stalled connection, and then I added a mode to test_heartbeats.py that uses that capability to make sure that the heartbeat is not sent if the delay is too long.

MORE IMPROVEMENTS:

The proxy approach enabled me to clean up the test_heartbeat module quite a bit. Now we are recovering the full heartbeat URL as well as the user agent from the actual request logs, not the dry run debug output.
Made the environment prefix an argument of attempt_heartbeat for simpler calling in the patched activate function.
Added -n/--name and -p/--prefix command-line arguments to the python -m anaconda_anon_usage.heartbeat command so we can exercise the above argument
Added timing information to the debug output

mcg1969 · 2025-02-01T23:41:14Z

Here is an example test_heartbeat output. The delay lines represent the case where an artificial 1s connection delay is inserted. The test verifies that the heartbeat request is not sent in that case.

Testing heartbeat
-----------------
Expected host: repo.anaconda.com:443
Expected path: /pkgs/main/noarch/activate-0.0.0-0.conda
Expected tokens: aau,c,s,e
| aau/0.5.0+26.ge99e782 c/Gw6W35u1xiObFwLRCdFb5b s/KIq7bAsMl1HOJ0Zp1VIrcP e/y_HTPaJT9NjV4kyFyXzBbN
hval  shell      envname    status
----- ---------- ---------- ----------
true  posix      base       OK
true  posix      base       OK
true  posix      testchild1 OK
true  posix      testchild1 OK
true  posix      testchild2 OK
true  posix      testchild2 OK
false posix      base       OK
false posix      base       OK
false posix      testchild1 OK
false posix      testchild1 OK
false posix      testchild2 OK
false posix      testchild2 OK
delay posix      base       OK
delay posix      base       OK
delay posix      testchild1 OK
delay posix      testchild1 OK
delay posix      testchild2 OK
delay posix      testchild2 OK
FAILURES: 0

mcg1969 · 2025-02-02T19:56:11Z

.pre-commit-config.yaml

@@ -20,7 +20,7 @@ repos:
    rev: v3.19.1
    hooks:
      - id: pyupgrade
-        args: [--py36-plus]
+        args: [--py36-plus, --keep-percent-format]


what can I say I like percents

mcg1969 · 2025-02-02T20:02:58Z

anaconda_anon_usage/heartbeat.py



-def attempt_heartbeat(channel=None, path=None, wait=False):
-    global DRY_RUN
-    line = "------------------------"


moved the print statements, which are only needed when calling from the command line, to the main() function where they belong

mcg1969 · 2025-02-02T20:03:39Z

anaconda_anon_usage/heartbeat.py

-    _print("anaconda-anon-usage heartbeat", standalone=True)
-    _print(line, standalone=True)
-
+def attempt_heartbeat(prefix=None, dry_run=False, channel=None, path=None):


added the prefix argument so that this one function controls the user agent

mcg1969 · 2025-02-02T20:05:17Z

anaconda_anon_usage/heartbeat.py

+    _print("anaconda-anon-usage heartbeat")
+    _print(line)
+
+    def environment_path(s):


TIL that the type argument in argparse arguments is a callable:
https://stackoverflow.com/a/37472037

mcg1969 · 2025-02-02T20:05:47Z

tests/integration/test_heartbeats.py

@@ -19,8 +20,6 @@
 if os.path.isfile("/etc/conda/machine_token"):
    expected += ("m",)

-os.environ["ANACONDA_ANON_USAGE_DEBUG"] = "1"
-os.environ["ANACONDA_HEARTBEAT_DRY_RUN"] = "1"


No need for these anymore, because we are mining the heartbeat information directly from the requests actually made.

mcg1969 · 2025-02-02T20:06:35Z

tests/requirements.txt

@@ -1,2 +1,3 @@
 pytest
 pytest-cov
+cryptography


The proxy tester does require the cryptography module in order to do certificate generation

mcg1969 · 2025-02-02T20:12:57Z

tests/integration/test_heartbeats.py

 need_header = True
-for hval in ("true", "false"):
-    os.environ["CONDA_ANACONDA_HEARTBEAT"] = hval
+for hval in ("true", "false", "delay"):


"delay" mode is enabled, but the network connection is blocked, so we're testing that no heartbeat was actually sent.

anaconda_anon_usage/heartbeat.py

jezdez · 2025-02-03T11:16:38Z

anaconda_anon_usage/heartbeat.py

+    _print("anaconda-anon-usage heartbeat")
+    _print(line)
+
+    def environment_path(s):


jezdez · 2025-02-03T11:18:48Z

tests/integration/proxy_tester.py

@@ -0,0 +1,535 @@
+#!/usr/bin/env python3


I'll need some time to check the AI's job to implement this correctly, my trust in AI slop is limited :)

I'm actually generating a bunch of tests for the script using Claude. I found an interesting case with this approach! I will keep you posted.

mcg1969 force-pushed the proxy-tester branch 2 times, most recently from 49cd556 to aa3423c Compare February 1, 2025 21:06

use proxy tester for better verification

87a163a

mcg1969 force-pushed the proxy-tester branch 2 times, most recently from 718481c to 3d8eba5 Compare February 1, 2025 22:21

mcg1969 changed the title ~~use proxy tester for better verification~~ Use proxy tester for better verification, and fix timeout support Feb 1, 2025

mcg1969 requested a review from jezdez February 1, 2025 22:27

mcg1969 force-pushed the proxy-tester branch 3 times, most recently from dd6d1bc to 864811f Compare February 1, 2025 23:08

mcg1969 force-pushed the proxy-tester branch 4 times, most recently from 1bea81a to 0a709e7 Compare February 2, 2025 01:12

Fix timeout handling, add timeout testing

71d4a87

mcg1969 force-pushed the proxy-tester branch 3 times, most recently from a3554dc to 3b1804b Compare February 2, 2025 19:22

timeout adjustments

17e2b8e

mcg1969 force-pushed the proxy-tester branch from 3b1804b to 17e2b8e Compare February 2, 2025 19:48

mcg1969 commented Feb 2, 2025

View reviewed changes

mcg1969 added 2 commits February 2, 2025 14:22

a little more cleanup

e9403a5

remove unused noevent kwarg

642ae05

jezdez reviewed Feb 3, 2025

View reviewed changes

jezdez approved these changes Feb 4, 2025

View reviewed changes

jezdez merged commit a286699 into main Feb 4, 2025
39 checks passed

jezdez deleted the proxy-tester branch February 4, 2025 00:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use proxy tester for better verification, and fix timeout support #129

Use proxy tester for better verification, and fix timeout support #129

mcg1969 commented Feb 1, 2025 •

edited

Loading

mcg1969 commented Feb 1, 2025

mcg1969 Feb 2, 2025

mcg1969 Feb 2, 2025

mcg1969 Feb 2, 2025

mcg1969 Feb 2, 2025

jezdez Feb 3, 2025

mcg1969 Feb 2, 2025

mcg1969 Feb 2, 2025

mcg1969 Feb 2, 2025

jezdez Feb 3, 2025

jezdez Feb 3, 2025

mcg1969 Feb 3, 2025

Use proxy tester for better verification, and fix timeout support #129

Use proxy tester for better verification, and fix timeout support #129

Conversation

mcg1969 commented Feb 1, 2025 • edited Loading

mcg1969 commented Feb 1, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcg1969 commented Feb 1, 2025 •

edited

Loading