chore(cre-deployment): lightweight changeset tests #16322

krehermann · 2025-02-11T15:19:50Z

Enable lighter weight ChangeSet testing. 5x speed up of go test deployment/keystone/... b/c the majority of our tests don't need real in memory nodes

Fleshing out the test.JDNodeService will also let us write tests in the CLD repo without relying on phony in memory nodes nor the real JD instances

Requires

Supports

github-actions · 2025-02-11T15:40:44Z

Flakeguard Summary

Ran new or updated tests between develop and 1b36669 (cre/contract-only-test-env).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

2 Results

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestConfigureOCR3	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	6.713333333s	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation
TestConfigureOCR3/fails_multiple_OCR3_contracts_but_unspecified_address	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	263.333333ms	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

github-actions · 2025-02-11T15:43:58Z

AER Report: CI Core ran successfully ✅

aer_workflow , commit

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

github-actions · 2025-02-11T19:48:00Z

Flakeguard Summary

Ran new or updated tests between develop and b1da8bf (cre/contract-only-test-env).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

2 Results

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestConfigureOCR3	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	7.943333333s	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation
TestConfigureOCR3/fails_multiple_OCR3_contracts_but_unspecified_address	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	396.666666ms	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

github-actions · 2025-02-11T20:10:58Z

Flakeguard Summary

Ran new or updated tests between develop and 2140beb (cre/contract-only-test-env).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

2 Results

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestConfigureOCR3	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	7.406666666s	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation
TestConfigureOCR3/fails_multiple_OCR3_contracts_but_unspecified_address	0%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	false	296.666666ms	@smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

krehermann · 2025-02-11T21:38:17Z

deployment/environment.go

@@ -275,6 +284,14 @@ func (n Nodes) DefaultF() uint8 {
 	return uint8(len(n) / 3)
 }

+func (n Nodes) IDs() []string {


krehermann · 2025-02-11T21:39:17Z

deployment/environment/test/jd.go

 	nodev1 "github.com/smartcontractkit/chainlink-protos/job-distributor/v1/node"
 	"github.com/smartcontractkit/chainlink/deployment"
 	"github.com/smartcontractkit/chainlink/deployment/environment/memory"
 	"github.com/smartcontractkit/chainlink/v2/core/services/keystore/keys/p2pkey"
 )

-var _ node.NodeServiceServer = (*JDNodeService)(nil)
+var _ nodev1.NodeServiceClient = (*JDNodeService)(nil)


implement the entire OffchainClient interface so we can use this in an Environment for tests

krehermann · 2025-02-11T21:47:22Z

deployment/keystone/changeset/test/dons.go

+	Name() string
+}
+
+var _ testDon = (*memoryDon)(nil)


these interface impls are the basic reason we can support either in-memory nodes or 'view-only' nodes

krehermann · 2025-02-11T21:48:38Z

deployment/keystone/changeset/add_capabilities_test.go

-			AssetDonConfig:  test.DonConfig{N: 4},
-			WriterDonConfig: test.DonConfig{N: 4},
+		te := test.SetupContractTestEnv(t, test.EnvWrapperConfig{
+			WFDonConfig:     test.DonConfig{Name: "wfDon", N: 4},


these all got a Name so the test framewok can be antagonistic to the details of the don in the new Get method

krehermann · 2025-02-11T21:50:49Z

deployment/keystone/changeset/test/env_setup.go

-	validateNodes(t, gotRegistry, cwNodes, expectedHashedCapabilities(t, gotRegistry, cwDon))
-	validateNodes(t, gotRegistry, assetNodes, expectedHashedCapabilities(t, gotRegistry, assetDon))
+	require.Len(t, gotNodes, len(dons.P2PIDs()))
+	validateNodes(t, gotRegistry, dons.Get(c.WFDonConfig.Name), expectedHashedCapabilities(t, gotRegistry, wfDonCapabilities))


all these call changed so we can provide the dons interface rather than directly access state about the kind of test env we have in our hands

krehermann · 2025-02-11T21:55:14Z

deployment/keystone/changeset/test/env_setup.go

+		chains,
+		nil,
+		dons.NodeList().IDs(),
+		envtest.NewJDService(dons.NodeList()),


this is where we are using the new JD test impl

AnieeG · 2025-02-12T20:13:32Z

deployment/environment/memory/node.go

+		CSAKey:         n.Keys.CSA.ID(),
+		PeerID:         n.Keys.PeerID,
+		AdminAddr:      admin,
+		MultiAddr:      n.Addr.String(),


is this multiaddress correct? Should it not be node's p2p listen address?

AnieeG · 2025-02-12T20:13:56Z

deployment/environment/memory/node.go

+					PeerId: n.Keys.PeerID.String(),
+				},
+				OcrKeyBundle:     keyBundle,
+				Multiaddr:        n.Addr.String(),


same question

AnieeG · 2025-02-12T20:15:00Z

deployment/environment/memory/node.go

+				},
+				OcrKeyBundle:     keyBundle,
+				Multiaddr:        n.Addr.String(),
+				Plugins:          nil,


okay to have nil plugins here? For CCIP I know in real node we have commit execute plugin enabled

cl-sonarqube-production · 2025-02-13T16:03:45Z

Quality Gate passed

Issues
8 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

krehermann added 8 commits February 8, 2025 15:56

chore(cre-deployment): contract only changeset tests

2ccc6b0

adapter from memory node to deployment node

6542b33

test

c6a44b6

refactor

688c607

refactoring setupTestEnv to support both kinds of dons

b30f1fb

5x test speed up

8aae062

use contract only tests

d6b3a00

rename func for clarity

1b36669

move files

b1da8bf

Merge branch 'develop' into cre/contract-only-test-env

2140beb

dedup; fix tests

b94dd95

krehermann marked this pull request as ready for review February 11, 2025 21:37

krehermann requested review from a team as code owners February 11, 2025 21:37

krehermann requested review from winder, asoliman92, makramkd, dimkouv, mateusz-sekara, 0xAustinWang and 0xnogo February 11, 2025 21:38

krehermann commented Feb 11, 2025

View reviewed changes

krehermann requested review from patrickhuie19, AnieeG and vyzaldysanchez February 11, 2025 21:44

krehermann commented Feb 11, 2025

View reviewed changes

krehermann changed the title ~~chore(cre-deployment): contract only changeset tests~~ chore(cre-deployment): lightweight changeset tests Feb 11, 2025

krehermann commented Feb 11, 2025

View reviewed changes

AnieeG reviewed Feb 12, 2025

View reviewed changes

krehermann added 2 commits February 12, 2025 16:36

address comments

d4f7566

Merge branch 'develop' into cre/contract-only-test-env

cfad1d7

krehermann requested a review from AnieeG February 12, 2025 23:41

fix print

b131d07

krehermann force-pushed the cre/contract-only-test-env branch from 60d68b7 to 39b7cba Compare February 13, 2025 02:20

Merge branch 'develop' into cre/contract-only-test-env

04cb25e

krehermann force-pushed the cre/contract-only-test-env branch from 39b7cba to 04cb25e Compare February 13, 2025 03:16

Merge branch 'develop' into cre/contract-only-test-env

80770f3

vyzaldysanchez approved these changes Feb 13, 2025

View reviewed changes

krehermann enabled auto-merge February 13, 2025 17:53

AnieeG approved these changes Feb 13, 2025

View reviewed changes

krehermann added this pull request to the merge queue Feb 13, 2025

Merged via the queue into develop with commit ab8e8fa Feb 13, 2025
184 checks passed

krehermann deleted the cre/contract-only-test-env branch February 13, 2025 21:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(cre-deployment): lightweight changeset tests #16322

chore(cre-deployment): lightweight changeset tests #16322

krehermann commented Feb 11, 2025 •

edited by jira bot

Loading

github-actions bot commented Feb 11, 2025

github-actions bot commented Feb 11, 2025 •

edited

Loading

github-actions bot commented Feb 11, 2025

github-actions bot commented Feb 11, 2025

krehermann Feb 11, 2025

krehermann Feb 11, 2025

krehermann Feb 11, 2025

krehermann Feb 11, 2025

krehermann Feb 11, 2025

krehermann Feb 11, 2025

AnieeG Feb 12, 2025

AnieeG Feb 12, 2025

AnieeG Feb 12, 2025

cl-sonarqube-production bot commented Feb 13, 2025

chore(cre-deployment): lightweight changeset tests #16322

chore(cre-deployment): lightweight changeset tests #16322

Conversation

krehermann commented Feb 11, 2025 • edited by jira bot Loading

Requires

Supports

github-actions bot commented Feb 11, 2025

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

github-actions bot commented Feb 11, 2025 • edited Loading

AER Report: CI Core ran successfully ✅

AER Report: Operator UI CI ran successfully ✅

github-actions bot commented Feb 11, 2025

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

github-actions bot commented Feb 11, 2025

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Feb 13, 2025

Quality Gate passed

krehermann commented Feb 11, 2025 •

edited by jira bot

Loading

github-actions bot commented Feb 11, 2025 •

edited

Loading