Docker Infra checklist

[meeDamian]

I've had a lot of stuff happening all around the place, so I want to create this issue to just gather it all together (Perhaps move it to some wiki later)…

🧩 meeDamian/simple-qemu

tl;dr: Fetches qemu source, and from it builds static binaries for amd64 host system. Result is pushed to Docker registry(ies), and binaries are uploaded to Github Release.

• Get rid of workflow generation scripts, once Actions allows for either:
  • optional jobs (not steps, nor workflows)
  • referencing jobs in different workflows

Click to expand already done

• Get it to build
• Verify it works
• Build multiple versions
  • [x]: v3.1.0 v3.1.1 v4.0.0 v4.1.0
• Create short/floating tags
  • v4, v3, v4.0, v4.1, v3.1, arm, aarch64, latest, arm32v7, arm64v8, enable, riscv32, riscv64, etc…
• Create permanent tags
• Push to 🐳 Hub
• Push to Registry It's borked. Perhaps later…
• Write proper README.md
• Create short script to generate the Simple tags section the way most repos have it
• Ask @lncm/lncm for code, and process review

🧩 lncm/berkeleydb

tl;dr: Pre-builds and packages BerkeleyDB v4.8.30.NC into a small container, that later can be reused in Dockerfile with sth like:

COPY --from=meedamian/berkeleydb:db-4.8.30.NC  /opt/  /opt/

• Document build process more in README.md(?)

Click to expand already done

• Push to Registry It's borked. Perhaps later…
• Setup build on Actions
• Build for amd64, arm32v7, and arm64v8
• Push to 🐳 Hub
• BDB doesn't use semver, but for consistency permanent tags can behave the same as above +build
• See if it's possible to help consumers emulating CPU arch choose the correct version based on manifest, w/o specifying it verbatim
• Ask @lncm/lncm for code, and process review
• Actions are enabled for the org (see Build process move to Github Actions docker-berkeleydb#11)
• Move back to lncm/ namespace (blocked by: Trigger build by pushing a git-tag docker-berkeleydb#13, AKA no-secrets-in-forks issue)

🧩 lncm/bitcoind

tl;dr: Docker image with Bitcoin Core client built for amd64, arm32v7, and arm64v8.

• Create a nowallet variant(?)
• Push to 🐳 Hub
• Document build process more in README.md
• Document new release process more in README.md (or another .md file)
• Move back to lncm/ namespace
• Use +build to distinguish internal builds(?)
• Ask @lncm/lncm for code, and process review

Click to expand already done

• Push to Registry It's borked. Perhaps later…
• Get it to build on a CI
• Split build into jobs
  • 1. Berkeleydb
  • 2. Finish at make
  • 3. Start at make check
  • 2. Build images
  • 3. Upload images
• Build a few last, non vulnerable, versions
  • builds latest ~3 on each push to master
• Build for amd64, arm32v7, and arm64v8
• Do not upload anything to image registry, until builds of all architecture variants complete green

🧩 lncm/lnd

tl;dr: Docker image of a Lightning Network client built for amd64, arm32v7, and arm64v8.

• Create a neutrino variant (if warrants a separate image)
• From Actions push to 🐳 Hub
• Move back to lncm/ namespace
• Ask @lncm/lncm for code and process review

Click to expand already done

• Push to Registry It's borked. Perhaps later…
• Move to Actions
• Build for amd64, arm32v7, and arm64v8
• Use Go v1.13
• Use -trimpath -mod=readonly
• In each build, build each binary twice:
  • alpine
  • debian:*-slim
  • compare checksums

🧩 lncm/invoicer

tl;dr: Utility living on top of lnd (and optionally bitcoind), that helps with creation of payment-accepting frontends

• Add tests
• Document a release process
• Get rid of the Makefile(?)

Click to expand already done

• Push to Registry It's borked. Perhaps later…
• Move to Actions
• Build for amd64, arm32v7, and arm64v8
• Use Go v1.13
• Use -trimpath -mod=readonly
• In each build, build each binary twice:
  • alpine
  • debian:*-slim
  • compare checksums
• From Actions push to 🐳 Hub
• Move back to lncm/ namespace once Actions are enabled for the org (see Build process move to Github Actions docker-berkeleydb#11)
• Get rid of *BSD platforms (@AnotherDroog?)
• Ask @lncm/lncm for code and process review

🧩 lncm/donations

tl;dr: A UI that lives on top of invoicer that facilitates receiving BTC & LN donations.

• Cleanup
  • Get rid of fluff

🧩 lncm/invoicer-ui

tl;dr: A UI that lives on top of invoicer that facilitates receiving BTC & LN in-store payments.

• Should probably be renamed to sth along the lines of merchant-ui
• is there anyone actually maintaining it?

🧩 lncm/noma

• @AnotherDroog : )

🧩 lncm/pi-factory

• @nolim1t : )

Other/General TODOs

• Create references from pre-existing issues to points on this list 😅
• Create template repo for stuff that needs emulation
  • Golang (pass arch to build using --build-arg)
  • General (register qemu using meedamian/simple-qemu)
• Look into DOCKER_CONTENT_TRUST=1 (currently @ DOCKER_CONTENT_TRUST 🤔 meeDamian/simple-qemu#5)

Click to expand already done

• Find a way to sync README.md between , and 🐳 Hub
  • done: https://github.com/meeDamian/sync-readme
• Write an all-in Action to create Release, and upload files to it
  • done: https://github.com/meeDamian/github-release
• Come up with a tagging scheme that never overrides non-short tags (Ex. override of v4 is desired, while override of v4.1.0-riscv64 should probably be avoided). First thought is to use semver-compliant +build scheme.
  • done: https://github.com/meeDamian/tag-suggestions

[nolim1t]

merchant-ui

• no active maintainers as far as I know

pi-factory

• legacy box project. Working with latest lnd and bitcoind. Also invoicer to follow

lnd

• have a neutrino build already on dockerhub

[meeDamian]

have a neutrino build already on dockerhub

I don't think we'll "have it" until it's automated, and auditable. For what I, and anyone, knows you could've added code that sends you all of the btc there once there's enough deposited on that node ;).

[nolim1t]

You're welcome to audit it

[AnotherDroog]

For what it’s worth, I’m happily using your lnd container @nolim1t

Of course auditing is a core concern for us. In that light, I’d like to see all components within one organization and the same git host.

That way at least all dependencies can be easily found