microg-build3 how to handle zips after they have been synced to download server?

[petefoth]

The retention policy on build servers is

• 2 x builds of the latest branch
• 1 x build of the previous branch (until we're happy that latest branch is stable).

On microg-build3, we have a lot of devices getting the second build of 22.1 so there are 2 x 22.1 and 1 x 21.0. When we wre halfway though the February builds they started to fail with (out of disk space` errors (see this comment .

I got round that problem by deleting everything in /mnt/persistent/zips (which ***should already be on the download server). THta may get us through to the end of the build run, but we need a 'solution' that's a bit more robust.

When we only had one build server microg-build2 - with loads of disk space, /mnt/persistent/zips acted as an unofficial backup of the download server: if we ever lost the download server, microg-build2would be have copy of everything that had been on the build server.

The easy solution to the current problem would be to delete build artefacts from the build server once they have been synced to the download server, but

• if we do it on both servers we have no backup of the download server
• if we do it only on microg-build3, the 'backupon microg-build2does not have the latest builds frommicrog-build3`

Proposed solution

• Create a new directory onmicrog-build2 (which currently has 2.5T free out a total 4TB on /mnt/persistent/)
• On completion of a build, once build artefacts have been synced to the download server, move them to the new backup directory. This is easy for builds on microg-build2, but we will need to use rsync or some other mechanism to get them from microg-build3 or microg-build2 (either directlly or via the download server). More thought needed on this

[petefoth]

To be discussed in this private issue