1.4 series is missing #16

Krzmbrzl · 2022-01-21T17:49:22Z

Kissaki · 2022-01-27T11:19:56Z

I will try to find some time for this… :)

Kissaki · 2022-02-07T18:33:08Z

Our PPA build pipeline uses dl.mumble.info to download source tarballs.

With the migration into subfolders there it looks the our PPA pipeline broke back then for 1.3 releases too.

Kissaki · 2022-02-10T22:51:51Z

I removed now unsupported Ubuntu versions. Our PPA is currently missing releases for Ubuntu versions Hirsute, Impish, Jammy (future LTS). The released and supported LTS versions are already there.

I may have fixed the PPA scripts
May/Should work for a stable release.
Snapshots are missing the tarballs.

For 1.4 we changed our build pipeline to CMake, so additional adjustments will have to be made on the build scripts.(?)

It’s unclear to me what the relationship between our four CIs is now, and which ones do the actual release, and whether the old PPA CI job on our older, internal CI is still being triggered.

Related resources

Krzmbrzl · 2022-02-11T06:52:55Z

What four CI? Cirrus, Azure, GitHub Actions and AppVeyor? Cirrus and GA disqualify as one has nothing to do with Linux and the other has only been created long after the PPA pipeline.
And AppVeyor is only used for creating signed Windows release packages. So not really related either. So that only leaves Azure as a potential candidate.

In terms of source tarballs: I think it would be better if the CI job checked out the repo via git and then simply creates a source tarball from that to upload on launchpad.

And in terms of triggering, I think it's not a good idea to make the PPA depend on any CI in the upstream repo.
If anything we should make it depend on release in that repo (probably possible via e.g. GA)

Krzmbrzl · 2022-02-11T06:54:05Z

Note also that there exists a PR in the main repo that is dedicated to get cmake deb packaging to work. It has stalled at figuring out the correct dependencies but other than that it already works, iirc

Kissaki · 2022-02-11T16:45:18Z

mumble-voip/mumble#5006

Krzmbrzl · 2022-02-11T16:46:25Z

Yes exactly 👍

natrius · 2022-08-15T16:41:42Z

Any chance to get an update to this @Kissaki ? Or is there not time at all? (completely understandable :) )

Kissaki · 2022-08-15T16:44:57Z

Update in what way?

I am not working on this. I am not familiar with CMake, and find it hard to get into. Same with my build environment.

Unfortunately, it doesn’t look like anyone else is picking it up either.

ViBE-HU · 2022-08-20T15:01:07Z

oh boy. it's about a decade since i started to add Mumble PPAs to my systems after installations to always get the fresh stuff. what's changed since the last has been released via ppa?

alxndr42 · 2022-08-20T15:35:27Z

From what I understand, the new(?) Mumble build process doesn't work with the existing PPA infrastructure.

FWIW, I ran the Docker build and just copied the binary out of the container image, out of desperation. This situation wouldn't be as bad if the Mumble team at least provided an official Linux binary.

Kissaki · 2022-08-20T16:56:11Z

There was a major switch to CMake build toolchain with 1.4. That means the previous logic does not work anymore.

natrius · 2022-08-22T15:09:20Z

I asked out of interest in #cmake:matrix.org and got the answer, if a developer wants to join the channel and asks for help about CPack, they(or at least one of them :D ) would be happy to support.
I pointed to this issue as well, just in case.

Krzmbrzl · 2022-08-22T16:43:49Z

there exists a PR in the main repo that is dedicated to get cmake deb packaging to work. It has stalled at figuring out the correct dependencies but other than that it already works

The problem wasn't CPack, but dependency management

davidebeatrici · 2022-08-22T18:09:41Z

I wonder if we should just set up our own APT repository, which could probably be used on most (if not all) Debian derivates.

That is, as long as dependencies are satisfied. If they aren't, APT (or DPKG) shows a warning when installing the relevant package.

jonasbadstuebner · 2022-08-23T09:27:17Z

+1

strongthony · 2022-08-29T16:30:28Z

I am in favor of hosting our own APT. Would that be too much work? Would we be able to do the same thing for DNF or whatever for Fedora/Cent based stuff?

Krzmbrzl · 2022-08-29T16:31:28Z

I think whether or not we want to host our own APT is pretty irrelevant until we are actually able to build DEB (or other) packages in the first place 🤷

Sailing74 · 2022-09-05T10:37:03Z

Just dropping by to mention that it'd be great to have working DEB packages (via Ubuntu PPA or some other means) again.

natrius · 2022-09-08T09:27:41Z

I just add this here, it seems mumble-voip/mumble#5006 might resolve the problem or at least is a big step forward, if i understood Krzmbrzl correct :)

Krzmbrzl · 2022-09-08T11:26:50Z

Yes, indeed. To my understanding of things, once mumble-voip/mumble#5006 is completed, all that is left to do is upload the generated .deb somewhere suitable and the PPA should be up and running again.

If you want to help move this forward, you could test the .deb package generated with the code from that PR. If you don't want to build Mumble yourself, let me know and I can give you the built package.

ethaldeman · 2023-01-25T22:40:08Z

I would be interested in testing the .deb package. While I have some experience compiling mumble, I think the safest bet is to provide me the .deb file.

Krzmbrzl · 2023-02-03T11:57:01Z

@ethaldeman thanks for volunteering. You can find the pre-built packages here.
Let me know how it goes :)

ethaldeman · 2023-02-04T16:08:08Z

@Krzmbrzl All these tests were made on Ubuntu 22.04 and a mumble client compiled from github's master branch is also present on the computer. The client appears to be fully functional. I confirmed audio in and out, positional audio, and text-to-speech are working. I also connected to two different mumble servers.

The server's deb successful installed, however I couldn't find anyway to interact with it beyond that. I tried running sudo dpkg-reconfigure mumble-mumble_server and the command returned without doing anything. I located the installed service mumble-server and that said it was running. The service said it executed /etc/init.d/mumble-server to start. Based on /etc/init.d/mumble-server, it appears the .ini file is located at /etc/mumble-server.ini, however that file did not exist even when the service was running.

ViBE-HU · 2023-02-04T16:14:43Z

i tried on Ubuntu 20.04 but it fails:

$ sudo dpkg -i Mumble-1.6.12-Linux-mumble_client.deb 
(Reading database ... 508985 files and directories currently installed.)
Preparing to unpack Mumble-1.6.12-Linux-mumble_client.deb ...
Unpacking mumble-mumble_client (1.6.12) ...
dpkg: error processing archive Mumble-1.6.12-Linux-mumble_client.deb (--install):
 trying to overwrite '/usr/bin/mumble', which is also in package mumble 1.3.4-1~ppa2~focal1
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 Mumble-1.6.12-Linux-mumble_client.deb

ethaldeman · 2023-02-04T16:25:26Z

@ViBE-HU trying to overwrite '/usr/bin/mumble', which is also in package mumble 1.3.4-1~ppa2~focal1

you need to uninstall your current 1.3.4 mumble installation to continue.

ViBE-HU · 2023-02-04T16:31:20Z

@ethaldeman you are right but it should not conflict. and here is another issue:

$ sudo dpkg -i Mumble-1.6.12-Linux-mumble_client.deb 
(Reading database ... 508956 files and directories currently installed.)
Preparing to unpack Mumble-1.6.12-Linux-mumble_client.deb ...
Unpacking mumble-mumble_client (1.6.12) ...
dpkg: dependency problems prevent configuration of mumble-mumble_client:
 mumble-mumble_client depends on libprotobuf23 (>= 3.12.4); however:
  Package libprotobuf23 is not installed.
 mumble-mumble_client depends on libqt5core5a (>= 5.15.1); however:
  Version of libqt5core5a:amd64 on system is 5.12.8+dfsg-0ubuntu2.1.
 mumble-mumble_client depends on libqt5dbus5 (>= 5.14.1); however:
  Version of libqt5dbus5:amd64 on system is 5.12.8+dfsg-0ubuntu2.1.
 mumble-mumble_client depends on libqt5gui5 (>= 5.14.1); however:
  Version of libqt5gui5:amd64 on system is 5.12.8+dfsg-0ubuntu2.1.
 mumble-mumble_client depends on libqt5network5 (>= 5.14.1); however:
  Version of libqt5network5:amd64 on system is 5.12.8+dfsg-0ubuntu2.1.
 mumble-mumble_client depends on libqt5widgets5 (>= 5.14.1); however:
  Version of libqt5widgets5:amd64 on system is 5.12.8+dfsg-0ubuntu2.1.

and if i try to start:

$ mumble
mumble: error while loading shared libraries: libPocoZip.so.80: cannot open shared object file: No such file or directory

ethaldeman · 2023-02-04T17:00:13Z

The method you used to install the
deb did not also install its dependencies. Mumble needs those to run. Running sudo apt-get -f install should install the dependencies. Going foward though installing a deb via sudo apt install ./name.deb should install dependencies at the same time.

ViBE-HU · 2023-02-04T17:04:46Z

i tried already via the store but it failed because of the dependencies. these packages are not available for 20.04 i guess. as far as i remember Mumble now use newer Qt:

$ sudo apt install ./Mumble-1.6.12-Linux-mumble_client.deb 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'mumble-mumble_client' instead of './Mumble-1.6.12-Linux-mumble_client.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mumble-mumble_client : Depends: libprotobuf23 (>= 3.12.4) but it is not installable
                        Depends: libqt5core5a (>= 5.15.1) but 5.12.8+dfsg-0ubuntu2.1 is to be installed
                        Depends: libqt5dbus5 (>= 5.14.1) but 5.12.8+dfsg-0ubuntu2.1 is to be installed
                        Depends: libqt5gui5 (>= 5.14.1) but 5.12.8+dfsg-0ubuntu2.1 is to be installed
                        Depends: libqt5network5 (>= 5.14.1) but 5.12.8+dfsg-0ubuntu2.1 is to be installed
                        Depends: libqt5widgets5 (>= 5.14.1) but 5.12.8+dfsg-0ubuntu2.1 is to be installed
E: Unable to correct problems, you have held broken packages.

Krzmbrzl · 2023-02-04T18:24:45Z

@ViBE-HU
it could be that the issue you are seeing is (partly) created by me having built the package on Ubuntu 22.04 and perhaps it picked the library versions from my system as the minimum requirement?
Maybe the deb package should be built for every Ubuntu version separately? Or maybe I have simply overlooked something 🤔

you are right but it should not conflict.

Why shouldn't it conflict? You have a completely unrelated package installed (from the package manager's POV) and now that new package tries to overwrite its files.
I don't think our self-made deb package aims for providing flawless upgrade paths from system packages. Definitely not at this early stage of development and I'm also not sure whether that is something we'll want long term (seems like it could create all sorts of subtle issues) 👀

@ethaldeman

however I couldn't find anyway to interact with it beyond that.

What about calling the installed binary from the command line directly? That should work, right?

The service said it executed /etc/init.d/mumble-server to start. Based on /etc/init.d/mumble-server, it appears the .ini file is located at /etc/mumble-server.ini

Uhm... yeah the ini file should be under /etc/mumble/mumble-server.ini iirc.
I have to wonder though whether the systemd service file is just buggy in general though... maybe that is unrelated to the deb package per se.

ethaldeman · 2023-02-04T21:06:48Z

It seems the Ubuntu computer I was testing on may have had some old mumble-server services still hanging around, making it unclear what the server .deb was really installing. So I uninstalled the server .deb and then purged all the mumble-server services related items I could find.

I then installed the .deb and executed the binary, located at /usr/bin/mumble-server directly; I should have thought of doing that earlier. It successfully starts a server. I was able to set a root password, create channels, and read setting from an ini. The .deb did not appear to create any services related to running a mumle-server, which may be intended but this is different from the previous Ubuntu .debs. It seems the the service stuff I described above is unrelated to this .deb, and just hang-overs from previous installs.

So in summary, both the client and server .deb appear to be functioning on Ubuntu 22.04 with the note that the server .deb longer installs a mumbler-server service. In there are any specific aspects you would like me to test, just let me know.

ViBE-HU · 2023-02-05T00:07:15Z

@ViBE-HU it could be that the issue you are seeing is (partly) created by me having built the package on Ubuntu 22.04 and perhaps it picked the library versions from my system as the minimum requirement? Maybe the deb package should be built for every Ubuntu version separately? Or maybe I have simply overlooked something thinking

you are right but it should not conflict.

Why shouldn't it conflict? You have a completely unrelated package installed (from the package manager's POV) and now that new package tries to overwrite its files. I don't think our self-made deb package aims for providing flawless upgrade paths from system packages. Definitely not at this early stage of development and I'm also not sure whether that is something we'll want long term (seems like it could create all sorts of subtle issues) eyes

@ethaldeman

however I couldn't find anyway to interact with it beyond that.

What about calling the installed binary from the command line directly? That should work, right?

The service said it executed /etc/init.d/mumble-server to start. Based on /etc/init.d/mumble-server, it appears the .ini file is located at /etc/mumble-server.ini

Uhm... yeah the ini file should be under /etc/mumble/mumble-server.ini iirc. I have to wonder though whether the systemd service file is just buggy in general though... maybe that is unrelated to the deb package per se.

not exactly clear how DEB packaging works but as far as i understand how dependencies works maybe yes(?)

i didn't know that this build is not related with the recent versions but the main issue are the dependencies.

Krzmbrzl · 2023-02-05T10:19:34Z

The .deb did not appear to create any services related to running a mumle-server, which may be intended but this is different from the previous Ubuntu .debs.

The package file should have installed /usr/etc/systemd/system/mumble-server.service. From my research on this, that should normally cause the service to be picked up automatically. Do you perhaps have to restart your computer in between or otherwise cause systemd to re-check available service files?

Krzmbrzl · 2023-02-05T10:24:58Z

@ViBE-HU I re-checked my packaging setup and those dependency versions are in fact those that I have written in the respective config. I have simply copied those from the mumble Ubuntu package, but that then presumably was the one from Ubuntu 22.04.
I guess we would have to build the DEB package on the oldest Ubuntu version that we want to support in order to make sure that the built binaries don't depend on newer features of the respective libraries (e.g. Qt). Then, if we adapt the dependency versions accordingly, I guess it should also run on newer Ubuntus.

For the time being though, I will not focus on making things work with anything before Ubuntu 22.04. We can focus on these things, once the package works fine on >= 22.04.

ethaldeman · 2023-02-05T15:43:10Z

I have confirmed the server .deb did install /usr/etc/systemd/system/mumble-server.service. However after restarting and running sudo systemctl daemon-reload, mumble-server.service is still not found.

sudo systemctl status mumble-server.service
Unit mumble-server.service could not be found.
sudo systemctl enable mumble-server.service
Failed to enable unit: Unit file mumble-server.service does not exist.

Krzmbrzl · 2023-02-05T16:03:45Z

Hm... then I guess I'll have to research a bit more and choose a different path for that 🤔

ethaldeman · 2023-02-05T16:10:09Z

While I don't know what best pratices are, /etc/systemd/system/mumble-server.service is a valid path. I moved the service file to there, and systemmd found the service. However the service did fail to start, but that could be caused by the service file not being where it expects itself to be.

natrius · 2023-02-05T16:21:15Z

Have you opened /usr/etc/systemd/system/mumble-server.service and checked where it leads to or where its trying to open mumble? And if the path itself is valid and so on? I installed paperless and hat to move my own paperless-service-files in there - so the path definitely works :)

Krzmbrzl · 2023-02-05T18:51:34Z

It could definitely be that the service file is corrupt. I have no clue about that thing, so the chances that I made some sort of mistake is somewhat high here ^^

ethaldeman · 2023-02-05T20:16:47Z

I dug into /usr/etc/systemd/system/mumble-server.service . It has the below command to start mumble-server. /usr/local/bin/mumble-server -ini /usr/local/etc/mumble/mumble-server.ini -fg First, that is not the path to the mumble-server installed by the deb, its /usr/bin/mumble-server. Second /usr/local/etc/mumble/mumble-server.ini does not exist. It seems in-order for mumble-server to start, the set ini must at least exist and be readable, even if it is empty. Third, the service is set to run by the user _mumble-server, which the deb installer didn't appear to create. I made the below changes

Updated the service file to have ExecStart=/usr/bin/mumble-server -ini /usr/local/etc/mumble/mumble-server.ini -fg
Created /usr/local/etc/mumble/mumble-server.ini as a file just containing the line database=/var/lib/mumble-server/mumble-server.sqlite
Created the _mumble-server user and ensured it had permissions to /usr/local/etc/mumble/mumble-server.ini and /var/lib/mumble-server/mumble-server.sqlite
Moved /usr/etc/systemd/system/mumble-server.service to /etc/systemd/system/mumble-server.service

After that I ran sudo systemctl enable mumble-server and sudo systemctl start mumble-server. I then had a working mumble-server service. Fixing the service file "might" be something I could submit as a PR, if I am provided a default ini to work from, but I don't even know where to begin to find where I need to make these changes in the source code or how to build a .deb file.

Krzmbrzl · 2023-02-05T21:00:12Z

@ethaldeman these are valuable insights!
We configure this file with the respective paths via cmake (see https://github.com/mumble-voip/mumble/blob/master/auxiliary_files/config_files/mumble-server.service.in which serves as the template) but apparently the paths used in there and the actual install paths for the binary and the ini file don't match (we do ship a default ini file, that should also be installed via the deb package).

As for the user, someone will have to look into how it is possible to make the deb create that on installation and how to generate the package via cpack in the desired way...

ethaldeman · 2023-02-08T23:36:32Z

While I still have not made much progress trying to understand how cmake actualy builds the server .deb, adding the user account should be achievable by having the .deb run adduser --system _mumble-server during installation.

Krzmbrzl · 2023-04-02T18:51:27Z

Alright, I had another look at this. The reason why the paths in the service file are off is because those assume that things get installed relative to the value of CMAKE_INSTALL_PREFIX (which is what happens when you do make install) but apparently CPack has its own per-generator defaults for what to use as an installation prefix (which incidentally in this case here is /usr/ rather than /usr/local/, which is the default for CMAKE_INSTALL_PREFIX on Unix). I am currently investigating whether there is a good way of figuring this out in advance and/or how to adapt to that on-the-fly in order to circumvent this issue.

(Maybe we can make use of CPACK_INSTALL_SCRIPTS to do the in-source patching for us. Though that will be a bit tedious since that might easily overlook a place where the paths require updating for things to work)

In terms of the missing user, I was told that sysusers should somehow take care of this. I am unsure though whether this is something that should happen automatically or if we have to create an additional config file for that to work...
Does anyone here happen to know how that is supposed to work?

Krzmbrzl · 2023-04-04T11:14:20Z

Update: It seems that we are already shipping a sysusers config file that seems like it should create the correct user entry: https://github.com/mumble-voip/mumble/blob/master/auxiliary_files/config_files/mumble-server.sysusers
And in principle that should also be installed by the package. So I guess that should take care of the user issue? perhaps after a restart?

ethaldeman · 2023-04-04T15:27:46Z

@Krzmbrzl Are the deb files you linked above still the best to use for testing, or is there a commit I should build from? If a commit, I will need general guidance for how to build the deb file unless they are a by product of the normal mumble build process. My Linux environment has undergone a lot of mumble deb testing and different versions, so I plan on testing in a clean Ubuntu 22.04 VM. I will install the deb and specifcy check if user _mumble-server is created.

In theory sysusers should be run at either boot or new package install.

Krzmbrzl · 2023-04-04T17:28:31Z

There have been only minor changes, but if you want to use to most up-to-date state (which I guess makes sense), you can simply build the package as follows:

Checkout my package branch at https://github.com/Krzmbrzl/mumble/tree/build-package\
Build Mumble as usual, but be sure to set -Dpackaging=ON and also set a build number via -DBUILD_NUMBER=42 when invoking cmake. What specific build number you are using doesn't matter at this point - it only prevents our build setup from complaining.
Once the regular build has finished, execute cpack -G DEB to build the deb packages. Once that finishes, you can find the packages in the build directory (which I assumed is your PWD)

In terms of the different install prefixes, I was told that there is no easy/standard way of doing this, so I'll have to improvise...

ethaldeman · 2023-04-05T01:51:37Z

I successfully built the deb files from your branch and ran sudo apt install ./mumble_upstream-1.6.42-Linux-mumble_server.deb, however it failed with the below error.

The following packages have unmet dependencies: mumble_upstream-mumble_server : Depends: libssl1.1 (>= 1.1.0) but it is not installable E: Unable to correct problems, you have held broken packages.

I also had this problem when I tried to install your deb file linked above. When I tested before on my Linux box, I didn't have this issue, but my brand-new VM is having this issue. A quick Goggle suggests Ubuntu 22.04 has dropped support for libssl1.1 and only supports libssl3 now.

Krzmbrzl · 2023-04-05T05:58:48Z

Ah yes, indeed. I have updated the dependency list to use libssl3 instead.

Krzmbrzl · 2023-04-05T06:15:22Z

I just saw that our sysusers config file gets installed to /usr/etc/sysconfig.d/mumble-server.conf as at least on my system the pkgconf call performed by cmake failed to return the proper sysusers directory (and apparently the default we choose in that case is no good)

ethaldeman · 2023-04-05T23:35:42Z

With the libssl3 change, I was able to install the server deb. The sysusers config was also installed to usr/etc/sysconfig.d/mumble-server.conf. for me. I also confirmed the user _mumble-server did not exist via getent passwd | grep "mumble". Based on sysusers documentation, I believe the correct path is /usr/lib/sysusers.d.

Krzmbrzl · 2023-04-08T12:10:50Z

The wrong install paths of the sysusers config file should be fixed once mumble-voip/mumble#6100 is merged.
Then the only thing left is the fix for the different install prefixes between cmake and cpack.

Krzmbrzl · 2023-12-30T09:09:41Z

Alright, I have just discovered CPACK_PACKAGING_INSTALL_PREFIX that should be able to help with the install path issues.

Kissaki self-assigned this Feb 7, 2022

Kissaki mentioned this issue May 26, 2022

Mumble PPA not update mumble-voip/mumble#5700

Closed

Kissaki removed their assignment May 26, 2022

Krzmbrzl mentioned this issue Oct 20, 2022

PPA for Ubuntu Jammy #17

Closed

Krzmbrzl mentioned this issue Feb 8, 2023

Backport "BUILD(crypto): Migrate to OpenSSL 3.0-compatible API" mumble-voip/mumble#5354

Closed

Hartmnt mentioned this issue Mar 2, 2023

Ubuntu PPA : Not updated mumble-voip/mumble#6074

Closed

Krzmbrzl mentioned this issue Aug 15, 2023

PPA on Pop-OS (Jammy) doesnt include a Release-File mumble-voip/mumble#6190

Closed

Hartmnt mentioned this issue Sep 18, 2023

Build number exceeds version format mumble-voip/mumble#5827

Closed

1.4 series is missing #16

1.4 series is missing #16

Comments

Krzmbrzl commented Jan 21, 2022

Kissaki commented Jan 27, 2022

Kissaki commented Feb 7, 2022

Kissaki commented Feb 10, 2022

Krzmbrzl commented Feb 11, 2022 • edited by Kissaki Loading

Krzmbrzl commented Feb 11, 2022

Kissaki commented Feb 11, 2022

Krzmbrzl commented Feb 11, 2022

natrius commented Aug 15, 2022

Kissaki commented Aug 15, 2022

ViBE-HU commented Aug 20, 2022

alxndr42 commented Aug 20, 2022

Kissaki commented Aug 20, 2022 • edited Loading

natrius commented Aug 22, 2022

Krzmbrzl commented Aug 22, 2022

davidebeatrici commented Aug 22, 2022

jonasbadstuebner commented Aug 23, 2022

strongthony commented Aug 29, 2022

Krzmbrzl commented Aug 29, 2022

Sailing74 commented Sep 5, 2022

natrius commented Sep 8, 2022

Krzmbrzl commented Sep 8, 2022

ethaldeman commented Jan 25, 2023

Krzmbrzl commented Feb 3, 2023 • edited Loading

ethaldeman commented Feb 4, 2023 • edited Loading

ViBE-HU commented Feb 4, 2023

ethaldeman commented Feb 4, 2023

ViBE-HU commented Feb 4, 2023

ethaldeman commented Feb 4, 2023

ViBE-HU commented Feb 4, 2023

Krzmbrzl commented Feb 4, 2023

ethaldeman commented Feb 4, 2023 • edited Loading

ViBE-HU commented Feb 5, 2023

Krzmbrzl commented Feb 5, 2023

Krzmbrzl commented Feb 5, 2023

ethaldeman commented Feb 5, 2023

Krzmbrzl commented Feb 5, 2023

ethaldeman commented Feb 5, 2023

natrius commented Feb 5, 2023 • edited Loading

Krzmbrzl commented Feb 5, 2023

ethaldeman commented Feb 5, 2023

Krzmbrzl commented Feb 5, 2023

ethaldeman commented Feb 8, 2023

Krzmbrzl commented Apr 2, 2023 • edited Loading

Krzmbrzl commented Apr 4, 2023

ethaldeman commented Apr 4, 2023

Krzmbrzl commented Apr 4, 2023

ethaldeman commented Apr 5, 2023

Krzmbrzl commented Apr 5, 2023

Krzmbrzl commented Apr 5, 2023

ethaldeman commented Apr 5, 2023

Krzmbrzl commented Apr 8, 2023

Krzmbrzl commented Dec 30, 2023

Krzmbrzl commented Feb 11, 2022 •

edited by Kissaki

Loading

Kissaki commented Aug 20, 2022 •

edited

Loading

Krzmbrzl commented Feb 3, 2023 •

edited

Loading

ethaldeman commented Feb 4, 2023 •

edited

Loading

ethaldeman commented Feb 4, 2023 •

edited

Loading

natrius commented Feb 5, 2023 •

edited

Loading

Krzmbrzl commented Apr 2, 2023 •

edited

Loading