Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F-Droid build (1.76.0) crashes #13850

Open
linsui opened this issue Oct 18, 2024 · 58 comments
Open

F-Droid build (1.76.0) crashes #13850

linsui opened this issue Oct 18, 2024 · 58 comments
Labels
bug Bug OS-android Issues involving Tailscale for Android packaging

Comments

@linsui
Copy link

linsui commented Oct 18, 2024

What is the issue?

Since 1.76 builds crashes. 1.76.1-t24929f6b6-ga20b1114dd5 apk can be downloaded from https://gitlab.com/fdroid/fdroiddata/-/jobs/8109300998/artifacts/file/tmp/com.tailscale.ipn_308.apk . Could you please take a look? Thanks

Steps to reproduce

No response

Are there any recent changes that introduced the issue?

No response

OS

Android

OS version

No response

Tailscale version

1.76.1-t24929f6b6-ga20b1114dd5

Other software

No response

Bug report

No response

@marek22k
Copy link

Same issue here on CalyxOS Android 14 FP4.

@Erisa Erisa added the OS-android Issues involving Tailscale for Android label Oct 18, 2024
@a-jackson
Copy link

Same on Graphene OS. Have downgraded back to 1.72.0

@bradfitz
Copy link
Member

How's it crash? Got any logs?

@a-jackson
Copy link

a-jackson commented Oct 18, 2024

It never opens. I tap the icon and then I just get the android popup for an app crashed.
Tailscale log 1f7f3b58cb0b.txt

@linsui
Copy link
Author

linsui commented Oct 18, 2024

Timestamp: 2024-10-18 15:27:33.875801175+0800
Process uptime: 1s
Cmdline: com.tailscale.ipn
pid: 16868, tid: 16893, name: m.tailscale.ipn >>> com.tailscale.ipn <<<
uid: 10230
tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
x0 0000000000000000 x1 00000000000041fd x2 0000000000000006 x3 0000000000000008
x4 0000000000000001 x5 0000000000000000 x6 0000000000000000 x7 0000000000000001
x8 0000000000000083 x9 000000000000db7c x10 000000000000000e x11 0000007412873f9c
x12 0000000000000001 x13 0000007411c5c294 x14 0000007411cc4c73 x15 000000000000000c
x16 0000007480e1f9f0 x17 0000007480f16090 x18 0000007411238000 x19 00000000000041e4
x20 0000007480f15f30 x21 0000007412edd8a0 x22 0000000000000013 x23 0000003cb9d69707
x24 8e41dc0be9fcd734 x25 000000be71521dac x26 0000000000000000 x27 0000000000000010
x28 00000040000021c0 x29 0000004000085c58
lr 0000007411d26554 sp 0000004000085c60 pc 0000007411d4ebf8 pst 0000000080001000
1 total frames
backtrace:
#00 pc 00000000004b1bf8 /data/app/~~BLbjkzLsGmwhSGSN4ZkCxw==/com.tailscale.ipn-lmzUtjCpuQmPqYjQEhOMKg==/base.apk (offset 0x2ac000) 

@a-jackson
Copy link

panic: binary built with tailscale_go build tag but failed to read build info or find tailscale.toolchain.rev in build info

Seems to be related to #13527

@bradfitz
Copy link
Member

What makes you think that? Any logs? What's "adb logcat" say while you attempt to start it?

@a-jackson
Copy link

My logs are attached to my previous comment. That's line 446 and 451

@bradfitz
Copy link
Member

Ah, thanks. I only saw the other commenter's logs.

I wish f-droid would test their builds before publishing them.

@licaon-kter
Copy link

F-Droid does not develop the app, only packages it.

If the upstream APKs work fine and the F-Droid one misbehaves, yes please open an issue on Gitlab.

@licaon-kter
Copy link

@linsui looking at your https://gitlab.com/fdroid/fdroiddata/-/commit/00d207cd1981c57fb57607ea6d9d156ecc6886c9#281743176537bad7e7758254b7095f352bea0c57_1554_1555 you mean that even if updating the go lib commit it still crashes?

@linsui
Copy link
Author

linsui commented Oct 18, 2024

I didn't update the commit. It's read from the go.toolchain.rev file.

@bradfitz
Copy link
Member

@linsui it sure looks like the problem was that prior to you disabling the build in https://gitlab.com/fdroid/fdroiddata/-/commit/00d207cd1981c57fb57607ea6d9d156ecc6886c9#281743176537bad7e7758254b7095f352bea0c57_1554_1555 , the tailscale-go srclibs hash was wrong and didn't match the repo's declared go.toolchain.rev.

It's not clear why you were bumping the tailscale-go git hash forward in a commit meant to only disable the build, but in any case, you bumped it to the correct value now, so it should be safe to un-disable the build now.

In summary, F-Droid was building with a bad mix of components and recent Tailscale now detects that mismatch and panicked as a result. That panic is meant to catch such build system screw-ups, but no automated or human testing is involved in F-Droid releases, so the mistake shipped to F-Droid users.

@licaon-kter
Copy link

licaon-kter commented Oct 18, 2024

was wrong and didn't match the repo's declared go.toolchain.rev.

read the recipe in full: https://gitlab.com/fdroid/fdroiddata/-/blob/00d207cd1981c57fb57607ea6d9d156ecc6886c9/metadata/com.tailscale.ipn.yml#L1543-L1573

the @... is just the clone point, we later checkout the commit based on your source code file

...
    srclibs:
...
      - tailscale-go@bf15628b759344c6fc7763795a405ba65b8be5d7
    prebuild:
...
    build:
      - export TOOLCHAINREV=$(cat ../go.toolchain.rev)
...
      - pushd $$tailscale-go$$
      - git checkout $TOOLCHAINREV
      - cd src
      - ./make.bash
      - popd
...

/LE: I only asked about the bumped commit because I was not expecting any change in the srclibs section, and also because I didn't know by heart the rest of the recipe

@licaon-kter
Copy link

licaon-kter commented Oct 18, 2024

the build log is here: https://f-droid.org/repo/com.tailscale.ipn_303.log.gz

...
2024-10-15 15:54:06,921 INFO: Running 'build' commands in build/com.tailscale.ipn/android
2024-10-15 15:54:06,921 DEBUG: Directory: build/com.tailscale.ipn/android
2024-10-15 15:54:06,921 DEBUG: > bash -e -u -o pipefail -x -c export TOOLCHAINREV=$(cat ../go.toolchain.rev); export TOOLCHAINDIR=/home/vagrant/build/srclib/tailscale-go; export ANDROID_NDK_ROOT=/opt/android-sdk/ndk/23.1.7779620; export PATH=$TOOLCHAINDIR/bin:$PATH; make -C .. env; pushd /home/vagrant/build/srclib/tailscale-go; git checkout $TOOLCHAINREV; cd src; ./make.bash; popd; make -C .. libtailscale
/etc/bash.bashrc: line 7: PS1: unbound variable
++ cat ../go.toolchain.rev
+ export TOOLCHAINREV=bf15628b759344c6fc7763795a405ba65b8be5d7
+ TOOLCHAINREV=bf15628b759344c6fc7763795a405ba65b8be5d7
+ export TOOLCHAINDIR=/home/vagrant/build/srclib/tailscale-go
+ TOOLCHAINDIR=/home/vagrant/build/srclib/tailscale-go
+ export ANDROID_NDK_ROOT=/opt/android-sdk/ndk/23.1.7779620
+ ANDROID_NDK_ROOT=/opt/android-sdk/ndk/23.1.7779620
+ export PATH=/home/vagrant/build/srclib/tailscale-go/bin:/opt/android-sdk/ndk/23.1.7779620:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/android-sdk/tools:/opt/android-sdk/platform-tools:/opt/gradle/bin
+ PATH=/home/vagrant/build/srclib/tailscale-go/bin:/opt/android-sdk/ndk/23.1.7779620:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/android-sdk/tools:/opt/android-sdk/platform-tools:/opt/gradle/bin
+ make -C .. env
make: Entering directory '/home/vagrant/build/com.tailscale.ipn'
PATH=/home/vagrant/build/com.tailscale.ipn/android/tool:/home/vagrant/build/com.tailscale.ipn/android/android/build/go/bin:/opt/android-sdk/cmdline-tools/latest/bin:/opt/android-sdk/platform-tools:/home/vagrant/build/srclib/tailscale-go/bin:/opt/android-sdk/ndk/23.1.7779620:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/android-sdk/tools:/opt/android-sdk/platform-tools:/opt/gradle/bin
ANDROID_SDK_ROOT=/opt/android-sdk
ANDROID_HOME=/opt/android-sdk
ANDROID_STUDIO_ROOT=
JAVA_HOME=
TOOLCHAINDIR=/home/vagrant/build/srclib/tailscale-go
AVD_IMAGE=system-images;android-33;google_apis;x86_64
make: Leaving directory '/home/vagrant/build/com.tailscale.ipn'
+ pushd /home/vagrant/build/srclib/tailscale-go
~/build/srclib/tailscale-go ~/build/com.tailscale.ipn/android
+ git checkout bf15628b759344c6fc7763795a405ba65b8be5d7
Previous HEAD position was 66fe5734c4 Merge pull request #85 from tailscale/update-go1.22.0-actions
HEAD is now at bf15628b75 [tailscale] runtime/debug: embed Tailscale toolchain git rev
+ cd src
+ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.22. (go1.22.7 linux/amd64)
...

maybe you can find other issues in it @bradfitz ?

@bradfitz
Copy link
Member

the @... is just the clone point, we later checkout the commit based on your source code file

How is it guaranteed that the clone point will contain code from the future to be able to git checkout to? In any case, it does appear that the git checkout bf15628b759344c6fc7763795a405ba65b8be5d7 worked.

Do you see that you're both building the tailscale/go toolchain but that it's also then later downloading it again (from our pre-built releases)?

HEAD is now at bf15628b75 [tailscale] runtime/debug: embed Tailscale toolchain git rev
+ cd src
+ ./make.bash
Building Go cmd/dist using /usr/lib/go-1.22. (go1.22.7 linux/amd64)
Building Go toolchain1 using /usr/lib/go-1.22.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for linux/amd64.
---
Installed Go for linux/amd64 in /home/vagrant/build/srclib/tailscale-go
Installed commands in /home/vagrant/build/srclib/tailscale-go/bin
+ popd
~/build/com.tailscale.ipn/android
+ make -C .. libtailscale
make: Entering directory '/home/vagrant/build/com.tailscale.ipn'
mkdir -p android/libs
./tool/go install golang.org/x/mobile/cmd/gobind
go: downloading golang.org/x/mobile v0.0.0-20240806205939-81131f6468ab
go: downloading golang.org/x/tools v0.24.0
go: downloading golang.org/x/sync v0.8.0
go: downloading golang.org/x/mod v0.20.0
./tool/go install golang.org/x/mobile/cmd/gomobile
# Downloading Go toolchain bf15628b759344c6fc7763795a405ba65b8be5d7
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  2 65.1M    2 1409k    0     0  1859k      0  0:00:35 --:--:--  0:00:35 1859k
 55 65.1M   55 36.4M    0     0  21.0M      0  0:00:03  0:00:01  0:00:02 35.9M
100 65.1M  100 65.1M    0     0  24.6M      0  0:00:02  0:00:02 --:--:-- 33.8M
go: downloading golang.org/x/sys v0.22.0
go: downloading golang.org/x/mod v0.19.0
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/go-json-experiment/json v0.0.0-20231102232822-2e55bd4e08b0
go: downloading go4.org/mem v0.0.0-20220726221520-4f986261bf13
go: downloading golang.org/x/net v0.27.0

That is, you're not entirely building this from source, as is the F-Droid way. You're still using our pre-built binaries in this recipe, for at least parts.

@licaon-kter
Copy link

How is it guaranteed that the clone point will contain code from the future to be able to git checkout to?

The srclib stage clones that repo at HEAD, then checks out that @commit. The commit mentioned in the srclib section is just a point it time that later we don't care about, for this app at least.

@bradfitz
Copy link
Member

Gotcha. In any case, it seems like the recipe is using a mix of Go toolchains.

That curl output in there from ./tool/go (https://github.com/tailscale/tailscale-android/blob/main/tool/go#L13-L71) means that TOOLCHAINDIR is not set, so it's falling back to downloading our toolchain from GitHub, not using the one you just built.

@linsui
Copy link
Author

linsui commented Oct 18, 2024

https://gitlab.com/fdroid/fdroiddata/-/jobs/8109300998#L6115 But TOOLCHAINDIR is set.

@chunjiw
Copy link

chunjiw commented Oct 18, 2024

I uninstalled Tailscale from F-Droid. If I try to install an earlier version, an error pops out saying "Update version code 254 is older than current 303". If I try to install in Google Play, says "another user has already installed an incompatible version". Is there another workaround at this moment?

@agottardo
Copy link
Contributor

@chunjiw looks like the F-Droid version remained installed? Perhaps try rebooting the phone after uninstalling.

We also publish APKs on our website nowadays so give that a try, these are the same builds as the Play Store: https://pkgs.tailscale.com/stable/#android (Even though the APK might work, I recommend switching to the Play Store when you get a chance, so you can get automatic updates which is better for security).

@agottardo agottardo changed the title F-Droid build crashes F-Droid build (1.76.0) crashes Oct 18, 2024
@chunjiw
Copy link

chunjiw commented Oct 18, 2024

I tried rebooting the phone, updating F-Droid, restarting F-Droid, nothing works. I also tried directly install APK as downloaded from link you provided (thank you 👍 ) but it says "App not installed as package conflicts with an existing package". Seems like F-Droid does not properly let the system know that this app is uninstalled.

@licaon-kter
Copy link

licaon-kter commented Oct 18, 2024

@chunjiw F-Droid does not do installs and uninstalls itself, only the system does this.

Go to Android Settings - Apps - Tailscale - Uninstall to retry

@chunjiw
Copy link

chunjiw commented Oct 18, 2024

Thank you for the suggestion! But I cannot even find Tailscale in Settings - Apps list.

@marek22k
Copy link

@licaon-kter @linsui Maybe the two broken versions could be removed from F-Droid? I somehow still see them in F-Droid.

https://f-droid.org/en/packages/com.tailscale.ipn/

@linsui
Copy link
Author

linsui commented Oct 21, 2024

They'll be removed. I don't know when.

Removed.

@DeWir3
Copy link

DeWir3 commented Oct 22, 2024

I uninstalled Tailscale from F-Droid. If I try to install an earlier version, an error pops out saying "Update version code 254 is older than current 303". If I try to install in Google Play, says "another user has already installed an incompatible version". Is there another workaround at this moment?

I am having the same issue. I uninstalled the F-Droid build but still cannot install any other build or any older versions of the F-Droid build due to conflicting signatures. I opened a new issue for that #13877

@chunjiw
Copy link

chunjiw commented Oct 22, 2024

@agottardo @licaon-kter My bad... Eventually I figured it out: I have Tailscale also installed in "Secure Folder" sandbox environment (Galaxy S24+). I believe it is a copy of the installed app. So when I uninstall the app before, I left the one in "Secure Folder" still installed.

Once I uninstalled both, I was able to reinstall from Play Store.

I want to say thank you for your help and patience when trouble shooting with me.

raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
We've suffered misalignment in versioning and toolchain usage due to the
shell invocations downstream of ./version/tailscale-version.sh, but also
the whole version data scheme in the Makefile was quite complicated, and
required synchronization in the build.grade.

- Makfile no longer needs to be version aware itself.
- A Makefile target tailscale.version refreshes a local cached output
  from tailscale.com/cmd/mkversion which is updated when go.mod / go.sum
  change.
- build.gradle loads tailscale.version to get the version string.
- ldflags are produced from tailscale.version via version-ldflags.sh

Updates tailscale/tailscale#13850

Signed-off-by: James Tucker <[email protected]>
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
We've suffered misalignment in versioning and toolchain usage due to the
shell invocations downstream of ./version/tailscale-version.sh, but also
the whole version data scheme in the Makefile was quite complicated, and
required synchronization in the build.grade.

- Makfile no longer needs to be version aware itself.
- A Makefile target tailscale.version refreshes a local cached output
  from tailscale.com/cmd/mkversion which is updated when go.mod / go.sum
  change.
- build.gradle loads tailscale.version to get the version string.
- ldflags are produced from tailscale.version via version-ldflags.sh

Updates tailscale/tailscale#13850

Signed-off-by: James Tucker <[email protected]>
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
The explicit target was removed during patch production, but the
dependency wasn't removed from the clean action.

Updates #546
Updates tailscale/tailscale#13850
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
The explicit target was removed during patch production, but the
dependency wasn't removed from the clean action.

Updates #546
Updates tailscale/tailscale#13850
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
The explicit target was removed during patch production, but the
dependency wasn't removed from the clean action.

Updates #546
Updates tailscale/tailscale#13850

Signed-off-by: James Tucker <[email protected]>
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
The explicit target was removed during patch production, but the
dependency wasn't removed from the clean action.

Updates #546
Updates tailscale/tailscale#13850

Signed-off-by: James Tucker <[email protected]>
raggi added a commit to tailscale/tailscale-android that referenced this issue Oct 31, 2024
The explicit target was removed during patch production, but the
dependency wasn't removed from the clean action.

Updates #546
Updates tailscale/tailscale#13850

Signed-off-by: James Tucker <[email protected]>
@AKDub
Copy link

AKDub commented Nov 19, 2024

How long until we are likely to see a version in F-Droid that is updatable to?

@linsui
Copy link
Author

linsui commented Dec 6, 2024

It works now. Do you want to switch to reproducible build? @bradfitz

@bradfitz
Copy link
Member

bradfitz commented Dec 6, 2024

Do you want to switch to reproducible build?

I don't understand the question; what does "reproducible build" mean in this context?

@licaon-kter
Copy link

@bradfitz
Copy link
Member

bradfitz commented Dec 6, 2024

Oh, great. I didn't know F-Droid had such an effort. We're huge fans of reproducible builds in general.

@licaon-kter
Copy link

I hope you are fans, because it might just work or add an extra layer that needs troubleshooting :)

@linsui
Copy link
Author

linsui commented Dec 6, 2024

Switching to reproducible build means the old users can't upgrade. The broken version forced users to uninstall so maybe it's a chance.

@linsui
Copy link
Author

linsui commented Dec 11, 2024

@bradfitz Any dicision? Should we try to switch to reproducible build?

@bradfitz
Copy link
Member

I'll leave the timing of that to @kari-ts.

/cc @raggi @agottardo @barnstar

@linsui
Copy link
Author

linsui commented Dec 21, 2024

@bradfitz @raggi @agottardo @barnstar Any news? Maybe we should release the fixed version on F-Droid without reproducible build?

@bradfitz
Copy link
Member

We had a discussion about it. The difficulty is all about making the sign work out such that the F-Droid binaries have the same signature as https://pkgs.tailscale.com/stable/tailscale-android-universal-1.78.3.apk

Google Play manages the signing keys for our official APKs, though, so we'd need to extract the signatures from the Google Play build (which we mirror at pkgs.tailscale.com, at least for the big universal APK)

It might be easier to start with just releasing the fixed version for now.

In parallel, if anybody wants to try to see if they can do the F-Droid reproducible thing and get the same bytes out, that'd be interesting.

@linsui
Copy link
Author

linsui commented Dec 21, 2024

OK, let's release the fixed version for now. :) I'm not sure if we can make the apk reproducible with the Google Play one if app bundle is involved.

@marek22k
Copy link

marek22k commented Dec 21, 2024

My opinion as a user: For reproducible builds: And is it desirable that Google keeps the signing keys for the F-Droid releases (if tailscale extracts the signature)?
As far as I know, the reproducible builds are there so that developers have more control (and the releases are faster in F-Droid) - they are not there to rely on Google instead of F-Droid.

@linsui
Copy link
Author

linsui commented Dec 21, 2024

Hmmm, tailscale/tailscale-android@1ffe285#diff-393313cfc99ef61e5782f45e70f091a14243d37dc36ac4c33859cb1b60a004ebL67 this line was removed by mistake and 1.78.2 can't be built... It's fixed in the branch. Should we update to 1.78.3?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug OS-android Issues involving Tailscale for Android packaging
Projects
None yet
Development

No branches or pull requests