Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False negatives #8

Open
merlos opened this issue Jan 9, 2025 · 0 comments
Open

False negatives #8

merlos opened this issue Jan 9, 2025 · 0 comments

Comments

@merlos
Copy link
Owner

merlos commented Jan 9, 2025

For this command:

 broken-links https://magasin.unicef.io --only-error --ignore-file ./.check-ignore 

External links such as
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9123525/
https://odk.org/
https://digitalprinciples.org/principles/establish-people-first-data-practices/,

Are getting a False however they work.

Output:

Evidence 1

Output of broken-links https://magasin.unicef.io --only-error --ignore-file ./.check-ignore

Page: https://magasin.unicef.io/get-started/index.html, Anchor: production deployment guides, Link: https://magasin.unicef.io/deployment.qmd, Working: False
Page: https://magasin.unicef.io/get-started/create-a-dashboard.html, Anchor: Custom deployment and setup, Link: https://magasin.unicef.io/deployment.qmd, Working: False
Page: https://magasin.unicef.io/implement/magasin-team.html, Anchor: Kaggle Learn, Link: https://magasin.unicef.io/[https:/www.kaggle.com/learn, Working: False
Page: https://magasin.unicef.io/implement/organizational-processes.html, Anchor: UNICEF Data quality framework, Link: https://data.unicef.org/resources/data-quality-framework/, Working: False
Page: https://magasin.unicef.io/implement/data-privacy.html, Anchor: General Data protection Regulation (GDPR), Link: https://gdpr.eu/, Working: False
Page: https://magasin.unicef.io/implement/data-privacy.html, Anchor: people-first data practices, Link: https://digitalprinciples.org/principles/establish-people-first-data-practices/, Working: False
Page: https://magasin.unicef.io/implement/data-privacy.html, Anchor: ODK, Link: https://odk.org/, Working: False
Page: https://magasin.unicef.io/implement/data-privacy.html, Anchor: human factor, Link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9123525/, Working: False
Page: https://magasin.unicef.io/implement/data-privacy.html, Anchor: Establish people-first data practices – Principles for Digital Development (digitalprinciples.org), Link: https://digitalprinciples.org/principles/establish-people-first-data-practices/, Working: False
Page: https://magasin.unicef.io/implement/deployment.html, Anchor: DaskHub helm chart, Link: https://artifacthub.io/packages/helm/dask/daskhub, Working: False
Page: https://magasin.unicef.io/contributing/documentation.html, Anchor: https://unicef.github.io/unicef/magasin, Link: https://ungithub.io/magasin, Working: False
Page: https://magasin.unicef.io/contributing/helm-repo-dev.html, Anchor: http://localhost:8000/index.yaml, Link: http://localhost:8000/index.yaml, Working: False
Page: https://magasin.unicef.io/contributing/repositories.html, Anchor: unicef/superset-dashboard-cloner, Link: https://github.com/unicef/superset-dashboard-cloner, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_core, Link: https://magasin.unicef.io/mag_core/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_dagster, Link: https://magasin.unicef.io/mag_dagster/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_daskhub, Link: https://magasin.unicef.io/mag_daskhub/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_drill, Link: https://magasin.unicef.io/mag_drill/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_minio, Link: https://magasin.unicef.io/mag_minio/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_superset, Link: https://magasin.unicef.io/mag_superset/, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_core, Link: https://magasin.unicef.io/mag_core, Working: False
Page: https://magasin.unicef.io/mag-cli/, Anchor: mag_superset, Link: https://magasin.unicef.io/mag_superset, Working: False
Page: https://magasin.unicef.io/mag_superset, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/about.html, Anchor: Magic Box, Link: https://www.unicef.org/innovation/Magicbox, Working: False
Page: https://magasin.unicef.io/get-started/, Anchor: production deployment guides, Link: https://magasin.unicef.io/deployment.qmd, Working: False
Page: https://magasin.unicef.io/deployment.qmd, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_core, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_superset/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_minio/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_drill/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_daskhub/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_dagster/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io/mag_core/, Anchor: @githubstatus, Link: https://twitter.com/githubstatus, Working: False
Page: https://magasin.unicef.io#applying-data-minimization-with-magasin, Anchor: Privacy policy, Link: https://magasin.unicef.io/privacy.html, Working: False
Page: https://magasin.unicef.io#applying-data-minimization-with-magasin, Anchor: Edit this page, Link: https://github.com/unicef/magasin/edit/main/docs/index.qmd, Working: False
Page: https://magasin.unicef.io#applying-data-minimization-with-magasin, Anchor: Report an issue, Link: https://github.com/unicef/magasin/issues/new, Working: False

Evidence 2

% curl https://odk.org/ -vv
*   Trying 141.193.213.10:443...
* Connected to odk.org (141.193.213.10) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=odk.org
*  start date: Nov 20 07:51:00 2024 GMT
*  expire date: Feb 18 07:50:59 2025 GMT
*  subjectAltName: host "odk.org" matched cert's "odk.org"
*  issuer: C=US; O=Let's Encrypt; CN=E6
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://odk.org/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: odk.org]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: odk.org
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/2 200 
< date: Thu, 09 Jan 2025 07:46:22 GMT
< content-type: text/html; charset=UTF-8
< vary: Accept-Encoding
< vary: Accept-Encoding
< vary: Accept-Encoding
< vary: Accept-Encoding,Cookie,X-WPENGINE-SEGMENT
< expires: Thu, 19 Nov 1981 08:52:00 GMT
< pragma: no-cache
< x-frame-options: DENY
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< link: <https://odk.org/>; rel=shortlink
< x-powered-by: WP Engine
< x-cacheable: SHORT
< cache-control: max-age=600, must-revalidate
< x-cache: HIT: 1
< x-cache-group: normal
< strict-transport-security: max-age=63072000
< cf-cache-status: DYNAMIC
< set-cookie: __cf_bm=g_uwSi8Kxdy0_g3HIb0oZReiFQGJCM6sSE.vJ19rO8Q-1736408782-1.0.1.1-CVuUJB4aKeizPNA6TXFQFPUlCv77TQR8SPJWpMhdQXvEc_tMwqxQJTIrVI5yNKQHz2DQQdKKPzCXpysm6wso6A; path=/; expires=Thu, 09-Jan-25 08:16:22 GMT; domain=.odk.org; HttpOnly; Secure; SameSite=None
< server: cloudflare
< cf-ray: 8ff2d0236c4173be-JNB
< alt-svc: h3=":443"; ma=86400
< 
<!DOCTYPE html>

Evidence 2

% curl -vv https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9123525/
*   Trying 130.14.29.110:443...
* Connected to www.ncbi.nlm.nih.gov (130.14.29.110) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.ncbi.nlm.nih.gov
*  start date: Oct 11 17:00:53 2024 GMT
*  expire date: Aug  2 21:41:30 2025 GMT
*  subjectAltName: host "www.ncbi.nlm.nih.gov" matched cert's "*.ncbi.nlm.nih.gov"
*  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9123525/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: www.ncbi.nlm.nih.gov]
* [HTTP/2] [1] [:path: /pmc/articles/PMC9123525/]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET /pmc/articles/PMC9123525/ HTTP/2
> Host: www.ncbi.nlm.nih.gov
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/2 301 
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< referrer-policy: origin-when-cross-origin
< content-security-policy: upgrade-insecure-requests
< date: Thu, 09 Jan 2025 07:48:53 GMT
< server: Apache
< strict-transport-security: max-age=31536000; includeSubDomains; preload
< referrer-policy: origin-when-cross-origin
< content-security-policy: upgrade-insecure-requests
< location: https://pmc.ncbi.nlm.nih.gov/articles/PMC9123525/
< content-length: 257
< content-type: text/html; charset=iso-8859-1
< vary: Accept-Encoding
< x-ua-compatible: IE=Edge
< x-xss-protection: 1; mode=block
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC9123525/">here</a>.</p>
</body></html>
* Connection #0 to host www.ncbi.nlm.nih.gov left intact
(venv) jmerlostevar@macaulay docs % ;2D

Evidence 3

curl https://digitalprinciples.org/principles/establish-people-first-data-practices/ -vv
*   Trying 141.193.213.21:443...
* Connected to digitalprinciples.org (141.193.213.21) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=digitalprinciples.org
*  start date: Dec 31 04:33:06 2024 GMT
*  expire date: Mar 31 05:33:02 2025 GMT
*  subjectAltName: host "digitalprinciples.org" matched cert's "digitalprinciples.org"
*  issuer: C=US; O=Google Trust Services; CN=WE1
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://digitalprinciples.org/principles/establish-people-first-data-practices/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: digitalprinciples.org]
* [HTTP/2] [1] [:path: /principles/establish-people-first-data-practices/]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET /principles/establish-people-first-data-practices/ HTTP/2
> Host: digitalprinciples.org
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/2 200 
< date: Thu, 09 Jan 2025 07:50:52 GMT
< content-type: text/html; charset=UTF-8
< vary: Accept-Encoding
< vary: Accept-Encoding
< vary: Accept-Encoding
< vary: Accept-Encoding,Cookie
< last-modified: Wed, 03 Apr 2024 11:11:57 GMT
< link: <https://digitalprinciples.org/wp-json/>; rel="https://api.w.org/"
< link: <https://digitalprinciples.org/wp-json/wp/v2/pages/540>; rel="alternate"; title="JSON"; type="application/json"
< link: <https://digitalprinciples.org/?p=540>; rel=shortlink
< x-powered-by: WP Engine
< x-cacheable: YES:15552000.000
< cache-control: max-age=15552000, must-revalidate
< x-cache: HIT: 6
< x-cache-group: normal
< cf-cache-status: DYNAMIC
< set-cookie: __cf_bm=6bZy1hTEGLT8CL6IHGx9YgSMAEt8HCpi3AOEnnRNKao-1736409052-1.0.1.1-RZXel0_9wBWnIk1EB.46qVOC3LZQl2IP_Fz7iJ2KXm8wV1aV124Ph8bjQKQ04z_cYZ15QM3iNKAERXtkzOAOZw; path=/; expires=Thu, 09-Jan-25 08:20:52 GMT; domain=.digitalprinciples.org; HttpOnly; Secure; SameSite=None
< server: cloudflare
< cf-ray: 8ff2d6c26b6593d1-JNB
< alt-svc: h3=":443"; ma=86400
< 
<!DOCTYPE html>
<html lang="en-US">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant