Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transition whatwg.org to new hosting #7

Closed
42 of 43 tasks
foolip opened this issue Aug 23, 2017 · 40 comments
Closed
42 of 43 tasks

Transition whatwg.org to new hosting #7

foolip opened this issue Aug 23, 2017 · 40 comments

Comments

@foolip
Copy link
Member

foolip commented Aug 23, 2017

To improve our uptime and allow for more admins, we are moving the hosting of whatwg.org from DreamHost to DigitalOcean. These are the domains:

@foolip
Copy link
Member Author

foolip commented Aug 23, 2017

Separate from the hosting question, these are static sites that could be hosted inside https://github.com/whatwg/whatwg.org/:

@annevk?

@annevk
Copy link
Member

annevk commented Aug 23, 2017

I'd like @sideshowbarker's opinion on everything help.whatwg.org.

I'd be okay with merging whatwg/resources.whatwg.org into whatwg/whatwg.org at some point, though leaving it alone for now seems fine too? Seems somewhat more complicated if we want to keep the history intact.

For the two others I created whatwg/whatwg.org#52.

@annevk
Copy link
Member

annevk commented Aug 23, 2017

As a next step I think we should try to move everything that's not *.spec.whatwg (so we will move spec.whatwg), blog.whatwg, or wiki.whatwg. Then we could move blog.whatwg and wiki.whatwg and maybe html.whatwg (it seems like we'll continue to have a need to execute scripts so we might want to make a longer term plan first), and last all the specs (that are all intertwined I think).

@foolip
Copy link
Member Author

foolip commented Aug 23, 2017

Yeah, let's leave resources.whatwg.org alone, because it's used by HTML it might get more traffic, so might not want it deployed in the same place. Will you put images.whatwg.org and n.whatwg.org into the repo though? We can set up the deploy for that before DNS is changed.

@foolip
Copy link
Member Author

foolip commented Aug 23, 2017

Oops, that's whatwg/whatwg.org#52

@sideshowbarker
Copy link
Contributor

help.whatwg.org can just be redirected

@foolip
Copy link
Member Author

foolip commented Aug 24, 2017

resources.whatwg.org is a tricky one, because it's used by all specs. If it's not on the same IP and using the same cert as the specs, then load time will be less awesome than it could be. Maybe we should try hosting everything static on a single machine, and just make it beefier?

@annevk
Copy link
Member

annevk commented Aug 24, 2017

I think we should host all static things on the same server instance. But note that any performance benefits won't come until Let's Encrypt actually offers multi-domain wildcard certificates.

@foolip
Copy link
Member Author

foolip commented Aug 24, 2017

We could also request a single cert for all the domains on a single machine.

@foolip
Copy link
Member Author

foolip commented Aug 24, 2017

The downside is that if the DNS of any one of those domains changes, I think certbot renew would start failing.

@foolip
Copy link
Member Author

foolip commented Aug 24, 2017

The current setup for spec.whatwg.org (nested inside whatwg.org source-wise) leads to fun:
https://whatwg.org/specs/web-forms/current-work/
https://spec.whatwg.org/web-forms/current-work/

These both work, but the latter looks like an accident. I suggest that we just let those URLs go 404, and preserve only the original URLs.

@annevk
Copy link
Member

annevk commented Aug 24, 2017

Yes, we should do that.

@foolip
Copy link
Member Author

foolip commented Aug 25, 2017

I'm extracting out spec.whatwg.org now.

@foolip
Copy link
Member Author

foolip commented Aug 25, 2017

Handing html-differences in whatwg/html-differences#5

@foolip
Copy link
Member Author

foolip commented Aug 30, 2017

I've added lists.whatwg.org to the list, it's currently broken by HSTS, and it would be nice to restore it somehow.

@annevk
Copy link
Member

annevk commented Aug 30, 2017

@foolip I suspect we can't get ownership of that until we have DNS control since it's highly likely a name that's tightly coupled to DreamHost's managed mailing list service. (When we take that over we should make sure that [email protected] still works. Same for [email protected].)

@foolip
Copy link
Member Author

foolip commented Aug 30, 2017

Right, so the challenge is how to get the archives from DreamHost and, presumably, to keep updating them if we're not shutting down all of the lists.

@annevk
Copy link
Member

annevk commented Aug 30, 2017

Well, the main list we'd end up keeping is also archived at https://lists.w3.org/Archives/Public/public-whatwg-archive/. If we can it might be better to just find a mapping from the old URL to those URLs and redirect everything there. The alternative would be to install the same mailing list software (unless we want to deal with local redirects).

@foolip
Copy link
Member Author

foolip commented Aug 30, 2017

That would work, if https://lists.w3.org/Archives/Public/public-whatwg-archive/ contains everything that our own archives do. Still probably a lot of work.

@annevk
Copy link
Member

annevk commented Aug 30, 2017

It should be complete, but yes.

@foolip
Copy link
Member Author

foolip commented Sep 4, 2017

@domenic, is any action needed for build.whatwg.org? At the very least we shouldn't mess up its DNS in eventual DNS handover.

@domenic
Copy link
Member

domenic commented Sep 4, 2017

It might be a good idea for me to move that all from my personal AWS to one of the droplets. It'd also be good to switch to LetsEncrypt.

If you're up for it, I'd appreciate the help doing so; the readme at https://github.com/domenic/wattsi-server should have most of the info. Then you could switch the DNS to point to the droplet and I could shut down my server completely.

Eventually we probably want to move that to a whatwg/build.whatwg.org repository.

@foolip
Copy link
Member Author

foolip commented Sep 5, 2017

I've added it to the list so that it's not forgotten.

@foolip
Copy link
Member Author

foolip commented Nov 7, 2017

The other alternative would be to make it part of build.whatwg.org, but then we'd have to worry about malicious input, and not just processing what gets merged into the spec.

@annevk
Copy link
Member

annevk commented Nov 7, 2017

The more we can outsource to Travis the better I think.

@domenic
Copy link
Member

domenic commented Nov 7, 2017

Yes, putting it in the Docker stuff for CI build is my plan. It sounds like I should work on that sooner rather than later.

@foolip
Copy link
Member Author

foolip commented Nov 9, 2017

Yeah, otherwise we'll have to either run PHP on the machine serving HTML, or on some other machine that accepts the PDF and deploys it.

Getting the pdfbuild repo working on Travis isn't trivial, but is probably possible in the end.

@izh1979
Copy link

izh1979 commented Nov 9, 2017

Or you can rewrite the script in some other language. ;-) It's not big, nor complex.
And it's easier when you have working variant. ;-)

@snuggs
Copy link
Member

snuggs commented Nov 10, 2017

@izh1979 @foolip where would said script be located? Or is this related to whatwg/html-build#120 (comment) & whatwg/html-build#122

Lastly, what is http://sgr-a.ru/h/whatwgpdf.php ?

P.S. @annevk 👍 for PDF'ing the standards.

Thanks in advance!

@annevk
Copy link
Member

annevk commented Nov 10, 2017

@snuggs the script used for the HTML Standard is hosted at https://github.com/izh1979/makepdf and that .php page runs it.

@foolip
Copy link
Member Author

foolip commented Nov 11, 2017

Well, this is amusing, while trying to figure out how to do this, I came across https://www.princexml.com/forum/topic/2666/using-better-compression-and-pdf-size-optimization which is @izh1979 asking about compressing PDFs :)

@izh1979
Copy link

izh1979 commented Nov 13, 2017

Yeah, I've generating PDFs for WHATWG HTML spec for about of 3 years. ;-)

foolip pushed a commit to whatwg/html-build that referenced this issue Dec 1, 2017
foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 5, 2017
foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 5, 2017
@foolip
Copy link
Member Author

foolip commented Dec 11, 2017

All that remains here now is lists.whatwg.org, which is tied together with the DNS. Somebody needs to learn how email actually works to do this job, maybe we should liaison with the IETF :)

foolip added a commit to whatwg/html that referenced this issue Dec 15, 2017
foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 15, 2017
foolip added a commit to whatwg/whatwg.org that referenced this issue Dec 15, 2017
annevk pushed a commit to whatwg/html that referenced this issue Dec 16, 2017
@foolip
Copy link
Member Author

foolip commented Dec 17, 2017

An update on lists.whatwg.org. https://packages.debian.org/sid/mailman3-suite looked very promising, but is only in Debian unstable, and installation didn't actually work out of the box on a fresh Debian unstable, neither an (upgraded) DigitalOcean droplet or in a debian:unstable Docker image. (I think it failed to create/start a postgres db, and is probably easy-ish to fix.)

Because Debian unstable also doesn't get security updates in the same way, it's probably best to follow http://docs.mailman3.org/en/latest/prodsetup.html or similar, and 213da82 is what I tried so far. With some more tinkering it's probably possible to make that work.

I haven't tried https://github.com/maxking/docker-mailman yet because it seems somewhat implausible to me that the total complexity will be lower, one still has to have a database and web server separately, and we don't want archives, so it'd pretty much just give us mailman3-core and the django stuff.

On setting up an email server (not just mailman), something very discouraging at https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-e-mail-server-with-dovecot:

Postfix has several hundred configuration parameters. If you want to administer a mail server that reliably delivers business requirements to a sizable organization, you should make yourself intimate with all of them (man 5 postconf). This tutorial will not be enough, on its own, to make you a competent professional email admin.

@domenic
Copy link
Member

domenic commented Dec 17, 2017

Oh, wow, thanks for looking into this further. Can you explain a bit more about the docker-mailman not working? It seems to have the database inside the container.

@foolip
Copy link
Member Author

foolip commented Dec 17, 2017

I haven't tried setting it up the Docker image. From my reading of https://hub.docker.com/r/alexbarcelo/mailman3/ last time, given the POSTGRES_HOST configuration I surmised that the database would be external, but maybe it'll use an internal one if not.

@foolip
Copy link
Member Author

foolip commented Jan 8, 2018

Filed #75 for what remains.

@foolip foolip closed this as completed Jan 8, 2018
@domenic
Copy link
Member

domenic commented Jan 8, 2018

Also filed #77

alice pushed a commit to alice/html that referenced this issue Jan 8, 2019
This is similar to whatwg/whatwg.org#113, and
stands in the way of new hosting for HTML:
whatwg/misc-server#7
alice pushed a commit to alice/html that referenced this issue Jan 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

6 participants