Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Los Angeles County, CA missing testing data since 2020-08-08 #118

Open
mikelehen opened this issue Sep 9, 2020 · 1 comment
Open

Los Angeles County, CA missing testing data since 2020-08-08 #118

mikelehen opened this issue Sep 9, 2020 · 1 comment

Comments

@mikelehen
Copy link

[also added to escalation tracker]

curl -X GET "https://api.covidcountydata.org/covid_us?location=eq.06037&variable=eq.positive_tests_total&offset=100&limit=100" \
 -H "Accept: application/json, application/vnd.pgrst.object+json, text/csv" \
 -H "Range-Unit: items" 

Returns data up through 2020-08-08 and then stops. Same for negative_tests_total and tests_total.

LA County dashboard seems to have data (I don't know if this is the best source): http://publichealth.lacounty.gov/media/coronavirus/data/index.htm

Can we get more recent data ingested?

@sglyon
Copy link
Member

sglyon commented Sep 9, 2020

Thanks for bringing this up.

The problem is that the server responsible for running our scrapers seems to have been blocked by the server hosting the LA county dashboard.

I just ran the scraper locally and it finished without an issue.

If you re-run your query you should see data through 2020-09-07.

Do you have experience getting around restrictions like our scraper server's IP being blocked? I've heard of things like rolling ip addresses, but haven't tried setting them up before.

Note that this applies to 5-10 of our scrapers, so we if find a solution for LA county we should see more reliable inserts for other datasets also

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants