Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find GitHub repo #5

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open

Find GitHub repo #5

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3fcd106
new python notebook for the task of pulling github repo information f…
MelanieIStefan Feb 29, 2016
21c1cbf
try to create new ipynb and save
MelanieIStefan Mar 11, 2016
bda1d7b
wrote up a list of aims
MelanieIStefan Mar 11, 2016
27282c6
change to data directory; list content
MelanieIStefan Mar 11, 2016
09212e1
libraries for xml parsing
MelanieIStefan Mar 15, 2016
cd71f02
successfully imported xml and created data frameo
MelanieIStefan Mar 15, 2016
bb141f1
find github url string from abstract
MelanieIStefan Mar 15, 2016
a33c210
start iterating through rows of abstract, but get stuck
MelanieIStefan Mar 16, 2016
b466633
finding github url for all abstracts and saving them to data frame
MelanieIStefan Mar 17, 2016
d705525
started playing around with pygithub
MelanieIStefan Mar 17, 2016
48ced8c
refrain from telling the world my password
MelanieIStefan Mar 17, 2016
282aa38
look into extracting github project from url (might need to just do a…
MelanieIStefan Mar 24, 2016
71154e6
play aorund with urlparse
MelanieIStefan Apr 1, 2016
5ae4b18
extracting github project works for all urls of the form https://gith…
MelanieIStefan Apr 6, 2016
dd67f23
github project extraction seems to work for most, but not all urls
MelanieIStefan Apr 6, 2016
3e967b3
select only urls that actually point to github projects
MelanieIStefan Apr 7, 2016
25310bf
can parse most github urls now
MelanieIStefan Apr 13, 2016
e352b11
confirmed that we can get a github repo for almost 29% of entries
MelanieIStefan Apr 14, 2016
608566c
(and 89% of abstracts with have a github url)
MelanieIStefan Apr 14, 2016
bf0e1f7
start looking into how to get contributor info
MelanieIStefan Apr 14, 2016
f811aa0
OK, I can get contributor names from github repos. Exceeding rate lim…
MelanieIStefan May 3, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 9 additions & 19 deletions src/find_github_repo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,15 +510,15 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from github import Github\n",
"\n",
"g = Github(\"MelanieIStefan\", \"hn8PJuM\")\n"
"g = Github()\n"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unfortunately still present in the history since you made a commit with it. I bet you can close out this key though so it can't be used in the future...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, did that yesterday, when you alerted me to the problem :)

]
},
{
Expand Down Expand Up @@ -555,7 +555,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 5,
"metadata": {
"collapsed": false
},
Expand All @@ -575,31 +575,21 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"ename": "UnknownObjectException",
"evalue": "404 {u'documentation_url': u'https://developer.github.com/v3', u'message': u'Not Found'}",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mUnknownObjectException\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m<ipython-input-12-6cba399cea58>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[0marchtk\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mg\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_repo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"archtk\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[1;32mprint\u001b[0m \u001b[0marchtk\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdescription\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/Repository.pyc\u001b[0m in \u001b[0;36mdescription\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m 185\u001b[0m \u001b[1;33m:\u001b[0m\u001b[0mtype\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mstring\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 186\u001b[0m \"\"\"\n\u001b[1;32m--> 187\u001b[1;33m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_completeIfNotSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_description\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 188\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_description\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalue\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 189\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/GithubObject.pyc\u001b[0m in \u001b[0;36m_completeIfNotSet\u001b[1;34m(self, value)\u001b[0m\n\u001b[0;32m 226\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_completeIfNotSet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 227\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mvalue\u001b[0m \u001b[1;32mis\u001b[0m \u001b[0mNotSet\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 228\u001b[1;33m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_completeIfNeeded\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 229\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 230\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_completeIfNeeded\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/GithubObject.pyc\u001b[0m in \u001b[0;36m_completeIfNeeded\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m 230\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_completeIfNeeded\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 231\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__completed\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 232\u001b[1;33m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__complete\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 233\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 234\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m__complete\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/GithubObject.pyc\u001b[0m in \u001b[0;36m__complete\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m 235\u001b[0m headers, data = self._requester.requestJsonAndCheck(\n\u001b[0;32m 236\u001b[0m \u001b[1;34m\"GET\"\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 237\u001b[1;33m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_url\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalue\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 238\u001b[0m )\n\u001b[0;32m 239\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_storeAndUseAttributes\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mheaders\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/Requester.pyc\u001b[0m in \u001b[0;36mrequestJsonAndCheck\u001b[1;34m(self, verb, url, parameters, headers, input, cnx)\u001b[0m\n\u001b[0;32m 169\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 170\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mrequestJsonAndCheck\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mverb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minput\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcnx\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 171\u001b[1;33m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__check\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrequestJson\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mverb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcnx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 172\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 173\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mrequestMultipartAndCheck\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mverb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0murl\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minput\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m/usr/local/lib/python2.7/dist-packages/github/Requester.pyc\u001b[0m in \u001b[0;36m__check\u001b[1;34m(self, status, responseHeaders, output)\u001b[0m\n\u001b[0;32m 177\u001b[0m \u001b[0moutput\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__structuredFromJson\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0moutput\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 178\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mstatus\u001b[0m \u001b[1;33m>=\u001b[0m \u001b[1;36m400\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 179\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__createException\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstatus\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mresponseHeaders\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 180\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mresponseHeaders\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 181\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;31mUnknownObjectException\u001b[0m: 404 {u'documentation_url': u'https://developer.github.com/v3', u'message': u'Not Found'}"
"name": "stdout",
"output_type": "stream",
"text": [
"Computation of Sensitivities in Model ODE Systems\n"
]
}
],
"source": [
"cosmos = g.get_repo(\"\")\n",
"cosmos = g.get_repo(\"BioprocessdesignLab/COSMOS\")\n",
"print cosmos.description"
]
}
Expand Down