forked from swcarpentry/python-novice-inflammation
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy path01-numpy.html
489 lines (475 loc) · 42.3 KB
/
01-numpy.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="generator" content="pandoc">
<title>Software Carpentry: Programming with Python</title>
<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap.css" />
<link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap-theme.css" />
<link rel="stylesheet" type="text/css" href="css/swc.css" />
<link rel="alternate" type="application/rss+xml" title="Software Carpentry Blog" href="http://software-carpentry.org/feed.xml"/>
<meta charset="UTF-8" />
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body class="lesson">
<div class="container card">
<div class="banner">
<a href="http://software-carpentry.org" title="Software Carpentry">
<img alt="Software Carpentry banner" src="img/software-carpentry-banner.png" />
</a>
</div>
<article>
<div class="row">
<div class="col-md-10 col-md-offset-1">
<a href="index.html"><h1 class="title">Programming with Python</h1></a>
<h2 class="subtitle">Analyzing Patient Data</h2>
<section class="objectives panel panel-warning">
<div class="panel-heading">
<h2 id="learning-objectives"><span class="glyphicon glyphicon-certificate"></span>Learning Objectives</h2>
</div>
<div class="panel-body">
<ul>
<li>Explain what a library is, and what libraries are used for.</li>
<li>Import a Python library and use the things it contains.</li>
<li>Read tabular data from a file into a program.</li>
<li>Assign values to variables.</li>
<li>Select individual values and subsections from data.</li>
<li>Perform operations on arrays of data.</li>
<li>Display simple graphs.</li>
</ul>
</div>
</section>
<p>Words are useful, but what’s more useful are the sentences and stories we build with them. Similarly, while a lot of powerful, general tools are built into languages like Python, specialized tools built up from these basic units live in <a href="reference.html#library">libraries</a> that can be called upon when needed.</p>
<p>In order to load our inflammation data, we need to access (<a href="reference.html#import">import</a> in Python terminology) a library called <a href="http://docs.scipy.org/doc/numpy/" title="NumPy Documentation">NumPy</a>. In general you should use this library if you want to do fancy things with numbers, especially if you have matrices or arrays. We can import NumPy using:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">import</span> numpy</code></pre></div>
<p>Importing a library is like getting a piece of lab equipment out of a storage locker and setting it up on the bench. Libraries provide additional functionality to the basic Python package, much like a new piece of equipment adds functionality to a lab space. Once you’ve imported the library, we can ask the library to read our data file for us:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">numpy.loadtxt(fname<span class="op">=</span><span class="st">'inflammation-01.csv'</span>, delimiter<span class="op">=</span><span class="st">','</span>)</code></pre></div>
<pre class="output"><code>array([[ 0., 0., 1., ..., 3., 0., 0.],
[ 0., 1., 2., ..., 1., 0., 1.],
[ 0., 1., 1., ..., 2., 1., 1.],
...,
[ 0., 1., 1., ..., 1., 1., 1.],
[ 0., 0., 0., ..., 0., 2., 0.],
[ 0., 0., 1., ..., 1., 1., 0.]])</code></pre>
<p>The expression <code>numpy.loadtxt(...)</code> is a <a href="reference.html#function-call">function call</a> that asks Python to run the <a href="reference.html#function">function</a> <code>loadtxt</code> which belongs to the <code>numpy</code> library. This <a href="reference.html#dotted-notation">dotted notation</a> is used everywhere in Python to refer to the parts of things as <code>thing.component</code>.</p>
<p><code>numpy.loadtxt</code> has two <a href="reference.html#parameter">parameters</a>: the name of the file we want to read, and the <a href="reference.html#delimiter">delimiter</a> that separates values on a line. These both need to be character strings (or <a href="reference.html#string">strings</a> for short), so we put them in quotes.</p>
<p>When we are finished typing and press Shift+Enter, the notebook runs our command. Since we haven’t told it to do anything else with the function’s output, the notebook displays it. In this case, that output is the data we just loaded. By default, only a few rows and columns are shown (with <code>...</code> to omit elements when displaying big arrays). To save space, Python displays numbers as <code>1.</code> instead of <code>1.0</code> when there’s nothing interesting after the decimal point.</p>
<p>Our call to <code>numpy.loadtxt</code> read our file, but didn’t save the data in memory. To do that, we need to <a href="reference.html#assignment">assign</a> the array to a <a href="reference.html#variable">variable</a>. A variable is just a name for a value, such as <code>x</code>, <code>current_temperature</code>, or <code>subject_id</code>. Python’s variables must begin with a letter and are <a href="reference.html#case-sensitive">case sensitive</a>. We can create a new variable by assigning a value to it using <code>=</code>. As an illustration, let’s step back and instead of considering a table of data, consider the simplest “collection” of data, a single value. The line below assigns the value <code>55</code> to a variable <code>weight_kg</code>:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">weight_kg <span class="op">=</span> <span class="dv">55</span></code></pre></div>
<p>Once a variable has a value, we can print it to the screen:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(weight_kg)</code></pre></div>
<pre class="output"><code>55</code></pre>
<p>and do arithmetic with it:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'weight in pounds:'</span>, <span class="fl">2.2</span> <span class="op">*</span> weight_kg)</code></pre></div>
<pre class="output"><code>weight in pounds: 121.0</code></pre>
<p>As the example above shows, we can print several things at once by separating them with commas.</p>
<p>We can also change a variable’s value by assigning it a new one:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">weight_kg <span class="op">=</span> <span class="fl">57.5</span>
<span class="bu">print</span>(<span class="st">'weight in kilograms is now:'</span>, weight_kg)</code></pre></div>
<pre class="output"><code>weight in kilograms is now: 57.5</code></pre>
<p>If we imagine the variable as a sticky note with a name written on it, assignment is like putting the sticky note on a particular value:</p>
<div class="figure">
<img src="fig/python-sticky-note-variables-01.svg" alt="Variables as Sticky Notes" />
<p class="caption">Variables as Sticky Notes</p>
</div>
<p>This means that assigning a value to one variable does <em>not</em> change the values of other variables. For example, let’s store the subject’s weight in pounds in a variable:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">weight_lb <span class="op">=</span> <span class="fl">2.2</span> <span class="op">*</span> weight_kg
<span class="bu">print</span>(<span class="st">'weight in kilograms:'</span>, weight_kg, <span class="st">'and in pounds:'</span>, weight_lb)</code></pre></div>
<pre class="output"><code>weight in kilograms: 57.5 and in pounds: 126.5</code></pre>
<div class="figure">
<img src="fig/python-sticky-note-variables-02.svg" alt="Creating Another Variable" />
<p class="caption">Creating Another Variable</p>
</div>
<p>and then change <code>weight_kg</code>:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">weight_kg <span class="op">=</span> <span class="fl">100.0</span>
<span class="bu">print</span>(<span class="st">'weight in kilograms is now:'</span>, weight_kg, <span class="st">'and weight in pounds is still:'</span>, weight_lb)</code></pre></div>
<pre class="output"><code>weight in kilograms is now: 100.0 and weight in pounds is still: 126.5</code></pre>
<div class="figure">
<img src="fig/python-sticky-note-variables-03.svg" alt="Updating a Variable" />
<p class="caption">Updating a Variable</p>
</div>
<p>Since <code>weight_lb</code> doesn’t “remember” where its value came from, it isn’t automatically updated when <code>weight_kg</code> changes. This is different from the way spreadsheets work.</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="whos-who-in-the-memory"><span class="glyphicon glyphicon-pushpin"></span>Who’s who in the memory</h2>
</div>
<div class="panel-body">
<p>You can use the <code>%whos</code> command at any time to see what variables you have created and what modules you have loaded into the computer’s memory. As this is an IPython command, it will only work if you are in an IPython terminal or the Jupyter Notebook.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="op">%</span>whos</code></pre></div>
<pre class="output"><code>Variable Type Data/Info
--------------------------------
numpy module <module 'numpy' from '/Us<...>kages/numpy/__init__.py'>
weight_kg float 100.0
weight_lb float 126.5</code></pre>
</div>
</aside>
<p>Just as we can assign a single value to a variable, we can also assign an array of values to a variable using the same syntax. Let’s re-run <code>numpy.loadtxt</code> and save its result:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">data <span class="op">=</span> numpy.loadtxt(fname<span class="op">=</span><span class="st">'inflammation-01.csv'</span>, delimiter<span class="op">=</span><span class="st">','</span>)</code></pre></div>
<p>This statement doesn’t produce any output because assignment doesn’t display anything. If we want to check that our data has been loaded, we can print the variable’s value:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(data)</code></pre></div>
<pre class="output"><code>[[ 0. 0. 1. ..., 3. 0. 0.]
[ 0. 1. 2. ..., 1. 0. 1.]
[ 0. 1. 1. ..., 2. 1. 1.]
...,
[ 0. 1. 1. ..., 1. 1. 1.]
[ 0. 0. 0. ..., 0. 2. 0.]
[ 0. 0. 1. ..., 1. 1. 0.]]</code></pre>
<p>Now that our data is in memory, we can start doing things with it. First, let’s ask what <a href="reference.html#type">type</a> of thing <code>data</code> refers to:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="bu">type</span>(data))</code></pre></div>
<pre class="output"><code><class 'numpy.ndarray'></code></pre>
<p>The output tells us that <code>data</code> currently refers to an N-dimensional array created by the NumPy library. These data correspond to arthritis patients’ inflammation. The rows are the individual patients and the columns are their daily inflammation measurements.</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="data-type"><span class="glyphicon glyphicon-pushpin"></span>Data Type</h2>
</div>
<div class="panel-body">
<p>A Numpy array contains one or more elements of the same type. <code>type</code> will only tell you that a variable is a NumPy array. We can also find the out the type of the data contained in the NumPy array.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(data.dtype)</code></pre></div>
<pre class="output"><code>dtype('float64')</code></pre>
<p>This tells us that the NumPy array’s elements are <a href="reference.html#floating-point%20number">floating-point numbers</a>.</p>
</div>
</aside>
<p>We can see what the array’s <a href="reference.html#shape">shape</a> is like this:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(data.shape)</code></pre></div>
<pre class="output"><code>(60, 40)</code></pre>
<p>This tells us that <code>data</code> has 60 rows and 40 columns. When we created the variable <code>data</code> to store our arthritis data, we didn’t just create the array, we also created information about the array, called <a href="reference.html#member">members</a> or attributes. This extra information describes <code>data</code> in the same way an adjective describes a noun. <code>data.shape</code> is an attribute of <code>data</code> which describes the dimensions of <code>data</code>. We use the same dotted notation for the attributes of variables that we use for the functions in libraries because they have the same part-and-whole relationship.</p>
<p>If we want to get a single number from the array, we must provide an <a href="reference.html#index">index</a> in square brackets, just as we do in math:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'first value in data:'</span>, data[<span class="dv">0</span>, <span class="dv">0</span>])</code></pre></div>
<pre class="output"><code>first value in data: 0.0</code></pre>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'middle value in data:'</span>, data[<span class="dv">30</span>, <span class="dv">20</span>])</code></pre></div>
<pre class="output"><code>middle value in data: 13.0</code></pre>
<p>The expression <code>data[30, 20]</code> may not surprise you, but <code>data[0, 0]</code> might. Programming languages like Fortran and MATLAB start counting at 1, because that’s what human beings have done for thousands of years. Languages in the C family (including C++, Java, Perl, and Python) count from 0 because that’s more convenient when indices are computed rather than constant (see <a href="http://exple.tive.org/blarg/2013/10/22/citation-needed/">Mike Hoye’s blog post</a> for historical details). As a result, if we have an M×N array in Python, its indices go from 0 to M-1 on the first axis and 0 to N-1 on the second. It takes a bit of getting used to, but one way to remember the rule is that the index is how many steps we have to take from the start to get the item we want.</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="in-the-corner"><span class="glyphicon glyphicon-pushpin"></span>In the Corner</h2>
</div>
<div class="panel-body">
<p>What may also surprise you is that when Python displays an array, it shows the element with index <code>[0, 0]</code> in the upper left corner rather than the lower left. This is consistent with the way mathematicians draw matrices, but different from the Cartesian coordinates. The indices are (row, column) instead of (column, row) for the same reason, which can be confusing when plotting data.</p>
</div>
</aside>
<p>An index like <code>[30, 20]</code> selects a single element of an array, but we can select whole sections as well. For example, we can select the first ten days (columns) of values for the first four patients (rows) like this:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(data[<span class="dv">0</span>:<span class="dv">4</span>, <span class="dv">0</span>:<span class="dv">10</span>])</code></pre></div>
<pre class="output"><code>[[ 0. 0. 1. 3. 1. 2. 4. 7. 8. 3.]
[ 0. 1. 2. 1. 2. 1. 3. 2. 2. 6.]
[ 0. 1. 1. 3. 3. 2. 6. 2. 5. 9.]
[ 0. 0. 2. 0. 4. 2. 2. 1. 6. 7.]]</code></pre>
<p>The <a href="reference.html#slice">slice</a> <code>0:4</code> means, “Start at index 0 and go up to, but not including, index 4.” Again, the up-to-but-not-including takes a bit of getting used to, but the rule is that the difference between the upper and lower bounds is the number of values in the slice.</p>
<p>We don’t have to start slices at 0:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(data[<span class="dv">5</span>:<span class="dv">10</span>, <span class="dv">0</span>:<span class="dv">10</span>])</code></pre></div>
<pre class="output"><code>[[ 0. 0. 1. 2. 2. 4. 2. 1. 6. 4.]
[ 0. 0. 2. 2. 4. 2. 2. 5. 5. 8.]
[ 0. 0. 1. 2. 3. 1. 2. 3. 5. 3.]
[ 0. 0. 0. 3. 1. 5. 6. 5. 5. 8.]
[ 0. 1. 1. 2. 1. 3. 5. 3. 5. 8.]]</code></pre>
<p>We also don’t have to include the upper and lower bound on the slice. If we don’t include the lower bound, Python uses 0 by default; if we don’t include the upper, the slice runs to the end of the axis, and if we don’t include either (i.e., if we just use ‘:’ on its own), the slice includes everything:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">small <span class="op">=</span> data[:<span class="dv">3</span>, <span class="dv">36</span>:]
<span class="bu">print</span>(<span class="st">'small is:'</span>)
<span class="bu">print</span>(small)</code></pre></div>
<pre class="output"><code>small is:
[[ 2. 3. 0. 0.]
[ 1. 1. 0. 1.]
[ 2. 2. 1. 1.]]</code></pre>
<p>Arrays also know how to perform common mathematical operations on their values. The simplest operations with data are arithmetic: add, subtract, multiply, and divide. When you do such operations on arrays, the operation is done on each individual element of the array. Thus:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">doubledata <span class="op">=</span> data <span class="op">*</span> <span class="fl">2.0</span></code></pre></div>
<p>will create a new array <code>doubledata</code> whose elements have the value of two times the value of the corresponding elements in <code>data</code>:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'original:'</span>)
<span class="bu">print</span>(data[:<span class="dv">3</span>, <span class="dv">36</span>:])
<span class="bu">print</span>(<span class="st">'doubledata:'</span>)
<span class="bu">print</span>(doubledata[:<span class="dv">3</span>, <span class="dv">36</span>:])</code></pre></div>
<pre class="output"><code>original:
[[ 2. 3. 0. 0.]
[ 1. 1. 0. 1.]
[ 2. 2. 1. 1.]]
doubledata:
[[ 4. 6. 0. 0.]
[ 2. 2. 0. 2.]
[ 4. 4. 2. 2.]]</code></pre>
<p>If, instead of taking an array and doing arithmetic with a single value (as above) you did the arithmetic operation with another array of the same shape, the operation will be done on corresponding elements of the two arrays. Thus:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">tripledata <span class="op">=</span> doubledata <span class="op">+</span> data</code></pre></div>
<p>will give you an array where <code>tripledata[0,0]</code> will equal <code>doubledata[0,0]</code> plus <code>data[0,0]</code>, and so on for all other elements of the arrays.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'tripledata:'</span>)
<span class="bu">print</span>(tripledata[:<span class="dv">3</span>, <span class="dv">36</span>:])</code></pre></div>
<pre class="output"><code>tripledata:
[[ 6. 9. 0. 0.]
[ 3. 3. 0. 3.]
[ 6. 6. 3. 3.]]</code></pre>
<p>Often, we want to do more than add, subtract, multiply, and divide values of data. NumPy knows how to do more complex operations on arrays. If we want to find the average inflammation for all patients on all days, for example, we can ask NumPy to compute <code>data</code>’s mean value:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(numpy.mean(data))</code></pre></div>
<pre class="output"><code>6.14875</code></pre>
<p><code>mean</code> is a <a href="reference.html#function">function</a> that takes an array as an <a href="reference.html#argument">argument</a>. If variables are nouns, functions are verbs: they do things with variables.</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="not-all-functions-have-input"><span class="glyphicon glyphicon-pushpin"></span>Not all functions have input</h2>
</div>
<div class="panel-body">
<p>Generally, a function uses inputs to produce outputs. However, some functions produce outputs without needing any input. For example, checking the current time doesn’t require any input.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">import</span> time
<span class="bu">print</span>(time.ctime())</code></pre></div>
<pre class="output"><code>'Sat Mar 26 13:07:33 2016'</code></pre>
<p>For functions that don’t take in any arguments, we still need parentheses (<code>()</code>) to tell Python to go and do something for us.</p>
</div>
</aside>
<p>NumPy has lots of useful functions that take an array as input. Let’s use three of those functions to get some descriptive values about the dataset. We’ll also use multiple assignment, a convenient Python feature that will enable us to do this all in one line.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">maxval, minval, stdval <span class="op">=</span> numpy.<span class="bu">max</span>(data), numpy.<span class="bu">min</span>(data), numpy.std(data)
<span class="bu">print</span>(<span class="st">'maximum inflammation:'</span>, maxval)
<span class="bu">print</span>(<span class="st">'minimum inflammation:'</span>, minval)
<span class="bu">print</span>(<span class="st">'standard deviation:'</span>, stdval)</code></pre></div>
<pre class="output"><code>maximum inflammation: 20.0
minimum inflammation: 0.0
standard deviation: 4.61383319712</code></pre>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="mystery-functions-in-ipython"><span class="glyphicon glyphicon-pushpin"></span>Mystery functions in IPython</h2>
</div>
<div class="panel-body">
<p>How did we know what functions NumPy has and how to use them? If you are working in the IPython/Jupyter Notebook there is an easy way to find out. If you type the name of something with a full-stop then you can use tab completion (e.g. type <code>numpy.</code> and then press tab) to see a list of all functions and attributes that you can use. After selecting one you can also add a question mark (e.g. <code>numpy.cumprod?</code>) and IPython will return an explanation of the method! This is the same as doing <code>help(numpy.cumprod)</code>.</p>
</div>
</aside>
<p>When analyzing data, though, we often want to look at partial statistics, such as the maximum value per patient or the average value per day. One way to do this is to create a new temporary array of the data we want, then ask it to do the calculation:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">patient_0 <span class="op">=</span> data[<span class="dv">0</span>, :] <span class="co"># 0 on the first axis, everything on the second</span>
<span class="bu">print</span>(<span class="st">'maximum inflammation for patient 0:'</span>, patient_0.<span class="bu">max</span>())</code></pre></div>
<pre class="output"><code>maximum inflammation for patient 0: 18.0</code></pre>
<p>Everything in a line of code following the ‘#’ symbol is a <a href="reference.html#comment">comment</a> that is ignored by the computer. Comments allow programmers to leave explanatory notes for other programmers or their future selves.</p>
<p>We don’t actually need to store the row in a variable of its own. Instead, we can combine the selection and the function call:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(<span class="st">'maximum inflammation for patient 2:'</span>, numpy.<span class="bu">max</span>(data[<span class="dv">2</span>, :]))</code></pre></div>
<pre class="output"><code>maximum inflammation for patient 2: 19.0</code></pre>
<p>What if we need the maximum inflammation for <em>all</em> patients (as in the next diagram on the left), or the average for each day (as in the diagram on the right)? As the diagram below shows, we want to perform the operation across an axis:</p>
<div class="figure">
<img src="fig/python-operations-across-axes.png" alt="Operations Across Axes" />
<p class="caption">Operations Across Axes</p>
</div>
<p>To support this, most array functions allow us to specify the axis we want to work on. If we ask for the average across axis 0 (rows in our 2D example), we get:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(numpy.mean(data, axis<span class="op">=</span><span class="dv">0</span>))</code></pre></div>
<pre class="output"><code>[ 0. 0.45 1.11666667 1.75 2.43333333 3.15
3.8 3.88333333 5.23333333 5.51666667 5.95 5.9
8.35 7.73333333 8.36666667 9.5 9.58333333
10.63333333 11.56666667 12.35 13.25 11.96666667
11.03333333 10.16666667 10. 8.66666667 9.15 7.25
7.33333333 6.58333333 6.06666667 5.95 5.11666667 3.6
3.3 3.56666667 2.48333333 1.5 1.13333333
0.56666667]</code></pre>
<p>As a quick check, we can ask this array what its shape is:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(numpy.mean(data, axis<span class="op">=</span><span class="dv">0</span>).shape)</code></pre></div>
<pre class="output"><code>(40,)</code></pre>
<p>The expression <code>(40,)</code> tells us we have an N×1 vector, so this is the average inflammation per day for all patients. If we average across axis 1 (columns in our 2D example), we get:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="bu">print</span>(numpy.mean(data, axis<span class="op">=</span><span class="dv">1</span>))</code></pre></div>
<pre class="output"><code>[ 5.45 5.425 6.1 5.9 5.55 6.225 5.975 6.65 6.625 6.525
6.775 5.8 6.225 5.75 5.225 6.3 6.55 5.7 5.85 6.55
5.775 5.825 6.175 6.1 5.8 6.425 6.05 6.025 6.175 6.55
6.175 6.35 6.725 6.125 7.075 5.725 5.925 6.15 6.075 5.75
5.975 5.725 6.3 5.9 6.75 5.925 7.225 6.15 5.95 6.275 5.7
6.1 6.825 5.975 6.725 5.7 6.25 6.4 7.05 5.9 ]</code></pre>
<p>which is the average inflammation per patient across all days.</p>
<p>The mathematician Richard Hamming once said, “The purpose of computing is insight, not numbers,” and the best way to develop insight is often to visualize data. Visualization deserves an entire lecture (or course) of its own, but we can explore a few features of Python’s <code>matplotlib</code> library here. While there is no “official” plotting library, this package is the de facto standard. First, we will import the <code>pyplot</code> module from <code>matplotlib</code> and use two of its functions to create and display a heat map of our data:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">import</span> matplotlib.pyplot
image <span class="op">=</span> matplotlib.pyplot.imshow(data)
matplotlib.pyplot.show()</code></pre></div>
<div class="figure">
<img src="fig/01-numpy_71_0.png" alt="Heatmap of the Data" />
<p class="caption">Heatmap of the Data</p>
</div>
<p>Blue regions in this heat map are low values, while red shows high values. As we can see, inflammation rises and falls over a 40-day period.</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="some-ipython-magic"><span class="glyphicon glyphicon-pushpin"></span>Some IPython magic</h2>
</div>
<div class="panel-body">
<p>If you’re using an IPython / Jupyter notebook, you’ll need to execute the following command in order for your matplotlib images to appear in the notebook when <code>show()</code> is called:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="op">%</span> matplotlib inline</code></pre></div>
<p>The <code>%</code> indicates an IPython magic function - a function that is only valid within the notebook environment. Note that you only have to execute this function once per notebook.</p>
</div>
</aside>
<p>Let’s take a look at the average inflammation over time:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">ave_inflammation <span class="op">=</span> numpy.mean(data, axis<span class="op">=</span><span class="dv">0</span>)
ave_plot <span class="op">=</span> matplotlib.pyplot.plot(ave_inflammation)
matplotlib.pyplot.show()</code></pre></div>
<div class="figure">
<img src="fig/01-numpy_73_0.png" alt="Average Inflammation Over Time" />
<p class="caption">Average Inflammation Over Time</p>
</div>
<p>Here, we have put the average per day across all patients in the variable <code>ave_inflammation</code>, then asked <code>matplotlib.pyplot</code> to create and display a line graph of those values. The result is roughly a linear rise and fall, which is suspicious: based on other studies, we expect a sharper rise and slower fall. Let’s have a look at two other statistics:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">max_plot <span class="op">=</span> matplotlib.pyplot.plot(numpy.<span class="bu">max</span>(data, axis<span class="op">=</span><span class="dv">0</span>))
matplotlib.pyplot.show()</code></pre></div>
<div class="figure">
<img src="fig/01-numpy_75_1.png" alt="Maximum Value Along The First Axis" />
<p class="caption">Maximum Value Along The First Axis</p>
</div>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">min_plot <span class="op">=</span> matplotlib.pyplot.plot(numpy.<span class="bu">min</span>(data, axis<span class="op">=</span><span class="dv">0</span>))
matplotlib.pyplot.show()</code></pre></div>
<div class="figure">
<img src="fig/01-numpy_75_3.png" alt="Minimum Value Along The First Axis" />
<p class="caption">Minimum Value Along The First Axis</p>
</div>
<p>The maximum value rises and falls perfectly smoothly, while the minimum seems to be a step function. Neither result seems particularly likely, so either there’s a mistake in our calculations or something is wrong with our data. This insight would have been difficult to reach by examining the data without visualization tools.</p>
<p>You can group similar plots in a single figure using subplots. This script below uses a number of new commands. The function <code>matplotlib.pyplot.figure()</code> creates a space into which we will place all of our plots. The parameter <code>figsize</code> tells Python how big to make this space. Each subplot is placed into the figure using its <code>add_subplot</code> <a href="reference.html#method">method</a>. The <code>add_subplot</code> method takes 3 parameters. The first denotes how many total rows of subplots there are, the second parameter refers to the total number of subplot columns, and the final parameter denotes which subplot your variable is referencing (left-to-right, top-to-bottom). Each subplot is stored in a different variable (<code>axes1</code>, <code>axes2</code>, <code>axes3</code>). Once a subplot is created, the axes can be titled using the <code>set_xlabel()</code> command (or <code>set_ylabel()</code>). Here are our three plots side by side:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">import</span> numpy
<span class="im">import</span> matplotlib.pyplot
data <span class="op">=</span> numpy.loadtxt(fname<span class="op">=</span><span class="st">'inflammation-01.csv'</span>, delimiter<span class="op">=</span><span class="st">','</span>)
fig <span class="op">=</span> matplotlib.pyplot.figure(figsize<span class="op">=</span>(<span class="fl">10.0</span>, <span class="fl">3.0</span>))
axes1 <span class="op">=</span> fig.add_subplot(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">1</span>)
axes2 <span class="op">=</span> fig.add_subplot(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">2</span>)
axes3 <span class="op">=</span> fig.add_subplot(<span class="dv">1</span>, <span class="dv">3</span>, <span class="dv">3</span>)
axes1.set_ylabel(<span class="st">'average'</span>)
axes1.plot(numpy.mean(data, axis<span class="op">=</span><span class="dv">0</span>))
axes2.set_ylabel(<span class="st">'max'</span>)
axes2.plot(numpy.<span class="bu">max</span>(data, axis<span class="op">=</span><span class="dv">0</span>))
axes3.set_ylabel(<span class="st">'min'</span>)
axes3.plot(numpy.<span class="bu">min</span>(data, axis<span class="op">=</span><span class="dv">0</span>))
fig.tight_layout()
matplotlib.pyplot.show()</code></pre></div>
<div class="figure">
<img src="fig/01-numpy_80_0.png" alt="The Previous Plots as Subplots" />
<p class="caption">The Previous Plots as Subplots</p>
</div>
<p>The <a href="reference.html#function-call">call</a> to <code>loadtxt</code> reads our data, and the rest of the program tells the plotting library how large we want the figure to be, that we’re creating three subplots, what to draw for each one, and that we want a tight layout. (Perversely, if we leave out that call to <code>fig.tight_layout()</code>, the graphs will actually be squeezed together more closely.)</p>
<aside class="callout panel panel-info">
<div class="panel-heading">
<h2 id="scientists-dislike-typing"><span class="glyphicon glyphicon-pushpin"></span>Scientists dislike typing</h2>
</div>
<div class="panel-body">
<p>We will always use the syntax <code>import numpy</code> to import NumPy. However, in order to save typing, it is <a href="http://www.scipy.org/getting-started.html#an-example-script">often suggested</a> to make a shortcut like so: <code>import numpy as np</code>. If you ever see Python code online using a NumPy function with <code>np</code> (for example, <code>np.loadtxt(...)</code>), it’s because they’ve used this shortcut.</p>
</div>
</aside>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="check-your-understanding"><span class="glyphicon glyphicon-pencil"></span>Check your understanding</h2>
</div>
<div class="panel-body">
<p>Draw diagrams showing what variables refer to what values after each statement in the following program:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">mass <span class="op">=</span> <span class="fl">47.5</span>
age <span class="op">=</span> <span class="dv">122</span>
mass <span class="op">=</span> mass <span class="op">*</span> <span class="fl">2.0</span>
age <span class="op">=</span> age <span class="op">-</span> <span class="dv">20</span></code></pre></div>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="sorting-out-references"><span class="glyphicon glyphicon-pencil"></span>Sorting out references</h2>
</div>
<div class="panel-body">
<p>What does the following program print out?</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">first, second <span class="op">=</span> <span class="st">'Grace'</span>, <span class="st">'Hopper'</span>
third, fourth <span class="op">=</span> second, first
<span class="bu">print</span>(third, fourth)</code></pre></div>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="slicing-strings"><span class="glyphicon glyphicon-pencil"></span>Slicing strings</h2>
</div>
<div class="panel-body">
<p>A section of an array is called a <a href="reference.html#slice">slice</a>. We can take slices of character strings as well:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">element <span class="op">=</span> <span class="st">'oxygen'</span>
<span class="bu">print</span>(<span class="st">'first three characters:'</span>, element[<span class="dv">0</span>:<span class="dv">3</span>])
<span class="bu">print</span>(<span class="st">'last three characters:'</span>, element[<span class="dv">3</span>:<span class="dv">6</span>])</code></pre></div>
<pre class="output"><code>first three characters: oxy
last three characters: gen</code></pre>
<p>What is the value of <code>element[:4]</code>? What about <code>element[4:]</code>? Or <code>element[:]</code>?</p>
<p>What is <code>element[-1]</code>? What is <code>element[-2]</code>? Given those answers, explain what <code>element[1:-1]</code> does.</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="thin-slices"><span class="glyphicon glyphicon-pencil"></span>Thin slices</h2>
</div>
<div class="panel-body">
<p>The expression <code>element[3:3]</code> produces an <a href="reference.html#empty-string">empty string</a>, i.e., a string that contains no characters. If <code>data</code> holds our array of patient data, what does <code>data[3:3, 4:4]</code> produce? What about <code>data[3:3, :]</code>?</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="check-your-understanding-plot-scaling"><span class="glyphicon glyphicon-pencil"></span>Check your understanding: plot scaling</h2>
</div>
<div class="panel-body">
<p>Why do all of our plots stop just short of the upper end of our graph? If we want to change this, we can use the <code>set_ylim(min, max)</code> method of each ‘axes’, for example:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python">axes3.set_ylim(<span class="dv">0</span>,<span class="dv">6</span>)</code></pre></div>
<p>Update your plotting code to automatically set a more appropriate scale (hint: you can make use of the <code>max</code> and <code>min</code> methods to help)</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="check-your-understanding-drawing-straight-lines"><span class="glyphicon glyphicon-pencil"></span>Check your understanding: drawing straight lines</h2>
</div>
<div class="panel-body">
<p>Why are the vertical lines in our plot of the minimum inflammation per day not perfectly vertical?</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="make-your-own-plot"><span class="glyphicon glyphicon-pencil"></span>Make your own plot</h2>
</div>
<div class="panel-body">
<p>Create a plot showing the standard deviation (<code>numpy.std</code>) of the inflammation data for each day across all patients.</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="moving-plots-around"><span class="glyphicon glyphicon-pencil"></span>Moving plots around</h2>
</div>
<div class="panel-body">
<p>Modify the program to display the three plots on top of one another instead of side by side.</p>
</div>
</section>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="stacking-arrays"><span class="glyphicon glyphicon-pencil"></span>Stacking arrays</h2>
</div>
<div class="panel-body">
<p>Arrays can be concatenated and stacked on top of one another, using NumPy’s <code>vstack</code> and <code>hstack</code> functions for vertical and horizontal stacking, respectively.</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">import</span> numpy
A <span class="op">=</span> numpy.array([[<span class="dv">1</span>,<span class="dv">2</span>,<span class="dv">3</span>], [<span class="dv">4</span>,<span class="dv">5</span>,<span class="dv">6</span>], [<span class="dv">7</span>, <span class="dv">8</span>, <span class="dv">9</span>]])
<span class="bu">print</span>(<span class="st">'A = '</span>)
<span class="bu">print</span>(A)
B <span class="op">=</span> numpy.hstack([A, A])
<span class="bu">print</span>(<span class="st">'B = '</span>)
<span class="bu">print</span>(B)
C <span class="op">=</span> numpy.vstack([A, A])
<span class="bu">print</span>(<span class="st">'C = '</span>)
<span class="bu">print</span>(C)</code></pre></div>
<pre class="output"><code>A =
[[1 2 3]
[4 5 6]
[7 8 9]]
B =
[[1 2 3 1 2 3]
[4 5 6 4 5 6]
[7 8 9 7 8 9]]
C =
[[1 2 3]
[4 5 6]
[7 8 9]
[1 2 3]
[4 5 6]
[7 8 9]]</code></pre>
<p>Write some additional code that slices the first and last columns of <code>A</code>, and stacks them into a 3x2 array. Make sure to <code>print</code> the results to verify your solution.</p>
</div>
</section>
</div>
</div>
</article>
<div class="footer">
<a class="label swc-blue-bg" href="http://software-carpentry.org">Software Carpentry</a>
<a class="label swc-blue-bg" href="https://github.com/swcarpentry/python-novice-inflammation">Source</a>
<a class="label swc-blue-bg" href="mailto:[email protected]">Contact</a>
<a class="label swc-blue-bg" href="LICENSE.html">License</a>
</div>
</div>
<!-- Javascript placed at the end of the document so the pages load faster -->
<script src="http://software-carpentry.org/v5/js/jquery-1.9.1.min.js"></script>
<script src="css/bootstrap/bootstrap-js/bootstrap.js"></script>
<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-37305346-2', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>