What is R?

First there was S, a general-purpose, interpreted, computer language especially designed for statistics. It came from the famous Bell Labs.

The original implementation of S is no longer commercially available. S together with additional functions was renamed S-PLUS. It is currently owned by TIBCO Software Inc, where it seems to have been replaced by or buried within a product called Spotfire.

R is free S. It is free as in free beer (you can download it with no charge) and free as in free speech (you can do whatever you want with it except make it non-free). More precisely, R is a dialect of the S language. R and S-PLUS are more or less compatible. Roughly 90% of things you want to do work in both. Most other things work with minor variations. R is available from the Comprehensive R Archive Network (CRAN).

R is the the language of choice for research statistics. If it's statistics, you can do it in R.

If you have the time and want to know more about R, the Introduction to R that comes with the R software is the first thing to read, but it is way more than you need to know for this course.

What is Rweb?

Free software is amazing. Creative programmers can use it to do anything they can think of. There's no vendor controlling use of the software to protect their profits.

Prof. Jeff Banfield at Montana State University put R on the web. You can run simple R commands from any computer connected to the internet. A similar program could be easily done for S-PLUS but would be illegal because the vendor couldn't profit from it.

The local Rweb server is at http://rweb.stat.umn.edu/Rweb. This link is also in the navigation section of every course web page.

There are two interfaces to Rweb. The simple one found by clicking on the Rweb link on the main Rweb page, is the only one we will explain. It has the virtue of being embeddable in web pages to make examples.

Here is a simple example (one of the examples on the Rweb page)

To see how the example works, just click the Submit button.

When you have seen the example, click the Back button on your web browser to return to this page.

For now, don't bother with what the example does. Just notice that it does some calculations on some data and draws a picture.

The Relation between R and Rweb

Rweb is just R. You type R statements into a web form. You submit them. They get executed on the server. The results get stuffed into a web page sent back to your computer. So Rweb is just R run over the web.

So mostly we will use R and Rweb interchangeably.

One important difference between Rweb and R is that the server remembers nothing between Rweb submissions. The entire calculation you want done must be submitted to Rweb in one web form. R run on your own computer does remember. You can build up a complicated analysis a little bit at a time.

Thus Rweb is fairly useless for really complicated problems, but is fine for (simple) coursework.

Variables and Assignment

Like all other computer languages, R has variables, which are referred to by variable names. Variable names may contain any letter, digit, the dot (.), or the underscore (_) and cannot begin with a digit. Names are case sensitive, thus fred, Fred, and FRED refer to different variables.

The assignment operator in R is an arrow <- constructed from two characters. An assignment statement looks like 

fred <- 4

or 

sally <- 2 + 2

or 

a.very.long.variable.name <- sqrt(16)

Each assigns the value of the expression on the right side of the assignment operator to the variable name on the left side. In each case the variable gets the value 4.

Output (Print and Plot)

In order to see any results from R. You have to execute a command that makes output, the most common being print and plot.

When a calculation is done or an assignment made, you don't see anything unless you ask explicitly.

prints the value (4) assigned to the variable sally.

If the print statement were omitted, there wouldn't be any point because you wouldn't see anything and Rweb would't remember the results for future use.

Actually this example can be shortened to

because an expression that is not an assignment usually prints its value so 

sally

does the same thing as 

print(sally)

If in doubt, put in the print.

Vectors

Not all R variable values are single numbers (in fact most aren't). Most R variables are vectors, which is R's name for a list of objects of the same type (often numbers but character variables and other types are possible).

There are many ways to create vectors in R. Many functions and operators return vector values if given vector values as arguments. Here we will only look at a few ways to create a vector and a few functions and operators that work vectorwise.

The c Function

The R function c (on-line help) combines or collects all its arguments into one vector, for example

The seq Function

The R function seq (on-line help) creates a sequence, for example

Rweb External Data Entry

Variables can also be read into Rweb from an external file, either a file on your own computer or one on the web. We'll only illustrate the latter. An example file is

http://www.stat.umn.edu/geyer/5101/examp/blurfle.txt

The file has the following properties.

This has the result that all of the variables must be vectors of the same length. This can usually be arranged somehow (if necessary pad the variables that are too short with NA values).

When a job is submitted to Rweb, the first thing it does is read the External Data Entry file (if there is one) and create the variables in it. The example blurfle.txt creates three variables, color, x, and y and prints them out.

R Data Entry

The same issues from the preceding section apply when you are not using Rweb but using R on your own computer. The pattern is a little different. Suppose you are in R. The following commands 

X <- read.table(url("http://www.stat.umn.edu/geyer/5101/examp/blurfle.txt"),
    header = TRUE)
attach(X)
mimic what Rweb does when it loads the data in the URL of the preceding section. After that the statements in the Rweb form in the preceding section work the same in your computer as they do on Rweb. (In Rweb you can actually see the two statements shown just above in each Rweb output page, but the read.table in Rweb does not read from the web but from an already downloaded copy of the file.)

You can also download the file yourself to your own computer before starting R and then just do 

X <- read.table("blurfle.txt", header = TRUE)
attach(X)
This assumes the file "blurfle.txt" has been downloaded to a directory or folder where R will look for it (the current working directory under Linux, the user's Documents folder by default under Windows, or the user's home directory under MacOS X, these can be changed using the menus of the R GUI app).

Or you can create your own data file with your own data in it.

Plain Text Files

We repeat information from the Rweb data entry section above.

The file has the following properties.

This has the result that all of the variables must be vectors of the same length. This can usually be arranged somehow (if necessary pad the variables that are too short with NA values).

You cannot use Microsoft Word or any similar product to create data files for entry into R. Just say no. It cannot be made to work. You will only frustrate yourself if you try.

If you are careful to save in plain text format, you can use Microsoft Notepad.

An alternative is to get and use RStudio, but that is very complicated and we are not going to teach you how to use it.

CSV (Comma Separated Values) Files

An alternative is to use Microsoft Excel or other spreadsheet program, such as LibreOffice Calc to enter your data. Then write it out as a CSV (comma separated values) file. This is not the default format. You have to choose this output format specially when you save the file.

Here's what a CSV file looks like

http://www.stat.umn.edu/geyer/5101/examp/blurfle.csv

It looks almost the same as blurfle.txt. The only difference is that commas instead of whitespace separate the values.

Now 

X <- read.csv("blurfle.csv")
attach(X)
reads the data into R.

For more info see the on-line help page for read.table, read.csv, and several related functions for reading in data.

Vectorwise Functions and Operators

It is an important and generally useful fact about R that most functions and operators work vectorwise (operating on each element of the vector).

Note that multiplication needs an explicit operator * as in most computer languages. The ^ operator is exponentiation: bob^2 is bob squared.

That's all for now (admittedly too brief, see Simple manipulations; numbers and vectors in the Introduction to R document if you need to know more, but don't look at it your first time through this).

Indexing Vectors

Indexing operations allow you to modify or pick out or remove specified elements of a vector.

Integer Indexing

The simplest form of indexing uses positive integers in the range from one to the length of the vector. For example

do what is obvious (after you get used to vector indexing). Not quite so obvious is that subscripts work the same way on the other side of the assignment operator.

Negative Integer Indexing

Negative index values indicate everything but

do the same thing (why? figure it out!).

Logical Indexing

Perhaps the most useful form of indexing uses logical vectors. First the example, then the explanation. 

bob[bob != 42]

is the (vector of) elements of bob not equal to 42.

(The operator != is not equal. Similarly <= is less than or equal and >= is greater than or equal.)

The result of 

bob != 42

is a logical vector (all elements having values TRUE or FALSE. Indexing with such a vector picks out the elements for which the index is TRUE.

When the logical vector is the result of a comparison (as here), it picks out the elements for which the comparison was TRUE.

That's all for this web page. If you need to know more, see Index vectors; selecting and modifying subsets of a data set in the Introduction to R document if you need to know more, but don't look at it your first time through this.

Functions

Built-in Functions

We've already mentioned a few R functions. There are lots and lots of others. By built-in functions, we mean those that you don't have to do anything special to use. Strictly, speaking R doesn't have any built-in functions. Any function is like any other function. None are more special than any other. But several packages called base, datasets, graphics, grDevices, methods, stats, and utils are automatically available with no special effort.

These functions are listed on the documentation for the base library, and so forth. All the libraries are listed on the package index.

Arguments

To use an R function, you just type the function name followed by the list of arguments in parentheses. We've already seen examples, like 

plot(x, y)

Named Arguments

Most R functions also have named arguments. The syntax for that is

The named arguments here, main, xlab and ylab can appear in any order so long as they are after the unnamed arguments.

This makes the functions much simpler to use. Many functions have dozens of arguments, and you only need to use a few (the others have default values or aren't used the way you are invoking the function).

If you actually know the order of all the arguments, then you don't need the name. For example, the three expressions 

rnorm(10, 0.0, 1.0)
rnorm(10, mean = 0.0, sd = 1.0)
rnorm(10)

all do the same thing (generate 10 independent and identically distributed standard normal random numbers) because the second argument is mean and the third is sd and the defaults for these arguments are 0.0 and 1.0, respectively.

Your choice.

Packages

Some functions are not available until the package containing it is added. For example 

library(exactRankTests)

adds the exactRankTests package, which does pretty much what the name suggests.

Other than needing a library command first, functions in such a package, such as the wilcox.exact function in the exactRankTests package are just like any other functions.

The list of all packages available on our Rweb server is here. It can also be found by going to the main Rweb page (follow the link in the navigation section of any course web page) then clicking on the link HTML documentation in the second paragraph and then on the link Packages on the main R documentation page.

Many more packages can be found at the contributed packages page at CRAN.

Writing Your Own

Defining Functions

The function function defines new functions. For example 

trim <- function(x, lower = 0.0, upper = 1.0) {
    inies <- x >= lower & x <= upper
    return(x[inies])
}

trims off the values of the argument x that are below or above the arguments lower and upper, respectively.

The lower = 0.0 and upper = 1.0 in the definition specify default values for these arguments that are used when the user does not supply values.

Let's check it out.

As the assignment suggests, an R function is just an R object like any other R object. As such, it can be assigned a variable name or used in any other way an R object can be used. In this example, trim is an R variable that happens to be a function and x is an R variable that happens to be a numeric vector.

This allows functions to be passed as arguments to other functions, a very useful technique that we will use often (that's the main reason we will want to define our own functions).

Returned Value

A return statement is not strictly necessary. Functions return the value of the last expression if there is no return statement. The curly brackets are not necessary if there is only one statement.

Thus 

trim <- function(x, lower = 0.0, upper = 1.0) x[x >= lower & x <= upper]

works just as well as the other definition. But it is a lot harder to read, and we generally won't use this trick.

Local Variables

Local variables are variables defined inside a function. They exist only inside the function and have no influence on anything outside the function.

The following example shows this behavior.

Inside the function x is defined to be the value of y, but outside the function x is unchanged.

Global Variables

Global variables are variables defined outside a function. They are not defined inside the function, either in the argument list or in the body. They can, however, be used inside the function.

This is sometimes very convenient, but can lead to confusing code. It probably shouldn't be overused. (Real programmers think global variables are evil, but they are part of the R way.)

More on Functions

This section only scratches the surface. There's a lot more to be said about R functions. The section on writing your own functions in the Introduction to R book is a good place to start.

Missing Data and Computer Arithmetic

The number system used by R has two sorts of accomodation to values the computer can't handle or at least isn't supposed to deal with.

NA: Not Available

Any data value, numeric or not, can be NA. This is what you use for missing data. Always use NA for this purpose. Never use 999 or some other code that is actually a number. Sad experience of many scientists shows this sort of code is always forgotten at some point and the data analysis thereby ruined.

NaN: Not a Number

This is a special value that only numeric variables can take. It is the result of an undefined operation like 0 / 0. It is produced by the low level arithmetic of all modern computers. R is just going along with the standard here.

Inf: Infinity

Numeric variables can also take the values -Inf and Inf. These are produced by the low level arithmetic of all modern computers by operations such as -1 / 0 and 1 / 0. R is just going along with the standard here.

You shouldn't think of these as real infinities, like in calculus, but rather that the correct calculation, if the computer could do it, would probably (but not certainly) be very large, larger than the largest numbers the computer can hold (about 10300) and of the sign of the infinity.

Control Structures

R is a Turing complete computer language. Anything you can do with a computer, you can do in R (if you are a sufficiently clever programmer). For those who don't want to use a computer except via a WIMP interface (mice and menus), this may seem irrelevant, but it is very important.

No computer software product, no matter how large, can implement everything. We will see that, even in an undergraduate-master's level course like this, there are many issues than cannot be explored using a canned program. Thus R, unlike other statistics programs (except for S-PLUS, which is equal in power to R), is able to explore these issues.

Those who find computer programming frightening may rest assured that the programming we do will be very simple, involving no more than writing your own functions and the two control structures described in this section.

For Loops

One thing computers are much better at than people is mindless repetition. The for control construct (on-line help) is the main way mindless repetition is done in R. The R expression 

for (i in 1:100) {
    ### some R statements that do some work here
}

does the same thing (whatever is done by the R statements inside) 100 times.

Here is a simple example. Suppose we want to examine the sample median as an estimator of the mean of a normal distribution. We will learn about the asymptotic distribution in Stat 5102, but here we will just use simulation.

The following code simulates the median of a random sample of size n from the normal distribution (it does not matter which one), and we do this repeatedly nsim times.

Comments

The statement theta.hat <- double(nsim) creates a vector of length nsim to hold the simulation results (that we have not done yet). Each time the for loop repeats, it calculates one result (one median of a sample of size n).

The statement x <- rnorm(n) simulates a random sample of size n, see the web page about probability distributions in R for more about simulation of random variables.

The expression median(x) calculates the median of the sample just simulated, and the whole statement theta.hat[i] <- median(x) assigns this calculation to the i-th element of the vector theta.hat.

To understand this one needs to know that each time the loop is executed the variable i takes a different value from the list specified in the for statement, which in this case is 1:nsim, the vector of integers 1, 2, …, nsim.

When the loop finishes the vector theta.hat contains nsim random (and independent) replications of the sample median of a sample of size n.

The line following the loop draws the histogram: the sampling distribution of the sample median for a random sample of size n from a normal population.

The last line adds the density of the asymptotic normal distribution of this estimator, from which we see the simulation agrees with the theory.

If, Else, and Ifelse

If

Besides mindless repetition, the thing that allows computers to think or at least appear smarter than the average bear is their ability to make decisions based on some criterion applied to their current state. The if control construct (on-line help) is the main way decisions are made in R.

As an example of decisions, we investigate a really bad idea that seems to occur naturally to many people exposed to introductory statistics.

There are two kinds of two-sample t test, the old-fashioned one that is exact under the assumption of normal populations and equal population variance and the newer one that is only approximate but does not need the assumption of equal population variance (on-line help for the t.test function).

Which to use? Perhaps we should use a test about population variances to decide. Or perhaps we should decide only to use the exact when the test about population variances says it is o. k. Let's try that.

Comments

The statement tstat <- rep(NA, nsim) creates a vector of length nsim all of whose elements are NA. In the following loop we will set some of them to be useful values. We initialize to NA so it will be clear which values were not set.

The for loop works just like the preceding example.

The if statement calculates the P-value for the test of equality of variance and if greater than 0.05 then we carry out the t test assuming equality of variance and record the test statistic.

After the loop finishes, some of the values of the vector tstat are independent random realizations of the null distribution of the test statistic when the hypothesis of equality of variances is false (because sigx and sigy are different.

The code following the loop plots the histogram of the simulation distribution and the density of the t distribution that the procedure assumes the test statistic has. Clearly it doesn't.

Else

There are lots of ways to make decisions in R. One alternative is the else control construct (on-line help) that is used with the if control construct.

This example does the same thing as in the preceding section.

The only difference is that instead of initializing the vector tstat to NA we do the assignment to NA in the if-else control construct: if the P-value is greater than 0.05 then we compute the test statistic and assign it to tstat[i] as in the preceding example, otherwise we assign tstat[i] the value NA.

Ifelse

The preceding examples in this section use the if control construct in ways that look like general purpose computer languages such as C or Java. Here is a way that is unique to R using vector operations.

The if construct in the loop is gone. We just save both the P-value and the t test statistic each time through the loop.

After the loop is done we use the ifelse function (on-line help) to make the tstat vector into what it is at the end of the loop in the other examples.

Logical Indexing

Another R way to make decisions that has no analog in conventional computing languages uses logical indexing (which is described in a section above). Here's how that works.

This example is just a little different from the others in that after the statement tstat <- tstat[pval > 0.05] the vector tstat contains only the values corresponding to P-values greater than 0.05. There are no NA values; we have just shortened the vector to omit those cases.