Lesson 2: The Statistical Process

Lesson Introduction

Statistics are used in every aspect of society. From formal scientific studies to consumer ratings on product reviews, every statistical analysis follows a pattern we will call the Statistical Process. This process will be introduced in this lesson and will be used throughout the course.

The Five Steps of the Statistical Process
	Design the Study
	Collect the Data
	Describe the Data
	Make Inference
	Take Action

The above icons have a letter and an image to help you remember each of the five steps.

Lesson Outcomes

By the end of this lesson you should be able to do the following.

State the five steps of the Statistical Process.
Distinguish between an observational study and an experiment.
Distinguish between a population and a sample.
Distinguish between a categorical and a quantitative variable.
Distinguish between a parameter and a statistic.
Distinguish between and give an example of each of the following sampling schemes:
Explain the significance of using a random sample.

The following Case Study will show you how these outcomes apply to a real life scenario.

Case Study: Daniel Can Discern More Truth

Case Study Objective: Demonstrate the five steps of the Statistical Process.

After taking Israel captive, Babylon’s King Nebuchadnezzar asked his chief officer to bring Israelite children who were “well favoured, and skilful in all wisdom, and cunning in knowledge, and understanding science…to stand in the king’s palaces” (Daniel 1:4). To aid their preparation, Nebuchadnezzar planned to feed them his meat and wine for three years (Daniel 1:5).

Daniel did not want to defile himself by partaking of the king’s meat and wine. He asked permission to eat pulse¹ and drink water instead. His supervisor, Melzar, was afraid to displease the king. He thought that after eating pulse and water, the selected Israelites would look worse than their peers, and he would be punished (Daniel 1:8-10).

Design the Study

With an understanding of the background of the situation, Daniel proposed an experiment. He said,

“Prove thy servants, I beseech thee, ten days; and let them give us pulse to eat, and water to drink. Then let our countenances be looked upon before thee, and the countenance of the children that eat of the portion of the king’s meat: and as thou seest, deal with thy servants” (Daniel 1:12-13).

In short, Daniel’s implied research question can be stated as: Will those who eat pulse and drink water appear healthier than those who eat the king’s meat and drink his wine?

Daniel’s study, which Melzar agreed to, provides an ancient example of a designed experiment because the diet individuals were given was selected by the researchers. If the researchers had just let individuals select their diet themselves, then it would have been an observational study. While both of these are useful types of studies, only the experiment can help determine cause and effect relationships.

Daniel’s experiment included two groups of people.

Group 1: Treatment	Group 2: Control
Those who had the experimental treatment of eating pulse and drinking water	Those who ate the standard food consisting of the king’s meat.

Note that if there was no control group, then there would be no way to compare the effect of the diets (the treatments). For Daniel, the control group (who ate the king’s meat and drank his wine) provided a basis for comparing the effect of the new treatment (i.e. eating pulse and drinking water.)

Daniel’s hypothesis is that the Israelite children who eat pulse and drink water will appear healthier in just ten days, compared to those who eat the king’s meat and drink his wine.

Collect the Data

When designing a study, much attention is given to the process by which data are collected. Similarly, when examining a study, it is also important to understand the data collection procedures. Typically data is collected from a sample, in other words a subset (or a portion), of a population.

In Daniel’s experiment, the population was all of the Israelite children that were “well favoured, and skilful in all wisdom, and cunning in knowledge, and understanding science” (Daniel 1:4). In Daniel’s experiment, a sample of this population consisting of at least Daniel, Hananiah, Mishael, and Azariah was given the pulse and water to eat and drink. The remainder of the population was given the king’s meat.

Daniel’s study design stated that data be collected at the end of 10 days on the “countenance of the children” (Daniel 1:12-13). Note that the scriptures do not provide enough detail to know how the “countenance of the children” was measured. It is likely that quanititative measurements were not made, just overall impressions that were categorical in nature, like “improved” or “not improved.”

Whatever data Melzar officially recorded would be used to compare the appearances of two groups of people: (1) Israelites who ate pulse and drank water versus (2) Israelites who ate the king’s meat and drank his wine.

Describe the Data

When we describe data, we use any tools appropriate to the situation. This can include verbal descriptions of the data or statistics and graphics to help visualize and summarize the sample data. The statistics and graphics from the sample data are then used to make conclusions about the population parameters in the “Make Inference” step of the Statistical Process.

For Daniel’s experiment, the data are described in Daniel 1:15 using only words instead of graphs and statistics: “And at the end of ten days [the] countenances [of those who ate pulse] appeared fairer and fatter in flesh than all the children which did eat the portion of the king’s meat.”

Make Inference

Inference is the process of using the information contained in a sample to make a general statement (i.e. to infer something) about the entire population. It applies the logic that if something is true for the sample, then it must also be true for the population. The methods we will learn in this course help us know when it is appropriate to make such a bold conclusions about the population and when it would be unwise to do so.

Melzar made an inference. Based on the results of the sample, he determined that (in general) those who eat pulse and drink water will be healthier than those who eat the king’s meat and drink his wine (Daniel 1:15-16.). This is a bold generalization about all of the Israelite children, that was made possible through the experiment performed on Daniel, Hananiah, Mishael, and Azariah.

Take Action

The goal of a statistical analysis is to determine which action to take in a particular situation. Actions can include many things: launching an internet ad campaign (or not), expressing gratitude (or not), getting vaccinated (or not), etc.

Melzar took action as described in Daniel 1:16: “Thus Melzar took away the portion of their meat, and the wine that they should drink; and gave [all the Israelite children] pulse.”

The experiment was a success! “Now at the end of the days that the king had said he should bring them in…the king communed with them; and among them all was found none like Daniel, Hananiah, Mishael, and Azariah… And in all matters of wisdom and understanding, that the king enquired of them, he found them ten times better than all the magicians and astrologers that were in all his realm” (Daniel 1:18-20).

Note

Daniel’s experience can also help you learn the Statistical Process. Look at the first letter of each of the steps in the Statistical Process. You can use the phrase “Daniel Can Discern More Truth” to help you to help you remember the five steps in the Statistical Process.


	Daniel	Design the Study
	Can	Collect the Data
	Discern	Describe the Data
	More	Make inferences
	Truth	Take action

The Statistical Process will be used throughout the course. Take time to memorize the five steps. Also take some time to study in more detail the concepts discussed in this Case Study using the Concepts and Definitions section below.

Concepts and Definitions

Types of Studies: Observational & Experimental

Overview

Most research projects can be classified into one of two basic study designs:

Observational Study	Designed Experiment
A type of statistical study design where researchers observe the responses of the individuals, without controlling the conditions experienced by the individuals.	A type of statistical study design where researchers manipulate the conditions experienced by the participants.

Examples

Observational Studies

Example 1

Study Design - Students at a university watch to see which gender is more likely to clean up after themselves after finishing their meal in the school’s cafeteria.

Explanation - This is observational because the students simply observed the behavior of students eating in the cafeteria.

Example 2

Study Design - Researchers follow a group of people over a long period of time and track whether or not the individuals develop lung cancer. They also record whether or not the individual smoked or not. They use this information to determine if the risk of developing cancer is higher for those that smoke than for those that do not.

Explanation - This is observational because the researchers did not determine which individuals smoked and which did not smoke. They simply observed the rates of cancer for those that smoked and those that did not smoke.

Example 3

Study Design - The administration of a university studies their student population to determine if those receiving scholarships typically maintain higher GPA’s than those who do not receive scholarships.

Explanation - This is observational because the university did not randomly assign scholarships to students, they simply observed the behavior of students that received scholarships as compared to those that did not receive scholarships.

Experimental Studies

Example 1

Study Design - A teacher decides to implement a new textbook in one section of a certain course while continuing the use of the old textbook in another section of the course. They hope to learn if the new textbook has higher student approval than the old textbook.

Explanation - This is a designed experiment because the teacher decided which textbook the students would use. The teacher manipulated the environment that the students experienced.

Example 2

Study Design - Researchers use a driving simulator to have students drive through a virtual road. They randomly assign some of the students to “text while driving” and assign the other students to just drive in order to determine if texting while driving has detrimental effects on driving.

Explanation - This is a designed experiment because the researchers assigned students to either “text while driving” or “not text”. They manipulated the environment that the students experienced.

Example 3

Study Design - A bank emails out two different types of offers. Several randomly selected individuals receive a letter with notice that they will be given a $100 cash reward for opening up a new account. Another randomly selected group of individuals is offered $150 cash reward for opening up a new account. The response rate of each group is recorded as well as how much money is deposited by each group to determine if the bank benefits more from one type of offer than the other.

Explanation - This is a designed experiment because the bank sent one type of offer to a selection of individuals and another type of offer to a different selection of individuals. They manipulated the type of offer that individuals received.

Explanation

Both observational studies and designed experiments follow the five step procedure of the Statistical Process.

Designed experiments are often performed by randomly assigning subjects to one of two groups, a treatment group and a control group. The experiment is conducted by applying different kinds of treatment to the subjects in each group, and comparing the results between the two groups.

Often, those in the treatment group receive the actual treatment. Those in the control group either do not receive the treatment or receive a different treatment. Sometimes a placebo is given to the control group, which is meant to make individuals think they are receiving the treatment when they are not. In this way researchers can determine the effect of the treatments compared to when no treatment is given.

Types of Variables: Quantitative & Categorical

Overview

There are two basic types of data, i.e. variables:

Quantitative Variables	Categorical Variables
Provide numeric measurements from each individual in the dataset, like height, weight, price, etc.	Categorize the individual observations into groups, like gender, nationality, job title, area code of a telephone number, etc.

Note: not all things are data. Social security numbers or email addresses are not data. Thus, they are neither quantitative nor categorical variables, just ID variables.

Examples

Quantitative Data

Height

A researcher takes a sample of men from a certain population and measures each male’s height in inches.

This is quantitative data because each individual was measured. Further, it would make sense to compute the average height of the males in the sample.

Weight

A hospital records the birth weight of every child that is born in ounces.

This is quantitative data because each individual was measured. Also, the average birth weight would be a useful summary of this data.

Price

A student records the price of different kinds of cereal in dollars.

This is quantitative data because each type of cereal was measured by its price. The average price of cereals from a certain store could be computed for this data.

GPA

A university records the GPA of each student on a scale from 0.000 to 4.000.

This is quantitative data because each student receives a measurement and the average GPA of the university could be determined from this data.

Categorical Data

Gender

A researcher samples fish from a river and counts up how many male fish and how many female fish were in the sample.

This is categorical data because each fish is categorized into one of two groups based on its gender. Computing the average gender would not make sense.

Nationality

A survey asks participants to select their nationality from a list of several options.

This is categorical data because each individual will be categorized into a certain type of nationality based on their response. It would not make sense to compute the average nationality.

Job Title

A recent publication shows average job satisfaction ratings from various job titles like plumber, doctor, statistician, accountant and lawyer.

Job title is a categorical variable that allowed researchers to group individuals into their occupations. It wouldn’t make sense to report the average job title. However, notice that job satisfaction rating is a quantitative variable, and by first categorizing individuals by job title, the researchers could then report the average job satisfaction rating for each category.

Area Code

Note that phone numbers of themselves are just unique identification numbers, they are not data. However, the area code of a phone number does contain information that categorizes individuals into areas or regions of the U.S. according to where their phone number is originally located.

Explanation

For a quantitative variable, it makes sense to apply arithmetic operations to the data, like calculating an average.

Categorical variables are labels and it does not make sense to do arithmetic with them. They simply categorize each observation into a distinct category.

Note: some variables, like shoe size, can be considered as either quantitative or categorical variables. The researcher must decide if they wish to compute the “average shoe size” or if they want to “categorize people into groups based on their shoe size.”

Populations, Parameters, Samples, & Statistics

Overview

Population	Parameter
The entire collection of individuals (or items) of interest.	A characteristic of the population. Typically unknown.

Sample	Statistic
A subset of individuals (or items) from the population.	A characteristic of the sample.

Sample Statistics allow us to estimate the unknown Population Parameters.

Examples

Example 1

A researcher wishes to know the average debt load of U.S. households. They take a sample of 1,000 households. They use the information they find in their sample of 1,000 U.S. households to make a claim about the average debt load of all U.S. households.

Population	Parameter
U.S. Households	Average debt load of U.S. households. (Unknown.)

Sample	Statistic
The 1,000 U.S. households contacted.	Average debt load of the 1,000 U.S. households contacted.

The average debt load from the sample of 1,000 households is used to estimate the average debt load of all U.S. households.

Example 2

A business wants to determine the percentage of their customers that would be interested in a new service they are considering providing. They email out surveys to a sample of 100 recent customers, but only 70 customers respond. They find that a certain percentage of the sample is interested in the new service. They use this information to make a conclusion about the percentage of all their customers that are interested in the new service.

Population	Parameter
All customers of the business.	Percentage of customers of the business that are interested in the new service. (Unknown.)

Sample	Statistic
The 70 customers of the business that were contacted and responded.	The percentage of the 70 customers that responsed that are interested in the new service.

The percentage of 70 customers that are interested in the new service will be used to estimate the percentage of all the customers of the business that would be interested in the new service.

Explanation

It is often impossible to conduct a census, which is an examination of the full population. Whenever a census is possible, the population parameter of interest can be known.

Instead, a more feasible approach is to select a sample from the population, which provides a more manageable group of items. The information gained from the sample, the sample statistic, is used to estimate the population parameter. This process of estimating the parameter from the statistic is called “making an inference,” i.e., a generalization from the sample to the population.

Types of Sampling: Random & Convenience

In statistics, the information from a sample is used to make inference about the population. Because of this, it is important that the sample that is obtained is found in an unbiased way.

Using random sampling is the best way to reduce bias.

Overview

Random Sampling Methods	Definition
Simple Random Sampling (SRS)	Every group of size $n$ is given an equal chance of selection.
Stratified Sampling	Simple random samples are drawn from each of separate homogeneous groups of the population called strata.
Systematic Sampling	Every $k^{\text{th}}$ item in the population is selected, beginning at a random starting point.
Cluster Sampling	Consists of taking all items in one or more randomly selected clusters, or blocks.

Non-Random Sampling Methods
Convenience Sampling	Selecting items that are relatively easy to obtain without the use of random selection to choose the sample. Should be avoided.

Unfortunately, most samples are obtained by convenience sampling, which can result in a very biased sample. A biased sample will make us think the truth about the population is very different from what it actually is.

Examples

Random Sampling Methods	Examples
Simple Random Sampling (SRS)	A random sample of $n=40$ BYU–Idaho students is obtained using a computer to randomly select names from a complete list of all currently enrolled students.
Stratified Sampling	A random sample of $n=40$ BYU–Idaho students is obtained by randomly sampling $10$ freshman, $10$ sophomores, $10$ juniors, and $10$ seniors from the complete list of BYU–Idaho students. (The freshman, sophomores, juniors, and seniors are homogeneous groups from the population.)
Systematic Sampling	A random sample of $n=40$ BYU–Idaho students is obtained by randomly selecting one name in the student directory, and then picking every $8$th person after that until $40$ have been selected.
Cluster Sampling	A random sample of $n=40$ BYU–Idaho students is obtained by taking all of the students in a randomly selected class from the list of all BYU–Idaho classes that have $40$ students in them. (The classes create natural $40$ student clusters, and just one of the clusters was sampled.)

Non-Random Sampling Methods
Convenience Sampling	A non-random (biased) sample of $n=40$ BYU–Idaho students is obtained by selecting the first $40$ students that you come across while walking across campus.

Explanation

Random Sampling Methods	Explanation
Simple Random Sampling (SRS)	This is the best random sampling technique and should be used whenever possible. However, a SRS can only be collected if there is an accessible list of all the individuals in the population. All the statistical procedures in this course assume that simple random sampling has been used.
Stratified Sampling	Stratified sampling works well when the items are similar within each stratum and tend to differ from one stratum to another.
Systematic Sampling	Systematic sampling works well when the items are in a random, sequential ordering. If the items are not arranged randomly, a systematic sample can miss important parts of the population.
Cluster Sampling	Cluster Sampling words well when the variation from one block to another is relatively low, compared to the variation within the block.

Non-Random Sampling Methods
Convenience Sampling	Selecting items that are relatively easy to obtain without the use of random selection to choose the sample. This is frequently done when a quick and simple sample is needed, but may not yield a sample that represents the population well. When possible, convenience samples should be avoided.

Worked Example

From Wikimedia Commons.

The Salk Polio Trials comprised one of the most famous designed experiments in history. Performed in the 1950’s, the experiment was aimed to discover a vaccine that would prevent polio. The disease had reached epidemic status in the U.S. and Europe by the 1950’s crippling on average 35,000 people a year according to the CDC.

The first trial of the experiment, which enrolled almost 1.1 million children in the study, ended up as a complete failure due to some serious flaws in the design. However, a second trial of the study was later implemented, which was very successful.

Design the Study

Jonas Salk enrolled hundreds of thousands of children in his second study. The children were randomly assigned to one of two groups. The first group was given the new polio vaccine. The second group was given a placebo. The placebo was an injection that looked just like the vaccine, but contained a harmless saline solution.

This study was double blind. Neither the children’s parents nor their doctors knew whether a particular child received the treatment or the placebo. Both parties were blinded to this information, but well informed of the details of the study prior to participating.

The research hypothesis of this study was that the infection rate of polio would be lower in the vaccine group than in the placebo group. If this were the case, then that would be sufficient evidence to conclude the vaccine worked.

Collect the Data

The researchers followed up with each child to determine if they contracted polio. They recorded the number of children in each group that developed polio during the study period. Not all of Salk’s experiments were double-blind. Here is a summary of the results from the regions where a double-blind study was conducted (Francis et al., 1955; Brownlee, 1955).

Children Who Developed Polio

	Yes	No	Total
Treatment Group	57	200,688	200,745
Placebo Group	142	201,087	201,229

Describe the Data

One way to summarize the data is to compute the proportion of children in each group that developed polio.

Vaccine Group	Placebo Group
$\frac{\huge 57}{200745} = 0.000 283 9$	$\frac{\huge 142}{201229} = 0.000 705 7$

The proportion of children in the placebo group that developed polio during the study period was more than double the proportion of children in the treatment group that developed polio during the study period. That suggests that the treatment is effective in reducing the proportion of children that will develop polio.

Make Inference

Using the methods that we will discuss later in the course, the researchers determined that there was sufficient evidence to conclude that the polio vaccine worked at reducing the rate of polio infections.

Take Action

Once it was clear that the vaccine was effective, children who were unvaccinated or had received the placebo were given Salk’s vaccine. Since 1954, there has been a marked decrease in the number of polio cases worldwide. Public health researchers continue to work to eradicate this disease around the world.

Note

You can read more about it at the World Health Organization’s website if you are interested.

Preparation Assignment

Click here to begin.

Homework Assignment

Click here to begin.

Lesson 3 >>

Random Sampling Methods	Definition
Simple Random Sampling (SRS)	Every group of size \(n\) is given an equal chance of selection.
Stratified Sampling	Simple random samples are drawn from each of separate homogeneous groups of the population called strata.
Systematic Sampling	Every \(k^{\text{th}}\) item in the population is selected, beginning at a random starting point.
Cluster Sampling	Consists of taking all items in one or more randomly selected clusters, or blocks.

Random Sampling Methods	Examples
Simple Random Sampling (SRS)	A random sample of \(n=40\) BYU–Idaho students is obtained using a computer to randomly select names from a complete list of all currently enrolled students.
Stratified Sampling	A random sample of \(n=40\) BYU–Idaho students is obtained by randomly sampling \(10\) freshman, \(10\) sophomores, \(10\) juniors, and \(10\) seniors from the complete list of BYU–Idaho students. (The freshman, sophomores, juniors, and seniors are homogeneous groups from the population.)
Systematic Sampling	A random sample of \(n=40\) BYU–Idaho students is obtained by randomly selecting one name in the student directory, and then picking every \(8\)th person after that until \(40\) have been selected.
Cluster Sampling	A random sample of \(n=40\) BYU–Idaho students is obtained by taking all of the students in a randomly selected class from the list of all BYU–Idaho classes that have \(40\) students in them. (The classes create natural \(40\) student clusters, and just one of the clusters was sampled.)