# Importance of an Experimental Design and Statistical Analysis

Experimental Design In Research,The importance of careful experimental design, Overview of statistical analysis,EDA always precedes formal (confirmatory) data analysis.

Scientific
learning is always an iterative process, as represented in. If we start at
Current State of Knowledge, the next step is choosing a current theory to test
or explore (or proposing a new theory). This step is often called **“Constructing
a Testable Hypothesis”**. Any hypothesis must be allowed for different possible
conclusions or it is pointless.

For
an exploratory goal, the different possible conclusions may be only vaguely
specified. In contrast, much of statistical theory focuses on a specific, so-called**
“null hypothesis”** (eg, reaction time is not affected by background noise) which
often represents **“nothing interesting going on”** usually in terms of some effect
being exactly equal to zero, as opposed to a more general, **“alternative
hypothesis”** (eg, reaction time changes as the level of background noise
changes), which encompasses any amount of change other than zero.

The next step
in the cycle is to **“Design an Experiment”**, followed by **“Perform the
Experiment”**, **“Perform Informal and Formal Statistical Analyses”**, and finally
**“Interpret and Report”,** which leads to possible modification of the “Current
State of knowledge”.

**The importance of careful experimental design**

Experimental
design is a careful balancing of several features including **"power"**,
generalizability, various forms of** "validity"**, practicality and cost.
A thoughtful balancing of these features in advance will result in an
experiment with the best chance of providing useful evidence to modify the
current state of knowledge in a particular scientific field.

On the other hand,
it is unfortunate that many experiments are designed with avoidable flaws. It
is only rarely in these circumstances that statistical analysis can rescue the
experimenter. This is an example of the old maxim “an ounce of prevention is
worth a pound of cure”.

** "Our goal is always to actively design an experiment that has
the best chance to produce meaningful, defensible evidence, rather than hoping
that good statistical analysis may be able to correct for defects after the fact."**

** Overview of statistical analysis**

Statistical
analysis of experiments starts with graphical and non-graphical exploratory
data analysis (EDA). EDA is useful for.

· 1:detection of mistakes

· 2:checking assumptions

· 3:determining relationships among the explanatory variables

· 4:assessing the direction and rough size of relationships between
explanatory and outcome variables, and

· 5:preliminary selection of appropriate models of the relationship
between an outcome variable and one or more explanatory variables.

**“EDA always precedes formal (confirmatory) data analysis.”**

Most formal (confirmatory) statistical analyzes are based on models. Statistical models are ideal, mathematical representations of observable characteristics. Models are best divided into two components. The structural component of the model (or structural model) specifies the relationships between explanatory variables and the mean (or other key feature) of the outcome variables.

The
**“random” or “error”** component of the model (or error model) characterizes the
deviations of the individual observations from the mean. (Here,** “error” **does
not indicate **“mistake”**.) The two model components are also called **“signal” and
“noise”** respectively. Statisticians realize that no mathematical models are
perfect representations of the real world, but some are close enough to reality
to be useful.

A full description of a model should include all assumptions
being made because statistical inference is impossible without assumptions, and
sufficient deviation of reality from the assumptions will invalidate any statistical
inferences. A slightly different point of view says that models describe how
the distribution of the outcome varies with changes in the explanatory
variables.

** "Statistical models have both a structural component and a
random component which describe means and the pattern of deviation from the
mean, respectively."**

A statistical test is always based on certain model assumptions about the population from which our sample comes. For example, a t-test includes the assumptions that the individual measurements are independent of each other, that the two groups being compared each have a Gaussian distribution, and that the standard deviations of the groups are equal.

The farther the truth is from
these assumptions, the more likely it is that the t-test will give a misleading
result. We will need to learn methods for assessing the truth of the
assumptions, and we need to learn how "robust" each test is to
assumption violation, ie, how far the assumptions can be **"bent"**
before misleading conclusions are likely.

**"Understanding the assumptions behind every statistical
analysis we learn is critical to judging whether or not the statistical
conclusions are believable."**

Statistical
analyzes can and should be framed and reported in different ways in different
circumstances. But all statistical statements should at least include
information about their level of uncertainty. The main reporting mechanisms you
will learn about here are confidence intervals for unknown quantities and
p-values and power estimates for specific hypotheses.

** "p-values are not the only way to express inferential
conclusions, and they are insufficient or even misleading in some cases."**

**An oversimplified concept map.**

**What you should learn here**

THe expectation is that many of you, coming into the course, have a **“ conceptmap ”**
similar. This is typical of what students remember from a first course in
statistics. By the end of the book and of course you should learn many things.
You should be able to speak and write clearly using the appropriate technical
language of statistics and experimental design.

You should know the definitions of the key terms and understand the sometimes-subtle differences between the meanings of these terms in the context of experimental design and analysis as opposed to their meanings in ordinary speech. You should understand a host of concepts and their interrelationships.

These concepts form a **“concept-map”** such
as the one in that shows the relationships between many of the main concepts
stressed in this course. The concepts and their relationships are the key to
the practical use of statistics in the social and other sciences. As a bonus to
the creation of your own concept map, you will find that these maps will stick
with you much longer than individual facts.

By
actively working with data, you will gain the experience that becomes **“
datasense ”**. This requires learning to use a specific statistical computer
package. Many excellent packages exist and are suitable for this purpose.

A reasonably complete concept map for this topic.

From SPSS, but this is in no way an endorsement of SPSS over other packages. You should be able to design an experiment and discuss the choices that can be made and their competing positive and negative effects on the quality and feasibility of the experiment. You should know some of the pitfalls of carrying out experiments. It is critical to learn how to perform exploratory data analysis, assess data quality, and consider data transformations.

You should also learn how to choose and perform the most common statistical analyses. And you should be able to assess whether the assumptions of the analysis are appropriate for the given data. You should know how to consider and compare alternative models. Finally, you should be able to interpret and report your results correctly so that you can assess how y our experimental results may have changed the state of knowledge in your field.

Give your opinion if have any.