Emergenetics Home Contact Us About Us Associate Website  
 EmergeneticsEmergenetics AssociatesEmergenetics AsiaEmergeneticsEmergenetics
 
 
Instrument Design
The general design of the STEP includes both a theoretical and an ad hoc practical approach. The basic theory consists of defining four internal thinking styles and three external behavior styles that can be easily identified and understood by most people. The practical approach includes examining underlying patterns that help isolate and identify effective individual teaching techniques. The overall objective was to develop a tool that is easy to understand and apply while avoiding theoretical psychobabble and arcane methodology.

The instrument contains two parts. The first section contains 84 items that are statistically related to the three derived behaviors and four thinking styles. These items reflected items drawn from a base of over 25,000 previous participants. Items were edited for students aged 9-18. The second section of the instrument contains 24 common teaching techniques gathered from a sample of middle and high school teachers. The instrument was written to a Flesch Reading Ease of 78.4 and a Flesch Kincaid Grade Level of 6.4 to accommodate a wide age range of students.
 
Dancing the Macarena is one way to fire the neurons in both hemispheres of your brain.

Even if you’re not ready to hit the dance floor, there are lots of ways to get your entire brain working. Find out what they are on page 239 of Emergenetics: Tap Into the New Science of Success
Know Your Brain
 


Subjects
Data was gathered from a sample of 2705 students representing 18 different teachers. There were 1333 girls and 1372 boys in the study: 508 were 8-10 years of age, 681 were 11-12, 490 were 13-14, 672 were 15-16, and 354 were 17-18.

Reliability
A major concern of test developers is whether each test item reliably measures what it is supposed to measure. If a test is well designed, construct scores from one part of the test should be about the same as scores measuring the same construct in another part of the test. Thus, test items intended to measure the same construct should give approximately the same results from beginning to end. Statistical procedures used in development of the STEP profile include split-half (overall) reliability, inter-item reliability (item by item) and test/re-test reliability. During development, the STEP was carefully measured for both inter-item and construct relationships. For example, if the response to question 14 was supposed to measure Assertiveness, the value of this response would be expected to increase with the total score for Assertiveness. If the item score and total score moved independently of each other, this item would be dropped from the test.

The measure of Split-half (overall reliability) used for the STEP is Coefficient Alpha. Coefficient Alpha refers to the average of all possible inter-item and split-half correlations, both good and bad. There were 8 to 12 items in each factor to maintain high test-retest reliability. Without relying on single indicators of reliability, which may contain large amounts of error, Coefficient Alpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are:
ConstructCoefficient Alpha
Analytical.79
Structural.63
Social.67
Conceptual.76
Expressiveness.68
Assertiveness.70
Flexibility.55

At this time, test/retest measures are being considered, but it’s too early in the development of the instrument to adequately perform this analysis.

Validation Results
The validity of a test refers to how well a test records what it is supposed to measure. Like reliability, there are several different types of validity. The STEP meets all requirements for face, content and construct validity.

Face validity refers to whether a test-taker perceives the test to be credible. If thinking styles and behaviors are to be measured, for example, asking questions about bank deposits or religious affiliations seriously threaten face validity. Irrelevant questions may stimulate respondents to question the validity of the entire test and thereby produce unreliable answers. Questions on the STEP were specifically chosen for their relevance to everyday classroom events and behaviors.

Content validity refers to the adequacy of the STEP to measure the behavior it is supposed to measure. Unlike a typing test which has a clear relationship between what the test measures and a specific skill, content validity is more difficult to obtain for a general personality instrument. Therefore, the content validity for the STEP relied on item relevancy, statistical data, and observation. From a relevancy standpoint, the specific questions in the test address situations and behaviors which almost every student ages 8-18 experiences. The generalizability of these behavior samples was validated through statistical factor analysis where specific patterns of items emerged as predicted. If the targeted behaviors were random or unrelated, no ordered patterns would have been observed.

The final form of validity is construct validity. Construct validity refers to whether the STEP measures the fundamental constructs it is supposed to measure: specifically four types of thinking styles and three kinds of behaviors. Differential construct validity can be obtained through correlational analysis. This examination should include evidence that constructs which are different should show significant divergent relationships and constructs which are similar should show significant convergent relationships. Constructs which have no relationship should have low or no significant correlations.

Relationships of the Attributes
The relationships between thinking attributes and behavioral attributes are the strength of the STEP. However, they also make understanding the STEP slightly more complex. The relationships between the specific STEP factors are shown in the following table. All correlations are significant at or above the p<.05 level.

 StructuralSocialConceptualFlexibilityAssertivenessExpressiveness
Analytical.47.22.56.42-.05.19
Structure .34.38.33-.22.07
Social  .45.46-.05.40
Conceptual   .63.12.51
Flexible    .19.48
Assertive     .44

It is significant to note the strength of the positive and negative relationships between the behaviors and the thinking styles. For example, although Analytical (thinking) and Structural (thinking) are positively correlated (.47), the relationship of Analytical with Assertive (behavior) is substantially less than the size of the relationship with Structure (-.05 vs. -.22). Contrast this with the positive values associated with Expressiveness (behavior).

Gender Differences and Age Variances
Individual results for the student showed that the means (rounded) varied by gender and changed with age. Boys tend to score higher in Analytical and Assertiveness while girls tended to score higher in Social. Other relationships can be seen in the following table. NOTE: the ratings were based on self-descriptions. They may or may not have any relationship with actual skills.

Comparison by Gender
AgeGenderAnalyticalConceptualStructuralSocialExpressivenessAssertivenessFlexibility
10Girl22322524231921
 Boy24322423232021
11Girl21322425231921
 Boy24332323242121
12Girl20312425241921
 Boy23312222232120
13Girl20322326252021
 Boy21302122242120
14Girl19312226252020
 Boy21302123242220
15Girl17302126242020
 Boy20302023242219
16Girl19312126242121
 Boy20312023242219
17Girl19322227252121
 Boy19302024242220
18Girl19292125232019
 Boy20302023232119

Comparison by Age (Girl)
As can be seen in the next table, girls’ Analytical, Conceptual and Structural scores tend to fall with age. Whether this is due to educational methodology, social pressures or biological maturity still needs to be investigated.
AgeAnalyticalConceptualStructuralSocialExpressivenessAssertivenessFlexibility
1022322524231921
1121322425231921
1220312425241921
1320322326252021
1419312226252020
1517302126242020
1619312126242121
1719322227252121
1819292125232019
Comparison by Age (Boy)
As can be seen in the following table, boys’ scores in Analytical, Conceptual, Structural also tend to fall with age. Like the girl data, whether this is due to educational methodology, social pressures or biological maturity still needs to be investigated.
AgeAnalyticalConceptualStructuralSocialExpressivenessAssertivenessFlexibility
1024322423232021
1124332323242121
1223312222232120
1321302122242120
1421302123242220
1520302023242219
1620312023242219
1719302024242220
1820302023232119

General Learning Environments vs. Teacher Role
When the eigenvalues of the varimax-rotated factor analytical model were examined, it showed the 24 teaching techniques clustered into four general learning environments. The environments, in turn, fell along two axes. One axis represented the role of the teacher; that is, it moved from a directive teaching style to a facilitative teaching style. The other axis represented environmental factors; i.e., moving from playful and active to quiet and introspective. These factors are illustrated below:
Playful -----General Environment------Quiet
Open Learning Environment
  • Students roam around the classroom and lay on the floor
  • There is music, fun and a noisy atmosphere
  • Kids are allowed to learn through hands-on experimentation
  • Students teach and help each other
  • There are no challenging questions
  • The teacher gives plenty of praise
  • There are few, if any hard rules
Structured Learning Environment
  • Students are relaxed
  • There is a serious learning atmosphere
  • There is liberal use of charts, graphs and audio visuals
  • Students teach and help each other
  • Students are challenged with questions
  • Kids get plenty of attention and praise
  • The overall atmosphere is quiet and structured
Cooperative Learning Environment
  • Students are challenged with questions and made to think
  • There is plenty of hands-on learning
  • Personal experimentation is encouraged
  • There is liberal use of charts and graphs
  • Students teach and help each other
Intensive Learning Environment
  • The atmosphere is serious and structured
  • Students are challenged to learn and work alone
  • There are encouraged to experiment and questioned on what they learned
  • Help is provided by the teacher
  • The atmosphere is quiet and moves at a fast pace

Conclusion
The STEP meets the expected reliability and validity measures recommended for this type of test. It provides valuable information about four different thinking styles and three behavioral styles using norms generated from analyzing thousands of profiles. It also relates students’ preferences for learning.

Back to the STEP Profile