 |
Subjects Data was gathered from a sample of 2705 students representing 18 different teachers. There were 1333 girls and 1372 boys in the study: 508 were 8-10 years of age, 681 were 11-12, 490 were 13-14, 672 were 15-16, and 354 were 17-18.
Reliability A major concern of test developers is whether each test item reliably measures what it is supposed to measure. If a test is well designed, construct scores from one part of the test should be about the same as scores measuring the same construct in another part of the test. Thus, test items intended to measure the same construct should give approximately the same results from beginning to end. Statistical procedures used in development of the STEP profile include split-half (overall) reliability, inter-item reliability (item by item) and test/re-test reliability. During development, the STEP was carefully measured for both inter-item and construct relationships. For example, if the response to question 14 was supposed to measure Assertiveness, the value of this response would be expected to increase with the total score for Assertiveness. If the item score and total score moved independently of each other, this item would be dropped from the test.
The measure of Split-half (overall reliability) used for the STEP is Coefficient Alpha. Coefficient Alpha refers to the average of all possible inter-item and split-half correlations, both good and bad. There were 8 to 12 items in each factor to maintain high test-retest reliability. Without relying on single indicators of reliability, which may contain large amounts of error, Coefficient Alpha provides an overall measure of the internal reliability of the test. The Coefficient Alphas for the STEP are:
| Construct | Coefficient Alpha | | Analytical | .79 | | Structural | .63 | | Social | .67 | | Conceptual | .76 | | Expressiveness | .68 | | Assertiveness | .70 | | Flexibility | .55 |
At this time, test/retest measures are being considered, but it’s too early in the development of the instrument to adequately perform this analysis.
Validation Results The validity of a test refers to how well a test records what it is supposed to measure. Like reliability, there are several different types of validity. The STEP meets all requirements for face, content and construct validity.
Face validity refers to whether a test-taker perceives the test to be credible. If thinking styles and behaviors are to be measured, for example, asking questions about bank deposits or religious affiliations seriously threaten face validity. Irrelevant questions may stimulate respondents to question the validity of the entire test and thereby produce unreliable answers. Questions on the STEP were specifically chosen for their relevance to everyday classroom events and behaviors.
Content validity refers to the adequacy of the STEP to measure the behavior it is supposed to measure. Unlike a typing test which has a clear relationship between what the test measures and a specific skill, content validity is more difficult to obtain for a general personality instrument. Therefore, the content validity for the STEP relied on item relevancy, statistical data, and observation. From a relevancy standpoint, the specific questions in the test address situations and behaviors which almost every student ages 8-18 experiences. The generalizability of these behavior samples was validated through statistical factor analysis where specific patterns of items emerged as predicted. If the targeted behaviors were random or unrelated, no ordered patterns would have been observed.
The final form of validity is construct validity. Construct validity refers to whether the STEP measures the fundamental constructs it is supposed to measure: specifically four types of thinking styles and three kinds of behaviors. Differential construct validity can be obtained through correlational analysis. This examination should include evidence that constructs which are different should show significant divergent relationships and constructs which are similar should show significant convergent relationships. Constructs which have no relationship should have low or no significant correlations.
Relationships of the Attributes The relationships between thinking attributes and behavioral attributes are the strength of the STEP. However, they also make understanding the STEP slightly more complex. The relationships between the specific STEP factors are shown in the following table. All correlations are significant at or above the p<.05 level.
| | Structural | Social | Conceptual | Flexibility | Assertiveness | Expressiveness | | Analytical | .47 | .22 | .56 | .42 | -.05 | .19 | | Structure | | .34 | .38 | .33 | -.22 | .07 | | Social | | | .45 | .46 | -.05 | .40 | | Conceptual | | | | .63 | .12 | .51 | | Flexible | | | | | .19 | .48 | | Assertive | | | | | | .44 |
It is significant to note the strength of the positive and negative relationships between the behaviors and the thinking styles. For example, although Analytical (thinking) and Structural (thinking) are positively correlated (.47), the relationship of Analytical with Assertive (behavior) is substantially less than the size of the relationship with Structure (-.05 vs. -.22). Contrast this with the positive values associated with Expressiveness (behavior).
Gender Differences and Age Variances Individual results for the student showed that the means (rounded) varied by gender and changed with age. Boys tend to score higher in Analytical and Assertiveness while girls tended to score higher in Social. Other relationships can be seen in the following table. NOTE: the ratings were based on self-descriptions. They may or may not have any relationship with actual skills.
| Comparison by Gender | | Age | Gender | Analytical | Conceptual | Structural | Social | Expressiveness | Assertiveness | Flexibility | | 10 | Girl | 22 | 32 | 25 | 24 | 23 | 19 | 21 | | | Boy | 24 | 32 | 24 | 23 | 23 | 20 | 21 | | 11 | Girl | 21 | 32 | 24 | 25 | 23 | 19 | 21 | | | Boy | 24 | 33 | 23 | 23 | 24 | 21 | 21 | | 12 | Girl | 20 | 31 | 24 | 25 | 24 | 19 | 21 | | | Boy | 23 | 31 | 22 | 22 | 23 | 21 | 20 | | 13 | Girl | 20 | 32 | 23 | 26 | 25 | 20 | 21 | | | Boy | 21 | 30 | 21 | 22 | 24 | 21 | 20 | | 14 | Girl | 19 | 31 | 22 | 26 | 25 | 20 | 20 | | | Boy | 21 | 30 | 21 | 23 | 24 | 22 | 20 | | 15 | Girl | 17 | 30 | 21 | 26 | 24 | 20 | 20 | | | Boy | 20 | 30 | 20 | 23 | 24 | 22 | 19 | | 16 | Girl | 19 | 31 | 21 | 26 | 24 | 21 | 21 | | | Boy | 20 | 31 | 20 | 23 | 24 | 22 | 19 | | 17 | Girl | 19 | 32 | 22 | 27 | 25 | 21 | 21 | | | Boy | 19 | 30 | 20 | 24 | 24 | 22 | 20 | | 18 | Girl | 19 | 29 | 21 | 25 | 23 | 20 | 19 | | | Boy | 20 | 30 | 20 | 23 | 23 | 21 | 19 |
| Comparison by Age (Girl) | | As can be seen in the next table, girls’ Analytical, Conceptual and Structural scores tend to fall with age. Whether this is due to educational methodology, social pressures or biological maturity still needs to be investigated. | | Age | Analytical | Conceptual | Structural | Social | Expressiveness | Assertiveness | Flexibility | | 10 | 22 | 32 | 25 | 24 | 23 | 19 | 21 | | 11 | 21 | 32 | 24 | 25 | 23 | 19 | 21 | | 12 | 20 | 31 | 24 | 25 | 24 | 19 | 21 | | 13 | 20 | 32 | 23 | 26 | 25 | 20 | 21 | | 14 | 19 | 31 | 22 | 26 | 25 | 20 | 20 | | 15 | 17 | 30 | 21 | 26 | 24 | 20 | 20 | | 16 | 19 | 31 | 21 | 26 | 24 | 21 | 21 | | 17 | 19 | 32 | 22 | 27 | 25 | 21 | 21 | | 18 | 19 | 29 | 21 | 25 | 23 | 20 | 19 |
| Comparison by Age (Boy) | | As can be seen in the following table, boys’ scores in Analytical, Conceptual, Structural also tend to fall with age. Like the girl data, whether this is due to educational methodology, social pressures or biological maturity still needs to be investigated. | | Age | Analytical | Conceptual | Structural | Social | Expressiveness | Assertiveness | Flexibility | | 10 | 24 | 32 | 24 | 23 | 23 | 20 | 21 | | 11 | 24 | 33 | 23 | 23 | 24 | 21 | 21 | | 12 | 23 | 31 | 22 | 22 | 23 | 21 | 20 | | 13 | 21 | 30 | 21 | 22 | 24 | 21 | 20 | | 14 | 21 | 30 | 21 | 23 | 24 | 22 | 20 | | 15 | 20 | 30 | 20 | 23 | 24 | 22 | 19 | | 16 | 20 | 31 | 20 | 23 | 24 | 22 | 19 | | 17 | 19 | 30 | 20 | 24 | 24 | 22 | 20 | | 18 | 20 | 30 | 20 | 23 | 23 | 21 | 19 |
General Learning Environments vs. Teacher Role When the eigenvalues of the varimax-rotated factor analytical model were examined, it showed the 24 teaching techniques clustered into four general learning environments. The environments, in turn, fell along two axes. One axis represented the role of the teacher; that is, it moved from a directive teaching style to a facilitative teaching style. The other axis represented environmental factors; i.e., moving from playful and active to quiet and introspective. These factors are illustrated below:
| Playful -----General Environment------Quiet |  | Open Learning Environment
- Students roam around the classroom and lay on the floor
- There is music, fun and a noisy atmosphere
- Kids are allowed to learn through hands-on experimentation
- Students teach and help each other
- There are no challenging questions
- The teacher gives plenty of praise
- There are few, if any hard rules
| Structured Learning Environment
- Students are relaxed
- There is a serious learning atmosphere
- There is liberal use of charts, graphs and audio visuals
- Students teach and help each other
- Students are challenged with questions
- Kids get plenty of attention and praise
- The overall atmosphere is quiet and structured
| Cooperative Learning Environment
- Students are challenged with questions and made to think
- There is plenty of hands-on learning
- Personal experimentation is encouraged
- There is liberal use of charts and graphs
- Students teach and help each other
| Intensive Learning Environment
- The atmosphere is serious and structured
- Students are challenged to learn and work alone
- There are encouraged to experiment and questioned on what they learned
- Help is provided by the teacher
- The atmosphere is quiet and moves at a fast pace
|
Conclusion The STEP meets the expected reliability and validity measures recommended for this type of test. It provides valuable information about four different thinking styles and three behavioral styles using norms generated from analyzing thousands of profiles. It also relates students’ preferences for learning.
Back to the STEP Profile |
|  |