Links to Appendices Below. NOTE: Use horizontal scroll bars to see complete row(s).
Appendix A – Ride Rating Rubric
Appendix C – Key Factors Rubric
Appendix D – Build Complexity Rubric
Appendix A – Ride Rating Rubric
| 1 | 2 | 3 | 4 | |
| Originality | Design clearly derivative or a copy of a design already used. Design is not animated with computer and robotic elements. No decorative elements or mini-figure use. | Design is animated with computer and robotic elements with one-step simple program. One decorative elements or mini-figure use. [If not animated, has some detailed build work.] | Design is animated with computer and robotic elements and multi-step program. Some decorative elements or mini-figure use. [If not animated, has detailed build work.] | Recorded own sound(s). Decorated with craft materials. Very inventive design. Creative use of mini-figures and additional ride elements. Design is animated with computer and robotic elements and complex program. |
| Functionality | Design neither safe nor interesting. Very unstable design. Not to scale. No concern for symmetry. Could not finish. | Design is safe or interesting. Design somewhat unstable. A few elements of appropriate scale or symmetry. Met some elements of challenge and works to some extent. | Design is safe and interesting. Design is basically stable. Design has some appropriate elements of symmetry and scale. Met basic requirements and is functional. | Design is safe and interesting. Design is very stable. Design symmetrical (if appropriate) and to scale. Use of gears and/or pulleys. Goes beyond basic requirements and works very well. |
Appendix B – EDP Codes
These codes describe the engineering design process. In the case of clear, overlapping design phases, code the verbal with [2:name] and then indicate the end with [2:END] Code the end of the session as [END], which is normally the start of the post-interview. If the subject starts building again during the post-interview, delay the end until building is complete and code that building.
BUILD-NORMAL – normal building, which includes looking for parts unless the looking for parts was researching the feasibility of a potential design or subsystem. If the subject is describing what they are doing now, count as building and not planning. The use of present tense verbs help indicate building. Count cleaning up and organizing as BUILD-NORMAL. Measuring a part by comparing it in place is part of building and not RESEARCH. Referring to a plan (drawing) to build is BUILD-NORMAL.
BUILD-REBUILD – rebuilding (fixing) something that built previously. This includes building it in a different way as well as reattaching a subsystem that fell off for example.
EVALUATE-PHYSICAL – evaluate model by testing physically. Trying out pieces in place (without connecting them) to see if they will work is counted as RESEARCH.
EVALUATE-VERBAL – evaluate model without any physical test by talking. Include comments about the process.
EVALUATE-VISUAL – evaluate model by looking without touching or talking.
EVALUATE-SYSTEM – evaluate the whole system including the program by running the program.
PLAN – subject was planning some aspect of their design, typically verbally. Do not count describing what the subject is currently building. That should be coded as building. If the subject is verbalizing what they are planning to build in the future, even if it is the immediate future, count as planning. The use of future tense verbs help indicate planning (for example, “I will …”, “I am going to”, etc.). Plan can include giving rationale for their plan, demonstrating their plan, or evaluating the plan itself (such as drawings). Sometimes planning can be inferred depending on the surrounding activity when the physical activity segment is coded as no_activity and the subject is not talking. If subject verbalizes the need for the next, identified, single part, count that as BUILD (for example, “I need another one of these”). If they verbalize the possible use of a single part (“I could use this one at the end” for example), count as PLAN. If they verbalize the need for more than one part, count as PLAN.
PROBLEM-SCOPING – subject tries to clarify the problem as defined by the researcher, typically by asking a question and/or gathering more information about the problem (not about a possible solution, which is research). Include questions about the process.
PROGRAM-NORMAL – Programming the robot. Connecting the USB cable and downloading programs (for NXT) are counted as PROGRAM-NORMAL.
PROGRAM-REPROGRAM – Fixing a previous program.
RESEARCH – researching a problem or possible solution. Looking for parts can be considering research if it is affecting major design decisions before building starts or during the build. Otherwise, consider it part of building. If there is a small, separate build to test out a possible solution, code that as research. When the small, separate build is evaluated, consider that research as well. Trying out pieces in place (without actually connecting them) with the intent of evaluating for suitability is considered research. However, do not count measuring a part by comparing it in place, which is part of building.
SHARE-OUT – the student is sharing out without being prompted. Normally not used since sharing out is part of the post interview. However, this should be used if the student is sharing out unprompted or is prompted but later starts building again. Specifically, use this code if the subject is making post-make drawings.
Appendix C – Key Factors Rubric
| Low | Medium | High | ||
| Structural Knowledge | Student does not show knowledge of LEGO connector parts and connection techniques. Student is unable to learn new parts and techniques when needed. No knowledge or use of programming. | Student has some knowledge of LEGO connector parts and connection techniques. Student sometimes learns new parts and techniques when needed. Some knowledge or use of programming. | Student has extensive knowledge of LEGO connector parts and connection techniques. Student consistently learns new parts and techniques as needed. Extensive knowledge or use of programming. | |
| Engineering Process Skills | Subskills | |||
| Math/Science | Student does not apply math or science to a problem. | Student sometimes applies math or science to successfully solve a problem. | Student frequently applies math or science to successfully solve a problem. | |
| Design Principles | Student does not mention or use scale, symmetry, or stability in their build. | Student sometimes mentions or uses one of scale, symmetry, or stability in their build. | Student frequently mentions and uses two or more of scale, symmetry, and stability in their build. | |
| OtherProcess | Student does not use other techniques and strategies such as control of variables, troubleshooting tactics, systemic testing, and engineering design process knowledge. | Student sometimes uses other techniques and strategies such as control of variables, troubleshooting tactics, systemic testing, and engineering design process knowledge. | Student frequently uses other techniques and strategies such as control of variables, troubleshooting tactics, systemic testing, and engineering design process knowledge. | |
| Cognitive Skills | Casual Reasoning | Student usually makes incorrect predictions and inferences. | Student sometimes makes correct predictions and inferences. | Student makes frequent and correct predictions and inferences. |
| Planning | Student is a serial builder using trial and error who seldom plans ahead. | Student shows evidence of near term planning ahead. Student uses a clear mix of trial and error and planning. Try and classify as serial builder or planner but put in this category if there is a clear mix of serial and plan ahead building styles. | Student shows evidence of planning the complete ride (as a system) ahead of time. Student consistently shows evidence of planning their next step. | |
| Cognitive Flexibility | Student shows evidence of cognitive inflexibility (non-optimal persistence). Student seldom rethinks strategies even when there are persistent failures. | Student shows some evidence of cognitive flexibility. Student sometimes rethinks strategies when there are persistent failures. | Student shows cognitive flexibility. Student frequently and creatively rethinks strategies when there are failures. |
Appendix D – Build Complexity Rubric
| Build Complexity | Low | Medium | High |
| The ride is not animated (no computer, no motor). There are no decorative elements or mini-figures. There are a small number of parts put together simply. | The ride is animated with one motor and computer. There is some decorative elements or mini-figure use. The ride uses a simple program. There are a moderate number of parts and subsystems. | The build is animated with a computer, motor, and uses gears. There are many decorative elements or mini-figures. The ride has multiple motors or has a sensor. The ride uses a complex program. There are a large number of parts and subsystems. |
Appendix E – Research Prompt
[Student’s name], I asked you to join me to help me with some of my homework for my own schoolwork. My homework is to better understand how kids design and build robots at different ages.
To better understand what you are thinking, I am going to ask you to talk out loud as you work so I understand what you are doing and thinking. I may also ask you other questions if I am not sure what you are doing or thinking. You are not being tested or evaluated in any way.
First, you will do a warm up task to practice talking aloud as you work.
“Oh, no, our house has no roof! Add a flat and sturdy roof to the house. You can use the LEGO pieces on the table any way you want. You can use this weight (show) to test your roof. Remember to talk aloud as you work to say what you are doing and thinking as you plan and build.”
Now, we will do the main project. Have you ever been to a fair or amusement park? What rides do you like? [Make sure student understands what an amusement park ride is.]
You will now build a model amusement park ride. It can be like a ride you have been on before or it can be one you make up using your own imagination. You may want to use paper to draw pictures or write words that help to plan what you are going to build. You can also tell me in your own words what you are planning to build, if you know that ahead of time. Your ride should be safe and interesting.
You should use LEGOs to build your ride but you can also add any of the other materials you see. [Show student LEGOs, craft materials, wooden blocks.] You may also use a computer laptop to program your ride with motors, sounds, or sensors.
You will have about 1 hour to build your model amusement park ride.
Are there any questions before you start?
