Skip to content

Question Preparation Guide

Jump to bottom
Felix-Deng edited this page May 1, 2023 · 4 revisions

General notes for preparing questions:

  • Avoid adding feature(s) that result in a change of less than 0.5% of the total mass, volume, and surface area of the part. The error tolerance that is granted for model evaluation is set to be 0.5%, accounting for any numerical error in Onshape's geometric solver. Therefore, missing feature(s) that result in less than 0.5% change of the part's properties will not be detected by the app.
  • When multiple parts are evaluated within the same Part Studio, avoid having parts with properties that are very closed to each other numerically. When evaluating, all parts' properties are first sorted in order, and they are then compared and checked in order. Parts with very similar mass properties may result in different ordering when minor numerical errors are presented by Onshape's geometric solver.

To create and add questions to this project through the Questioner app, here are the question types that are currently supported:

  • Single-part part studio questions (SPSS)
  • Multi-part part studio questions (MPSS)
  • Assembly mating questions (ASMB)
  • Multi-step part studio questions (MSPS)

To ensure consistency in the collected data for research, we should avoid making changes to any published questions in the app once they are made available to the public (i.e., once data collection starts). However, the ability to updates to a question is still implemented to allow updates with small but necessary changes to the questions, but this feature should be used extra caution. For more instructions on this feature, go to this section.

Single-Part Part Studio Questions

Every question should be prepared with two Onshape documents: one for the (internal) source model and one for the public version.

For the source version:

  • The reference part should be built in this document
  • Material of the part should be assigned
  • This model should NOT be made public and should NOT be shared with anyone other than the question creator(s)

In the public version, there should be 3 - 4 elements, and they should also be prepared/created in the following order:

  1. A reference Part Studio where the reference part should be imported from the source document as a derived feature. This reference Part Studio should show the expected solution to the question, while importing as a derived feature (from a non-public document) hides the original feature list.
  2. An industry-standard manufacturing drawing created in an Onshape Drawing using the reference Part Studio. All geometric dimensions and the material type should be created and labelled as if this is an official question for the Onshape certification exam.
  3. (Optional) An alternative format of the manufacturing drawing in portrait form. Since the app will be opened by the users as a right-panel extension in their Onshape documents, images (the question drawings) are most likely better presented in a portrait style. However, all dimensions and the material type should be the same.
  4. Export the drawing element of the portrait format (the standard format if the portrait format is not available) as a JPEG image and store it as a new element (tab) within the same Onshape document. Once the question is added to the app through the admin portal, an API request will be called to retrieve this image from the document to be presented in the app's page.

After preparation of all points listed above, (1) create a version and (2) make ONLY the public version to be public with link document permission enabled. The users will be directed to the public version (if they choose to) to view the drawings in more details. Then, the question can be added through the admin portal of the app following the instructions on the website.

Multi-Part Part Studio Questions

Every question should be prepared with two Onshape documents: one for the (internal) source model and one for the public version.

For the source version:

  • All reference parts should be built in this document
  • Material of all parts should be assigned
  • This model should NOT be made public and should NOT be shared with anyone other than the question creator(s)

In the public version, there should be 3 - 5 elements, and they should also be prepared/created in the following order:

  1. A reference Part Studio where the reference part should be imported from the source document as either one derived feature or various derived features (one for each part). This reference Part Studio should show the expected solution to the question, while importing as derived feature(s) (from a non-public document) hides the original feature list.
  2. (Optional) A starting Part Studio where all starting parts should be imported as derived feature(s). When the users start an MPPS question, all parts in this starting Part Studio will be packed and imported to the user's working Onshape document with one derived feature. The number of parts will still be the same. If a starting Part Studio is not used/needed, simply leave the Starting element ID field blank in the admin portal of the app.
  3. An industry-standard manufacturing drawing created in an Onshape Drawing using the reference Part Studio. All geometric dimensions and the material type should be created and labelled as if this is an official question for the Onshape certification exam.
  4. (Optional) An alternative format of the manufacturing drawing in portrait form. Since the app will be opened by the users as a right-panel extension in their Onshape documents, images (the question drawings) are most likely better presented in a portrait style. However, all dimensions and the material type should be the same.
  5. Export the drawing element of the portrait format (the standard format if the portrait format is not available) as a JPEG image and store it as a new element (tab) within the same Onshape document. Once the question is added to the app through the admin portal, an API request will be called to retrieve this image from the document to be presented in the app's page.

After preparation of all points listed above, (1) create a version and (2) make ONLY the public version to be public with link document permission enabled. The users will be directed to the public version (if they choose to) to view the drawings in more details. Then, the question can be added through the admin portal of the app following the instructions on the website.

Assembly Mating Questions

Every question should be prepared with two Onshape documents: one for the (internal) source model and one for the public version.

For the source version:

  • All starting parts should be built in this document
  • Material of all parts should be assigned
  • The final (correct) assembly should be created with all starting parts in this document; you may also ask the users to use standard parts from Onshape Assembly as part of the question, but these should be clearly stated/instructed in the question
  • This model should NOT be made public and should NOT be shared with anyone other than the question creator(s)

In the public version, there should be 4 - 5 elements, and they should also be prepared/created in the following order:

  1. A starting Part Studio where all starting parts should be imported from the source document with Derived features. This starting Part Studio will be imported to the user's working Assembly as a whole as assembly instances, when they start the question.
    • If multiple instances of the same part are required, you can either import multiple derived parts into the starting Part Studio, or you can simply states the requirement in the question and ask the users to replicate the part themselves.
    • When the user starts a question, the entire Part Studio will be added into the user's working Assembly in the same space configuration as you see in the starting Part Studio. You may also want to also use the Transform feature in Onshape Part Studio to move the parts around in space, such that the starting Assembly for the students does not look too similar to the final expected Assembly.
  2. A reference assembly of the final expected Assembly should be prepared in the following ways:
    • Once all parts are properly mated in an Assembly in the source version, duplicate the Assembly in the source version and delete all the mate features in the duplicated Assembly. Make sure to do this in the source version document, otherwise all the mates will be visible through micro versions.
    • Copy the new Assembly (with all mate features deleted) from the source version to the public version. This can be done by right-clicking at the tab in the bottom of the source-version document, choose "Copy to clipboard". In the public-version document, click the "+" button at the bottom left corner of the page (where you will go to create new Part Studios and Assemblies), and you should see an option to "Paste tab".
    • Once the new Assembly is pasted in the source version, you should be careful NOT to move any parts around, as this will affect the geometric measurements of the assembly.
    • This reference assembly will be used to (1) take the thumbnail image for the question, and (2) take the reference principal inertia of the whole assembly for submission evaluation of the question.
  3. An industry-standard manufacturing drawing created in an Onshape Drawing using the reference Assembly. All geometric relations and interaction of parts should be outlined and labelled as if this is an official question for the Onshape certification exam.
  4. (Optional) An alternative format of the manufacturing drawing in portrait form. Since the app will be opened by the users as a right-panel extension in their Onshape documents, images (the question drawings) are most likely better presented in a portrait style. However, all dimensions and relations should be the same.
  5. Export the drawing element of the portrait format (the standard format if the portrait format is not available) as a JPEG image and store it as a new element (tab) within the same Onshape document. Once the question is added to the app through the admin portal, an API request will be called to retrieve this image from the document to be presented in the app's page.

After preparation of all points listed above, (1) create a version and (2) make ONLY the public version to be public with link document permission enabled. The users will be directed to the public version (if they choose to) to view the drawings in more details. Then, the question can be added through the admin portal of the app following the instructions on the website.

Multi-Step Part Studio Questions

A multi-step part studio question (MSPS) is essentially multiple single- or multi-part part studio questions (remember to check or uncheck the Is multi part field when adding the question) being presented subsequently as steps, with the option of having a starting part before presenting the first step. Every MSPS question should have ≥1 step and preferably ≤4 steps; adding too many steps may take too much respond time for the API process which will eventually terminate before completing.

Every question should be prepared with two Onshape documents: one for the (internal) source model and one for the public version.

For the source version:

  • All reference parts should be built in this document
  • Material of all parts should be assigned
  • This model should NOT be made public and should NOT be shared with anyone other than the question creator(s)

In the public version, there should be 3 elements for every step (preferably grouped into folders by steps), and they should also be prepared/created in the following order:

  1. (Optional) A starting Part Studio where all starting parts should be imported as derived feature(s). When the users start an MSPS question, all parts in this starting Part Studio will be packed and imported to the user's working Onshape document with one derived feature. The number of parts will still be the same. If a starting Part Studio is not used/needed, simply leave the Starting element ID field blank in the admin portal of the app.
  2. For every step of the MSPS question, prepare the following:
    1. A reference Part Studio where the reference part should be imported from the source document as either one derived feature or various derived features (one for each part). This reference Part Studio should show the expected solution to the question, while importing as derived feature(s) (from a non-public document) hides the original feature list.
    2. An industry-standard manufacturing drawing created in an Onshape Drawing using the reference Part Studio. All geometric dimensions and the material type should be created and labelled as if this is an official question for the Onshape certification exam.
    3. (Optional) An alternative format of the manufacturing drawing in portrait form. Since the app will be opened by the users as a right-panel extension in their Onshape documents, images (the question drawings) are most likely better presented in a portrait style. However, all dimensions and the material type should be the same.
    4. Export the drawing element of the portrait format (the standard format if the portrait format is not available) as a JPEG image and store it as a new element (tab) within the same Onshape document. Once the question is added to the app through the admin portal, an API request will be called to retrieve this image from the document to be presented in the app's page.

After preparation of all points listed above, (1) create a version and (2) make ONLY the public version to be public with link document permission enabled. The users will be directed to the public version (if they choose to) to view the drawings in more details. Then, the question can be added through the admin portal of the app following the instructions on the website.

Question Update

As part of the main purpose of the informatics framework is enable research into design behaviours of different users on the same modelling question in CAD, every user should have access to the same question to ensure data consistency (i.e., a controlled experiment). In general, changes should not be made to a question once it is made public and data collection starts. If a question is found with major errors after it is published and attempted, the entire question should be unpublished (or deleted), and a new version of the question should be created for the app.

However, it is also possible that there could be instances when small changes (e.g., missing a small dimension, adding a minimal clarification) are necessary to ensure the question can be completed with minimal ambiguity after it is published. This should also preferably be done as early as possible, when not that many people have attempted the question yet.

To update a question:

  1. Make all the necessary updates/changes in the source and/or public Onshape document for the question.
  2. Create a new version in the public-version document.
  3. Go to the admin portal of the app and update any modified information of the question that you would like to change. This includes at least the new version ID of the question document.
  4. Save the new changes in the question.
  5. Back to the page where a list of all questions in the database is shown, select the question that you just saved the updates, and choose "Force update selected questions" from the "Action" dropdown list at the top. Hit "Go" to initiate the updating action. If this step is not done, none of the saved changes will have an effect on the actual question data in the database.

Note: the force update action should also be done to one question at a time. If the API response time takes too long to complete, the website may terminate the updating process automatically.

Clone this wiki locally