[SYNPY-1861] remove submissionInstructionsMessage and submissionReceiptMessage from required attributes#1401
[SYNPY-1861] remove submissionInstructionsMessage and submissionReceiptMessage from required attributes#1401jaymedina wants to merge 3 commits into
Conversation
…m required attributes for Evaluation creation PUT request body. add test coverage. update documentation.
There was a problem hiding this comment.
Pull request overview
This pull request updates the Evaluation OOP model so that submission_instructions_message and submission_receipt_message are treated as truly optional when building the REST request body for create/update operations, aligning client-side validation with the Synapse REST API’s accepted payloads.
Changes:
- Removed
submission_instructions_messageandsubmission_receipt_messagefrom the set of attributes required byEvaluation.to_synapse_request(). - Updated
to_synapse_request()to omit submission message fields from the request body when they areNone. - Added/updated unit and integration tests to verify evaluations (and submissions against them) can be created/updated without these optional message fields.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
synapseclient/models/evaluation.py |
Makes submission message fields optional in request validation and omits them from request bodies when unset. |
tests/unit/synapseclient/models/unit_test_evaluation.py |
Updates required-field tests and adds a unit test asserting request bodies omit optional message keys when absent. |
tests/integration/synapseclient/models/async/test_evaluation_async.py |
Adds integration coverage for creating and updating evaluations without optional message fields. |
tests/integration/synapseclient/models/async/test_submission_async.py |
Adds integration coverage for creating/retrieving submissions against an evaluation created without optional message fields. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jaymedina
left a comment
There was a problem hiding this comment.
Note: This comment has been generated with AI assistance and reviewed by me ✍️
Reviewed evaluation.py, unit_test_evaluation.py, test_evaluation_async.py, and test_submission_async.py. The core logic change is correct — the two message fields are properly removed from the required list and are now conditionally included in the request body only when non-None. Docstrings and examples are updated consistently. Two minor findings left as inline comments.
linglp
left a comment
There was a problem hiding this comment.
@jaymedina Hi Jenny! It looks good overall. I found some issues in the tests and docstring that should probably be fixed before merging.
| assert created_submission.created_on is not None | ||
| assert created_submission.version_number is not None | ||
|
|
||
| async def test_store_submission_to_evaluation_without_message_fields_async( |
There was a problem hiding this comment.
Nit: the submission tests shouldn't care whether submission_instructions_message or submission_receipt_message are set. What about just removing the message fields from test_evaluation, delete test_store_submission_to_evaluation_without_message_fields_async
There was a problem hiding this comment.
Note: This comment has been generated with AI assistance and reviewed by me ✍️
The intent of test_store_submission_to_evaluation_without_message_fields_async is to ensure that the submission service doesn't require submission_instructions_message or submission_receipt_message to be set on the evaluation — since those attributes surface to submitters, it's worth explicitly verifying that submissions can be created against an evaluation that omits them. Happy to keep the test and clarify the comment if that makes the intent clearer.
| @@ -46,6 +46,28 @@ async def test_create_evaluation(self): | |||
| assert created_evaluation.owner_id is not None # Check that owner_id is set | |||
There was a problem hiding this comment.
I think the entire TestEvaluationValidation class should also probably be moved to unit test
| @@ -46,6 +46,28 @@ async def test_create_evaluation(self): | |||
| assert created_evaluation.owner_id is not None # Check that owner_id is set | |||
There was a problem hiding this comment.
Also, test_create_evaluation_missing_required_fields only covers when the description is missing. But missing name and missing content_source aren't tested.
| @@ -46,6 +46,28 @@ async def test_create_evaluation(self): | |||
| assert created_evaluation.owner_id is not None # Check that owner_id is set | |||
There was a problem hiding this comment.
I think def _build_evaluation_with_all_fields(self) -> Evaluation and probably some other tests related to test_to_synapse_request_create should also be updated.
| f"Your evaluation object is missing the '{attribute}' attribute. This attribute is required to {request_type.value} an evaluation" | ||
| ) | ||
|
|
||
| # Build a request body for storing a brand new evaluation |
There was a problem hiding this comment.
I saw that under to_synapse_request, we have:
for attribute in required_attributes:
if not getattr(self, attribute):
raise ValueError(
f"Your evaluation object is missing the '{attribute}' attribute. This attribute is required to {request_type.value} an evaluation"
)
But this would only report the first missing field. I think the consistent pattern in the code base is to check per field:
if not self.x: raise ValueError(...)
| name: Optional[str] = None | ||
| """The name of this Evaluation.""" | ||
| """**Required to store.** The name of this Evaluation.""" |
There was a problem hiding this comment.
For discussion, should there be an Optional[str] here if it is "required"? Is Required to store a new semantic within the client?
There was a problem hiding this comment.
I think the idea is that "name" is required when calling evaluation.store(). But I think we don't have the convention of doing something like **Required to store.** in the code base. My understanding is that we will just raise an error like here
There was a problem hiding this comment.
I agree with Lingling here on both counts.
There was a problem hiding this comment.
Should type hinting be...
name: None
"""The name of this Evaluation."""
instead of
name: Optional[str] = None
| ) | ||
| if self.submission_receipt_message is not None: | ||
| request_body["submissionReceiptMessage"] = self.submission_receipt_message | ||
|
|
There was a problem hiding this comment.
Shouldn't delete_none_keys(request_body) be used here?
…valuationCreation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Problem:
The Evaluation OOP model's
store()method incorrectly requiredsubmission_instructions_messageandsubmission_receipt_messageto be set before creating or updating an evaluation. These are optional display messagesshown to submitters. The Synapse REST API accepts PUT requests without them.
Solution:
submission_instructions_messageandsubmission_receipt_messagefrom the required attributes validated into_synapse_request().Testing:
Tests are passing locally: