Compatibility Fixes Needed for reproduce() Function in Latest Acto Version (Version committed on March 27, 2025)

[Nidar53]

Description:

While attempting to use the reproduce() function with the latest Acto code, I had to apply the following changes to get it working:
• Removed the line:
• preload_images_ = []
• Updated the call to:
• errors = acto.run()
These manual adjustments suggest that the reproduce.py script might be out of sync with recent changes in the Acto API or internal logic.

 acto = Acto(
        workdir_path=workdir_path,
        operator_config=config,
        cluster_runtime=kwargs["cluster_runtime"],
        # preload_images_=[], <------------------------  Comment this line
        context_file=context_cache,
        helper_crd=None,
        num_workers=1,
        num_cases=1,
        dryrun=False,
        analysis_only=False,
        is_reproduce=True,
        input_model=input_model,
        apply_testcase_f=apply_testcase_f,
        acto_namespace=acto_namespace,
    )
    
    # errors = acto.run(modes=["normal"]) <----------- Comment this line
    errors = acto.run()  <--------------------  Updated  to 
    return [error for error in errors if error is not None]

Question:

Are these modifications expected when using the current version of Acto, or is there an updated reproduction script available?

Thank you

[tylergu]

Hi @Nidar53 thanks for the diligent investigation. Yes your changes would be sufficient to adapt to the recent Acto class' interface changes.

It seems that you have figured out how to fix this. Do you mind opening a PR for this? I can also fix this later today.

[tylergu]

I tried to fix it in the PR #407 , do you mind to give it a try?

[Nidar53]

Thanks a lot, I really appreciate your consideration.
However, when I run the acto.reproduce command using the latest Acto codebase with the Zookeeper test data (zkop-454), the process crashes after deployment (Operator deployed).

Observed Errors:

1.TypeError:
TypeError: apply_repro_testcase() got an unexpected keyword argument 'constraints'
o The apply_repro_testcase() function in reproduce.py passes the argument constraints, which is no longer expected.
o Temporary fix I applied: set constraints=None to bypass the error, At line 35 in the source code of the reproduce command.. However, I think the best practice would be to remove or update the relevant parts of the code if necessary.

def apply_repro_testcase(
    value_with_schema: ValueWithSchema,
    _: list,
    testcase: TestCase,
    __: bool = False,
       constraints=None,  <---------- add 
) -> jsonpatch.JsonPatch:

2.AttributeError:
AttributeError: 'ReproInputModel' object has no attribute 'metadata'
This occurs at: engine.py (line 1189)
"num_total_testcases": self.input_model.metadata,
But the class ReproInputModel does not define metadata, which leads to a crash.

Analysis:

• ReproInputModel
• engine.py
They are likely out of sync due to recent internal changes in Acto.

Note:

After adding constraints=None , reproduce() works correctly for some operators (like rabbitmq-operator), but not with others like zookeeper-operator — which still throws errors later in the process.

Question:

How would you recommend updating engine.py, reproduce.py and ReproInputModel to align with the latest engine logic and ensure that acto.reproduce() works as expected?

[tylergu]

Hi @Nidar53 , thanks for the comments. I fixed the function signature for the apply_repro_testcase function---we now have a function type for it to have some type validation.

The missing metadata is also fixed now.

[tylergu]

Let me know if you are still encountering problems, I just tried the python3 -m acto.reproduce --reproduce-dir test/e2e_tests/test_data/zkop-454 --config data/zookeeper-operator/config.json on my end and I can see that the config oracle is reporting the problem by looking at the generation-002-runtime.json in the testrun directory.

[Nidar53]

Thanks, it works now!
I really appreciate your help and consideration.

[tianyin]

Great! Let us know what we can help with and we'd love to hear your use cases.

[Nidar53]

Thank you!
I’m currently working on verifying Kubernetes operators in a telco cloud context as part of my Master’s thesis, and Acto has been really helpful in catching subtle inconsistencies.
Right now, I’m focusing on getting familiar with Acto by trying to reproduce existing bugs. After that, I plan to evaluate its performance in the telco cloud context.
I would be grateful if you could share some documentation on Acto.

[tianyin]

cool!

I'm not sure whether we have more docs to share -- right now it's mostly a research tool. But do let us know if you run into problems when using it.

verifying Kubernetes operators

Note that Acto is mostly a testing tool at this point. You can check out https://github.com/anvil-verifier/anvil/ if by "verify" you really mean "formal verification".

~t