Plan, Apply, & ChangeSets

[rawkode]

Comtrya is a pretty simple task-runner right now. It uses the query function to determine if an action actually needs to run, though it's usage is sparse atm.

We'd like to provide a better --dry-run that didn't just show what manifests will run, but which manifests need to run to get the system into the desired state.

This discussion is around an enhancement proposal for actions to support ChangeSets.

I'm proposing that we modify the Action trait to force all Actions to implement the following:

In-order to minimise the changes to the codebase upfront, we'll provide a default implementation that says that changes are required.

pub trait Action {
    // Unchanged
    fn run(&self, manifest: &Manifest, context: &Context) -> Result<ActionResult>;
    fn dry_run(&self, manifest: &Manifest, context: &Context) -> Result<ActionResult>;

    // New
    fn changeset(&self, manifest: &Manifest, context: &Context) -> Option<ActionChangeSet> {
      Some(ActionChangeSet {
        changes: vec![
          ActionChange: {
             action: self.Name(),
             change: String::from("No ChangeSet implementation, assuming always needs executed")
           }
        ]
      })
    }
}

pub struct ActionChangeSet {
  pub changes: []ActionChange
}

pub struct ActionChange {
  pub action: String // Which action: "package.install"
  pub change: String // What needs to happen: "The requested packages, vim and emacs, are currently missing"
}

[felipesere]

🤔 Hmmm...
I wonder if we'd want to feed the ActionChangeSet back into run(...)?
That would force the (internal) mechanic of "plan first, then run it".

On the structures themselves, particularly ActionChange, I think we can start with that and see where it goes? I don't have it fully formed in my head what ActionChange in order to be executable by itself.

Is it worth spiking this out on a branch to get a feel for it? Maybe for file.link and directory.copy?

[rawkode]

I completely agree. I had 3 or 4 different ideas for ActionChange that actually contained the diffs that would be executed, similarly to what you're describing; and keeping that generic for all actions for a little tricky. Though we could trait them and let each action have its own representation provided we can get a string to show a user.

I think spiking it on a branch and keeping comms open makes a lot of sense. Maybe we'll find a way to break up the work for us both?

Let me know how you want to proceed

[rawkode]

Leaving this rough mind dump here, needs structure and thought on more:

use serde::{Deserialize, Serialize};

//
// Actions aren't really too smart. They're primarily compositions of base actions,
// lets think of them as Atoms. Creating a file, changing the attributions, permissions,
// Making an HTTP call, running a command, script, etc ... POSIX?
//
// So instead of allowing each of the contributors that brings a new Action to manually
// handle all these common Atoms, we should provide the bedrock / substrate.
//
// Should we rename Actions to Molecules?
//
// The current changeset proposal would be one or more Atoms.
// Actions would emit Atoms that can be played or reverted.
// This will also make testing easier, as we can verify that the Actions emit
// the correct set of atoms.
//

pub trait Atom {
    fn apply(); // Apply new to old
    fn revert(); // Revert new to old
    fn diff(); // Print a string that describes what it will do. Show old and new values
}

pub struct FileAtom {
    filename: String,
}

pub struct FileCreate {
    file: FileAtom,
}

pub struct FileSetContents {
    file: FileAtom,
    contents: String,
}

pub struct FileAppendContents {
    file: FileAtom,
    contents: String,
}

pub struct FileOwner {
    file: FileAtom,
    owner: String,
}

pub struct FileGroup {
    file: FileAtom,
    group: String,
}

pub struct FilePermissions {
    file: FileAtom,
    chmod: u32,
}

[kraileth]

I like that chemistry analogy described here. Thinking a little bit about it though, I'm not entirely sure whether it really works.

If I knew nothing about Comtrya and first read about it using atoms, I would expect that it was about atomic operations which are executed entirely or not at all. In your example you have both cases that should be atomic and others that are very likely not. Changing file permissions is probably atomic as it should be a single syscall - but I don't see how one would be able to guarantee "atomicity" (is that even a word?) of e.g. running an external script without adding lots and lots of complexity.

So for one thing renaming actions to atoms might be misleading for newcomers. The second thing that came to my mind is much less of a problem, but I'd still like to mention it: Speaking of atoms allows to identify them as building blocks for molecules. However an atom does not exactly do anything whereas an action obviously does! But then each and every metaphor has its limits and eventually you'll just have to pick one.

Some more food for thought:

Staying in the chemistry field, reactions come to mind. Unfortunately they also bear the problem of being misleading: Comtrya's actions don't react to anything but are a driving force on their own.

Another thing could be speaking of elements instead of basic actions. Elements can cause reactions (and thus change) after all and would do away with the problem of "atomic" vs. "atom". Elements could then be combined into compounds.

[rawkode]

Thanks for the feedback!

Just to clarify on my end, I don't believe we want to change actions to be called anything else. I plan on the public manifest format to stay the same for end users. However, as I was working on the FreeBSD pkg provider, I found it a little cumbersome juggling the different commands and validating output for what I expect. It was a good thing, because it was a bit of a "oh, do I expect new contributors to do this?" And I really don't want that to be the case.

I provided a run_command helper last time to make this easier for contributions, but it's not enough.

So the idea of Atoms (totally open to new name) was to provide the building blocks for all future actions. Those building blocks are create file, set permissions, make an HTTP call, execute a command, etc. Actions, hopefully, only need to execute a set of data structures (Atoms) and the runtime will execute it. I took this analogy a little further on Discord and proposed receiving an anonymous function (Electron) as part of the atom to verify the command output or http response, which could end the runtime for that manifest.

I hope this makes more sense? Definitely would love more feedback, and very very happy to get naming suggestions for the internal API

Thank you 🙏

[kraileth]

My pleasure! As I cannot help with the coding work, I'd at least like to provide some input here and there.

Agreed: Identifying the tedious things that could probably be solved properly once and then be re-used is a great thing. This is in fact also one reason why I proposed taking FreeBSD into account early: It's different enough from Linux and macOS to likely lead to such discoveries.

I like the idea of the "Atoms" and "Electrons" a lot (whatever they might be called in the end). Throwing in another random thought: What about sticking with actions for the basic building blocks and tasks for the more advanced things we also call "actions" right now?

Also I just opened #80 to provide an example of what (part of) service management could look like on my main platform. Hope that it helps with a little inspiration on this topic.