[raft/store] Update Transact signature

[MariemBaccari]

Transact currently takes a raw func(ctx, R) error closure which works for the sqlstore but can't be replicated by the Raftstore since it needs to serialize the request, propose it and then apply it in a different thread and different nodes.

This PR changes the interface to accept a store.OperationRequest instead, a value that carries an operation ID and can be selected using a Registry which holds the encoding and execution logic.
The OperationID method would be generated automatically.

To keep changes minimal in this PR, the FuncOperation[R] adapter wraps the existing closures as a stub OperationRequest. Follow-up PRs will remove FuncOperation[R] and extract the actions for all handlers (see aux example in #1582).

The PR also adds TransactWithResult[R, Res], a generic function wrapping Transact which serves as a helper for result type asserts to avoid code duplication on the handler side. The reason why we need a wrapper instead of making Transact generic is that Go does not currently allow generic methods.

[mickmis]

IMO the newly added abstractions have room for improvement. They expose concepts that look too specific to an implementation. As suggested in a comment, consider pushing raft-specific logic into the raft code.

[mickmis]

• functions on those interface need to be documented correctly
• it is unclear from this PR why we need a separate ActionMetadata interface

[mickmis]

What is this? It's not used anywhere, not described in PR description, and not documented. Remove it if not necessary?

[mickmis]

Why does this get a new return parameter for the result? This looks like a new feature separate from the objective of this PR.

[MariemBaccari]

With the old closure-based signature, the handlers captured results through closure variables. With the change of signature, we don't have that anymore. The only way to return a result is through the return value of Transact itself. I think this change is inseparable from the parameter change as it's the consequence of it.

[mickmis]

That's a lot of any types. We'd like to avoid that in general, especially here both types are not unknown IIUC. This could indicate shaky abstractions.

[mickmis]

Naming: why not Transaction? This is what it is after all, a (store) transaction on a repository R.

[mickmis]

Checking the other PR (#1580): I think I get why this is introduced here, but I think the abstractions can be improved. To me it looks like the payload, the request type, and the readonly flags are all functions of the DSS operation (i.e. the HTTP endpoint called). Given that, what about storing alongside the transaction function only the operation, and inferring all of the raft-specific details in the raft code?
That would also eliminate the additions to the generated code in #1580.

[MariemBaccari]

I just pushed another proposal that aims to implement your suggestion. We could pass Transact the restapi request directly and have a Dispatch function that maps restapi requests to their corresponding execution functions. You can refer to #1525 and #1582 to have an idea of the implications of this on both the raftstore and sqlstore implementations. #1582 implements the aux store using this design.
On the one hand, we eliminate the generated methods and the Action structs and interfaces, simplifying my initial proposal but on the other hand we lose the typing at the Transact signature...
What do you think ?

[mickmis]

Building on this, how about:

Get the openapi-to-go-server generate this (e.g.):

func (p *PutDSSInstancesHeartbeatRequest) OperationID() string {
 	// this is what you already had with RequestType, but renamed to align with terminology
	return "PutDSSInstancesHeartbeat"
}

func (p *PutDSSInstancesHeartbeatRequest) MarshalJSON() ([]byte, error) {
	return json.Marshal(p)
}

And the transact interface:

type OperationRequest interface {
	OperationID() string
	json.Marshaler
}

type Store[R any] interface {
	Transact(ctx context.Context, req OperationRequest, f func(context.Context, R) error) error
	// meaning no need to have a return type anymore, c.f. other comment
}

This IMO would keep the change minimal, and keep abstractions acceptable. Would that work? (my minimal validation seems to indicate that yes)

[MariemBaccari]

This is similar to my first proposal (#1574) which introduced a request parameter alongside f. It was discarded by @barroco as it introduces a split between parameters only used by Raft (req) and the ones only used by sqlstore (f). The aim of the current proposal is to avoid that split.

To explain why f needs to be removed from the signature, I think we need to look at how the Raftstore works:
When a node receives a request, the Transact method should serialize it, and propose it to Raft which would return the result. In separate threads, the proposal is received as a committed entry and applied by all nodes (including the one that proposed it). This means that the f closure is not available not only to the other nodes but also to the current node since the applying of the operation happens in a different thread. Any workaround that I can think of aiming to keep f in the handlers and the signature would lead to code duplication and risks having diverging implementations between the raftstore and the sqlstore or even amongst the Raft nodes themselves.
This also explains why a return value to Transact is necessary.

Regarding the MarshalJSON, I think that the recent PR #1587 makes us want to avoid generating that since we will probably be implementing custom serialization in the future as an optimization.
Given that, the serialization methods can't be defined on the request type since we would be defining them in a separate package as the receiver.

We would then have a minimal interface:

type OperationRequest interface {
    OperationID() string
}

And a struct:

type OperationHandler[R any] struct {
    Serialize   func(OperationRequest) ([]byte, error)
    Deserialize func([]byte) (OperationRequest, error)
    Execute     func(ctx context.Context, r R, req OperationRequest) (any, error)
}

Which allows us to have each service defining a registry of request types to handling methods which is used by both the sqlstore and raftstore.

Giving us the following Transact:

Transact(ctx context.Context, req OperationRequest) (any, error)

Conceptually, I think this makes sense as the result is part of the "operation execution" contract in the sense that we give Transact an operation and it is natural to expect a transaction result.
Also, a request is a natural unit of work so passing it directly to Transact would mean "execute this operation against the store". The f closure itself is an implementation detail that is here only because the sqlstore needs it. It's not inherently part of the operation's identity and it's tailored to how one specific store works.

[mickmis]

FTR, discussed offline: LGTM modulo OperationHandler.(De)Serialize to use interfaces.

[MariemBaccari]

I just pushed the changes almost as we discussed:

One detail I overlooked during our conversation is that embedding the interfaces in OperationRequest would require implementing those methods in the package that now exclusively has generated content.

Embedding them in OperationHandler doesn't seem to make sense because they assume that the receiver is the value being encoded. If we go with that, we would need to make the handler carry the request as a field, making it stateful and, in my opinion, a bit more confusing to use since it's supposed to be a one-time defined per-type registry / toolbox.

What do you think ?

[mickmis]

Good idea for the transition 👍

[mickmis]

LGTM

[mickmis]

Shouldn't this rather be an interface?

[MariemBaccari]

That would require implementing a concrete type for every single operation, wouldn't that get too verbose ?

[mickmis]

Not necessarily, with an interface you have more flexibility, e.g.:

type OperationHandler[R any] interface {
	Encode(req OperationRequest) ([]byte, error)
	Decode(buf []byte) (OperationRequest, error)
	Execute(ctx context.Context, repo R, request OperationRequest) (any, error)
}

type mySwitchOpHandler[R any] struct {}

func (h *mySwitchOpHandler) Encode(req OperationRequest) ([]byte, error) {
	switch req.OperationID() {
	...
	}
} 

// same as current implementation, but behind the interface
type myFuncOpHandler[R any] struct {
	encode func(req OperationRequest) ([]byte, error)
}

func (h *myFuncOpHandler) Encode(req OperationRequest) ([]byte, error) {
	return h.encode(req)
} 

[MariemBaccari]

My understanding is that the interface is only useful if we're expecting multiple implementations which it seems like it's not the case here. The plan is that each service will have its registry with one entry of OperationHandler per type of operation. Have I overlooked a use case that can benefit from the interface ?

[mickmis]

Have I overlooked a use case that can benefit from the interface ?

Nope for what we need it is equivalent.
It is Just that (IMO) an interface represents better what OperationHandler is. The fact that it is a struct with only function fields hints to that.
But we are in the opinion and nit territory here (which I probably should have mentioned from the start), so feel free to disagree and go ahead with your current version, my positive review stands regardless (just LMK if that's the case so that I can merge the PR).

[MariemBaccari]

I'd suggest we go with the current version and, perhaps after the implementation of aux, see if we should better reconsider it ?