New Capability: Message Processing

[sigpwned]

Users should be able to use Lammy to create message processing capabilities.

• Input types
  • Message - Message record with attributes, body, etc.
  • Bean - Just the body, deserialized
• Batch processing style
  • Batch - everything at once, entire batch succeeds or fails
  • Minibatch - one per message, each message succeeds or fails independently
• Function style
  • Processor - List of inputs to List of outputs
  • Consumer - List of inputs to no output
• Destination style (processors only)
  • Managed - One destination for all messages, sending managed by AWS
  • Manual - Dynamic destination per message, sending managed by framework
• Output batching style (processors only)
  • Unbatched - One batch per message
  • Batch - Outputs from all microbatches are combined independent of source
  • Microbatch -- Outputs from each microbatch are combined
• Message source types
  • SQS
  • SNS
  • Kinesis Firehose
  • Kinesis?
  • EventBridge?
  • CloudWatch Logs?
  • DynamoDB Streams?
• Message destination types
  • SQS
  • SNS
  • Kinesis Firehose
  • Kinesis?
  • EventBridge?

[sigpwned]

More thoughts on requirements and approach:

Lammy messaging

MessageProcessor and MessageConsumer
Use baked-in serialization
Lambda-facing request type is SQS, SNS, etc.
Lambda-facing response is void for both, since we either consume the message or send to destination and return nothing.

Message will be a model type in Lammy. It will have id, headers, and body. Body is string.
MessageEncoder / MessageDecoder — convert from AWS SQS/SNS/etc request type to Message, and from Message to SQS/SNS/etc message type for making API requests. (Deserializer relationship with Sink?)

MessageFilter interfaces for request and response processing.

MessageBean is same as Message, but body is a bean. Generic type.

MessageBeanConsumer and MessageBeanProcessor.

We will use MessageSerializer for the mapping. Default implementations for CustomPojoSerializer, ContextAwareCustomPojoSerializer.

MessageBeanFilter for request and response types. Coexists with MessageFilter.

MessageSender (?) MessageSink (?) is responsible for sending the message to the appropriate destination.

Most of the detail-oriented work will be in building the Sinks with batching.