From 9407a60138156cf6ad18495b915009ecf354c96f Mon Sep 17 00:00:00 2001 From: Stewart Gebbie Date: Mon, 22 Nov 2021 14:50:17 +0200 Subject: [PATCH 1/4] Drafting an RFC for Unix Domain Socket Support This RFC will provide details around the intention for adding support for Unix Domain Sockets to the Pony runtime library and the `net` package. --- text/0000-unix-domain-sockets.md | 64 ++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 text/0000-unix-domain-sockets.md diff --git a/text/0000-unix-domain-sockets.md b/text/0000-unix-domain-sockets.md new file mode 100644 index 00000000..b51563e6 --- /dev/null +++ b/text/0000-unix-domain-sockets.md @@ -0,0 +1,64 @@ +- Feature Name: Unix Domain Socket Support +- Start Date: 2021-11-22 +- RFC PR: (leave this empty) +- Pony Issue: (leave this empty) + +# Summary + +The intent is to extend to runtime library, together with the `net` package, to +add support for Unix domain sockets. That is, currently there is support for TCP +and UDP sockets, but not for Unix domain sockets. + +# Motivation + +Unix domain sockets, and in particular, named Unix domain sockets can provide +interprocess communications support that is well suited to modular component +development under Unix. Additionally, integration with existing services +sometimes requires interaction via named unix domain sockets. + +# Detailed design + +TODO + +This is the bulk of the RFC. Explain the design in enough detail for somebody familiar with the language to understand, and for somebody familiar with the compiler to implement. This should get into specifics and corner-cases, and include examples of how the feature is used. + +# How We Teach This + +TODO + +What names and terminology work best for these concepts and why? How is this idea best presented? As a continuation of existing Pony patterns, or as a wholly new one? + +Would the acceptance of this proposal mean the Pony guides must be re-organized or altered? Does it change how Pony is taught to new users at any level? + +How should this feature be introduced and taught to existing Pony users? + +# How We Test This + +TODO + +How do we assure that the initial implementation works? How do we assure going forward that the new functionality works after people make changes? Do we need unit tests? Something more sophisticated? What's the scope of testing? Does this change impact the testing of other parts of Pony? Is our standard CI coverage sufficient to test this change? Is manual intervention required? + +In general this section should be able to serve as acceptance criteria for any implementation of the RFC. + +# Drawbacks + +TODO + +Why should we *not* do this? Things you might want to note: + +* Breaks existing code +* Introduces instability into the compiler and or runtime which will result in bugs we are going to spend time tracking down +* Maintenance cost of added code + +# Alternatives + +TODO + +What other designs have been considered? What is the impact of not doing this? +None is not an acceptable answer. There is always to option of not implementing the RFC. + +# Unresolved questions + +TODO + +What parts of the design are still TBD? From 72ea78fa68c13a839a005da769d5525b28c8663d Mon Sep 17 00:00:00 2001 From: Stewart Gebbie Date: Tue, 23 Nov 2021 11:38:16 +0200 Subject: [PATCH 2/4] Initial rough draft of a unix domain socket RFC. The intent would be to add support for creating and using Unix domain sockets from within Pony programs. In particular, this RFC suggests adding this support to the `net` package. --- text/0000-unix-domain-sockets.md | 81 +++++++++++++++++++++++--------- 1 file changed, 60 insertions(+), 21 deletions(-) diff --git a/text/0000-unix-domain-sockets.md b/text/0000-unix-domain-sockets.md index b51563e6..747c300a 100644 --- a/text/0000-unix-domain-sockets.md +++ b/text/0000-unix-domain-sockets.md @@ -16,49 +16,88 @@ interprocess communications support that is well suited to modular component development under Unix. Additionally, integration with existing services sometimes requires interaction via named unix domain sockets. +Part of the motivation for supporting Unix domain sockets is the support for +connection oriented boundary preserving reads and writes. That is, the +`SOCK_SEQPACKET` type, under the `AF_UNXI` domain, provides reliable and +sequential datagram style sockets. This is useful for implementing system +services. + # Detailed design -TODO +We will provide implementations of: -This is the bulk of the RFC. Explain the design in enough detail for somebody familiar with the language to understand, and for somebody familiar with the compiler to implement. This should get into specifics and corner-cases, and include examples of how the feature is used. +- `UnixConnection` +- `UnixConnectionNotify` +- `UnixListener` +- `UnixListenerNotify` -# How We Teach This +When creating a new connection the socket type can be set to one of: + +- `SOCK_STREAM` +- `SOCK_DGRAM` +- `SOCK_SEQPACKET` + +When creating a new listener the socket type can be set to one of: + +- `SOCK_STREAM` +- `SOCK_SEQPACKET` + +Beyond the above coarse grained API types, it is expected that the actual +methods that are exposed will be similar to those of the TCP or UDP +implementations. -TODO +TODO/WIP - this section does not contains enough detail yet. -What names and terminology work best for these concepts and why? How is this idea best presented? As a continuation of existing Pony patterns, or as a wholly new one? +# How We Teach This -Would the acceptance of this proposal mean the Pony guides must be re-organized or altered? Does it change how Pony is taught to new users at any level? +As with TCP and UDP sockets, the in-code documentation would provide sufficient +examples of the usage. -How should this feature be introduced and taught to existing Pony users? +Most of the API patterns, in terms of notifiers etc., will be common to the Unix +domain sockets and the other supported socket types. # How We Test This -TODO +It will be possible to implement unit tests to create unix domain sockets and +then exercise both the server side listening code and the client side connection +code. -How do we assure that the initial implementation works? How do we assure going forward that the new functionality works after people make changes? Do we need unit tests? Something more sophisticated? What's the scope of testing? Does this change impact the testing of other parts of Pony? Is our standard CI coverage sufficient to test this change? Is manual intervention required? +Additionally, the build pipelines will be able to test this on all of the +supported Unix operating systems. -In general this section should be able to serve as acceptance criteria for any implementation of the RFC. +While the final implementation may benefit from reusing some o the existing TCP +or UDP code, the impact should be mitigated by existing testing. # Drawbacks -TODO +## OS Specific -Why should we *not* do this? Things you might want to note: +By definition, the Unix domain sockets are specific to particular supported +operating systems. -* Breaks existing code -* Introduces instability into the compiler and or runtime which will result in bugs we are going to spend time tracking down -* Maintenance cost of added code +However, this would not impact TCP or UDP, and the programmer would be aware of +using OS specific features. On other OSes a suitable error can be returned. # Alternatives -TODO - -What other designs have been considered? What is the impact of not doing this? -None is not an acceptable answer. There is always to option of not implementing the RFC. +TODO/WIP - more alternatives probably need to be considered. # Unresolved questions -TODO +## Naming + +Should the new components be named: `UnixConnection`, `UnixListener` etc.? Or, +should the components be named: `UNIXConnection`, `UNIXListener` etc.? + +## Abstract Socket Addresses + +Should there be explicit support for the Linux specific abstract socket +addresses? Or, do we simply allow names to start with `\0` in order to enable +the abstract naming convention? + +## Sharing interface definitions -What parts of the design are still TBD? +Given the similarities between the TCP domain and the Unix domain (when using +the stream or sequential-packet socket types), or between the UDP domain and the +Unix domain (when using the datagram or sequential-packet socket types), should +there be an attempt to pull out common interface types? From 875292a9c6682481b6305812c9016087f6ee7450 Mon Sep 17 00:00:00 2001 From: Stewart Gebbie Date: Tue, 23 Nov 2021 17:55:21 +0200 Subject: [PATCH 3/4] Alternative: use ASIO directly. Using ASIO directly from Pony code might reduce the need for changing the runtime code. --- text/0000-unix-domain-sockets.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/0000-unix-domain-sockets.md b/text/0000-unix-domain-sockets.md index 747c300a..bca8d6ba 100644 --- a/text/0000-unix-domain-sockets.md +++ b/text/0000-unix-domain-sockets.md @@ -82,6 +82,17 @@ using OS specific features. On other OSes a suitable error can be returned. TODO/WIP - more alternatives probably need to be considered. +## Direct use of ASIO + +It might be possible to implement the mechanism needed to unix domain sockets by +using the asio API directly. That way, `libponyrt` might not need to be changed. +Instead, calls from Pony code to `libc` could potentially be used, along with +making use of `_event_notify`. + +For prior-art and an example of direct calls to ASIO, see the experimental +project: +[Lori](https://github.com/seantallen-org/lori/blob/main/lori/pony_asio.pony). + # Unresolved questions ## Naming From bf73480e4b7096756ccc4d64fae992397b7ad759 Mon Sep 17 00:00:00 2001 From: Stewart Gebbie Date: Tue, 23 Nov 2021 22:11:14 +0200 Subject: [PATCH 4/4] Use compile time errors for OSes without unix sockets. Note that we can make use of the `compile_error` directive. --- text/0000-unix-domain-sockets.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-unix-domain-sockets.md b/text/0000-unix-domain-sockets.md index bca8d6ba..53303a74 100644 --- a/text/0000-unix-domain-sockets.md +++ b/text/0000-unix-domain-sockets.md @@ -76,7 +76,8 @@ By definition, the Unix domain sockets are specific to particular supported operating systems. However, this would not impact TCP or UDP, and the programmer would be aware of -using OS specific features. On other OSes a suitable error can be returned. +using OS specific features. On other OSes a suitable compile time error can be +returned by making use of the `ifdef ... else compile_error ... end` directive. # Alternatives