Network optimization with sendmmsg (part 3) - #204
Conversation
Requires Java 25 for FFM support. Only works on Linux. The old implementation will be used otherwise.
BalaM314
changed the title
|
Flame graphs: patched, unpatched, patched-2, unpatched-2 were taken back to back in the same world, toggling the optimization on and off. We had ~3000 units (mostly 1000 each of dagger, mace, crawler) and 30 players. (We see similar numbers in real gameplay, for example in the map The Rift) There was also a bunch of smites running to generate artificial lag. |
| java{ | ||
| sourceCompatibility = 1.8 | ||
| targetCompatibility = 1.8 | ||
| sourceCompatibility = JavaVersion.VERSION_25 |
There was a problem hiding this comment.
This is a complete non-starter; targeting 25 breaks compilation for Android, iOS and desktop. You will need to find some way to optionally load the 25-target code without changing the main build script or target.
| ArcNet.handleError(e); | ||
| con.close(DcReason.error); | ||
| } | ||
| } else { |
There was a problem hiding this comment.
Lots of incorrect formatting in the PR (extra spaces everywhere)
| public void bind(InetSocketAddress tcpPort, InetSocketAddress udpPort) throws IOException{ | ||
| close(); | ||
| try { | ||
| multi = new MultiUdpSender(); |
There was a problem hiding this comment.
Should be gated behind OS.javaVersion >= 25
| public void bind(DatagramChannel channel) throws Exception { | ||
| //Channel must be already bound | ||
|
|
||
| var fdField = channel.getClass().getDeclaredField("fd"); |
There was a problem hiding this comment.
Using Reflect.get() would be cleaner, wouldn't it?
There was a problem hiding this comment.
Oh, yeah
I didn't have Arc while I was prototyping this
| //It's complaining because it thinks the GC could mess with the data while we're accessing it | ||
| //This is safe because the method is marked as critical, so the GC will not run while the method is running | ||
| //and the method immediately copies the data out into the kernel buffer before returning | ||
| //TODO: Does this have to be marked as critical to block GC while this is running? |
There was a problem hiding this comment.
Using Reference.reachabilityFence() can help?
3 participants
Use sendmmsg syscall directly instead of sending UDP data to each connection.
Currently, UDP data that needs to be sent to all connections (like entity snapshots) is sent by serializing once, compressing once, then calling DatagramSocket.send() on each individual connection. This results in O(pe) syscalls. With the volume of data being sent, syscall overhead is significant. There also may be some overhead on each call going through the JVM classes, according to my flame graphs.
The sendmmsg syscall allows sending multiple UDP messages with one syscall. We call this through Java FFM, cutting a factor of p in the number of syscalls.
this is still very much a prototype, but it's a functional prototype (uploaded to fish servers and we're seeing a performance improvement). It's probably at least a little broken and unsafe.
Caveats: