Skip to content

[P2] Add performance benchmarks (JMH) #318

@sfloess

Description

@sfloess

Problem

No performance benchmarks exist to measure and track performance characteristics.

Impact

  • Severity: MEDIUM
  • Category: Performance/Quality
  • Unknown performance characteristics
  • Cannot detect performance regressions
  • No data for optimization decisions
  • Difficult to compare with alternatives

Key Performance Metrics Needed

1. Application Lifecycle

  • Application startup time: Deploy → Start (warm JVM)
  • Application cold start: Deploy → Start (cold JVM)
  • Application shutdown time: Stop → Undeploy
  • Hot reload latency: Old version → New version

2. ClassLoader Operations

  • ClassLoader creation overhead
  • Class loading time (first load vs cached)
  • ClassLoader cleanup/GC impact

3. Resource Monitoring

  • Monitoring overhead (CPU impact of monitoring itself)
  • Snapshot collection latency
  • Metrics aggregation performance

4. Message Bus

  • Publish latency (message → delivered)
  • Throughput (messages/second)
  • Subscriber overhead (N subscribers impact)

5. REST API

  • Deploy endpoint latency
  • List applications latency (N apps)
  • Concurrent request handling

Recommended Framework

Use JMH (Java Microbenchmark Harness):

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class ApplicationLifecycleBenchmark {
    
    private ApplicationManager manager;
    
    @Setup
    public void setup() {
        manager = new ApplicationManager();
    }
    
    @Benchmark
    public void measureApplicationStartup() {
        manager.deploy(descriptor);
        manager.start("test-app");
    }
}

Module Structure

Create new module:

jplatform-benchmarks/
├── pom.xml (JMH dependencies)
└── src/main/java/
    └── org/flossware/jplatform/benchmarks/
        ├── ApplicationLifecycleBenchmark.java
        ├── ClassLoaderBenchmark.java
        ├── MessageBusBenchmark.java
        └── ResourceMonitorBenchmark.java

Running Benchmarks

mvn clean install -pl jplatform-benchmarks
java -jar jplatform-benchmarks/target/benchmarks.jar

CI/CD Integration

Run benchmarks on release and track trends:

- name: Run Performance Benchmarks
  run: mvn -pl jplatform-benchmarks jmh:run -Djmh.resultFormat=JSON
  
- name: Store Benchmark Results
  uses: benchmark-action/github-action-benchmark@v1
  with:
    tool: 'jmh'
    output-file-path: jplatform-benchmarks/target/jmh-result.json

Performance Targets

Set baselines (example):

  • Application startup: < 100ms (warm JVM)
  • Message bus publish: < 1ms (p99)
  • ClassLoader creation: < 50ms
  • REST API deploy: < 200ms

Priority

P2 - Medium - Should be added for v1.3 or v2.0.

Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions