Fetch env file from coordinator

[sjg20]

Description

Adds a way for labgrid-client to fetch the env file from the coordinator instead of requiring every user to keep a local copy in sync.

• The coordinator gains a new GetEnvironment RPC and an --environment FILE flag pointing at the YAML env file to serve. The file is read fresh on each request, so admins can edit it in place.
• The client treats the literal string coordinator: as a special LG_ENV / --config value. When set, it issues a one-shot GetEnvironment RPC, writes the YAML under $XDG_CACHE_HOME/labgrid (or ~/.cache/labgrid), and loads it like any other env file. Per-user paths inside the served env are still resolvable via the existing LG_* template substitution.
• Default behaviour is unchanged: with no --environment flag on the coordinator, GetEnvironment returns an empty string and clients keep loading env from a local file.

What is the feature used for? Triage and console access from a laptop, scaling to larger labs with many clients, and removing the out-of-band env-file distribution step (shared FS, git checkout, scp, etc).

How does labgrid benefit? Lower friction for casual remote use - a remote user only needs labgrid, network access to the coordinator and three env vars (PATH, LG_COORDINATOR, LG_ENV=coordinator:) to run labgrid-client console -p <board> from anywhere.

How was it verified? Ran the coordinator with --environment <file> and a client with LG_ENV=coordinator: against it locally; confirmed the YAML is fetched, cached under ~/.cache/labgrid, and that subsequent commands resolve LG_* template paths correctly. Confirmed that with no --environment flag, behaviour is unchanged.

No hardware-specific changes.

Checklist

• Documentation for the feature
• Tests for the feature
• Add a section on how to use the feature to doc/usage.rst
• PR has been tested
• Man pages have been regenerated

[codecov]

Codecov Report

❌ Patch coverage is 2.70270% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.9%. Comparing base (db1cba3) to head (a71588f).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
labgrid/remote/client.py	4.3%	22 Missing ⚠️
labgrid/remote/coordinator.py	0.0%	14 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #1866     +/-   ##
========================================
- Coverage    46.0%   45.9%   -0.1%     
========================================
  Files         180     180             
  Lines       14439   14472     +33     
========================================
+ Hits         6654    6655      +1     
- Misses       7785    7817     +32     

Flag	Coverage Δ
3.10	45.9% <2.7%> (-0.1%)	⬇️
3.11	45.9% <2.7%> (-0.1%)	⬇️
3.12	45.9% <2.7%> (-0.1%)	⬇️
3.13	45.9% <2.7%> (-0.1%)	⬇️
3.14	45.9% <2.7%> (-0.1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[flxzt]

from the experience of using labgrid, this change sounds great. Reducing complexity on user-side is something I would welcome very much.

[Bastian-Krause]

I haven't looked in detail at this, but I'm currently experimenting with a rather similar change at https://github.com/Bastian-Krause/labgrid/commits/bst/coordinator-remote-env/. It's still a prototype, things might change, but I think that's the general direction we'd like to head.

I'll open a RFC PR with this next week.

[Emantor]

Both of these approaches lock the environment down to one target, which limits the usefulness to single target tests. While this is definitely useful on it's own, we should rethink the concept to make multi-target environments possible as well.

[sjg20]

Both of these approaches lock the environment down to one target, which limits the usefulness to single target tests. While this is definitely useful on it's own, we should rethink the concept to make multi-target environments possible as well.

Thanks for looking at this.

I am actually using this approach with about 30 targets. Do you mean a single exporter?

I have some local changes which allow each exporter to send an environment and have the coordinator merge them. I haven't sent it yet since I thought this PR was a good starting point. Let me know if I should send that too.

But in any case this PR is likely just a starting point.

[Emantor]

You are very correct, sorry, that was my fault. I will try to find time to take a closer look some time next week.

[Bastian-Krause]

Both of these approaches lock the environment down to one target, which limits the usefulness to single target tests. While this is definitely useful on it's own, we should rethink the concept to make multi-target environments possible as well.

That's not the case for my approach: The remote env is applied per place (optionally combined with a local env), so multi target envs still work. For pytest, a minimal example (relying solely on the remote envi) would be:

targets:
  test1:
    resources:
      RemotePlace:
        name: test1
  test2:
    resources:
      RemotePlace:
        name: test2

But let's continue the discussion in the draft PR I'm going to open.

[Bastian-Krause]

@sjg20 Could you give an example of an environment you give the exporter? Does it contain all your targets (including drivers and resources)? Looking at the env skeleton in the docs, do you also store options, features, paths, images, tools and imports in there?

[sjg20]

@sjg20 Could you give an example of an environment you give the exporter? Does it contain all your targets (including drivers and resources)? Looking at the env skeleton in the docs, do you also store options, features, paths, images, tools and imports in there?

Yes, please see here for exporter and environment:

asc.txt
asc2.txt

[jluebbe]

asc2.txt

This contains many different targets in a single environment file.
Each environment is intended to describe a single test environment, not a whole lab.
It abstracts different test environments from the perspective of a test suite/client program.
As such it doesn't make sense to have only one, as then you lose the ability to cleanly switch between test environments.

I think I explained this to @sjg20 before, but don't have time to dig it up at the moment.

To be explicit: I'm going to NACK any change which is motivated only by not using labgrid's abstractions as they are intended to be used.

I see the need to share more of the test environment's configuration between users and @Bastian-Krause's approach will give us a way to do that, while not breaking with existing abstractions.

[sjg20]

asc2.txt

This contains many different targets in a single environment file. Each environment is intended to describe a single test environment, not a whole lab. It abstracts different test environments from the perspective of a test suite/client program. As such it doesn't make sense to have only one, as then you lose the ability to cleanly switch between test environments.

I think I explained this to @sjg20 before, but don't have time to dig it up at the moment.

Yes, it is here:

#1375

[Bastian-Krause]

I haven't looked in detail at this, but I'm currently experimenting with a rather similar change at https://github.com/Bastian-Krause/labgrid/commits/bst/coordinator-remote-env/. It's still a prototype, things might change, but I think that's the general direction we'd like to head.

I'll open a RFC PR with this next week.

Here it is: #1889