Add initial version of flexmeasures-openadr3 plugin#1
Conversation
|
Migrate to using ingestion redis queue |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
There was a problem hiding this comment.
I'd like some clarification about dependencies. I'm not sure what the uv.lock file is used for?
Maybe it is useful for running only this plugin, and it installs FlexMeasures + dependencies?
How compatible is that with running multiple plugins?
In our practical experience, we have a FlexMeasures docker image, and we might create a dedicated one with some custom plugin with some extra requirements installed. And we might install extra plugins (like this one) even at the deployment step (e.g. with ansible).
There was a problem hiding this comment.
The uv.lock file locks dependencies to a specific version to ensure a consistent environment.
Running multiple plugins should be relatively straightforward, flexmeasures has also updated to using UV for dependency management.
In the dockerfile in the repository you can see how the plugin can be installed the same way it was done before (just with a uv pip wrapper to ensure we are in a venv).
Furthermore, older plugins that just run on pip can also be installed in this manner
| ) | ||
| db.session.commit() | ||
| finally: | ||
| self.schedule(ven_client, sensor_config, replace_existing=False) |
There was a problem hiding this comment.
The chain of scheduled ingestion jobs would break if one job fails before this point. The finally safeguards against many sources of failure, but if a worker is restarted during job handling before reaching this point, that is problematic.
I'm flagging this as a potential follow-up to move to rq.cron.CronScheduler, which will require a dedicated rq-cron-scheduler service in docker-compose.yml running something like:
from rq.cron import CronScheduler
cron = CronScheduler(connection=app.redis_connection)
cron.start()
And then registering recurring jobs with something like:
cron.register(
self._execute,
queue_name=app.queues[FETCH_EVENTS_QUEUE_NAME],
cron_string="0 0 * * *",
kwargs={"ven_id": ven_client.id, "config_name": sensor_config.name},
)
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
|
Apologies for the delay, it's been a few hectic weeks in preparation of the residential flexibility event next week. I'm going to be going over the codebase once more on the 22nd of June to process all feedback, take another look with fresh eyes at the codebase and answer any outstanding questions. I'll be in touch! |
|
A few tests seem to have broken in the meantime with the new release, I'll fix this and the pipeline sometime this week when I find a quiet moment again. Along with that, I'll look into implementing a better cronscheduler while i'm at it. |
|
Just a heads-up: We will support secret storage and token refreshing strartegies in FlexMeasures v1.0, see the merged PR and the docs here. That implementation is partly inspired by code in this PR, so maybe we can simplify this PR? we can also do that in a follow-up, of course. We could create a version which is compatible with FM v0.33, then one that works with FM v1.0. Would love to hear your thoughts. |
4 participants
No description provided.