Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions concoursetools/additional.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ class InOnlyConcourseResource(ConcourseResource[DatetimeVersion]):
:meth:`~concoursetools.resource.ConcourseResource.download_version`.

The correct use case of this resource is to execute a
:concourse:`put-step.put-step`, and then place parameters
in the :concourse:`put-step.schema.put.get_params` section:
:concourse:`steps.put`, and then place parameters
in the :concourse:`steps.put#get_params` section:

.. code:: yaml

Expand Down Expand Up @@ -286,7 +286,7 @@ class MultiVersionConcourseResource(TriggerOnChangeConcourseResource[MultiVersio

.. tip::
This resource class is best suited to resources used in conjunction
with the :concourse:`set-pipeline-step`.
with the :concourse:`steps.set_pipeline` step.
"""
def __init__(self, key: str, sub_version_class: type[SortableVersionT]):
self.key = key
Expand Down
23 changes: 13 additions & 10 deletions concoursetools/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@
Build metadata is deliberately not passed to :meth:`~concoursetools.resource.ConcourseResource.fetch_new_versions`,
as none of this metadata is passed to the check environment by Concourse, to avoid antipatterns.

See the Concourse :concourse:`implementing-resource-types.resource-metadata` documentation for more information.
See the Concourse :concourse:`resource types metadata <resource_types.implementing#metadata>` documentation
for more information.
"""
from __future__ import annotations

Expand All @@ -32,8 +33,9 @@ class BuildMetadata: # pylint: disable=invalid-name
:param BUILD_NAME: The build number within the build's job.
:param BUILD_JOB_NAME: The name of the build's job.
:param BUILD_PIPELINE_NAME: The name of the pipeline that the build's job lives in.
:param BUILD_PIPELINE_INSTANCE_VARS: The instance vars of the :concourse:`instanced pipeline <instanced-pipelines>`
that the build's job lives in, serialized as JSON.
:param BUILD_PIPELINE_INSTANCE_VARS: The instance vars of the :concourse:`instanced pipeline
<pipelines.grouping-pipelines#managing-instanced-pipelines>` that the build's
job lives in, serialized as JSON.

.. note::
A few variables are often present in the build environment, but are **not** documented by Concourse:
Expand Down Expand Up @@ -66,22 +68,22 @@ def BUILD_CREATED_BY(self) -> str:

.. warning::
By default this information is **not** available. To enable it, you need to set
:concourse:`resources.schema.resource.expose_build_created_by` in your resource schema.
:concourse:`resources#expose_build_created_by` in your resource schema.
"""
try:
return os.environ["BUILD_CREATED_BY"]
except KeyError as error:
raise PermissionError("The 'BUILD_CREATED_BY' variable has not been made available. This must be enabled "
"with the 'expose_build_created_by' variable within the resource schema: "
"https://concourse-ci.org/resources.html#schema.resource.expose_build_created_by") from error
"https://concourse-ci.org/docs/resources/#expose_build_created_by") from error

@property
def is_one_off_build(self) -> bool:
"""
Return :data:`True` if this build is one-off, and :data:`False` otherwise.

A build is a "one-off" is it is triggered via the Concourse CLI
:concourse:`execute command <tasks.running-tasks>`.
:concourse:`execute command <tasks#running-tasks-with-fly-execute>`.
It is determined by the absence of all of the following attributes:

* ``BUILD_JOB_NAME``
Expand All @@ -96,14 +98,15 @@ def is_one_off_build(self) -> bool:

@property
def is_instanced_pipeline(self) -> bool:
"""Return :data:`True` if this is an :concourse:`instanced pipeline <instanced-pipelines>`."""
"""Return :data:`True` if this is an :concourse:`instanced pipeline <pipelines.grouping-pipelines#managing-instanced-pipelines>`."""
return self.BUILD_PIPELINE_INSTANCE_VARS is not None

def instance_vars(self) -> dict[str, object]:
"""
Return the instance vars set on this pipeline as a mapping.

When working with an :concourse:`instanced pipeline <instanced-pipelines>`, it is much more convenient to
When working with an :concourse:`instanced pipeline
<pipelines.grouping-pipelines#managing-instanced-pipelines>`, it is much more convenient to
work with the instance vars as a mapping instead of a JSON string.

.. note::
Expand All @@ -113,7 +116,7 @@ def instance_vars(self) -> dict[str, object]:
:Example:

If a instanced pipeline has been created from within another pipeline
(using the :concourse:`set-pipeline-step`), such as this:
(using the :concourse:`steps.set_pipeline` step), such as this:

.. code:: yaml

Expand Down Expand Up @@ -169,7 +172,7 @@ def format_string(self, string: str, additional_values: dict[str, str] | None =
Format a string with metadata using standard bash ``$`` notation.

Only a handful of "safe" values will be interpolated, not arbitrary attributes on the instance.
These are the :concourse:`original environment variables <implementing-resource-types.resource-metadata>`,
These are the :concourse:`original environment variables <resource_types.implementing#metadata>`,
including :attr:`BUILD_CREATED_BY` if it exists. object missing environment variable (such as in the case of a
one-off build) will be empty. A ``$BUILD_URL`` variable is also added for ease.

Expand Down
18 changes: 9 additions & 9 deletions concoursetools/parsing.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

def parse_check_payload(raw_json: str) -> tuple[ResourceConfig, VersionConfig | None]:
"""
Parse raw input JSON for a :concourse:`check payload <implementing-resource-types.resource-check>`.
Parse raw input JSON for a :concourse:`check payload <resource-types.implementing#check-check-for-new-versions>`.

:param raw_json: A JSON string of the following form:

Expand Down Expand Up @@ -46,7 +46,7 @@ def parse_check_payload(raw_json: str) -> tuple[ResourceConfig, VersionConfig |

def parse_in_payload(raw_json: str) -> tuple[ResourceConfig, VersionConfig, Params]:
"""
Parse raw input JSON for an :concourse:`in payload <implementing-resource-types.resource-in>`.
Parse raw input JSON for an :concourse:`in payload <resource-types.implementing#in-fetch-a-given-resource>`.

:param raw_json: A JSON string of the following form:

Expand Down Expand Up @@ -78,7 +78,7 @@ def parse_in_payload(raw_json: str) -> tuple[ResourceConfig, VersionConfig, Para

def parse_out_payload(raw_json: str) -> tuple[ResourceConfig, Params]:
"""
Parse raw input JSON for an :concourse:`out payload <implementing-resource-types.resource-out>`.
Parse raw input JSON for an :concourse:`out payload <resource-types.implementing#out-update-a-resource>`.

:param raw_json: A JSON string of the following form:

Expand Down Expand Up @@ -116,7 +116,7 @@ def parse_metadata(metadata_pairs: list[MetadataPair]) -> Metadata:

def format_check_output(version_configs: list[VersionConfig], **json_kwargs: Any) -> str:
"""
Format :concourse:`check output <implementing-resource-types.resource-check>` as a JSON string.
Format :concourse:`check output <resource-types.implementing#check-check-for-new-versions>` as a JSON string.

:param version_configs: A list of version configurations.
:param json_kwargs: Additional keyword arguments to pass to :func:`json.dumps`.
Expand All @@ -136,8 +136,8 @@ def format_check_output(version_configs: list[VersionConfig], **json_kwargs: Any

def format_in_out_output(version_config: VersionConfig, metadata: Metadata, **json_kwargs: Any) -> str:
"""
Format :concourse:`in output <implementing-resource-types.resource-in>` or
:concourse:`out output <implementing-resource-types.resource-out>` as a JSON string.
Format :concourse:`in output <resource-types.implementing#in-fetch-a-given-resource>` or
:concourse:`out output <resource-types.implementing#out-update-a-resource>` as a JSON string.

:param version_config: A version configuration.
:param metadata: A key-value mapping of metadata.
Expand Down Expand Up @@ -175,7 +175,7 @@ def format_metadata(metadata: Metadata) -> list[MetadataPair]:

def format_check_input(resource_config: ResourceConfig, version_config: VersionConfig | None = None, **json_kwargs: Any) -> str:
"""
Format :concourse:`check input <implementing-resource-types.resource-check>` as a JSON string.
Format :concourse:`check input <resource-types.implementing#check-check-for-new-versions>` as a JSON string.

:param resource_config: A resource configuration.
:param version_config: A version configuration, or :data:`None` if no version currently exists.
Expand All @@ -201,7 +201,7 @@ def format_check_input(resource_config: ResourceConfig, version_config: VersionC

def format_in_input(resource_config: ResourceConfig, version_config: VersionConfig, params: Params | None = None, **json_kwargs: Any) -> str:
"""
Format :concourse:`in input <implementing-resource-types.resource-in>` as a JSON string.
Format :concourse:`in input <resource-types.implementing#in-fetch-a-given-resource>` as a JSON string.

:param resource_config: A resource configuration.
:param version_config: A version configuration.
Expand Down Expand Up @@ -232,7 +232,7 @@ def format_in_input(resource_config: ResourceConfig, version_config: VersionConf

def format_out_input(resource_config: ResourceConfig, params: Params | None = None, **json_kwargs: Any) -> str:
"""
Format :concourse:`out input <implementing-resource-types.resource-out>` as a JSON string.
Format :concourse:`out input <resource-types.implementing#out-update-a-resource>` as a JSON string.

:param resource_config: A resource configuration.
:param params: Optional parameters to be passed.
Expand Down
8 changes: 4 additions & 4 deletions concoursetools/resource.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@
Configuring the same resource in any pipeline on any Concourse cluster will
behave the exact same way. Concourse will continuously check each configured
resource to discover new versions. These versions then flow through the pipeline
via :concourse:`get steps <get-step.get-step>` configured on jobs.
via :concourse:`get steps <steps.get>` configured on jobs.

Find out more about resources in the :concourse:`Concourse resource documentation <resources>`.

To learn more about how Concourse resource types are actually implemented under the hood,
check out :concourse:`implementing-resource-types` in Concourse.
check out :concourse:`implementing resource types <resource-types.implementing>` in Concourse.
"""
from __future__ import annotations

Expand All @@ -32,7 +32,7 @@ class ConcourseResource(ABC, Generic[VersionT]):
"""
Represents an external input or output to a pipeline.

The :concourse:`resource-types.schema.resource_type.source`
The :concourse:`resource-types#source`
defined in a Concourse :concourse:`pipeline <pipelines>` is
parsed into JSON by Concourse, and will be passed to the initialiser
of the :class:`ConcourseResource` class.
Expand Down Expand Up @@ -94,7 +94,7 @@ def certs_dir(self) -> Path:

This folder may not always exist, depending on how the Concourse runner was configured.

See the :concourse:`implementing-resource-types.resource-certs` documentation for more information.
See the :concourse:`resource-types.implementing#certificate-propagation` documentation for more information.
"""
return Path("/etc/ssl/certs")

Expand Down
6 changes: 3 additions & 3 deletions concoursetools/testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -395,13 +395,13 @@ class FileTestResourceWrapper(TestResourceWrapper[Version]):
This is best to use if your resource uses external scripts for any of the steps.

:param inner_resource_config: The JSON configuration for the resource.
:param check_script: The path to the external script for the :concourse:`check <implementing-resource-types.resource-check>`.
:param check_script: The path to the external script for the :concourse:`check <resource-types.implementing#check-check-for-new-versions>`.
Setting to :data:`None` (default) means that :meth:`fetch_new_versions`
raises :class:`NotImplementedError`.
:param in_script: The path to the external script for the :concourse:`check <implementing-resource-types.resource-in>`.
:param in_script: The path to the external script for the :concourse:`check <resource-types.implementing#in-fetch-a-given-resource>`.
Setting to :data:`None` (default) means that :meth:`download_version`
raises :class:`NotImplementedError`.
:param out_script: The path to the external script for the :concourse:`check <implementing-resource-types.resource-out>`.
:param out_script: The path to the external script for the :concourse:`check <resource-types.implementing#out-update-a-resource>`.
Setting to :data:`None` (default) means that :meth:`publish_new_version`
raises :class:`NotImplementedError`.
:param directory_dict: The initial state of the resource directory. See :class:`~concoursetools.mocking.TemporaryDirectoryState`
Expand Down
6 changes: 3 additions & 3 deletions concoursetools/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ class breaks the :wikipedia:`Liskov substitution principle`. If you wish to util
ResourceConfig = dict[str, Any]
"""
Represents arbitrary configuration passed to a Concourse resource.
See the :concourse:`config-basics.schema.config` schema for more information.
See the :concourse:`config-basics#config-schema` for more information.
"""

Params = dict[str, Any]
"""
Represents arbitrary parameters passed to a Concourse resource.
See the :concourse:`config-basics.schema.config` schema for more information.
See the :concourse:`config-basics#config-schema` for more information.
"""

Metadata = dict[str, str]
Expand All @@ -48,7 +48,7 @@ class MetadataPair(TypedDict):
VersionConfig = dict[str, str]
"""
Represents a version of a Concourse resource.
See the :concourse:`config-basics.schema.version` schema for more information.
See the :concourse:`config-basics#version-schema` for more information.
"""

VersionT = TypeVar("VersionT", bound="VersionProtocol")
Expand Down
2 changes: 1 addition & 1 deletion concoursetools/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A resource version represents the exact state of a resource at a given point in time. The resource is responsible for
defining what a version *actually is*, and this is defined with Concourse Tools using a :class:`Version` subclass.

More information on versions can be found in the :concourse:`Concourse documentation <resource-versions>`.
More information on versions can be found in the :concourse:`Concourse documentation <resources.resource-versions>`.
"""
from __future__ import annotations

Expand Down
8 changes: 4 additions & 4 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
accessible-pygments==0.0.5
alabaster==1.0.0
babel==2.17.0
babel==2.18.0
beautifulsoup4==4.14.3
certifi==2026.1.4
charset-normalizer==3.4.4
Expand All @@ -10,14 +10,14 @@ idna==3.11
imagesize==1.4.1
Jinja2==3.1.6
MarkupSafe==3.0.3
packaging==25.0
packaging==26.0
Pygments==2.19.2
requests==2.32.5
roman-numerals==4.1.0
snowballstemmer==3.0.1
soupsieve==2.8.1
soupsieve==2.8.3
Sphinx==9.1.0
sphinx-autodoc-typehints==3.6.2
sphinx-autodoc-typehints==3.6.3
sphinx-basic-ng==1.0.0b2
sphinxcontrib-applehelp==2.0.0
sphinxcontrib-devhelp==2.0.0
Expand Down
6 changes: 3 additions & 3 deletions docs/source/changelog/0.8.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ Build Metadata string formatting
Both :meth:`~concoursetools.resource.ConcourseResource.download_version` and
:meth:`~concoursetools.resource.ConcourseResource.publish_new_version` methods are passed an instance of
:class:`~concoursetools.metadata.BuildMetadata` to interface with the
:concourse:`implementing-resource-types.resource-metadata`. This allows developers to easily interface with this
metadata from within the resource code.
:concourse:`resource metadata <resource-types.implementing#metadata>`. This allows developers to easily interface with
this metadata from within the resource code.

However, many resources want to make this metadata available to the end user in their pipelines, by allowing them to
reference the variables in commit messages, slack notifications and emails. Previously, resources would need to use
Expand Down Expand Up @@ -245,7 +245,7 @@ Allowed ``source`` to be empty in resource configuration


Recall that the parameters of the ``__init__`` method of your :class:`~concoursetools.resource.ConcourseResource`
subclass are taken from the :concourse:`resource-types.schema.resource_type.source` block of the pipeline YAML.
subclass are taken from the :concourse:`resource-types#source` block of the pipeline YAML.
This means that a resource which looks like this:

.. code:: python
Expand Down
4 changes: 2 additions & 2 deletions docs/source/examples/branches.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ On occasion, you may wish your resource to emit versions when something has chan

But what happens when a check yields two versions from different branches? We could figure out the "latest" version by sorting by commit date, but if we are tracking multiple branches then we don't want to ignore a commit on one just because a more recent commit has been made on another. Previous in Concourse this was "fixed" by passing ``version: every`` to the resource, but this means that if multiple commits are pushed to a branch at once, then each commit will emit a build, which may not be what we want.

What we really want is to iterate over all branches in our repository, and spin up a branch-specific pipeline for each one using the :concourse:`set-pipeline-step`. This is easy to do, but we need to make sure that new pipelines are added when new branches appear, and that old pipelines are deleted when their branches are removed. We specifically require a resource which will trigger a pipeline whenever something has changed. The generic resource for this pattern is the :class:`~concoursetools.additional.TriggerOnChangeConcourseResource`, but because the "state" is made up of several "sub versions", it makes sense to instead utilise the :class:`~concoursetools.additional.MultiVersionConcourseResource`, which takes care of much of the boiler plate for us, and also allow us to automatically download these subversions as JSON so that we may iterate over them.
What we really want is to iterate over all branches in our repository, and spin up a branch-specific pipeline for each one using the :concourse:`steps.set_pipeline` step. This is easy to do, but we need to make sure that new pipelines are added when new branches appear, and that old pipelines are deleted when their branches are removed. We specifically require a resource which will trigger a pipeline whenever something has changed. The generic resource for this pattern is the :class:`~concoursetools.additional.TriggerOnChangeConcourseResource`, but because the "state" is made up of several "sub versions", it makes sense to instead utilise the :class:`~concoursetools.additional.MultiVersionConcourseResource`, which takes care of much of the boiler plate for us, and also allow us to automatically download these subversions as JSON so that we may iterate over them.

Branch Version
--------------
Expand Down Expand Up @@ -66,7 +66,7 @@ We only need to implement the :meth:`~concoursetools.additional.MultiVersionConc

The logic required here is incredibly minimal, and we only need to return each available subversion. The returned set will then be sorted by the resource when converting it to its final version, which is why we needed the subversion to be sortable and hashable.

When the branches change (either with new ones added or existing ones removed) a new version will be emitted and the pipeline will be triggered. Users can then iterate over them in their pipeline using a combination of the :concourse:`across-step` and the :concourse:`set-pipeline-step`:
When the branches change (either with new ones added or existing ones removed) a new version will be emitted and the pipeline will be triggered. Users can then iterate over them in their pipeline using a combination of the :concourse:`steps.modifier-and-hooks.across` step modifier and the :concourse:`steps.set_pipeline` step:

.. code:: yaml

Expand Down
Loading