feat(backend): implement proxy cache repository filter API#23527
feat(backend): implement proxy cache repository filter API#23527stonezdj wants to merge 1 commit into
Conversation
Introduce repository filter pattern and kind on proxy cache projects to explicitly allow selected repositories. Requests that do not match the filter are denied before Harbor proxies content from upstream. Fixes goharbor#13231 Signed-off-by: stonezdj <stone.zhang@broadcom.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #23527 +/- ##
==========================================
- Coverage 66.38% 66.34% -0.04%
==========================================
Files 1072 1073 +1
Lines 117541 117665 +124
Branches 2965 2965
==========================================
+ Hits 78024 78062 +38
- Misses 35224 35306 +82
- Partials 4293 4297 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
918d806 to
e4763ca
Compare
There was a problem hiding this comment.
Pull request overview
Adds configurable repository allowlist filtering for proxy-cache projects.
Changes:
- Adds doublestar and regex repository matchers.
- Exposes and validates filter metadata through project APIs.
- Enforces filtering for manifest proxying and adds matcher tests.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
api/v2.0/swagger.yaml |
Documents filter fields. |
src/lib/pattern/matcher.go |
Implements repository matching. |
src/lib/pattern/matcher_test.go |
Tests matcher behavior. |
src/pkg/project/models/pro_meta.go |
Defines metadata keys. |
src/server/middleware/repoproxy/proxy.go |
Filters manifest requests. |
src/server/middleware/repoproxy/proxy_test.go |
Tests proxy filter matching. |
src/server/v2.0/handler/model/project.go |
Hides fields for regular projects. |
src/server/v2.0/handler/project.go |
Validates project filter metadata. |
src/server/v2.0/handler/project_metadata.go |
Supports metadata API updates. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Apply repository filter: if proxy_cache_filter_pattern is set, the repository must match | ||
| if filterPattern, ok := p.GetMetadata(proModels.ProMetaProxyCacheFilterPattern); ok && filterPattern != "" { |
| // see https://github.com/goharbor/harbor/issues/12940 to get more info | ||
| if params.Project.Metadata != nil && p.IsProxy() { | ||
| params.Project.Metadata.EnableContentTrust = nil | ||
| if err := validateProxyCacheRepositoryFilter(params.Project.Metadata); err != nil { |
| metas[proModels.ProMetaMaxUpstreamConn] = strconv.FormatInt(v, 10) | ||
| case proModels.ProMetaProxyCacheFilterPattern: | ||
| // plain pattern string; empty means match all — no further validation needed at metadata level | ||
| if err := pattern.ValidateRepositoryFilter(value, pattern.KindDoublestar); err != nil { |
There was a problem hiding this comment.
Is hard code the KindDoublestar intended?
|
Comparing this against the merged proposal (goharbor/community#280), a few deviations: 1. Pattern validation at the metadata endpoint ignores the configured kind The proposal requires: "Invalid pattern in In case proModels.ProMetaProxyCacheFilterPattern:
if err := pattern.ValidateRepositoryFilter(value, pattern.KindDoublestar); err != nil {With 2. Metadata endpoint accepts the keys on non-proxy projects Proposal: "For non-proxy projects, these keys are ignored on create/update and omitted from response." Create/Update strip them and 3. Regex "auto-anchoring" isn't equivalent to real anchoring Proposal: regex is "always auto-anchored." 4. No audit entry when the filter changes Proposal, Audit section: "Record project update audit entries when filter set is changed." Only the blocked-pull log is implemented. 5. Blob path is not filtered (matches the proposal's implementation section, but not its Abstract) Enforcement is in |
|
backward-compatibility analysis
|
| if err != nil { | ||
| return false, err | ||
| } | ||
| match := re.FindStringIndex(value) |
There was a problem hiding this comment.
FindStringIndex returns the leftmost match, not necessarily a full-string match, so this can return false negatives.
Example: pattern a|ab against value ab — the leftmost match is a ([0,1]), so this returns false even though the pattern matches the whole string.
| // filterPattern is the plain pattern string; filterKind is "doublestar" or "regex" (defaults to doublestar when empty). | ||
| // If filterPattern is empty, all repositories are allowed. | ||
| // An invalid pattern is treated as no match. | ||
| func matchRepositoryFilter(repository, filterPattern, filterKind string) bool { |
There was a problem hiding this comment.
the matchRepositoryFilter is applied only for manifest, should we apply it for blob?
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package pattern | ||
|
|
||
| import ( | ||
| "regexp" | ||
| "strings" | ||
|
|
||
| "github.com/bmatcuk/doublestar" | ||
|
|
||
| "github.com/goharbor/harbor/src/lib/errors" | ||
| ) | ||
|
|
||
| const ( | ||
| // KindRegex indicates regular expression pattern matching | ||
| KindRegex = "regex" | ||
| // KindDoublestar indicates doublestar (glob) pattern matching | ||
| KindDoublestar = "doublestar" | ||
| ) | ||
|
|
||
| // RepositoryFilter represents a repository filter configuration with pattern and kind | ||
| type RepositoryFilter struct { | ||
| // Filter is the pattern expression to match against | ||
| Filter string | ||
| // Kind is the type of pattern matching: "regex" or "doublestar" | ||
| Kind string | ||
| } | ||
|
|
||
| // NewRepositoryFilter creates a RepositoryFilter from separate pattern and kind strings. | ||
| // If kind is empty, it defaults to KindDoublestar. | ||
| func NewRepositoryFilter(filterPattern, kind string) *RepositoryFilter { | ||
| return &RepositoryFilter{ | ||
| Filter: strings.TrimSpace(filterPattern), | ||
| Kind: strings.TrimSpace(kind), | ||
| } | ||
| } | ||
|
|
||
| // Match returns true if the value matches the filter pattern. | ||
| // Returns true if the filter is empty. | ||
| func (rf *RepositoryFilter) Match(value string) (bool, error) { | ||
| if rf == nil || rf.Filter == "" { | ||
| return true, nil | ||
| } | ||
| matcher := NewMatcher(rf.Kind) | ||
| return matcher.Match(value, rf.Filter) | ||
| } | ||
|
|
||
| // ValidateRepositoryFilter validates the repository filter kind and pattern. | ||
| // Empty pattern is valid and means all repositories are allowed. | ||
| func ValidateRepositoryFilter(filterPattern, kind string) error { | ||
| rf := NewRepositoryFilter(filterPattern, kind) | ||
| if rf.Kind != "" && rf.Kind != KindRegex && rf.Kind != KindDoublestar { | ||
| return errors.Errorf("unsupported repository filter kind %q", kind) | ||
| } | ||
| _, err := rf.Match("") | ||
| return err | ||
| } | ||
|
|
||
| // Matcher is an interface for matching strings against patterns | ||
| type Matcher interface { | ||
| // Match returns true if the value matches the pattern | ||
| Match(value, pattern string) (bool, error) | ||
| } | ||
|
|
||
| // RegexMatcher implements Matcher using regular expressions | ||
| type RegexMatcher struct{} | ||
|
|
||
| // Match matches value against a regular expression pattern | ||
| func (r *RegexMatcher) Match(value, pattern string) (bool, error) { | ||
| pattern = strings.TrimSpace(pattern) | ||
| if pattern == "" { | ||
| return true, nil | ||
| } | ||
| re, err := regexp.Compile(pattern) | ||
| if err != nil { | ||
| return false, err | ||
| } | ||
| match := re.FindStringIndex(value) | ||
| return match != nil && match[0] == 0 && match[1] == len(value), nil | ||
| } | ||
|
|
||
| // DoublestarMatcher implements Matcher using doublestar (glob) patterns | ||
| type DoublestarMatcher struct{} | ||
|
|
||
| // Match matches value against a doublestar (glob) pattern | ||
| func (d *DoublestarMatcher) Match(value, pattern string) (bool, error) { | ||
| pattern = strings.TrimSpace(pattern) | ||
| if pattern == "" { | ||
| return true, nil | ||
| } | ||
| return doublestar.Match(pattern, value) | ||
| } | ||
|
|
||
| // NewMatcher creates a new Matcher based on the specified kind. | ||
| // Defaults to KindDoublestar when kind is empty or unrecognised. | ||
| func NewMatcher(kind string) Matcher { | ||
| switch kind { | ||
| case KindRegex: | ||
| return &RegexMatcher{} | ||
| case KindDoublestar: | ||
| fallthrough | ||
| default: | ||
| return &DoublestarMatcher{} | ||
| } | ||
| } |
There was a problem hiding this comment.
func ValidateRepositoryFilter(filterPattern, kind string) error {
filterPattern = strings.TrimSpace(filterPattern)
if filterPattern == "" {
return nil
}
switch kind {
case KindRegex:
_, err := regexp.Compile(filterPattern)
return err
case KindDoublestar, "":
if !doublestar.ValidatePattern(filterPattern) {
return doublestar.ErrBadPattern
}
return nil
default:
return errors.Errorf("unsupported repository filter kind %q", kind)
}
}
Introduce repository filter pattern and kind on proxy cache projects to explicitly allow selected repositories. Requests that do not match the filter are denied before Harbor proxies content from upstream.
Fixes #13231
Thank you for contributing to Harbor!
Comprehensive Summary of your change
Issue being fixed
Fixes #(issue)
Please indicate you've done the following: