Feat: Item test fixture by AltayAkkus · Pull Request #563 · internetarchive/Zeno

AltayAkkus · 2026-02-22T21:52:43Z

When you want to create a models.Item for tests
previously

Zeno/internal/pkg/postprocessor/extractor/html_test.go

Lines 15 to 56 in d475a60

    
           func setupURL(html string) *models.URL { 
        
           	resp := &http.Response{ 
        
           		Body:   io.NopCloser(bytes.NewBufferString(html)), 
        
           		Header: make(http.Header), 
        
           	} 
        
           	resp.Header.Set("Content-Type", "text/html; charset=utf8") 
        
           	newURL, err := models.NewURL("http://ex.com") 
        
           	if err != nil { 
        
           		panic(err) 
        
           	} 
        
           	newURL.SetResponse(resp) 
        
           	if err := generalarchiver.ProcessBody(&newURL, false, false, 0, os.TempDir(), nil); err != nil { 
        
           		panic(err) 
        
           	} 
        
           	return &newURL 
        
           } 
        
           func setupItem(html string) *models.Item { 
        
           	newURL := setupURL(html) 
        
           	return models.NewItem(newURL, "") 
        
           } 
        
           func TestHTMLOutlinks(t *testing.T) { 
        
           	html := ` 
        
           	<html> 
        
           		<head></head> 
        
           		<body> 
        
           			<a href="http://example.com">ex</a> 
        
           			<a href="http://archive.org">ar</a> 
        
           			<p>test</p> 
        
           			<a href="https://web.archive.org">wa</a> 
        
           			<a onclick="window.location='http://foo.com'">click me</a> 
        
           			<a ondblclick="window.location='https://bar.com'">double click me</a> 
        
           			<iframe title="Internet Archive" src="https://web.archive.org"></iframe> 
        
           			<img src="world-map.jpg" usemap="#worldmap" alt="World Map"> 
        
           			<map name="worldmap"> 
        
           			  <area shape="rect" coords="34,44,270,350" href="https://example.com/usa" alt="USA"> 
        
           			</map> 
        
           		</body> 
        
           	</html>` 
        
           	URL := setupURL(html)

now

//go:embed testdata/ina_api_response.json
var inaFixture []byte

func TestExtractAssets_HydrateItemFixture(t *testing.T) {
	item := testutil.HydrateItem(t, inaFixture)
	assets, _, err := ExtractAssetsOutlinks(item)
         ...
}

You can create the JSON files by hand (look at the ina_api_response.json for ereference), define everything you need.
Or you can create them using this curl drop-in alias which just spits the JSON out.

codecov-commenter · 2026-02-27T02:19:25Z

Codecov Report

❌ Patch coverage is 33.33333% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.50%. Comparing base (aa1fbdb) to head (345e840).
⚠️ Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/pkg/postprocessor/testutil/item.go	33.33%	26 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #563      +/-   ##
==========================================
- Coverage   56.61%   56.50%   -0.11%     
==========================================
  Files         131      133       +2     
  Lines        6620     6769     +149     
==========================================
+ Hits         3748     3825      +77     
- Misses       2498     2555      +57     
- Partials      374      389      +15

Flag	Coverage Δ
e2etests	`41.96% <ø> (-0.22%)`	⬇️
unittests	`29.28% <33.33%> (+0.42%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AltayAkkus · 2026-06-10T13:34:43Z

Merged into #551

AltayAkkus added 2 commits February 22, 2026 21:37

feat: add ItemFixture and HydrateItem for testing models.Item

3ff149d

feat: add example dehydrated Item test for INA

ee377a1

AltayAkkus mentioned this pull request Feb 22, 2026

Refactor asset extractors #551

Draft

fix test

345e840

Merge branch 'main' into feat-item-test-fixture

7f514e4

AltayAkkus closed this Jun 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: Item test fixture#563

Feat: Item test fixture#563
AltayAkkus wants to merge 4 commits into
internetarchive:mainfrom
AltayAkkus:feat-item-test-fixture

AltayAkkus commented Feb 22, 2026

Uh oh!

codecov-commenter commented Feb 27, 2026

Uh oh!

AltayAkkus commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	func setupURL(html string) *models.URL {
	resp := &http.Response{
	Body: io.NopCloser(bytes.NewBufferString(html)),
	Header: make(http.Header),
	}
	resp.Header.Set("Content-Type", "text/html; charset=utf8")

	newURL, err := models.NewURL("http://ex.com")
	if err != nil {
	panic(err)
	}
	newURL.SetResponse(resp)
	if err := generalarchiver.ProcessBody(&newURL, false, false, 0, os.TempDir(), nil); err != nil {
	panic(err)
	}
	return &newURL
	}

	func setupItem(html string) *models.Item {
	newURL := setupURL(html)
	return models.NewItem(newURL, "")
	}

	func TestHTMLOutlinks(t *testing.T) {
	html := `
	<html>
	<head></head>
	<body>
	<a href="http://example.com">ex</a>
	<a href="http://archive.org">ar</a>
	<p>test</p>
	<a href="https://web.archive.org">wa</a>
	<a onclick="window.location='http://foo.com'">click me</a>
	<a ondblclick="window.location='https://bar.com'">double click me</a>
	<iframe title="Internet Archive" src="https://web.archive.org"></iframe>
	<img src="world-map.jpg" usemap="#worldmap" alt="World Map">
	<map name="worldmap">
	<area shape="rect" coords="34,44,270,350" href="https://example.com/usa" alt="USA">
	</map>
	</body>
	</html>`
	URL := setupURL(html)

Conversation

AltayAkkus commented Feb 22, 2026

Uh oh!

codecov-commenter commented Feb 27, 2026

Codecov Report

Uh oh!

AltayAkkus commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants