Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/palette.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,7 @@
## 2024-08-01 - ๋„ค์ดํ‹ฐ๋ธŒ ๋ธŒ๋ผ์šฐ์ € UI์˜ ๋‹คํฌ ๋ชจ๋“œ ์ง€์› ๊ฐ•์ œ
**ํ•™์Šต:** CSS ๋ฏธ๋””์–ด ์ฟผ๋ฆฌ(`@media (prefers-color-scheme: dark)`)๋ฅผ ํ†ตํ•ด ๋‹คํฌ ๋ชจ๋“œ๋ฅผ ์ง€์›ํ•˜๋”๋ผ๋„, ๋ธŒ๋ผ์šฐ์ €์˜ ๋„ค์ดํ‹ฐ๋ธŒ UI ์š”์†Œ(์Šคํฌ๋กค๋ฐ”, ๊ธฐ๋ณธ ํผ ์ปจํŠธ๋กค, ๊ธฐ๋ณธ ๋ฐฑ๊ทธ๋ผ์šด๋“œ ๋“ฑ)๋Š” ํ…Œ๋งˆ ๋ณ€๊ฒฝ์„ ์ธ์‹ํ•˜์ง€ ๋ชปํ•ด ์–ด๋‘์šด ํ…Œ๋งˆ ํ™˜๊ฒฝ์—์„œ ๋ฐ์€ ์Šคํฌ๋กค๋ฐ”๊ฐ€ ํ‘œ์‹œ๋˜๋Š” ๋“ฑ ์‹œ๊ฐ์  ๋ถˆ์ผ์น˜๋ฅผ ์ดˆ๋ž˜ํ•ฉ๋‹ˆ๋‹ค.
**์กฐ์น˜:** ํ•ญ์ƒ HTML ๋ฌธ์„œ์˜ `<head>` ์˜์—ญ์— `<meta name="color-scheme" content="light dark">` ๋ฉ”ํƒ€ ํƒœ๊ทธ๋ฅผ ๋ช…์‹œ์ ์œผ๋กœ ์ถ”๊ฐ€ํ•˜์—ฌ ๋ธŒ๋ผ์šฐ์ € ์ˆ˜์ค€์—์„œ ์‚ฌ์šฉ์ž์˜ ์‹œ์Šคํ…œ ํ…Œ๋งˆ(๋‹คํฌ ๋ชจ๋“œ ๋“ฑ)๋ฅผ ์™„์ „ํžˆ ์ƒ์†๋ฐ›์•„ ์ผ๊ด€์„ฑ ์žˆ๋Š” ๋„ค์ดํ‹ฐ๋ธŒ UI๋ฅผ ๋ Œ๋”๋งํ•˜๋„๋ก ๋ณด์žฅํ•˜์‹ญ์‹œ์˜ค.

## 2026-07-16 - ๋ฃจํŠธ ๋””๋ ‰ํ† ๋ฆฌ์˜ ๋นˆ ์ด๋ฆ„ ์ฒ˜๋ฆฌ ๋ฐ ์ ‘๊ทผ์„ฑ ํ–ฅ์ƒ
**Learning:** ํŒŒ์ผ ์‹œ์Šคํ…œ ๋ฃจํŠธ ๋””๋ ‰ํ† ๋ฆฌ(์˜ˆ: Linux/Mac์˜ `/` ๋˜๋Š” Windows์˜ `C:\`)๋ฅผ ์ฒ˜๋ฆฌํ•  ๋•Œ, Java์˜ `File.getName()`์€ ๋นˆ ๋ฌธ์ž์—ด์„ ๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค. ์ด๋กœ ์ธํ•ด ์ƒ์„ฑ๋œ HTML์˜ `<title>`๊ณผ `<h1>` ์š”์†Œ๊ฐ€ ๋น„์–ด ์žˆ๊ฒŒ ๋˜์–ด, ์Šคํฌ๋ฆฐ ๋ฆฌ๋” ์‚ฌ์šฉ์ž๊ฐ€ ํŽ˜์ด์ง€์˜ ๋งฅ๋ฝ์„ ์ดํ•ดํ•˜๊ธฐ ์–ด๋ ค์›Œ์ง‘๋‹ˆ๋‹ค.
**Action:** ๋””๋ ‰ํ† ๋ฆฌ ์ด๋ฆ„์„ ๋ Œ๋”๋งํ•  ๋•Œ `name`์ด ๋น„์–ด ์žˆ๋Š” ๊ฒฝ์šฐ `absolutePath`๋กœ ํด๋ฐฑ(`curr_dir.name.ifEmpty { curr_dir.absolutePath }`)ํ•˜๋„๋ก ๊ตฌํ˜„ํ•˜์—ฌ ํ•ญ์ƒ ์˜๋ฏธ ์žˆ๋Š” ์ œ๋ชฉ๊ณผ ์ œ๋ชฉ ํƒœ๊ทธ๊ฐ€ ์ƒ์„ฑ๋˜๋„๋ก ๋ณด์žฅํ•˜์‹ญ์‹œ์˜ค.
15 changes: 15 additions & 0 deletions .jules/sentinel.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,3 +83,18 @@
**Vulnerability:** ์ •์  HTML ์ƒ์„ฑ ๋„๊ตฌ์—์„œ ๋งค๋ฒˆ ๋‹ค๋ฅธ Nonce๋ฅผ ๋™์ ์œผ๋กœ ์ƒ์„ฑํ•˜์—ฌ CSP์— ์ ์šฉํ•˜๋Š” ๊ฒƒ์€, ์บ์‹ฑ ํšจ์œจ์„ ์ €ํ•˜์‹œํ‚ฌ ๋ฟ๋งŒ ์•„๋‹ˆ๋ผ ์ •์  ๋ฐฐํฌ ํ™˜๊ฒฝ(์˜ˆ: GitHub Pages ๋“ฑ)์—์„œ ์˜ฌ๋ฐ”๋ฅธ ๋ณด์•ˆ ์ •์ฑ… ์ˆ˜๋ฆฝ์„ ๋ฐฉํ•ดํ•  ์ˆ˜ ์žˆ๋Š” ์•ˆํ‹ฐ ํŒจํ„ด์ž…๋‹ˆ๋‹ค.
**Learning:** ์ •์ ์œผ๋กœ ๊ณ ์ •๋œ ์ธ๋ผ์ธ ์Šคํƒ€์ผ์ด๋‚˜ ์Šคํฌ๋ฆฝํŠธ์—๋Š” ๋‚œ์ˆ˜ํ™”๋œ Nonce๋ณด๋‹ค ์ฝ˜ํ…์ธ  ์ž์ฒด์˜ ํ•ด์‹œ(SHA-256 ๋“ฑ)๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์ด ์•ˆ์ „ํ•˜๊ณ  ์ผ๊ด€๋œ ๋ฐฉ์‹์ž„์„ ๋ฐฐ์› ์Šต๋‹ˆ๋‹ค.
**Prevention:** ์ž๋™ ์ƒ์„ฑ๋˜๋Š” ์ •์  HTML์˜ ์ฝ˜ํ…์ธ  ๋ณด์•ˆ ์ •์ฑ…(CSP)์—๋Š” `style-src 'sha256-<HASH>'` ๋ฐฉ์‹์„ ์ ์šฉํ•˜๊ณ , `<style>` ํƒœ๊ทธ์—์„œ ๋ถˆํ•„์š”ํ•œ `nonce` ์†์„ฑ์„ ์ œ๊ฑฐํ•˜์—ฌ ๋ธŒ๋ผ์šฐ์ €์˜ ๋ฌด๊ฒฐ์„ฑ ๊ฒ€์ฆ ๊ธฐ๋Šฅ์„ ์ ๊ทน ํ™œ์šฉํ•˜์‹ญ์‹œ์˜ค.

## 2026-07-16 - TOCTOU Symlink Attack in Temporary File Creation
**Vulnerability:** A TOCTOU (Time-of-Check to Time-of-Use) vulnerability existed when creating temporary files for `index.html`. A malicious actor could swap the target directory with a symlink after the crawler's initial checks but before the temporary file was created and moved into it.
**Learning:** Checking properties on an arbitrary file path (like whether it is a symlink) and later executing file operations on that same path using string representations is unsafe. The underlying file system object can change between the check and the use.
**Prevention:** Always re-evaluate paths to their real targets immediately before sensitive operations. In Java/Kotlin, converting a `Path` using `.toRealPath(LinkOption.NOFOLLOW_LINKS)` immediately before operations like `Files.createTempFile` and `Files.move` ensures that symlink swaps are caught or strictly prevented during those critical filesystem calls.

## 2026-07-16 - Safe File Handling with FileChannel and Exception Boundaries
**Vulnerability:** Opening files using standard `File` or `Path` wrappers without enforcing no-follow links during the open can lead to symlink traversal vulnerabilities or race conditions where a file might be swapped with a symlink before it is completely read. Stalled reads could also hold the crawler indefinitely.
**Learning:** For reading potentially sensitive or untrusted configuration files (like `.html4ignore`), use race-resistant descriptor-level operations such as `java.nio.channels.FileChannel.open` with `StandardOpenOption.READ` and `LinkOption.NOFOLLOW_LINKS`. This guarantees that the opened descriptor points to a regular file and won't follow symlinks. Ensure the block is wrapped in a general `Exception` boundary so failures (like IO exceptions, missing files) fall back to secure default handling.
**Prevention:** Instead of multiple filesystem check calls (e.g. `isFile`, `isSymbolicLink`), directly open a FileChannel with `NOFOLLOW_LINKS` and wrap the sequence with a robust exception handler and size limitations to prevent DoS attacks through large files or stalled streams.

## 2026-07-16 - Safe File Handling with Files.isRegularFile Guard
**Vulnerability:** Even when opening files with NOFOLLOW_LINKS, attempting to open named pipes (FIFOs), character devices, or other special files for reading can block the process indefinitely if there is no writer. This results in an application freeze (DoS), halting the directory crawling.
**Learning:** Never directly open potentially non-regular files without ensuring they are standard files first. Using `Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)` strictly validates that the file is not a directory, symlink, pipe, or device before you attempt a file descriptor operation on it. This combination eliminates block-read starvation.
**Prevention:** Guard all `FileChannel.open` invocations for user-controlled filenames (like `.html4ignore`) with an explicit `isRegularFile` check to reject special types, accompanied by a resilient exception fallback handler in case of TOCTOU.
84 changes: 50 additions & 34 deletions src/main/kotlin/html4tree/main.kt
Original file line number Diff line number Diff line change
Expand Up @@ -181,35 +181,49 @@ fun process_ignore_file(curr_dir: File, dirFilesNames: Array<String>? = null): S
// ๋ณด์•ˆ ํ–ฅ์ƒ: .html4ignore ํŒŒ์ผ์ด ์ผ๋ฐ˜ ํŒŒ์ผ์ธ์ง€ ํ™•์ธํ•˜๊ณ , ์‹ฌ๋ณผ๋ฆญ ๋งํฌ์ธ ๊ฒฝ์šฐ ๋ฌด์‹œํ•˜์—ฌ DoS ๋ฐ ๊ฒฝ๋กœ ์กฐ์ž‘์„ ๋ฐฉ์ง€ํ•ฉ๋‹ˆ๋‹ค.
// ๋ณด์•ˆ ํ–ฅ์ƒ: ํŒŒ์ผ ํฌ๊ธฐ(1MB ์ œํ•œ) ๋ฐ ์ค„ ์ˆ˜(1000์ค„), ์ •๊ทœ์‹ ๊ธธ์ด(100์ž)๋ฅผ ์ œํ•œํ•˜์—ฌ ReDoS ๋ฐ ๋ฉ”๋ชจ๋ฆฌ ๊ณ ๊ฐˆ(OOM) ๋ฐฉ์ง€
// ๋ณด์•ˆ ํ–ฅ์ƒ: ๊ถŒํ•œ์ด ์—†๋Š” ํŒŒ์ผ ์ ‘๊ทผ ์‹œ ๋ฐœ์ƒํ•˜๋Š” ์˜ˆ์™ธ(DoS)๋ฅผ ๋ฐฉ์ง€ํ•˜๊ธฐ ์œ„ํ•ด canRead() ์ถ”๊ฐ€ ํ™•์ธ
if(ignore_file.isFile && !Files.isSymbolicLink(ignore_file.toPath()) && ignore_file.canRead() && ignore_file.length() <= 1048576){
val ignored_matchers = mutableListOf<java.nio.file.PathMatcher>()

ignore_file.useLines { lines ->
for ((lineIndex, it) in lines.withIndex()) {
// ์ค„ ์ˆ˜ ์ œํ•œ์ด ํŒจํ„ด ์ˆ˜๋„ ํ•จ๊ป˜ ์ƒํ•œ(์ค„๋‹น ์ตœ๋Œ€ 1๊ฐœ ํŒจํ„ด)ํ•˜๋ฏ€๋กœ ๋ณ„๋„ ํŒจํ„ด ์นด์šดํ„ฐ๋Š” ๋ถˆํ•„์š”
if (lineIndex >= 1000) break
val pattern = it.trim()
if (pattern.isNotEmpty() && pattern.length <= 100) {
try {
ignored_matchers.add(java.nio.file.FileSystems.getDefault().getPathMatcher("glob:$pattern"))
} catch (_: java.util.regex.PatternSyntaxException) {
}
}
}
}

// โšก Bolt Performance Optimization: ๋””๋ ‰ํ† ๋ฆฌ ๋ชฉ๋ก์„ Set์— ์ถ”๊ฐ€ํ•˜๊ธฐ ์œ„ํ•ด ํ•„ํ„ฐ๋ง๋งŒ ํ•  ๋•Œ๋Š” ์ •๋ ฌ์ด ๋ถˆํ•„์š”ํ•˜๋ฏ€๋กœ .sorted()๋ฅผ ์ œ๊ฑฐํ•˜์—ฌ O(N log N) ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ๋ฐฉ์ง€ํ•ฉ๋‹ˆ๋‹ค.
val list = dirFilesNames ?: curr_dir.list()
list?.forEach {
val current = it
val pathCurrent = java.nio.file.Paths.get(current)
for (matcher in ignored_matchers) {
if (matcher.matches(pathCurrent)) {
files_to_exclude.add(current)
break
}
}
}
try {
val path = ignore_file.toPath()
// ๐Ÿ›ก๏ธ Sentinel: Enforce non-blocking open, check if regular file, and no-follow links
if (Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)) {
val channel = java.nio.channels.FileChannel.open(path, java.nio.file.StandardOpenOption.READ, LinkOption.NOFOLLOW_LINKS)
try {
val size = channel.size()
if (size <= 1048576) {
val ignored_matchers = mutableListOf<java.nio.file.PathMatcher>()
val scanner = java.util.Scanner(java.nio.channels.Channels.newInputStream(channel), "UTF-8")
var lineIndex = 0
while (scanner.hasNextLine() && lineIndex < 1000) {
val pattern = scanner.nextLine().trim()
if (pattern.isNotEmpty() && pattern.length <= 100) {
try {
ignored_matchers.add(java.nio.file.FileSystems.getDefault().getPathMatcher("glob:$pattern"))
} catch (_: java.util.regex.PatternSyntaxException) {
}
}
lineIndex++
}

// โšก Bolt Performance Optimization: ๋””๋ ‰ํ† ๋ฆฌ ๋ชฉ๋ก์„ Set์— ์ถ”๊ฐ€ํ•˜๊ธฐ ์œ„ํ•ด ํ•„ํ„ฐ๋ง๋งŒ ํ•  ๋•Œ๋Š” ์ •๋ ฌ์ด ๋ถˆํ•„์š”ํ•˜๋ฏ€๋กœ .sorted()๋ฅผ ์ œ๊ฑฐํ•˜์—ฌ O(N log N) ์˜ค๋ฒ„ํ—ค๋“œ๋ฅผ ๋ฐฉ์ง€ํ•ฉ๋‹ˆ๋‹ค.
val list = dirFilesNames ?: curr_dir.list()
if (list != null) {
list.forEach { item ->
val current = item
val pathCurrent = java.nio.file.Paths.get(current)
for (matcher in ignored_matchers) {
if (matcher.matches(pathCurrent)) {
files_to_exclude.add(current)
break
}
}
}
}
}
} finally {
channel.close()
}
}
} catch (_: java.lang.Exception) {
// Fall through to default exclusions
}

if ("index.html" !in files_to_exclude)
Expand All @@ -230,8 +244,10 @@ fun process_ignore_file(curr_dir: File, dirFilesNames: Array<String>? = null): S
}

fun write_index_file(curr_dir: File, content: String) {
val indexPath = curr_dir.toPath().resolve("index.html")
val tempPath = Files.createTempFile(curr_dir.toPath(), ".index-", ".html")
// ๐Ÿ›ก๏ธ Sentinel: Validate the real path immediately before temp-file creation and move to prevent TOCTOU.
val realPath = curr_dir.toPath().toRealPath(LinkOption.NOFOLLOW_LINKS)
val indexPath = realPath.resolve("index.html")
val tempPath = Files.createTempFile(realPath, ".index-", ".html")
try {
Files.write(tempPath, content.toByteArray(Charsets.UTF_8))
Files.move(tempPath, indexPath, StandardCopyOption.REPLACE_EXISTING)
Expand Down Expand Up @@ -327,12 +343,12 @@ ${cssContent} </style>
<meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src '${styleHash}'; base-uri 'none'; form-action 'none';">
<!-- ๋ณด์•ˆ ํ–ฅ์ƒ: ๋ฆฌํผ๋Ÿฌ๋ฅผ ํ†ตํ•œ ๋””๋ ‰ํ† ๋ฆฌ ๊ฒฝ๋กœ ๋…ธ์ถœ ๋ฐฉ์ง€ -->
<meta name="referrer" content="no-referrer">
<title>${curr_dir.getName().escapeHtml()}</title>
<title>${curr_dir.name.ifEmpty { curr_dir.absolutePath }.escapeHtml()}</title>
${css}
</head>
<body>
<main>
<h1>${curr_dir.getName().escapeHtml()}</h1>
<h1>${curr_dir.name.ifEmpty { curr_dir.absolutePath }.escapeHtml()}</h1>
<nav aria-label="๋””๋ ‰ํ† ๋ฆฌ ๋ชฉ๋ก">
<ul role="list">
<li><a class="dir-link" href="./.." aria-label="์ƒ์œ„ ๋””๋ ‰ํ† ๋ฆฌ๋กœ ์ด๋™" title="์ƒ์œ„ ๋””๋ ‰ํ† ๋ฆฌ๋กœ ์ด๋™"><span class="icon" aria-hidden="true">&#x21B0;</span> <span>..</span></a></li>
Expand All @@ -342,7 +358,7 @@ ${cssContent} </style>
val l = StringBuilder()

val filesList = dirFiles ?: curr_dir.listFiles()
val dir_files: MutableList<File> = filesList?.toMutableList() ?: mutableListOf()
val dir_files: MutableList<File> = if (filesList != null) filesList.toMutableList() else mutableListOf()
dir_files.sortWith(compareBy ({it.name}) )
dir_files.forEach {
val fileName = it.getName()
Expand Down
82 changes: 82 additions & 0 deletions src/test/kotlin/html4tree/MainTest.kt
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ class MainTest {
val hiddenFile = File(tempDir, ".hidden_file.txt")
hiddenFile.createNewFile()

// Also create a file that starts with a dot but is processed in the loop branches
val dotIgnoredFile = File(tempDir, ".ignored")
dotIgnoredFile.createNewFile()

val hiddenDir = File(tempDir, ".hidden_dir")
hiddenDir.mkdir()
val fileInHiddenDir = File(hiddenDir, "file_in_hidden_dir.txt")
Expand Down Expand Up @@ -383,6 +387,21 @@ class MainTest {
assertTrue(indexFile.readText().contains("<html lang=\"ko\">"))
}

@Test
fun testProcessDirWithEmptyNameFallback() {
val tempDirFallback = Files.createTempDirectory("test-fallback").toFile()
val fakeRoot = object : File(tempDirFallback, "fakeRoot") {
override fun getName() = ""
}
fakeRoot.mkdir()
process_dir(fakeRoot)
val indexFile = File(fakeRoot, "index.html")
assertTrue(indexFile.exists())
val htmlContent = indexFile.readText()
assertTrue(htmlContent.contains("<title>${fakeRoot.absolutePath.escapeHtml()}</title>"))
assertTrue(htmlContent.contains("<h1>${fakeRoot.absolutePath.escapeHtml()}</h1>"))
}

@Test
fun testGoWithSymlink() {
val subdir = File(tempDir, "subdir")
Expand Down Expand Up @@ -554,6 +573,20 @@ class MainTest {
assertTrue(excluded.contains("test.txt"))
}

@Test
fun testGoWithDirectoryExclusion() {
val ignoreFile = File(tempDir, ".html4ignore")
ignoreFile.writeText("subdirToIgnore")
val subdir = File(tempDir, "subdirToIgnore")
subdir.mkdir()
File(tempDir, "test.txt").createNewFile()
go(tempDir.absolutePath, -1)
val indexFile = File(tempDir, "index.html")
assertTrue(indexFile.exists())
val htmlContent = indexFile.readText()
assertFalse(htmlContent.contains("subdirToIgnore"))
}

@Test
fun testProcessIgnoreFileHiddenFiles() {
// .myhidden/.hiddendir are NOT in the static sensitive-file list,
Expand Down Expand Up @@ -625,6 +658,55 @@ class MainTest {
assertTrue(excluded.contains("index.html"))
}

@Test
fun testProcessIgnoreFileFallbackGenericException() {
// Create an invalid path that throws a generic exception (e.g. InvalidPathException if mapped in channel open)
val badDir = object : File(tempDir, "baddir") {
override fun toPath(): java.nio.file.Path {
throw RuntimeException("Simulated exception")
}
}
val excluded = process_ignore_file(badDir, null)
assertTrue(excluded.contains("index.html"))
}

@Test
fun testProcessIgnoreFileUnreadableException() {
val unreadableFile = File(tempDir, ".html4ignore")
unreadableFile.writeText("*.ignore")
unreadableFile.setReadable(false)
val excluded = process_ignore_file(tempDir, null)
assertTrue(excluded.contains("index.html"))
unreadableFile.setReadable(true)
}

@Test
fun testProcessIgnoreFileRegularFileCheck() {
val nonRegularFile = File(tempDir, ".html4ignore")
nonRegularFile.mkdir()
val excluded = process_ignore_file(tempDir, null)
assertTrue(excluded.contains("index.html"))
}

@Test
fun testUrlEncodePathFallback() {
// Just calling urlEncodePath directly on a string that isn't modified
val str = "abc"
assertEquals("abc", str.urlEncodePath())
}

@Test
fun testProcessIgnoreFileFallbackIOException() {
val ignoreFile = File(tempDir, ".html4ignore")
ignoreFile.mkdir()
File(tempDir, "test.txt").createNewFile()

// Should fall back to default exclusions gracefully
val excluded = process_ignore_file(tempDir, null)
assertFalse(excluded.contains("test.txt"))
assertTrue(excluded.contains("index.html"))
}

@Test
fun testProcessIgnoreFileLargeSize() {
val ignoreFile = File(tempDir, ".html4ignore")
Expand Down
Loading