sync: rvck #283: 【Backport】返合字符串string处理的重要优化#237
Merged
Conversation
mainline inclusion from mainline-6.6-rc2 commit 83feeb195592b533541ff6399c8084e5b7c59677 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- printbuf now needs to know the number of characters that would have been written if the buffer was too small, like snprintf(); this changes string_get_size() to return the the return value of snprintf(). Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.8-rc7 commit f0b7f8ade9d2532a7d7da40eb297570d48dd2147 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- The new flags parameter allows controlling - Whether or not the units suffix is separated by a space, for compatibility with sort -h - Whether or not to append a B suffix - we're not always printing bytes. Co-developed-by: Kent Overstreet <kent.overstreet@linux.dev> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Kent Overstreet <kent.overstreet@linux.dev> Link: https://lore.kernel.org/r/20240229205345.93902-1-andriy.shevchenko@linux.intel.com Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.8 commit 29d8568849fe5937e14f5f7763931bb2decf298d category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Convert test_string.c to KUnit so it can be easily run with everything else. Additional text context is retained for failure reporting. For example, when forcing a bad match, we can see the loop counters reported for the memset() tests: [09:21:52] # test_memset64: ASSERTION FAILED at lib/string_kunit.c:93 [09:21:52] Expected v == 0xa2a1a1a1a1a1a1a1ULL, but [09:21:52] v == -6799976246779207263 (0xa1a1a1a1a1a1a1a1) [09:21:52] 0xa2a1a1a1a1a1a1a1ULL == -6727918652741279327 (0xa2a1a1a1a1a1a1a1) [09:21:52] i:0 j:0 k:0 [09:21:52] [FAILED] test_memset64 Currently passes without problems: $ ./tools/testing/kunit/kunit.py run string ... [09:37:40] Starting KUnit Kernel (1/1)... [09:37:40] ============================================================ [09:37:40] =================== string (6 subtests) ==================== [09:37:40] [PASSED] test_memset16 [09:37:40] [PASSED] test_memset32 [09:37:40] [PASSED] test_memset64 [09:37:40] [PASSED] test_strchr [09:37:40] [PASSED] test_strnchr [09:37:40] [PASSED] test_strspn [09:37:40] ===================== [PASSED] string ====================== [09:37:40] ============================================================ [09:37:40] Testing complete. Ran 6 tests: passed: 6 [09:37:40] Elapsed time: 6.730s total, 0.001s configuring, 6.562s building, 0.131s running Link: https://lore.kernel.org/r/20240301202732.2688342-1-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.8 commit fb57550fcbd868391a84411b0a99b2978656cdc1 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Convert test-string_helpers.c to KUnit so it can be easily run with everything else. Failure reporting doesn't need to be open-coded in most places, for example, forcing a failure in the expected output for upper/lower testing looks like this: [12:18:43] # test_upper_lower: EXPECTATION FAILED at lib/string_helpers_kunit.c:579 [12:18:43] Expected dst == strings_upper[i].out, but [12:18:43] dst == "ABCDEFGH1234567890TEST" [12:18:43] strings_upper[i].out == "ABCDEFGH1234567890TeST" [12:18:43] [FAILED] test_upper_lower Currently passes without problems: $ ./tools/testing/kunit/kunit.py run string_helpers ... [12:23:55] Starting KUnit Kernel (1/1)... [12:23:55] ============================================================ [12:23:55] =============== string_helpers (3 subtests) ================ [12:23:55] [PASSED] test_get_size [12:23:55] [PASSED] test_upper_lower [12:23:55] [PASSED] test_unescape [12:23:55] ================= [PASSED] string_helpers ================== [12:23:55] ============================================================ [12:23:55] Testing complete. Ran 3 tests: passed: 3 [12:23:55] Elapsed time: 6.709s total, 0.001s configuring, 6.591s building, 0.066s running Link: https://lore.kernel.org/r/20240301202732.2688342-2-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc3 commit 9259a4721699947ceb397037991c0e4acc496b21 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Currently, str*cmp functions (strcmp, strncmp, strcasecmp and strncasecmp) are not covered with tests. Extend the `string_kunit.c` test by adding the test cases for them. This patch adds 8 more test cases: 1) strcmp test 2) strcmp test on long strings (2048 chars) 3) strncmp test 4) strncmp test on long strings (2048 chars) 5) strcasecmp test 6) strcasecmp test on long strings 7) strncasecmp test 8) strncasecmp test on long strings These test cases aim at covering as many edge cases as possible, including the tests on empty strings, situations when the different symbol is placed at the end of one of the strings, etc. Signed-off-by: Ivan Orlov <ivan.orlov0322@gmail.com> Reviewed-by: Andy Shevchenko <andy@kernel.org> Link: https://lore.kernel.org/r/20240417233033.717596-1-ivan.orlov0322@gmail.com Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc5 commit b03442f761aae4bbb093a281ad2205bc346188f5 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- In preparation for moving the strscpy_kunit.c tests into string_kunit.c, rename "tc" to "strscpy_check" for better readability. Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Tested-by: Ivan Orlov <ivan.orlov0322@gmail.com> Link: https://lore.kernel.org/r/20240419140155.3028912-1-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc5 commit bb8d9b742aa7c576d39b354612224b3c6bfd3cbc category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Move the strscpy() tests into string_kunit.c. Remove the separate Kconfig and Makefile rule. Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Tested-by: Ivan Orlov <ivan.orlov0322@gmail.com> Link: https://lore.kernel.org/r/20240419140155.3028912-2-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc5 commit 6e4ef1429f3be236e145c6115b539acdbd2e299c category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- The test naming convention differs between string_kunit.c and strcat_kunit.c. Move "test" to the beginning of the function name. Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Tested-by: Ivan Orlov <ivan.orlov0322@gmail.com> Link: https://lore.kernel.org/r/20240419140155.3028912-3-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc5 commit bd678f7d9b72ab8b6978dac92b841e46f4b935a3 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Move the strcat() tests into string_kunit.c. Remove the separate Kconfig and Makefile rule. Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Tested-by: Ivan Orlov <ivan.orlov0322@gmail.com> Link: https://lore.kernel.org/r/20240419140155.3028912-4-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc5 commit dde915c5cba1fe49e980efe72662d9bc2a6b7ffd category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- The KUnit convention for test names is AREA_test_WHAT. Adjust the string test names to follow this pattern. Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Tested-by: Ivan Orlov <ivan.orlov0322@gmail.com> Link: https://lore.kernel.org/r/20240419140155.3028912-5-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc6 commit 0efc5990bca540b8d438fda23db3a72efa733eb0 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Another ambiguous use of strncpy() is to copy from strings that may not be NUL-terminated. These cases depend on having the destination buffer be explicitly larger than the source buffer's maximum size, having the size of the copy exactly match the source buffer's maximum size, and for the destination buffer to get explicitly NUL terminated. This usually happens when parsing protocols or hardware character arrays that are not guaranteed to be NUL-terminated. The code pattern is effectively this: char dest[sizeof(src) + 1]; strncpy(dest, src, sizeof(src)); dest[sizeof(dest) - 1] = '\0'; In practice it usually looks like: struct from_hardware { ... char name[HW_NAME_SIZE] __nonstring; ... }; struct from_hardware *p = ...; char name[HW_NAME_SIZE + 1]; strncpy(name, p->name, HW_NAME_SIZE); name[NW_NAME_SIZE] = '\0'; This cannot be replaced with: strscpy(name, p->name, sizeof(name)); because p->name is smaller and not NUL-terminated, so FORTIFY will trigger when strnlen(p->name, sizeof(name)) is used. And it cannot be replaced with: strscpy(name, p->name, sizeof(p->name)); because then "name" may contain a 1 character early truncation of p->name. Provide an unambiguous interface for converting a maybe not-NUL-terminated string to a NUL-terminated string, with compile-time buffer size checking so that it can never fail at runtime: memtostr() and memtostr_pad(). Also add KUnit tests for both. Link: https://lore.kernel.org/r/20240410023155.2100422-1-keescook@chromium.org Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-6.9-rc6 commit c01c41e5009c04515d81a87f6278c413914920ce category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- It is more logical to have the strtomem() test in string_kunit.c instead of the memcpy() suite. Move it to live with memtostr(). Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit ae45f896a40a07449d9b45d0395fb7245fdd75fc category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add a KUnit test for strlen() to verify correctness across different string lengths and memory alignments. Use vmalloc() to place the NUL character at the page boundary to ensure over-reads are detected. Suggested-by: Kees Cook <kees@kernel.org> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Reviewed-by: Kees Cook <kees@kernel.org> Link: https://patch.msgid.link/20260130025018.172925-2-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit 263dca234e5cc12aa8b434592ceb655538bf4ea4 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add a KUnit test for strnlen() to verify correctness across different string lengths and memory alignments. Use vmalloc() to place the NUL character at the page boundary to ensure over-reads are detected. Suggested-by: Andy Shevchenko <andy@kernel.org> Suggested-by: Kees Cook <kees@kernel.org> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Reviewed-by: Kees Cook <kees@kernel.org> Link: https://patch.msgid.link/20260130025018.172925-3-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit 27b2810a4a3dcd1545ec8bafc82f967eda591c47 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add a KUnit test for strrchr() to verify correctness across different string lengths and memory alignments. Use vmalloc() to place the NUL character at the page boundary to ensure over-reads are detected. Suggested-by: Kees Cook <kees@kernel.org> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Reviewed-by: Kees Cook <kees@kernel.org> Link: https://patch.msgid.link/20260130025018.172925-4-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit 0020240a431187628e2636284023e63b9b7a2aa1 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Introduce a benchmarking framework to the string_kunit test suite to measure the execution efficiency of string functions. The implementation is inspired by crc_benchmark(), measuring throughput (MB/s) and latency (ns/call) across a range of string lengths. It includes a warm-up phase, disables preemption during measurement, and uses a fixed seed for reproducible results. This framework allows for comparing different implementations (e.g., generic C vs. architecture-optimized assembly) within the KUnit environment. Initially, provide a benchmark for strlen(). Suggested-by: Andy Shevchenko <andy@kernel.org> Suggested-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Reviewed-by: Kees Cook <kees@kernel.org> Link: https://patch.msgid.link/20260130025018.172925-5-jiangfeng@kylinos.cn [pjw@kernel.org: fixed a checkpatch issue] Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit e73bcb3708a69369d506e5bc6a63d4fc13d8e28a category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Extend the string benchmarking suite to include strnlen(), strchr(), and strrchr(). For character search functions strchr() and strrchr(), the benchmark targets the NUL character. This ensures the entire string is scanned, providing a consistent measure of full-length processing efficiency comparable to strlen(). Suggested-by: Andy Shevchenko <andy@kernel.org> Suggested-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Acked-by: Andy Shevchenko <andy@kernel.org> Reviewed-by: Kees Cook <kees@kernel.org> Link: https://patch.msgid.link/20260130025018.172925-6-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit 5ba15d419fab848a3813eb56bbcad00e291fbc49 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add an optimized strnlen() implementation for RISC-V. This version includes a generic optimization and a Zbb-powered optimization using the 'orc.b' instruction, derived from the strlen() implementation. Benchmark results (QEMU TCG, rv64): Length | Original (MB/s) | Optimized (MB/s) | Improvement -------|-----------------|------------------|------------ 16 B | 179 | 309 | +72.6% 512 B | 347 | 1562 | +350.1% 4096 B | 356 | 1878 | +427.5% Suggested-by: Qingfang Deng <dqfext@gmail.com> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Link: https://patch.msgid.link/20260130025018.172925-7-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit adf542133960d402f63c976b00e46be4d986d4c3 category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add an assembly implementation of strchr() for RISC-V. By eliminating stack frame management (prologue/epilogue) and optimizing the function entries, the assembly version provides significant relative gains for short strings where the fixed overhead of the C function is most prominent. As string length increases, performance converges with the generic C implementation. Benchmark results (QEMU TCG, rv64): Length | Original (MB/s) | Optimized (MB/s) | Improvement -------|-----------------|------------------|------------ 1 B | 21 | 22 | +4.8% 7 B | 113 | 121 | +7.1% 16 B | 195 | 202 | +3.6% 512 B | 376 | 389 | +3.5% 4096 B | 394 | 393 | -0.3% Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Tested-by: Joel Stanley <joel@jms.id.au> Link: https://patch.msgid.link/20260130025018.172925-8-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
mainline inclusion from mainline-7.0-rc6 commit bef64bcb940269a503d12eb1bc180d1aa9adf74d category: feature bugzilla: RVCK-Project/rvck#280 -------------------------------- Add an assembly implementation of strrchr() for RISC-V. This implementation minimizes instruction count and avoids unnecessary memory access to the stack. The performance benefits are most visible on small workloads (1-16 bytes) where the architectural savings in function overhead outweigh the execution time of the scan loop. Benchmark results (QEMU TCG, rv64): Length | Original (MB/s) | Optimized (MB/s) | Improvement -------|-----------------|------------------|------------ 1 B | 20 | 21 | +5.0% 7 B | 111 | 120 | +8.1% 16 B | 189 | 199 | +5.3% 512 B | 361 | 382 | +5.8% 4096 B | 388 | 391 | +0.8% Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn> Tested-by: Joel Stanley <joel@jms.id.au> Link: https://patch.msgid.link/20260130025018.172925-9-jiangfeng@kylinos.cn Signed-off-by: Paul Walmsley <pjw@kernel.org> Signed-off-by: Gao Rui <gao.rui@zte.com.cn> [sync from rvck.] Signed-off-by: WeiHao Li <cn.liweihao@gmail.com>
|
开始测试 log: https://github.com/RVCK-Project/rvck-olk/actions/runs/27003292939 参数解析结果
测试完成 详细结果:
Kunit Test Resultkunit test failed
Kernel Build Result
Check Patch Result
LAVA Check (qemu)
result: Lava check done!
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync commits from rvck PR 283.
Source PR: RVCK-Project/rvck#283