Initial QSfera import
This commit is contained in:
Курнат Андрей
@@ -0,0 +1,3 @@
|
||||
.DS_Store
|
||||
*.out
|
||||
*.test
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
The goals and overview of this package can be found in the README.md file,
|
||||
start by reading that.
|
||||
|
||||
The goal of this package is to determine the display (column) width of a
|
||||
string, UTF-8 bytes, or runes, as would happen in a monospace font, especially
|
||||
in a terminal.
|
||||
|
||||
When troubleshooting, write Go unit tests instead of executing debug scripts.
|
||||
The tests can return whatever logs or output you need. If those tests are
|
||||
only for temporary troubleshooting, clean up the tests after the debugging is
|
||||
done.
|
||||
|
||||
(Separate executable debugging scripts are messy, tend to have conflicting
|
||||
dependencies and are hard to cleanup.)
|
||||
|
||||
If you make changes to the trie generation in internal/gen, it can be invoked
|
||||
by running `go generate` from the top package directory.
|
||||
|
||||
## Pull Requests and branches
|
||||
|
||||
For PRs (pull requests), you can use the gh CLI tool. Compare the current branch with main. Reviewing a PR and reviewing a branch are about the same, but the PR may add context.
|
||||
|
||||
Understand the goals of the PR. Note any API changes, especially breaking changes.
|
||||
|
||||
Look for thoroughness of tests, as well as GoDoc comments.
|
||||
|
||||
Retrieve and consider the comments on the PR, which may have come from GitHub Copilot or Cursor BugBot. Think like GitHub Copilot or Cursor BugBot.
|
||||
|
||||
Offer to optionally post a brief summary of the review to the PR, via the gh CLI tool.
|
||||
|
||||
## Comparisons to go-runewidth
|
||||
|
||||
We originally attempted to make this package compatible with go-runewidth.
|
||||
However, we found that there were too many differences in the handling of
|
||||
certain characters and properties.
|
||||
|
||||
We believe, preliminarily, that our choices are more correct and complete,
|
||||
by using more complete categories such as Unicode Cf (format) for zero-width
|
||||
and Mn (Nonspacing_Mark) for combining marks.
|
||||
+115
@@ -0,0 +1,115 @@
|
||||
# Changelog
|
||||
|
||||
## [0.10.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.9.0...v0.10.0)
|
||||
|
||||
### Added
|
||||
- New `ControlSequences` option to treat ECMA-48/ANSI escape sequences as zero-width. (#20)
|
||||
- `TruncateString` and `TruncateBytes` now preserve trailing ANSI escape sequences (such as SGR resets) when `ControlSequences` is true, preventing color bleed in terminal output.
|
||||
|
||||
### Changed
|
||||
- Removed `stringish` dependency; generic type constraints are now inline `~string | []byte`.
|
||||
- Upgraded uax29 dependency to v2.6.0 for ANSI escape sequence support in the grapheme iterator.
|
||||
|
||||
## [0.9.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.8.0...v0.9.0)
|
||||
|
||||
### Changed
|
||||
- Unicode 17 support: East Asian Width and emoji data updated to Unicode 17.0.0. (#18)
|
||||
- Upgraded uax29 dependency to v2.5.0 (Unicode 17 grapheme segmentation).
|
||||
|
||||
## [0.8.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.7.0...v0.8.0)
|
||||
|
||||
### Changed
|
||||
- Performance: ASCII fast path that applies to any run of printable
|
||||
ASCII. 2x-10x faster for ASCII text vs v0.7.0. (#16)
|
||||
- Upgraded uax29 dependency to v2.4.0 for Unicode 16 support. Text that includes
|
||||
Indic_Conjunct_Break may segment differently (and more correctly). (#15)
|
||||
|
||||
## [0.7.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.6.2...v0.7.0)
|
||||
|
||||
### Added
|
||||
- New `TruncateString` and `TruncateBytes` methods to truncate strings to a
|
||||
maximum display width, with optional tail (like an ellipsis). (#13)
|
||||
|
||||
## [0.6.2]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.6.1...v0.6.2)
|
||||
|
||||
### Changed
|
||||
- Internal: reduced property categories for simpler trie.
|
||||
|
||||
## [0.6.1]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.6.0...v0.6.1)
|
||||
|
||||
### Changed
|
||||
- Perf improvements: replaced the ASCII lookup table with a simple
|
||||
function. A bit more cache-friendly. More inlining.
|
||||
- Bug fix: single regional indicators are now treated as width 2, since that
|
||||
is what actual terminals do.
|
||||
|
||||
## [0.6.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.5.0...v0.6.0)
|
||||
|
||||
### Added
|
||||
- New `StringGraphemes` and `BytesGraphemes` methods, for iterating over the
|
||||
widths of grapheme clusters.
|
||||
|
||||
### Changed
|
||||
- Fast ASCII lookups
|
||||
|
||||
## [0.5.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.4.1...v0.5.0)
|
||||
|
||||
### Added
|
||||
- Unicode 16 support
|
||||
- Improved emoji presentation handling per Unicode TR51
|
||||
|
||||
### Changed
|
||||
- Corrected VS15 (U+FE0E) handling: now preserves base character width (no-op) per Unicode TR51
|
||||
- Performance optimizations: reduced property lookups
|
||||
|
||||
### Fixed
|
||||
- VS15 variation selector now correctly preserves base character width instead of forcing width 1
|
||||
|
||||
## [0.4.1]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.4.0...v0.4.1)
|
||||
|
||||
### Changed
|
||||
- Updated uax29 dependency
|
||||
- Improved flag handling
|
||||
|
||||
## [0.4.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.3.1...v0.4.0)
|
||||
|
||||
### Added
|
||||
- Support for variation selectors (VS15, VS16) and regional indicator pairs (flags)
|
||||
|
||||
## [0.3.1]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.3.0...v0.3.1)
|
||||
|
||||
### Added
|
||||
- Fuzz testing support
|
||||
|
||||
### Changed
|
||||
- Updated stringish dependency
|
||||
|
||||
## [0.3.0]
|
||||
|
||||
[Compare](https://github.com/clipperhouse/displaywidth/compare/v0.2.0...v0.3.0)
|
||||
|
||||
### Changed
|
||||
- Dropped compatibility with go-runewidth
|
||||
- Trie implementation cleanup
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Matt Sherman
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+170
@@ -0,0 +1,170 @@
|
||||
# displaywidth
|
||||
|
||||
A high-performance Go package for measuring the monospace display width of strings, UTF-8 bytes, and runes.
|
||||
|
||||
[](https://pkg.go.dev/github.com/clipperhouse/displaywidth)
|
||||
[](https://github.com/clipperhouse/displaywidth/actions/workflows/gotest.yml)
|
||||
[](https://github.com/clipperhouse/displaywidth/actions/workflows/gofuzz.yml)
|
||||
|
||||
## Install
|
||||
```bash
|
||||
go get github.com/clipperhouse/displaywidth
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/clipperhouse/displaywidth"
|
||||
)
|
||||
|
||||
func main() {
|
||||
width := displaywidth.String("Hello, 世界!")
|
||||
fmt.Println(width)
|
||||
|
||||
width = displaywidth.Bytes([]byte("🌍"))
|
||||
fmt.Println(width)
|
||||
|
||||
width = displaywidth.Rune('🌍')
|
||||
fmt.Println(width)
|
||||
}
|
||||
```
|
||||
|
||||
For most purposes, you should use the `String` or `Bytes` methods. They sum
|
||||
the widths of grapheme clusters in the string or byte slice.
|
||||
|
||||
> Note: in your application, iterating over runes to measure width is likely incorrect;
|
||||
the smallest unit of display is a grapheme, not a rune.
|
||||
|
||||
### Iterating over graphemes
|
||||
|
||||
If you need the individual graphemes:
|
||||
|
||||
```go
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/clipperhouse/displaywidth"
|
||||
)
|
||||
|
||||
func main() {
|
||||
g := displaywidth.StringGraphemes("Hello, 世界!")
|
||||
for g.Next() {
|
||||
width := g.Width()
|
||||
value := g.Value()
|
||||
// do something with the width or value
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
Create the options you need, and then use methods on the options struct.
|
||||
|
||||
```go
|
||||
var myOptions = displaywidth.Options{
|
||||
EastAsianWidth: true,
|
||||
ControlSequences: true,
|
||||
}
|
||||
|
||||
width := myOptions.String("Hello, 世界!")
|
||||
```
|
||||
|
||||
#### ControlSequences
|
||||
|
||||
`ControlSequences` specifies whether to ignore ECMA-48 escape sequences
|
||||
when calculating the display width. When `false` (default), ANSI escape
|
||||
sequences are treated as just a series of characters. When `true`, they are
|
||||
treated as a single zero-width unit.
|
||||
|
||||
#### EastAsianWidth
|
||||
|
||||
`EastAsianWidth` defines how
|
||||
[East Asian Ambiguous characters](https://www.unicode.org/reports/tr11/#Ambiguous)
|
||||
are treated.
|
||||
|
||||
When `false` (default), East Asian Ambiguous characters are treated as width 1.
|
||||
When `true`, they are treated as width 2.
|
||||
|
||||
You may wish to configure this based on environment variables or locale.
|
||||
`go-runewidth`, for example, does so
|
||||
[during package initialization](https://github.com/mattn/go-runewidth/blob/master/runewidth.go#L26C1-L45C2). `displaywidth` does not do this automatically, we prefer to leave it to you.
|
||||
|
||||
|
||||
## Technical standards and compatibility
|
||||
|
||||
This package implements the Unicode East Asian Width standard
|
||||
([UAX #11](https://www.unicode.org/reports/tr11/tr11-43.html)), and handles
|
||||
[version selectors](https://en.wikipedia.org/wiki/Variation_Selectors_(Unicode_block)),
|
||||
and [regional indicator pairs](https://en.wikipedia.org/wiki/Regional_indicator_symbol)
|
||||
(flags). We implement [Unicode TR51](https://www.unicode.org/reports/tr51/tr51-27.html)
|
||||
for emojis. We are keeping an eye on
|
||||
[emerging standards](https://www.jeffquast.com/post/state-of-terminal-emulation-2025/).
|
||||
|
||||
For control sequences, we implement the [ECMA-48](https://ecma-international.org/publications-and-standards/standards/ecma-48/) standard for 7-bit ASCII control sequences.
|
||||
|
||||
`clipperhouse/displaywidth`, `mattn/go-runewidth`, and `rivo/uniseg` will
|
||||
give the same outputs for most real-world text. Extensive details are in the
|
||||
[compatibility analysis](comparison/COMPATIBILITY_ANALYSIS.md).
|
||||
|
||||
## Prior Art
|
||||
|
||||
[mattn/go-runewidth](https://github.com/mattn/go-runewidth)
|
||||
|
||||
[rivo/uniseg](https://github.com/rivo/uniseg)
|
||||
|
||||
[x/text/width](https://pkg.go.dev/golang.org/x/text/width)
|
||||
|
||||
[x/text/internal/triegen](https://pkg.go.dev/golang.org/x/text/internal/triegen)
|
||||
|
||||
## Benchmarks
|
||||
|
||||
```bash
|
||||
cd comparison
|
||||
go test -bench=. -benchmem
|
||||
```
|
||||
|
||||
```
|
||||
goos: darwin
|
||||
goarch: arm64
|
||||
pkg: github.com/clipperhouse/displaywidth/comparison
|
||||
cpu: Apple M2
|
||||
|
||||
BenchmarkString_Mixed/clipperhouse/displaywidth-8 5784 ns/op 291.69 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_Mixed/mattn/go-runewidth-8 14751 ns/op 114.36 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_Mixed/rivo/uniseg-8 19360 ns/op 87.14 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkString_ASCII/clipperhouse/displaywidth-8 54.60 ns/op 2344.32 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_ASCII/mattn/go-runewidth-8 1195 ns/op 107.08 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_ASCII/rivo/uniseg-8 1578 ns/op 81.13 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 5837 ns/op 289.01 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_EastAsian/mattn/go-runewidth-8 24418 ns/op 69.09 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_EastAsian/rivo/uniseg-8 19339 ns/op 87.23 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3225 ns/op 224.51 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_Emoji/mattn/go-runewidth-8 4851 ns/op 149.25 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkString_Emoji/rivo/uniseg-8 6591 ns/op 109.85 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3385 ns/op 498.34 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkRune_Mixed/mattn/go-runewidth-8 5354 ns/op 315.07 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3397 ns/op 496.56 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15673 ns/op 107.64 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 255.7 ns/op 500.53 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkRune_ASCII/mattn/go-runewidth-8 261.5 ns/op 489.55 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1371 ns/op 528.22 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkRune_Emoji/mattn/go-runewidth-8 2267 ns/op 319.43 MB/s 0 B/op 0 allocs/op
|
||||
|
||||
BenchmarkTruncateWithTail/clipperhouse/displaywidth-8 3229 ns/op 54.82 MB/s 192 B/op 14 allocs/op
|
||||
BenchmarkTruncateWithTail/mattn/go-runewidth-8 8408 ns/op 21.05 MB/s 192 B/op 14 allocs/op
|
||||
|
||||
BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8 3554 ns/op 64.43 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8 11189 ns/op 20.47 MB/s 0 B/op 0 allocs/op
|
||||
```
|
||||
|
||||
Here are some notes on [how to make Unicode things fast](https://clipperhouse.com/go-unicode/).
|
||||
+3
@@ -0,0 +1,3 @@
|
||||
package displaywidth
|
||||
|
||||
//go:generate go run -C internal/gen .
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
package displaywidth
|
||||
|
||||
import (
|
||||
"github.com/clipperhouse/uax29/v2/graphemes"
|
||||
)
|
||||
|
||||
// Graphemes is an iterator over grapheme clusters.
|
||||
//
|
||||
// Iterate using the Next method, and get the width of the current grapheme
|
||||
// using the Width method.
|
||||
type Graphemes[T ~string | []byte] struct {
|
||||
iter *graphemes.Iterator[T]
|
||||
options Options
|
||||
}
|
||||
|
||||
// Next advances the iterator to the next grapheme cluster.
|
||||
func (g *Graphemes[T]) Next() bool {
|
||||
return g.iter.Next()
|
||||
}
|
||||
|
||||
// Value returns the current grapheme cluster.
|
||||
func (g *Graphemes[T]) Value() T {
|
||||
return g.iter.Value()
|
||||
}
|
||||
|
||||
// Width returns the display width of the current grapheme cluster.
|
||||
func (g *Graphemes[T]) Width() int {
|
||||
return graphemeWidth(g.Value(), g.options)
|
||||
}
|
||||
|
||||
// StringGraphemes returns an iterator over grapheme clusters for the given
|
||||
// string.
|
||||
//
|
||||
// Iterate using the Next method, and get the width of the current grapheme
|
||||
// using the Width method.
|
||||
func StringGraphemes(s string) Graphemes[string] {
|
||||
return DefaultOptions.StringGraphemes(s)
|
||||
}
|
||||
|
||||
// StringGraphemes returns an iterator over grapheme clusters for the given
|
||||
// string, with the given options.
|
||||
//
|
||||
// Iterate using the Next method, and get the width of the current grapheme
|
||||
// using the Width method.
|
||||
func (options Options) StringGraphemes(s string) Graphemes[string] {
|
||||
g := graphemes.FromString(s)
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
return Graphemes[string]{iter: g, options: options}
|
||||
}
|
||||
|
||||
// BytesGraphemes returns an iterator over grapheme clusters for the given
|
||||
// []byte.
|
||||
//
|
||||
// Iterate using the Next method, and get the width of the current grapheme
|
||||
// using the Width method.
|
||||
func BytesGraphemes(s []byte) Graphemes[[]byte] {
|
||||
return DefaultOptions.BytesGraphemes(s)
|
||||
}
|
||||
|
||||
// BytesGraphemes returns an iterator over grapheme clusters for the given
|
||||
// []byte, with the given options.
|
||||
//
|
||||
// Iterate using the Next method, and get the width of the current grapheme
|
||||
// using the Width method.
|
||||
func (options Options) BytesGraphemes(s []byte) Graphemes[[]byte] {
|
||||
g := graphemes.FromBytes(s)
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
return Graphemes[[]byte]{iter: g, options: options}
|
||||
}
|
||||
+1699
File diff suppressed because it is too large
Load Diff
+367
@@ -0,0 +1,367 @@
|
||||
package displaywidth
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/clipperhouse/uax29/v2/graphemes"
|
||||
)
|
||||
|
||||
// Options allows you to specify the treatment of ambiguous East Asian
|
||||
// characters and ANSI escape sequences.
|
||||
type Options struct {
|
||||
// EastAsianWidth specifies whether to treat ambiguous East Asian characters
|
||||
// as width 1 or 2. When false (default), ambiguous East Asian characters
|
||||
// are treated as width 1. When true, they are width 2.
|
||||
EastAsianWidth bool
|
||||
|
||||
// ControlSequences specifies whether to ignore ECMA-48 escape sequences
|
||||
// when calculating the display width. When false (default), ANSI escape
|
||||
// sequences are treated as just a series of characters. When true, they are
|
||||
// treated as a single zero-width unit.
|
||||
//
|
||||
// Note that this option is about *sequences*. Individual control characters
|
||||
// are already treated as zero-width. With this option, ANSI sequences such as
|
||||
// "\x1b[31m" and "\x1b[0m" do not count towards the width of a string.
|
||||
ControlSequences bool
|
||||
}
|
||||
|
||||
// DefaultOptions is the default options for the display width
|
||||
// calculation, which is EastAsianWidth false and ControlSequences false.
|
||||
var DefaultOptions = Options{EastAsianWidth: false, ControlSequences: false}
|
||||
|
||||
// String calculates the display width of a string,
|
||||
// by iterating over grapheme clusters in the string
|
||||
// and summing their widths.
|
||||
func String(s string) int {
|
||||
return DefaultOptions.String(s)
|
||||
}
|
||||
|
||||
// String calculates the display width of a string, for the given options, by
|
||||
// iterating over grapheme clusters in the string and summing their widths.
|
||||
func (options Options) String(s string) int {
|
||||
width := 0
|
||||
pos := 0
|
||||
|
||||
for pos < len(s) {
|
||||
// Try ASCII optimization
|
||||
asciiLen := printableASCIILength(s[pos:])
|
||||
if asciiLen > 0 {
|
||||
width += asciiLen
|
||||
pos += asciiLen
|
||||
continue
|
||||
}
|
||||
|
||||
// Not ASCII, use grapheme parsing
|
||||
g := graphemes.FromString(s[pos:])
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
start := pos
|
||||
|
||||
for g.Next() {
|
||||
v := g.Value()
|
||||
width += graphemeWidth(v, options)
|
||||
pos += len(v)
|
||||
|
||||
// Quick check: if remaining might have printable ASCII, break to outer loop
|
||||
if pos < len(s) && s[pos] >= 0x20 && s[pos] <= 0x7E {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// Defensive, should not happen: if no progress was made,
|
||||
// skip a byte to prevent infinite loop. Only applies if
|
||||
// the grapheme parser misbehaves.
|
||||
if pos == start {
|
||||
pos++
|
||||
}
|
||||
}
|
||||
|
||||
return width
|
||||
}
|
||||
|
||||
// Bytes calculates the display width of a []byte,
|
||||
// by iterating over grapheme clusters in the byte slice
|
||||
// and summing their widths.
|
||||
func Bytes(s []byte) int {
|
||||
return DefaultOptions.Bytes(s)
|
||||
}
|
||||
|
||||
// Bytes calculates the display width of a []byte, for the given options, by
|
||||
// iterating over grapheme clusters in the slice and summing their widths.
|
||||
func (options Options) Bytes(s []byte) int {
|
||||
width := 0
|
||||
pos := 0
|
||||
|
||||
for pos < len(s) {
|
||||
// Try ASCII optimization
|
||||
asciiLen := printableASCIILength(s[pos:])
|
||||
if asciiLen > 0 {
|
||||
width += asciiLen
|
||||
pos += asciiLen
|
||||
continue
|
||||
}
|
||||
|
||||
// Not ASCII, use grapheme parsing
|
||||
g := graphemes.FromBytes(s[pos:])
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
start := pos
|
||||
|
||||
for g.Next() {
|
||||
v := g.Value()
|
||||
width += graphemeWidth(v, options)
|
||||
pos += len(v)
|
||||
|
||||
// Quick check: if remaining might have printable ASCII, break to outer loop
|
||||
if pos < len(s) && s[pos] >= 0x20 && s[pos] <= 0x7E {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// Defensive, should not happen: if no progress was made,
|
||||
// skip a byte to prevent infinite loop. Only applies if
|
||||
// the grapheme parser misbehaves.
|
||||
if pos == start {
|
||||
pos++
|
||||
}
|
||||
}
|
||||
|
||||
return width
|
||||
}
|
||||
|
||||
// Rune calculates the display width of a rune. You
|
||||
// should almost certainly use [String] or [Bytes] for
|
||||
// most purposes.
|
||||
//
|
||||
// The smallest unit of display width is a grapheme
|
||||
// cluster, not a rune. Iterating over runes to measure
|
||||
// width is incorrect in many cases.
|
||||
func Rune(r rune) int {
|
||||
return DefaultOptions.Rune(r)
|
||||
}
|
||||
|
||||
// Rune calculates the display width of a rune, for the given options.
|
||||
//
|
||||
// You should almost certainly use [String] or [Bytes] for most purposes.
|
||||
//
|
||||
// The smallest unit of display width is a grapheme cluster, not a rune.
|
||||
// Iterating over runes to measure width is incorrect in many cases.
|
||||
func (options Options) Rune(r rune) int {
|
||||
if r < utf8.RuneSelf {
|
||||
return asciiWidth(byte(r))
|
||||
}
|
||||
|
||||
// Surrogates (U+D800-U+DFFF) are invalid UTF-8.
|
||||
if r >= 0xD800 && r <= 0xDFFF {
|
||||
return 0
|
||||
}
|
||||
|
||||
var buf [4]byte
|
||||
n := utf8.EncodeRune(buf[:], r)
|
||||
|
||||
// Skip the grapheme iterator
|
||||
return graphemeWidth(buf[:n], options)
|
||||
}
|
||||
|
||||
const _Default property = 0
|
||||
|
||||
// TruncateString truncates a string to the given maxWidth, and appends the
|
||||
// given tail if the string is truncated.
|
||||
//
|
||||
// It ensures the visible width, including the width of the tail, is less than or
|
||||
// equal to maxWidth.
|
||||
//
|
||||
// When [Options.ControlSequences] is true, ANSI escape sequences that appear
|
||||
// after the truncation point are preserved in the output. This ensures that
|
||||
// escape sequences such as SGR resets are not lost, preventing color bleed
|
||||
// in terminal output.
|
||||
func (options Options) TruncateString(s string, maxWidth int, tail string) string {
|
||||
maxWidthWithoutTail := maxWidth - options.String(tail)
|
||||
|
||||
var pos, total int
|
||||
g := graphemes.FromString(s)
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
for g.Next() {
|
||||
gw := graphemeWidth(g.Value(), options)
|
||||
if total+gw <= maxWidthWithoutTail {
|
||||
pos = g.End()
|
||||
}
|
||||
total += gw
|
||||
if total > maxWidth {
|
||||
if options.ControlSequences {
|
||||
// Build result with trailing ANSI escape sequences preserved
|
||||
var b strings.Builder
|
||||
b.Grow(len(s) + len(tail)) // at most original + tail
|
||||
b.WriteString(s[:pos])
|
||||
b.WriteString(tail)
|
||||
rem := graphemes.FromString(s[pos:])
|
||||
rem.AnsiEscapeSequences = true
|
||||
for rem.Next() {
|
||||
v := rem.Value()
|
||||
if len(v) > 0 && v[0] == 0x1B {
|
||||
b.WriteString(v)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
return s[:pos] + tail
|
||||
}
|
||||
}
|
||||
// No truncation
|
||||
return s
|
||||
}
|
||||
|
||||
// TruncateString truncates a string to the given maxWidth, and appends the
|
||||
// given tail if the string is truncated.
|
||||
//
|
||||
// It ensures the total width, including the width of the tail, is less than or
|
||||
// equal to maxWidth.
|
||||
func TruncateString(s string, maxWidth int, tail string) string {
|
||||
return DefaultOptions.TruncateString(s, maxWidth, tail)
|
||||
}
|
||||
|
||||
// TruncateBytes truncates a []byte to the given maxWidth, and appends the
|
||||
// given tail if the []byte is truncated.
|
||||
//
|
||||
// It ensures the visible width, including the width of the tail, is less than or
|
||||
// equal to maxWidth.
|
||||
//
|
||||
// When [Options.ControlSequences] is true, ANSI escape sequences that appear
|
||||
// after the truncation point are preserved in the output. This ensures that
|
||||
// escape sequences such as SGR resets are not lost, preventing color bleed
|
||||
// in terminal output.
|
||||
func (options Options) TruncateBytes(s []byte, maxWidth int, tail []byte) []byte {
|
||||
maxWidthWithoutTail := maxWidth - options.Bytes(tail)
|
||||
|
||||
var pos, total int
|
||||
g := graphemes.FromBytes(s)
|
||||
g.AnsiEscapeSequences = options.ControlSequences
|
||||
|
||||
for g.Next() {
|
||||
gw := graphemeWidth(g.Value(), options)
|
||||
if total+gw <= maxWidthWithoutTail {
|
||||
pos = g.End()
|
||||
}
|
||||
total += gw
|
||||
if total > maxWidth {
|
||||
if options.ControlSequences {
|
||||
// Build result with trailing ANSI escape sequences preserved
|
||||
result := make([]byte, 0, len(s)+len(tail)) // at most original + tail
|
||||
result = append(result, s[:pos]...)
|
||||
result = append(result, tail...)
|
||||
rem := graphemes.FromBytes(s[pos:])
|
||||
rem.AnsiEscapeSequences = true
|
||||
for rem.Next() {
|
||||
v := rem.Value()
|
||||
if len(v) > 0 && v[0] == 0x1B {
|
||||
result = append(result, v...)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
result := make([]byte, 0, pos+len(tail))
|
||||
result = append(result, s[:pos]...)
|
||||
result = append(result, tail...)
|
||||
return result
|
||||
}
|
||||
}
|
||||
// No truncation
|
||||
return s
|
||||
}
|
||||
|
||||
// TruncateBytes truncates a []byte to the given maxWidth, and appends the
|
||||
// given tail if the []byte is truncated.
|
||||
//
|
||||
// It ensures the total width, including the width of the tail, is less than or
|
||||
// equal to maxWidth.
|
||||
func TruncateBytes(s []byte, maxWidth int, tail []byte) []byte {
|
||||
return DefaultOptions.TruncateBytes(s, maxWidth, tail)
|
||||
}
|
||||
|
||||
// graphemeWidth returns the display width of a grapheme cluster.
|
||||
// The passed string must be a single grapheme cluster.
|
||||
func graphemeWidth[T ~string | []byte](s T, options Options) int {
|
||||
// Optimization: no need to look up properties
|
||||
switch len(s) {
|
||||
case 0:
|
||||
return 0
|
||||
case 1:
|
||||
return asciiWidth(s[0])
|
||||
}
|
||||
|
||||
// Multi-byte grapheme clusters led by a C0 control (0x00-0x1F)
|
||||
if s[0] <= 0x1F {
|
||||
return 0
|
||||
}
|
||||
|
||||
p, sz := lookup(s)
|
||||
prop := property(p)
|
||||
|
||||
// Variation Selector 16 (VS16) requests emoji presentation
|
||||
if prop != _Wide && sz > 0 && len(s) >= sz+3 {
|
||||
vs := s[sz : sz+3]
|
||||
if isVS16(vs) {
|
||||
prop = _Wide
|
||||
}
|
||||
// VS15 (0x8E) requests text presentation but does not affect width,
|
||||
// in my reading of Unicode TR51. Falls through to return the base
|
||||
// character's property.
|
||||
}
|
||||
|
||||
if options.EastAsianWidth && prop == _East_Asian_Ambiguous {
|
||||
prop = _Wide
|
||||
}
|
||||
|
||||
if prop > upperBound {
|
||||
prop = _Default
|
||||
}
|
||||
|
||||
return propertyWidths[prop]
|
||||
}
|
||||
|
||||
func asciiWidth(b byte) int {
|
||||
if b <= 0x1F || b == 0x7F {
|
||||
return 0
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
// printableASCIILength returns the length of consecutive printable ASCII bytes
|
||||
// starting at the beginning of s.
|
||||
func printableASCIILength[T string | []byte](s T) int {
|
||||
i := 0
|
||||
for ; i < len(s); i++ {
|
||||
b := s[i]
|
||||
// Printable ASCII is 0x20-0x7E (space through tilde)
|
||||
if b < 0x20 || b > 0x7E {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// If the next byte is non-ASCII (>= 0x80), back off by 1. The grapheme
|
||||
// parser may group the last ASCII byte with subsequent non-ASCII bytes,
|
||||
// such as combining marks.
|
||||
if i > 0 && i < len(s) && s[i] >= 0x80 {
|
||||
i--
|
||||
}
|
||||
|
||||
return i
|
||||
}
|
||||
|
||||
// isVS16 checks if the slice matches VS16 (U+FE0F) UTF-8 encoding
|
||||
// (EF B8 8F). It assumes len(s) >= 3.
|
||||
func isVS16[T ~string | []byte](s T) bool {
|
||||
return s[0] == 0xEF && s[1] == 0xB8 && s[2] == 0x8F
|
||||
}
|
||||
|
||||
// propertyWidths is a jump table of sorts, instead of a switch
|
||||
var propertyWidths = [4]int{
|
||||
_Default: 1,
|
||||
_Zero_Width: 0,
|
||||
_Wide: 2,
|
||||
_East_Asian_Ambiguous: 1,
|
||||
}
|
||||
|
||||
const upperBound = property(len(propertyWidths) - 1)
|
||||
Reference in New Issue
Block a user