Initial QSfera import
This commit is contained in:
+234
@@ -0,0 +1,234 @@
|
||||
# ksuid [](https://goreportcard.com/report/github.com/segmentio/ksuid) [](https://godoc.org/github.com/segmentio/ksuid) [](https://circleci.com/gh/segmentio/ksuid.svg?style=shield)
|
||||
|
||||
ksuid is an efficient, comprehensive, battle-tested Go library for
|
||||
generating and parsing a specific kind of globally unique identifier
|
||||
called a *KSUID*. This library serves as its reference implementation.
|
||||
|
||||
## Install
|
||||
```sh
|
||||
go get -u github.com/segmentio/ksuid
|
||||
```
|
||||
|
||||
## What is a KSUID?
|
||||
|
||||
KSUID is for K-Sortable Unique IDentifier. It is a kind of globally
|
||||
unique identifier similar to a [RFC 4122 UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), built from the ground-up to be "naturally"
|
||||
sorted by generation timestamp without any special type-aware logic.
|
||||
|
||||
In short, running a set of KSUIDs through the UNIX `sort` command will result
|
||||
in a list ordered by generation time.
|
||||
|
||||
## Why use KSUIDs?
|
||||
|
||||
There are numerous methods for generating unique identifiers, so why KSUID?
|
||||
|
||||
1. Naturally ordered by generation time
|
||||
2. Collision-free, coordination-free, dependency-free
|
||||
3. Highly portable representations
|
||||
|
||||
Even if only one of these properties are important to you, KSUID is a great
|
||||
choice! :) Many projects chose to use KSUIDs *just* because the text
|
||||
representation is copy-and-paste friendly.
|
||||
|
||||
### 1. Naturally Ordered By Generation Time
|
||||
|
||||
Unlike the more ubiquitous UUIDv4, a KSUID contains a timestamp component
|
||||
that allows them to be loosely sorted by generation time. This is not a strong
|
||||
guarantee (an invariant) as it depends on wall clocks, but is still incredibly
|
||||
useful in practice. Both the binary and text representations will sort by
|
||||
creation time without any special sorting logic.
|
||||
|
||||
### 2. Collision-free, Coordination-free, Dependency-free
|
||||
|
||||
While RFC 4122 UUIDv1s *do* include a time component, there aren't enough
|
||||
bytes of randomness to provide strong protection against collisions
|
||||
(duplicates). With such a low amount of entropy, it is feasible for a
|
||||
malicious party to guess generated IDs, creating a problem for systems whose
|
||||
security is, implicitly or explicitly, sensitive to an adversary guessing
|
||||
identifiers.
|
||||
|
||||
To fit into a 64-bit number space, [Snowflake IDs](https://blog.twitter.com/2010/announcing-snowflake)
|
||||
and its derivatives require coordination to avoid collisions, which
|
||||
significantly increases the deployment complexity and operational burden.
|
||||
|
||||
A KSUID includes 128 bits of pseudorandom data ("entropy"). This number space
|
||||
is 64 times larger than the 122 bits used by the well-accepted RFC 4122 UUIDv4
|
||||
standard. The additional timestamp component can be considered "bonus entropy"
|
||||
which further decreases the probability of collisions, to the point of physical
|
||||
infeasibility in any practical implementation.
|
||||
|
||||
### Highly Portable Representations
|
||||
|
||||
The text *and* binary representations are lexicographically sortable, which
|
||||
allows them to be dropped into systems which do not natively support KSUIDs
|
||||
and retain their time-ordered property.
|
||||
|
||||
The text representation is an alphanumeric base62 encoding, so it "fits"
|
||||
anywhere alphanumeric strings are accepted. No delimiters are used, so
|
||||
stringified KSUIDs won't be inadvertently truncated or tokenized when
|
||||
interpreted by software that is designed for human-readable text, a common
|
||||
problem for the text representation of RFC 4122 UUIDs.
|
||||
|
||||
## How do KSUIDs work?
|
||||
|
||||
Binary KSUIDs are 20-bytes: a 32-bit unsigned integer UTC timestamp and
|
||||
a 128-bit randomly generated payload. The timestamp uses big-endian
|
||||
encoding, to support lexicographic sorting. The timestamp epoch is adjusted
|
||||
to March 5th, 2014, providing over 100 years of life. The payload is
|
||||
generated by a cryptographically-strong pseudorandom number generator.
|
||||
|
||||
The text representation is always 27 characters, encoded in alphanumeric
|
||||
base62 that will lexicographically sort by timestamp.
|
||||
|
||||
## High Performance
|
||||
|
||||
This library is designed to be used in code paths that are performance
|
||||
critical. Its code has been tuned to eliminate all non-essential
|
||||
overhead. The `KSUID` type is derived from a fixed-size array, which
|
||||
eliminates the additional reference chasing and allocation involved in
|
||||
a variable-width type.
|
||||
|
||||
The API provides an interface for use in code paths which are sensitive
|
||||
to allocation. For example, the `Append` method can be used to parse the
|
||||
text representation and replace the contents of a `KSUID` value
|
||||
without additional heap allocation.
|
||||
|
||||
All public package level "pure" functions are concurrency-safe, protected
|
||||
by a global mutex. For hot loops that generate a large amount of KSUIDs
|
||||
from a single Goroutine, the `Sequence` type is provided to elide the
|
||||
potential contention.
|
||||
|
||||
By default, out of an abundance of caution, the cryptographically-secure
|
||||
PRNG is used to generate the random bits of a KSUID. This can be relaxed
|
||||
in extremely performance-critical code using the included `FastRander`
|
||||
type. `FastRander` uses the standard PRNG with a seed generated by the
|
||||
cryptographically-secure PRNG.
|
||||
|
||||
*_NOTE:_ While there is no evidence that `FastRander` will increase the
|
||||
probability of a collision, it shouldn't be used in scenarios where
|
||||
uniqueness is important to security, as there is an increased chance
|
||||
the generated IDs can be predicted by an adversary.*
|
||||
|
||||
## Battle Tested
|
||||
|
||||
This code has been used in production at Segment for several years,
|
||||
across a diverse array of projects. Trillions upon trillions of
|
||||
KSUIDs have been generated in some of Segment's most
|
||||
performance-critical, large-scale distributed systems.
|
||||
|
||||
## Plays Well With Others
|
||||
|
||||
Designed to be integrated with other libraries, the `KSUID` type
|
||||
implements many standard library interfaces, including:
|
||||
|
||||
* `Stringer`
|
||||
* `database/sql.Scanner` and `database/sql/driver.Valuer`
|
||||
* `encoding.BinaryMarshal` and `encoding.BinaryUnmarshal`
|
||||
* `encoding.TextMarshal` and `encoding.TextUnmarshal`
|
||||
(`encoding/json` friendly!)
|
||||
|
||||
## Command Line Tool
|
||||
|
||||
This package comes with a command-line tool `ksuid`, useful for
|
||||
generating KSUIDs as well as inspecting the internal components of
|
||||
existing KSUIDs. Machine-friendly output is provided for scripting
|
||||
use cases.
|
||||
|
||||
Given a Go build environment, it can be installed with the command:
|
||||
|
||||
```sh
|
||||
$ go install github.com/segmentio/ksuid/cmd/ksuid
|
||||
```
|
||||
|
||||
## CLI Usage Examples
|
||||
|
||||
### Generate a KSUID
|
||||
|
||||
```sh
|
||||
$ ksuid
|
||||
0ujsswThIGTUYm2K8FjOOfXtY1K
|
||||
```
|
||||
|
||||
### Generate 4 KSUIDs
|
||||
|
||||
```sh
|
||||
$ ksuid -n 4
|
||||
0ujsszwN8NRY24YaXiTIE2VWDTS
|
||||
0ujsswThIGTUYm2K8FjOOfXtY1K
|
||||
0ujssxh0cECutqzMgbtXSGnjorm
|
||||
0ujsszgFvbiEr7CDgE3z8MAUPFt
|
||||
```
|
||||
|
||||
### Inspect the components of a KSUID
|
||||
|
||||
```sh
|
||||
$ ksuid -f inspect 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
||||
|
||||
REPRESENTATION:
|
||||
|
||||
String: 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
||||
Raw: 0669F7EFB5A1CD34B5F99D1154FB6853345C9735
|
||||
|
||||
COMPONENTS:
|
||||
|
||||
Time: 2017-10-09 21:00:47 -0700 PDT
|
||||
Timestamp: 107608047
|
||||
Payload: B5A1CD34B5F99D1154FB6853345C9735
|
||||
```
|
||||
|
||||
### Generate a KSUID and inspect its components
|
||||
|
||||
```sh
|
||||
$ ksuid -f inspect
|
||||
|
||||
REPRESENTATION:
|
||||
|
||||
String: 0ujzPyRiIAffKhBux4PvQdDqMHY
|
||||
Raw: 066A029C73FC1AA3B2446246D6E89FCD909E8FE8
|
||||
|
||||
COMPONENTS:
|
||||
|
||||
Time: 2017-10-09 21:46:20 -0700 PDT
|
||||
Timestamp: 107610780
|
||||
Payload: 73FC1AA3B2446246D6E89FCD909E8FE8
|
||||
|
||||
```
|
||||
|
||||
### Inspect a KSUID with template formatted inspection output
|
||||
|
||||
```sh
|
||||
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' 0ujtsYcgvSTl8PAuAdqWYSMnLOv
|
||||
2017-10-09 21:00:47 -0700 PDT: B5A1CD34B5F99D1154FB6853345C9735
|
||||
```
|
||||
|
||||
### Inspect multiple KSUIDs with template formatted output
|
||||
|
||||
```sh
|
||||
$ ksuid -f template -t '{{ .Time }}: {{ .Payload }}' $(ksuid -n 4)
|
||||
2017-10-09 21:05:37 -0700 PDT: 304102BC687E087CC3A811F21D113CCF
|
||||
2017-10-09 21:05:37 -0700 PDT: EAF0B240A9BFA55E079D887120D962F0
|
||||
2017-10-09 21:05:37 -0700 PDT: DF0761769909ABB0C7BB9D66F79FC041
|
||||
2017-10-09 21:05:37 -0700 PDT: 1A8F0E3D0BDEB84A5FAD702876F46543
|
||||
```
|
||||
|
||||
### Generate KSUIDs and output JSON using template formatting
|
||||
|
||||
```sh
|
||||
$ ksuid -f template -t '{ "timestamp": "{{ .Timestamp }}", "payload": "{{ .Payload }}", "ksuid": "{{.String}}"}' -n 4
|
||||
{ "timestamp": "107611700", "payload": "9850EEEC191BF4FF26F99315CE43B0C8", "ksuid": "0uk1Hbc9dQ9pxyTqJ93IUrfhdGq"}
|
||||
{ "timestamp": "107611700", "payload": "CC55072555316F45B8CA2D2979D3ED0A", "ksuid": "0uk1HdCJ6hUZKDgcxhpJwUl5ZEI"}
|
||||
{ "timestamp": "107611700", "payload": "BA1C205D6177F0992D15EE606AE32238", "ksuid": "0uk1HcdvF0p8C20KtTfdRSB9XIm"}
|
||||
{ "timestamp": "107611700", "payload": "67517BA309EA62AE7991B27BB6F2FCAC", "ksuid": "0uk1Ha7hGJ1Q9Xbnkt0yZgNwg3g"}
|
||||
```
|
||||
|
||||
## Implementations for other languages
|
||||
|
||||
- Python: [svix-ksuid](https://github.com/svixhq/python-ksuid/)
|
||||
- Ruby: [ksuid-ruby](https://github.com/michaelherold/ksuid-ruby)
|
||||
- Java: [ksuid](https://github.com/ksuid/ksuid)
|
||||
- Rust: [rksuid](https://github.com/nharring/rksuid)
|
||||
- dotNet: [Ksuid.Net](https://github.com/JoyMoe/Ksuid.Net)
|
||||
|
||||
## License
|
||||
|
||||
ksuid source code is available under an MIT [License](/LICENSE.md).
|
||||
Reference in New Issue
Block a user