Initial QSfera import
This commit is contained in:
Курнат Андрей
+749
@@ -0,0 +1,749 @@
|
||||
# Migrating from Legacy JetStream API to `jetstream` Package
|
||||
|
||||
This guide helps you migrate from the legacy JetStream API in the `nats` package
|
||||
(`nats.JetStreamContext`) to the new `jetstream` package
|
||||
(`github.com/nats-io/nats.go/jetstream`).
|
||||
|
||||
- [Why Migrate?](#why-migrate)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Stream Management](#stream-management)
|
||||
- [Consumer Management](#consumer-management)
|
||||
- [Publishing](#publishing)
|
||||
- [Consuming Messages](#consuming-messages)
|
||||
- [Replacing js.Subscribe()](#replacing-jssubscribe)
|
||||
- [Replacing js.PullSubscribe()](#replacing-jspullsubscribe)
|
||||
- [Ordered Consumers](#ordered-consumers)
|
||||
- [Push Consumers](#push-consumers)
|
||||
- [Subscription Options Mapping](#subscription-options-mapping)
|
||||
- [Error Handling in Consume/Messages](#error-handling-in-consumemessages)
|
||||
- [Message Acknowledgement](#message-acknowledgement)
|
||||
- [KeyValue Store](#keyvalue-store)
|
||||
- [Object Store](#object-store)
|
||||
|
||||
## Why Migrate?
|
||||
|
||||
The legacy JetStream API (`nats.JetStreamContext`) is deprecated. The `jetstream`
|
||||
package provides a cleaner, more predictable API with several key improvements:
|
||||
|
||||
- **Explicit resource management.** Streams and consumers are created and managed
|
||||
explicitly. The legacy `js.Subscribe()` implicitly created consumers behind
|
||||
the scenes, leading to surprising behavior.
|
||||
|
||||
- **Pull consumers as the default.** Pull consumers with `Consume()` and
|
||||
`Messages()` provide the same continuous message delivery as the legacy push-based
|
||||
`Subscribe()`, but with better flow control and no slow consumer issues.
|
||||
|
||||
- **`context.Context` throughout.** All API calls accept `context.Context` for
|
||||
timeout and cancellation, replacing the mix of `MaxWait`, `AckWait`, and
|
||||
`Context()` options.
|
||||
|
||||
- **Clear interface separation.** Instead of one large `JetStreamContext` interface,
|
||||
functionality is split across focused interfaces: `JetStream`, `Stream` and
|
||||
`Consumer`.
|
||||
|
||||
## Getting Started
|
||||
|
||||
The core NATS connection remains unchanged. Only the JetStream initialization
|
||||
differs:
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
nc, _ := nats.Connect(nats.DefaultURL)
|
||||
```
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
js, _ := nc.JetStream()
|
||||
|
||||
// With domain
|
||||
js, _ := nc.JetStream(nats.Domain("hub"))
|
||||
|
||||
// With custom API prefix
|
||||
js, _ := nc.JetStream(nats.APIPrefix("myprefix"))
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
js, _ := jetstream.New(nc)
|
||||
|
||||
// With domain
|
||||
js, _ := jetstream.NewWithDomain(nc, "hub")
|
||||
|
||||
// With custom API prefix
|
||||
js, _ := jetstream.NewWithAPIPrefix(nc, "myprefix")
|
||||
```
|
||||
|
||||
### Initialization Options
|
||||
|
||||
| Legacy | New |
|
||||
|-----------------------------------|--------------------------------------------|
|
||||
| `nats.Domain(domain)` | `jetstream.NewWithDomain(nc, domain)` |
|
||||
| `nats.APIPrefix(prefix)` | `jetstream.NewWithAPIPrefix(nc, prefix)` |
|
||||
| `nats.PublishAsyncMaxPending(n)` | `jetstream.WithPublishAsyncMaxPending(n)` |
|
||||
| `nats.PublishAsyncErrHandler(cb)` | `jetstream.WithPublishAsyncErrHandler(cb)` |
|
||||
|
||||
## Stream Management
|
||||
|
||||
`StreamConfig` is essentially the same struct — it just lives in the `jetstream`
|
||||
package now. The new API takes `StreamConfig` by value (not pointer) and
|
||||
management methods return a `Stream` handle instead of `*StreamInfo`.
|
||||
|
||||
| Legacy | New | Notes |
|
||||
|---------------------------------|-------------------------------------|-------------------------------------------------------------------|
|
||||
| `js.AddStream(cfg)` | `js.CreateStream(ctx, cfg)` | Also: `CreateOrUpdateStream()` |
|
||||
| `js.UpdateStream(cfg)` | `js.UpdateStream(ctx, cfg)` | |
|
||||
| `js.DeleteStream(name)` | `js.DeleteStream(ctx, name)` | |
|
||||
| `js.StreamInfo(name)` | `s.Info(ctx)` / `s.CachedInfo()` | Get stream handle first via `js.Stream(ctx, name)` |
|
||||
| `js.PurgeStream(name, opts...)` | `s.Purge(ctx, opts...)` | Options: `WithPurgeSubject`, `WithPurgeSequence`, `WithPurgeKeep` |
|
||||
| `js.GetMsg(name, seq)` | `s.GetMsg(ctx, seq)` | |
|
||||
| `js.GetLastMsg(name, subj)` | `s.GetLastMsgForSubject(ctx, subj)` | |
|
||||
| `js.DeleteMsg(name, seq)` | `s.DeleteMsg(ctx, seq)` | Also: `s.SecureDeleteMsg()` |
|
||||
| `js.Streams()` | `js.ListStreams(ctx)` | Returns lister with `.Info()` channel and `.Err()` |
|
||||
| `js.StreamNames()` | `js.StreamNames(ctx)` | Returns lister with `.Name()` channel and `.Err()` |
|
||||
|
||||
The key architectural difference is that stream-specific operations (purge, get/delete
|
||||
messages) now live on the `Stream` interface instead of the top-level context. Get
|
||||
a stream handle first, then operate on it:
|
||||
|
||||
```go
|
||||
s, _ := js.Stream(ctx, "ORDERS")
|
||||
s.Purge(ctx)
|
||||
msg, _ := s.GetMsg(ctx, 100)
|
||||
```
|
||||
|
||||
## Consumer Management
|
||||
|
||||
The biggest conceptual change: in the legacy API, `js.Subscribe()` would
|
||||
implicitly create consumers. In the new API, consumer creation is always explicit
|
||||
and separate from message consumption.
|
||||
|
||||
| Legacy | New | Notes |
|
||||
|------------------------------------------|----------------------------------------|----------------------------------------------------------------|
|
||||
| `js.AddConsumer(stream, cfg)` | `js.CreateConsumer(ctx, stream, cfg)` | Also: `CreateOrUpdateConsumer()`, `UpdateConsumer()` |
|
||||
| `js.Subscribe(subj, handler)` (implicit) | No equivalent | Must create consumer explicitly first |
|
||||
| `js.ConsumerInfo(stream, name)` | `cons.Info(ctx)` / `cons.CachedInfo()` | Get consumer handle first via `js.Consumer(ctx, stream, name)` |
|
||||
| `js.DeleteConsumer(stream, name)` | `js.DeleteConsumer(ctx, stream, name)` | |
|
||||
| `js.Consumers(stream)` | `s.ListConsumers(ctx)` | Returns lister with `.Info()` channel and `.Err()` |
|
||||
| `js.ConsumerNames(stream)` | `s.ConsumerNames(ctx)` | Returns lister with `.Name()` channel and `.Err()` |
|
||||
|
||||
Consumer management is available at two levels:
|
||||
|
||||
- On `JetStream` — requires stream name as parameter (e.g. `js.CreateConsumer(ctx, "ORDERS", cfg)`), bypassing the need to fetch a stream
|
||||
- On `Stream` — no stream name needed (e.g. `s.CreateConsumer(ctx, cfg)`)
|
||||
|
||||
The new API provides three creation methods:
|
||||
|
||||
- `CreateConsumer` — fails if the consumer already exists with different config
|
||||
- `UpdateConsumer` - fails if the consumer does not exist
|
||||
- `CreateOrUpdateConsumer` — creates or updates as needed
|
||||
|
||||
**Additional notes on consumer behavior:**
|
||||
|
||||
- The default ack policy changed between the APIs. In the legacy API,
|
||||
`AddConsumer()` defaulted to `AckNone`. In the new API, the default is
|
||||
`AckExplicit`.
|
||||
|
||||
- In the legacy API, `sub.Unsubscribe()` on an implicitly created
|
||||
consumer would automatically delete that consumer on the server. The new API
|
||||
does not perform any automatic cleanup - consumers must be deleted explicitly
|
||||
via `DeleteConsumer()`, or via `InactiveThreshold` on the consumer
|
||||
config to let the server remove it automatically after a period of inactivity.
|
||||
|
||||
Push consumers use separate methods: `CreatePushConsumer`, `CreateOrUpdatePushConsumer`,
|
||||
`UpdatePushConsumer`, and `PushConsumer` (for getting a handle).
|
||||
|
||||
```go
|
||||
s, _ := js.Stream(ctx, "ORDERS")
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
})
|
||||
```
|
||||
|
||||
## Publishing
|
||||
|
||||
Publishing is largely the same, with the addition of `context.Context` for
|
||||
synchronous operations.
|
||||
|
||||
### Synchronous Publish
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
ack, _ := js.Publish("ORDERS.new", []byte("hello"))
|
||||
ack, _ = js.PublishMsg(&nats.Msg{
|
||||
Subject: "ORDERS.new",
|
||||
Data: []byte("hello"),
|
||||
})
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
ack, _ := js.Publish(ctx, "ORDERS.new", []byte("hello"))
|
||||
ack, _ = js.PublishMsg(ctx, &nats.Msg{
|
||||
Subject: "ORDERS.new",
|
||||
Data: []byte("hello"),
|
||||
})
|
||||
```
|
||||
|
||||
### Async Publish
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
ackF, _ := js.PublishAsync("ORDERS.new", []byte("hello"))
|
||||
|
||||
select {
|
||||
case ack := <-ackF.Ok():
|
||||
fmt.Println(ack.Sequence)
|
||||
case err := <-ackF.Err():
|
||||
fmt.Println(err)
|
||||
}
|
||||
|
||||
// Wait for all pending acks
|
||||
<-js.PublishAsyncComplete()
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
// Async publish does not take context (returns immediately)
|
||||
ackF, _ := js.PublishAsync("ORDERS.new", []byte("hello"))
|
||||
|
||||
select {
|
||||
case ack := <-ackF.Ok():
|
||||
fmt.Println(ack.Sequence)
|
||||
case err := <-ackF.Err():
|
||||
fmt.Println(err)
|
||||
}
|
||||
|
||||
<-js.PublishAsyncComplete()
|
||||
```
|
||||
|
||||
### Publish Options
|
||||
|
||||
| Legacy | New |
|
||||
|------------------------------------------|---------------------------------------------------|
|
||||
| `nats.MsgId(id)` | `jetstream.WithMsgID(id)` |
|
||||
| `nats.ExpectStream(name)` | `jetstream.WithExpectStream(name)` |
|
||||
| `nats.ExpectLastSequence(seq)` | `jetstream.WithExpectLastSequence(seq)` |
|
||||
| `nats.ExpectLastSequencePerSubject(seq)` | `jetstream.WithExpectLastSequencePerSubject(seq)` |
|
||||
| `nats.ExpectLastMsgId(id)` | `jetstream.WithExpectLastMsgID(id)` |
|
||||
| `nats.RetryWait(dur)` | `jetstream.WithRetryWait(dur)` |
|
||||
| `nats.RetryAttempts(n)` | `jetstream.WithRetryAttempts(n)` |
|
||||
| `nats.StallWait(dur)` | `jetstream.WithStallWait(dur)` |
|
||||
|
||||
## Consuming Messages
|
||||
|
||||
This is the most significant area of change. The legacy API offered many
|
||||
subscription flavors (`Subscribe`, `SubscribeSync`, `QueueSubscribe`,
|
||||
`ChanSubscribe`, `PullSubscribe`) that blurred the line between consumer
|
||||
creation, stream lookup and message consumption. The new API separates these
|
||||
concerns: first create a consumer, then choose how to receive messages.
|
||||
|
||||
With the exception of PullSubscribe, all legacy subscription flavors utilized push consumers under the hood. The new API recommends pull consumers for all use cases, as they provide better flow control and no risk of slow consumer issues. Pull-based consumption is available via `Consume()` and `Messages()`, which maintain persistent pull subscriptions with pre-buffering for efficient continuous delivery. Push consumers are still supported for users who prefer that model, but pull consumers are the recommended default.
|
||||
|
||||
### Replacing `js.Subscribe()`
|
||||
|
||||
The legacy `js.Subscribe()` created a push consumer behind the scenes (unless
|
||||
explicitly specified otherwise via `nats.Bind()` or `nats.Durable()`) and
|
||||
delivered messages either via a callback. In the new API, the recommended
|
||||
replacement is a **pull consumer** with `Consume()` or `Messages()`. These
|
||||
provide the same continuous delivery with better flow control.
|
||||
|
||||
#### Legacy: callback subscription
|
||||
|
||||
```go
|
||||
sub, _ := js.Subscribe("ORDERS.*", func(msg *nats.Msg) {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data))
|
||||
msg.Ack()
|
||||
}, nats.Durable("processor"), nats.ManualAck)
|
||||
defer sub.Unsubscribe()
|
||||
```
|
||||
|
||||
#### New: callback with `Consume()`
|
||||
|
||||
`Consume()` is the closest equivalent to `js.Subscribe()` — it delivers messages
|
||||
to a callback function continuously.
|
||||
|
||||
```go
|
||||
s, _ := js.Stream(ctx, "ORDERS")
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
FilterSubject: "ORDERS.*",
|
||||
})
|
||||
|
||||
cc, _ := cons.Consume(func(msg jetstream.Msg) {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data()))
|
||||
msg.Ack()
|
||||
})
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
> Note: `ManualAck()` is not needed — messages are never auto-acknowledged in
|
||||
> the new API.
|
||||
|
||||
#### New: iterator with `Messages()`
|
||||
|
||||
`Messages()` provides an iterator-based approach, useful when you want explicit
|
||||
control over when the next message is fetched.
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
FilterSubject: "ORDERS.*",
|
||||
})
|
||||
|
||||
iter, _ := cons.Messages()
|
||||
for {
|
||||
msg, err := iter.Next()
|
||||
if err != nil {
|
||||
// handle error
|
||||
}
|
||||
fmt.Printf("Received: %s\n", string(msg.Data()))
|
||||
msg.Ack()
|
||||
}
|
||||
// Call iter.Stop() when done
|
||||
```
|
||||
|
||||
Both `Consume()` and `Messages()` maintain overlapping pull requests to the
|
||||
server, providing efficient continuous delivery without gaps.
|
||||
|
||||
#### Legacy: synchronous subscription
|
||||
|
||||
```go
|
||||
sub, _ := js.SubscribeSync("ORDERS.*", nats.Durable("processor"))
|
||||
msg, _ := sub.NextMsg(time.Second)
|
||||
```
|
||||
|
||||
**New:** Use `Messages()` and call `Next()`:
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
FilterSubject: "ORDERS.*",
|
||||
})
|
||||
|
||||
iter, _ := cons.Messages()
|
||||
msg, _ := iter.Next()
|
||||
```
|
||||
|
||||
#### Legacy: queue subscription
|
||||
|
||||
```go
|
||||
// Multiple instances share work via a queue group
|
||||
sub, _ := js.QueueSubscribe("ORDERS.*", "workers", handler,
|
||||
nats.Durable("processor"))
|
||||
```
|
||||
|
||||
**New with pull consumers:** With pull consumers, there is no need for an
|
||||
explicit queue group. Multiple application instances (or goroutines) calling
|
||||
`Consume()` or `Messages()` on the same durable consumer will naturally
|
||||
distribute messages among themselves — the server tracks pending acknowledgements
|
||||
and avoids delivering the same message to multiple consumers:
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
})
|
||||
|
||||
cc, _ := cons.Consume(handler)
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
**New with push consumers:** If you need push-based queue semantics, set
|
||||
`DeliverGroup` on a push consumer — this is the direct equivalent of the legacy
|
||||
queue group:
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdatePushConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
DeliverSubject: "deliver.orders",
|
||||
DeliverGroup: "workers",
|
||||
})
|
||||
|
||||
cc, _ := cons.Consume(handler)
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
> **Note:** Push consumers with `DeliverGroup` cannot be flow controlled. If you
|
||||
> experience slow consumer issues, consider using pull-based consumers instead —
|
||||
> multiple instances on the same durable consumer achieve the same work
|
||||
> distribution without the slow consumer risk.
|
||||
|
||||
#### Legacy: channel subscription
|
||||
|
||||
```go
|
||||
ch := make(chan *nats.Msg, 64)
|
||||
sub, _ := js.ChanSubscribe("ORDERS.*", ch, nats.Durable("processor"))
|
||||
|
||||
for msg := range ch {
|
||||
msg.Ack()
|
||||
}
|
||||
```
|
||||
|
||||
**New:** There is no direct channel-based equivalent. Use `Consume()` or
|
||||
`Messages()` instead.
|
||||
|
||||
### Replacing `js.PullSubscribe()`
|
||||
|
||||
The legacy pull subscription required creating a subscription and then calling
|
||||
`Fetch()` in a loop.
|
||||
|
||||
#### Legacy: pull subscribe + fetch loop
|
||||
|
||||
```go
|
||||
sub, _ := js.PullSubscribe("ORDERS.*", "processor")
|
||||
|
||||
for {
|
||||
msgs, _ := sub.Fetch(10, nats.MaxWait(5*time.Second))
|
||||
for _, msg := range msgs {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data))
|
||||
msg.Ack()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**New with `Fetch()`/`FetchNoWait()` (one-off batch):**
|
||||
|
||||
If you specifically need one-off batch fetching, `Fetch()` is available directly
|
||||
on the consumer — no separate subscription step:
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
FilterSubject: "ORDERS.*",
|
||||
})
|
||||
|
||||
// non-blocking, returns a `FetchResult` that provides messages and error
|
||||
msgs, _ := cons.Fetch(10, jetstream.FetchMaxWait(5*time.Second))
|
||||
for msg := range msgs.Messages() {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data()))
|
||||
msg.Ack()
|
||||
}
|
||||
if msgs.Error() != nil {
|
||||
// handle error
|
||||
}
|
||||
```
|
||||
|
||||
> **Warning:** `Fetch()`, `FetchNoWait()`, and `FetchBytes()` are one-off,
|
||||
> single pull requests. They do not perform pre-buffering optimizations. For
|
||||
> continuous message processing, always prefer `Consume()` or `Messages()`.
|
||||
> When using `FetchBytes()`, the requested byte size must stay under the
|
||||
> client's max pending bytes limit (64MB by default), otherwise it will trigger
|
||||
> slow consumer errors on the underlying subscription.
|
||||
|
||||
### Ordered Consumers
|
||||
|
||||
Ordered consumers provide strictly ordered, gap-free message delivery. The library
|
||||
automatically recreates the underlying consumer on sequence gaps or heartbeat
|
||||
failures.
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
sub, _ := js.Subscribe("ORDERS.*", handler, nats.OrderedConsumer())
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
cons, _ := js.OrderedConsumer(ctx, "ORDERS", jetstream.OrderedConsumerConfig{
|
||||
FilterSubjects: []string{"ORDERS.*"},
|
||||
})
|
||||
|
||||
// Use the same consumption methods as regular consumers
|
||||
cc, _ := cons.Consume(func(msg jetstream.Msg) {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data()))
|
||||
})
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
### Push Consumers
|
||||
|
||||
Pull consumers are recommended for most use cases, but push consumers are also
|
||||
supported. Push consumers require `DeliverSubject` in their config and only
|
||||
support `Consume()` (not `Fetch()` or `Messages()`).
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
sub, _ := js.Subscribe("ORDERS.*", handler,
|
||||
nats.Durable("processor"),
|
||||
nats.DeliverSubject("deliver.orders"),
|
||||
nats.IdleHeartbeat(30*time.Second),
|
||||
)
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
cons, _ := s.CreateOrUpdatePushConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Durable: "processor",
|
||||
FilterSubject: "ORDERS.*",
|
||||
DeliverSubject: "deliver.orders",
|
||||
IdleHeartbeat: 30 * time.Second,
|
||||
})
|
||||
|
||||
cc, _ := cons.Consume(func(msg jetstream.Msg) {
|
||||
fmt.Printf("Received: %s\n", string(msg.Data()))
|
||||
msg.Ack()
|
||||
})
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
### Subscription Options Mapping
|
||||
|
||||
Most legacy `SubOpt` options map directly to `ConsumerConfig` fields. Since
|
||||
consumer creation is explicit, these are set at creation time rather than passed
|
||||
as subscription options.
|
||||
|
||||
| Legacy SubOpt | New ConsumerConfig field |
|
||||
|-------------------------------------|---------------------------------------------------------------------------|
|
||||
| `nats.Durable("name")` | `Durable: "name"` |
|
||||
| `nats.ConsumerName("name")` | `Name: "name"` |
|
||||
| `nats.Description("desc")` | `Description: "desc"` |
|
||||
| `nats.DeliverAll()` | `DeliverPolicy: jetstream.DeliverAllPolicy` |
|
||||
| `nats.DeliverLast()` | `DeliverPolicy: jetstream.DeliverLastPolicy` |
|
||||
| `nats.DeliverLastPerSubject()` | `DeliverPolicy: jetstream.DeliverLastPerSubjectPolicy` |
|
||||
| `nats.DeliverNew()` | `DeliverPolicy: jetstream.DeliverNewPolicy` |
|
||||
| `nats.StartSequence(seq)` | `DeliverPolicy: jetstream.DeliverByStartSequencePolicy, OptStartSeq: seq` |
|
||||
| `nats.StartTime(t)` | `DeliverPolicy: jetstream.DeliverByStartTimePolicy, OptStartTime: &t` |
|
||||
| `nats.AckExplicit()` | `AckPolicy: jetstream.AckExplicitPolicy` |
|
||||
| `nats.AckAll()` | `AckPolicy: jetstream.AckAllPolicy` |
|
||||
| `nats.AckNone()` | `AckPolicy: jetstream.AckNonePolicy` |
|
||||
| `nats.ManualAck()` | Not needed (messages are never auto-acked) |
|
||||
| `nats.MaxDeliver(n)` | `MaxDeliver: n` |
|
||||
| `nats.MaxAckPending(n)` | `MaxAckPending: n` |
|
||||
| `nats.BackOff(durations)` | `BackOff: durations` |
|
||||
| `nats.ReplayOriginal()` | `ReplayPolicy: jetstream.ReplayOriginalPolicy` |
|
||||
| `nats.ReplayInstant()` | `ReplayPolicy: jetstream.ReplayInstantPolicy` |
|
||||
| `nats.RateLimit(bps)` | `RateLimit: bps` |
|
||||
| `nats.HeadersOnly()` | `HeadersOnly: true` |
|
||||
| `nats.InactiveThreshold(dur)` | `InactiveThreshold: dur` |
|
||||
| `nats.ConsumerFilterSubjects(s...)` | `FilterSubjects: s` |
|
||||
| `nats.ConsumerReplicas(n)` | `Replicas: n` |
|
||||
| `nats.ConsumerMemoryStorage()` | `MemoryStorage: true` |
|
||||
|
||||
The following options have no direct equivalent — use the consumer handle
|
||||
directly instead:
|
||||
|
||||
| Legacy SubOpt | New equivalent |
|
||||
|-------------------------------|---------------------------------------------------------------------|
|
||||
| `nats.Bind(stream, consumer)` | `js.Consumer(ctx, stream, consumer)` or `s.Consumer(ctx, consumer)` |
|
||||
| `nats.BindStream(stream)` | Use `js.Stream(ctx, stream)` to get a stream handle |
|
||||
| `nats.OrderedConsumer()` | `js.OrderedConsumer(ctx, stream, cfg)` |
|
||||
|
||||
### Consume/Messages Options
|
||||
|
||||
`Consume()` and `Messages()` accept options that control pull request behavior:
|
||||
|
||||
| Option | Description |
|
||||
|----------------------------|--------------------------------------------------------------|
|
||||
| `PullMaxMessages(n)` | Max messages buffered (default: 500) |
|
||||
| `PullMaxBytes(n)` | Max bytes buffered (mutually exclusive with PullMaxMessages) |
|
||||
| `PullExpiry(dur)` | Pull request timeout (default: 30s) |
|
||||
| `PullHeartbeat(dur)` | Idle heartbeat interval |
|
||||
| `PullThresholdMessages(n)` | Refill threshold (default: 50% of max) |
|
||||
| `PullThresholdBytes(n)` | Byte-based refill threshold |
|
||||
| `StopAfter(n)` | Auto-stop after N messages |
|
||||
| `ConsumeErrHandler(fn)` | Custom error handler |
|
||||
|
||||
### Error Handling in Consume/Messages
|
||||
|
||||
Both `Consume()` and `Messages()` handle server-sent status messages internally.
|
||||
Some errors are terminal (stop consumption), while others are recoverable
|
||||
(consumption continues).
|
||||
|
||||
**Terminal errors** — consumption stops automatically:
|
||||
|
||||
- `ErrConsumerDeleted` — the consumer was deleted on the server
|
||||
- `ErrBadRequest` — invalid request (e.g. misconfigured consumer)
|
||||
- Connection closed — for `Consume()` this surfaces as `ErrConnectionClosed`;
|
||||
for `Messages()`, `Next()` returns `ErrMsgIteratorClosed`
|
||||
|
||||
**Recoverable errors** — reported via error handler, consumption continues:
|
||||
|
||||
- `ErrNoHeartbeat` — missed idle heartbeats from server; a new pull request
|
||||
is issued automatically
|
||||
- `ErrConsumerLeadershipChanged` — consumer moved to a different server in the
|
||||
cluster; pending counts are reset
|
||||
- `nats.ErrNoResponders` — no JetStream service available (temporary)
|
||||
|
||||
#### Error handling with `Consume()`
|
||||
|
||||
Use `ConsumeErrHandler` to be notified about both terminal and recoverable errors:
|
||||
|
||||
```go
|
||||
cc, _ := cons.Consume(func(msg jetstream.Msg) {
|
||||
msg.Ack()
|
||||
}, jetstream.ConsumeErrHandler(func(cc jetstream.ConsumeContext, err error) {
|
||||
if errors.Is(err, jetstream.ErrConsumerDeleted) ||
|
||||
errors.Is(err, jetstream.ErrBadRequest) {
|
||||
log.Fatalf("terminal consumer error: %v", err)
|
||||
}
|
||||
log.Printf("recoverable consumer error: %v", err)
|
||||
}))
|
||||
defer cc.Stop()
|
||||
```
|
||||
|
||||
#### Error handling with `Messages()`
|
||||
|
||||
With `Messages()`, terminal errors are returned directly by `Next()`. By default,
|
||||
`ErrNoHeartbeat` is also returned by `Next()` (controlled by
|
||||
`WithMessagesErrOnMissingHeartbeat`), but it is not terminal — you can continue
|
||||
calling `Next()`:
|
||||
|
||||
```go
|
||||
iter, _ := cons.Messages()
|
||||
for {
|
||||
msg, err := iter.Next()
|
||||
if err != nil {
|
||||
if errors.Is(err, jetstream.ErrMsgIteratorClosed) {
|
||||
// iterator was stopped (either explicitly or due to connection close)
|
||||
break
|
||||
}
|
||||
if errors.Is(err, jetstream.ErrNoHeartbeat) {
|
||||
// recoverable — new pull request is issued, keep going
|
||||
log.Println("missed heartbeat, re-pulling")
|
||||
continue
|
||||
}
|
||||
// ErrConsumerDeleted, ErrBadRequest are terminal
|
||||
log.Fatalf("terminal error: %v", err)
|
||||
}
|
||||
msg.Ack()
|
||||
}
|
||||
```
|
||||
|
||||
## Message Acknowledgement
|
||||
|
||||
Ack methods are similar, with minor naming changes. The main difference is that
|
||||
message fields are accessed via methods instead of struct fields.
|
||||
|
||||
| Legacy | New |
|
||||
|-------------------------|------------------------------|
|
||||
| `msg.Ack()` | Unchanged |
|
||||
| `msg.AckSync()` | `msg.DoubleAck(ctx)` |
|
||||
| `msg.Nak()` | Unchanged |
|
||||
| `msg.NakWithDelay(dur)` | Unchanged |
|
||||
| `msg.InProgress()` | Unchanged |
|
||||
| `msg.Term()` | Unchanged |
|
||||
| N/A | `msg.TermWithReason(reason)` |
|
||||
| `msg.Metadata()` | Unchanged |
|
||||
|
||||
### Accessing Message Data
|
||||
|
||||
**Legacy:** Direct struct fields on `*nats.Msg`:
|
||||
|
||||
```go
|
||||
fmt.Println(string(msg.Data))
|
||||
fmt.Println(msg.Subject)
|
||||
fmt.Println(msg.Header.Get("key"))
|
||||
```
|
||||
|
||||
**New:** Methods on `jetstream.Msg` interface:
|
||||
|
||||
```go
|
||||
fmt.Println(string(msg.Data()))
|
||||
fmt.Println(msg.Subject())
|
||||
fmt.Println(msg.Headers().Get("key"))
|
||||
```
|
||||
|
||||
## KeyValue Store
|
||||
|
||||
The KV API is nearly identical. The main changes are:
|
||||
|
||||
1. All methods take `context.Context` as the first parameter
|
||||
2. New `CreateOrUpdateKeyValue()` and `UpdateKeyValue()` methods
|
||||
3. Types live in the `jetstream` package
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
js, _ := nc.JetStream()
|
||||
kv, _ := js.CreateKeyValue(&nats.KeyValueConfig{
|
||||
Bucket: "profiles",
|
||||
})
|
||||
|
||||
kv.Put("sue.color", []byte("blue"))
|
||||
entry, _ := kv.Get("sue.color")
|
||||
fmt.Println(string(entry.Value()))
|
||||
|
||||
watcher, _ := kv.Watch("sue.*")
|
||||
defer watcher.Stop()
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
js, _ := jetstream.New(nc)
|
||||
kv, _ := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{
|
||||
Bucket: "profiles",
|
||||
})
|
||||
|
||||
kv.Put(ctx, "sue.color", []byte("blue"))
|
||||
entry, _ := kv.Get(ctx, "sue.color")
|
||||
fmt.Println(string(entry.Value()))
|
||||
|
||||
watcher, _ := kv.Watch(ctx, "sue.*")
|
||||
defer watcher.Stop()
|
||||
```
|
||||
|
||||
### KV Management Methods
|
||||
|
||||
| Legacy | New |
|
||||
|-----------------------------|---------------------------------------|
|
||||
| `js.KeyValue(bucket)` | `js.KeyValue(ctx, bucket)` |
|
||||
| `js.CreateKeyValue(cfg)` | `js.CreateKeyValue(ctx, cfg)` |
|
||||
| N/A | `js.UpdateKeyValue(ctx, cfg)` |
|
||||
| N/A | `js.CreateOrUpdateKeyValue(ctx, cfg)` |
|
||||
| `js.DeleteKeyValue(bucket)` | `js.DeleteKeyValue(ctx, bucket)` |
|
||||
| `js.KeyValueStoreNames()` | `js.KeyValueStoreNames(ctx)` |
|
||||
| `js.KeyValueStores()` | `js.KeyValueStores(ctx)` |
|
||||
|
||||
## Object Store
|
||||
|
||||
Same pattern as KV — all methods gain `context.Context`, types move to `jetstream`
|
||||
package.
|
||||
|
||||
**Legacy:**
|
||||
|
||||
```go
|
||||
js, _ := nc.JetStream()
|
||||
os, _ := js.CreateObjectStore(&nats.ObjectStoreConfig{
|
||||
Bucket: "configs",
|
||||
})
|
||||
|
||||
os.PutString("config-1", "data")
|
||||
result, _ := os.Get("config-1")
|
||||
data, _ := io.ReadAll(result)
|
||||
```
|
||||
|
||||
**New:**
|
||||
|
||||
```go
|
||||
js, _ := jetstream.New(nc)
|
||||
os, _ := js.CreateObjectStore(ctx, jetstream.ObjectStoreConfig{
|
||||
Bucket: "configs",
|
||||
})
|
||||
|
||||
os.PutString(ctx, "config-1", "data")
|
||||
result, _ := os.Get(ctx, "config-1")
|
||||
data, _ := io.ReadAll(result)
|
||||
```
|
||||
|
||||
### Object Store Management Methods
|
||||
|
||||
| Legacy | New |
|
||||
|--------------------------------|------------------------------------------|
|
||||
| `js.ObjectStore(bucket)` | `js.ObjectStore(ctx, bucket)` |
|
||||
| `js.CreateObjectStore(cfg)` | `js.CreateObjectStore(ctx, cfg)` |
|
||||
| N/A | `js.UpdateObjectStore(ctx, cfg)` |
|
||||
| N/A | `js.CreateOrUpdateObjectStore(ctx, cfg)` |
|
||||
| `js.DeleteObjectStore(bucket)` | `js.DeleteObjectStore(ctx, bucket)` |
|
||||
+1092
File diff suppressed because it is too large
Load Diff
+158
@@ -0,0 +1,158 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type (
|
||||
apiResponse struct {
|
||||
Type string `json:"type"`
|
||||
Error *APIError `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// apiPaged includes variables used to create paged responses from the JSON API
|
||||
apiPaged struct {
|
||||
Total int `json:"total"`
|
||||
Offset int `json:"offset"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
)
|
||||
|
||||
// Request API subjects for JetStream.
|
||||
const (
|
||||
// DefaultAPIPrefix is the default prefix for the JetStream API.
|
||||
DefaultAPIPrefix = "$JS.API."
|
||||
|
||||
// jsDomainT is used to create JetStream API prefix by specifying only Domain
|
||||
jsDomainT = "$JS.%s.API."
|
||||
|
||||
// jsExtDomainT is used to create a StreamSource External APIPrefix
|
||||
jsExtDomainT = "$JS.%s.API"
|
||||
|
||||
// apiAccountInfo is for obtaining general information about JetStream.
|
||||
apiAccountInfo = "INFO"
|
||||
|
||||
// apiConsumerCreateT is used to create consumers.
|
||||
apiConsumerCreateT = "CONSUMER.CREATE.%s.%s"
|
||||
|
||||
// apiConsumerCreateWithFilterSubjectT is used to create consumers with a filter subject.
|
||||
// it accepts stream name, consumer name and filter subject
|
||||
apiConsumerCreateWithFilterSubjectT = "CONSUMER.CREATE.%s.%s.%s"
|
||||
|
||||
// apiConsumerInfoT is used to retrieve consumer information.
|
||||
apiConsumerInfoT = "CONSUMER.INFO.%s.%s"
|
||||
|
||||
// apiRequestNextT is the prefix for the request next message(s) for a consumer in worker/pull mode.
|
||||
apiRequestNextT = "CONSUMER.MSG.NEXT.%s.%s"
|
||||
|
||||
// apiConsumerDeleteT is used to delete consumers.
|
||||
apiConsumerDeleteT = "CONSUMER.DELETE.%s.%s"
|
||||
|
||||
// apiConsumerPauseT is used to pause a consumer.
|
||||
apiConsumerPauseT = "CONSUMER.PAUSE.%s.%s"
|
||||
|
||||
// apiConsumerListT is used to return all detailed consumer information
|
||||
apiConsumerListT = "CONSUMER.LIST.%s"
|
||||
|
||||
// apiConsumerNamesT is used to return a list with all consumer names for the stream.
|
||||
apiConsumerNamesT = "CONSUMER.NAMES.%s"
|
||||
|
||||
// apiStreams can lookup a stream by subject.
|
||||
apiStreams = "STREAM.NAMES"
|
||||
|
||||
// apiStreamCreateT is the endpoint to create new streams.
|
||||
apiStreamCreateT = "STREAM.CREATE.%s"
|
||||
|
||||
// apiStreamInfoT is the endpoint to get information on a stream.
|
||||
apiStreamInfoT = "STREAM.INFO.%s"
|
||||
|
||||
// apiStreamUpdateT is the endpoint to update existing streams.
|
||||
apiStreamUpdateT = "STREAM.UPDATE.%s"
|
||||
|
||||
// apiStreamDeleteT is the endpoint to delete streams.
|
||||
apiStreamDeleteT = "STREAM.DELETE.%s"
|
||||
|
||||
// apiStreamPurgeT is the endpoint to purge streams.
|
||||
apiStreamPurgeT = "STREAM.PURGE.%s"
|
||||
|
||||
// apiStreamListT is the endpoint that will return all detailed stream information
|
||||
apiStreamListT = "STREAM.LIST"
|
||||
|
||||
// apiMsgGetT is the endpoint to get a message.
|
||||
apiMsgGetT = "STREAM.MSG.GET.%s"
|
||||
|
||||
// apiDirectMsgGetT is the endpoint to perform a direct get of a message.
|
||||
apiDirectMsgGetT = "DIRECT.GET.%s"
|
||||
|
||||
// apiDirectMsgGetLastBySubjectT is the endpoint to perform a direct get of a message by subject.
|
||||
apiDirectMsgGetLastBySubjectT = "DIRECT.GET.%s.%s"
|
||||
|
||||
// apiMsgDeleteT is the endpoint to remove a message.
|
||||
apiMsgDeleteT = "STREAM.MSG.DELETE.%s"
|
||||
|
||||
// apiConsumerUnpinT is the endpoint to unpin a consumer.
|
||||
apiConsumerUnpinT = "CONSUMER.UNPIN.%s.%s"
|
||||
)
|
||||
|
||||
func (js *jetStream) apiRequestJSON(ctx context.Context, subject string, resp any, data ...[]byte) (*jetStreamMsg, error) {
|
||||
jsMsg, err := js.apiRequest(ctx, subject, data...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(jsMsg.Data(), resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return jsMsg, nil
|
||||
}
|
||||
|
||||
// a RequestWithContext with tracing via TraceCB
|
||||
func (js *jetStream) apiRequest(ctx context.Context, subj string, data ...[]byte) (*jetStreamMsg, error) {
|
||||
subj = js.apiSubject(subj)
|
||||
var req []byte
|
||||
if len(data) > 0 {
|
||||
req = data[0]
|
||||
}
|
||||
if js.opts.ClientTrace != nil {
|
||||
ctrace := js.opts.ClientTrace
|
||||
if ctrace.RequestSent != nil {
|
||||
ctrace.RequestSent(subj, req)
|
||||
}
|
||||
}
|
||||
resp, err := js.conn.RequestWithContext(ctx, subj, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if js.opts.ClientTrace != nil {
|
||||
ctrace := js.opts.ClientTrace
|
||||
if ctrace.ResponseReceived != nil {
|
||||
ctrace.ResponseReceived(subj, resp.Data, resp.Header)
|
||||
}
|
||||
}
|
||||
|
||||
return js.toJSMsg(resp), nil
|
||||
}
|
||||
|
||||
func (js *jetStream) apiSubject(subj string) string {
|
||||
if js.opts.apiPrefix == "" {
|
||||
return subj
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString(js.opts.apiPrefix)
|
||||
b.WriteString(subj)
|
||||
return b.String()
|
||||
}
|
||||
+541
@@ -0,0 +1,541 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/internal/syncx"
|
||||
"github.com/nats-io/nuid"
|
||||
)
|
||||
|
||||
type (
|
||||
|
||||
// Consumer contains methods for fetching/processing messages from a stream,
|
||||
// as well as fetching consumer info.
|
||||
//
|
||||
// This package provides two implementations of Consumer interface:
|
||||
//
|
||||
// - Standard named/ephemeral pull consumers. These consumers are created using
|
||||
// CreateConsumer method on Stream or JetStream interface. They can be
|
||||
// explicitly configured (using [ConsumerConfig]) and managed by the user,
|
||||
// either from this package or externally.
|
||||
//
|
||||
// - Ordered consumers. These consumers are created using OrderedConsumer
|
||||
// method on Stream or JetStream interface. They are managed by the library
|
||||
// and provide a simple way to consume messages from a stream. Ordered
|
||||
// consumers are ephemeral in-memory pull consumers and are resilient to
|
||||
// deletes and restarts. They provide limited configuration options
|
||||
// using [OrderedConsumerConfig].
|
||||
//
|
||||
// Consumer provides methods for optimized continuous consumption of messages
|
||||
// using Consume and Messages methods, as well as simple one-off messages
|
||||
// retrieval using Fetch and Next methods.
|
||||
Consumer interface {
|
||||
// Fetch is used to retrieve up to a provided number of messages from a
|
||||
// stream. This method will send a single request and deliver either all
|
||||
// requested messages unless the timeout is met earlier. Fetch timeout
|
||||
// defaults to 30 seconds and can be configured using FetchMaxWait
|
||||
// option.
|
||||
//
|
||||
// By default, Fetch uses a 5s idle heartbeat for requests longer than
|
||||
// 10 seconds. For shorter requests, the idle heartbeat is disabled.
|
||||
// This can be configured using FetchHeartbeat option. If a client does
|
||||
// not receive a heartbeat message from a stream for more than 2 times
|
||||
// the idle heartbeat setting, Fetch will return [ErrNoHeartbeat].
|
||||
//
|
||||
// Fetch is non-blocking and returns MessageBatch, exposing a channel
|
||||
// for delivered messages.
|
||||
//
|
||||
// Messages channel is always closed, thus it is safe to range over it
|
||||
// without additional checks. After the channel is closed,
|
||||
// MessageBatch.Error() should be checked to see if there was an error
|
||||
// during message delivery (e.g. missing heartbeat).
|
||||
//
|
||||
// NOTE: Fetch has worse performance when used to continuously retrieve
|
||||
// messages in comparison to Messages or Consume methods, as it does not
|
||||
// perform any optimizations (e.g. overlapping pull requests) and new
|
||||
// subscription is created for each execution.
|
||||
Fetch(batch int, opts ...FetchOpt) (MessageBatch, error)
|
||||
|
||||
// FetchBytes is used to retrieve up to a provided number of bytes from
|
||||
// the stream. This method will send a single request and deliver the
|
||||
// provided number of bytes unless the timeout is met earlier. FetchBytes
|
||||
// timeout defaults to 30 seconds and can be configured using
|
||||
// FetchMaxWait option.
|
||||
//
|
||||
// By default, FetchBytes uses a 5s idle heartbeat for requests longer than
|
||||
// 10 seconds. For shorter requests, the idle heartbeat is disabled.
|
||||
// This can be configured using FetchHeartbeat option. If a client does
|
||||
// not receive a heartbeat message from a stream for more than 2 times
|
||||
// the idle heartbeat setting, FetchBytes will return [ErrNoHeartbeat].
|
||||
//
|
||||
// FetchBytes is non-blocking and returns MessageBatch, exposing a channel
|
||||
// for delivered messages.
|
||||
//
|
||||
// Messages channel is always closed, thus it is safe to range over it
|
||||
// without additional checks. After the channel is closed,
|
||||
// MessageBatch.Error() should be checked to see if there was an error
|
||||
// during message delivery (e.g. missing heartbeat).
|
||||
//
|
||||
// NOTE: FetchBytes has worse performance when used to continuously
|
||||
// retrieve messages in comparison to Messages or Consume methods, as it
|
||||
// does not perform any optimizations (e.g. overlapping pull requests)
|
||||
// and new subscription is created for each execution.
|
||||
FetchBytes(maxBytes int, opts ...FetchOpt) (MessageBatch, error)
|
||||
|
||||
// FetchNoWait is used to retrieve up to a provided number of messages
|
||||
// from a stream. Unlike Fetch, FetchNoWait will only deliver messages
|
||||
// that are currently available in the stream and will not wait for new
|
||||
// messages to arrive, even if batch size is not met.
|
||||
//
|
||||
// FetchNoWait is non-blocking and returns MessageBatch, exposing a
|
||||
// channel for delivered messages.
|
||||
//
|
||||
// Messages channel is always closed, thus it is safe to range over it
|
||||
// without additional checks. After the channel is closed,
|
||||
// MessageBatch.Error() should be checked to see if there was an error
|
||||
// during message delivery (e.g. missing heartbeat).
|
||||
//
|
||||
// NOTE: FetchNoWait has worse performance when used to continuously
|
||||
// retrieve messages in comparison to Messages or Consume methods, as it
|
||||
// does not perform any optimizations (e.g. overlapping pull requests)
|
||||
// and new subscription is created for each execution.
|
||||
FetchNoWait(batch int) (MessageBatch, error)
|
||||
|
||||
// Consume will continuously receive messages and handle them
|
||||
// with the provided callback function. Consume can be configured using
|
||||
// PullConsumeOpt options:
|
||||
//
|
||||
// - Error handling and monitoring can be configured using ConsumeErrHandler
|
||||
// option, which provides information about errors encountered during
|
||||
// consumption (both transient and terminal)
|
||||
// - Consume can be configured to stop after a certain number of
|
||||
// messages have been received using StopAfter option.
|
||||
// - Consume can be optimized for throughput or memory usage using
|
||||
// PullExpiry, PullMaxMessages, PullMaxBytes and PullHeartbeat options.
|
||||
// Unless there is a specific use case, these options should not be used.
|
||||
//
|
||||
// Consume returns a ConsumeContext, which can be used to stop or drain
|
||||
// the consumer.
|
||||
Consume(handler MessageHandler, opts ...PullConsumeOpt) (ConsumeContext, error)
|
||||
|
||||
// Messages returns MessagesContext, allowing continuous iteration
|
||||
// over messages in a stream. Messages can be configured using
|
||||
// PullMessagesOpt options:
|
||||
//
|
||||
// - Messages can be optimized for throughput or memory usage using
|
||||
// PullExpiry, PullMaxMessages, PullMaxBytes and PullHeartbeat options.
|
||||
// Unless there is a specific use case, these options should not be used.
|
||||
// - WithMessagesErrOnMissingHeartbeat can be used to enable/disable
|
||||
// erroring out on MessagesContext.Next when a heartbeat is missing.
|
||||
// This option is enabled by default.
|
||||
Messages(opts ...PullMessagesOpt) (MessagesContext, error)
|
||||
|
||||
// Next is used to retrieve the next message from the consumer. This
|
||||
// method will block until the message is retrieved or the timeout
|
||||
// is reached.
|
||||
Next(opts ...FetchOpt) (Msg, error)
|
||||
|
||||
// Info fetches current ConsumerInfo from the server.
|
||||
Info(context.Context) (*ConsumerInfo, error)
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this consumer.
|
||||
// This method does not perform any network requests. The cached
|
||||
// ConsumerInfo is updated on every call to Info and Update.
|
||||
CachedInfo() *ConsumerInfo
|
||||
}
|
||||
|
||||
PushConsumer interface {
|
||||
// Consume will continuously receive messages and handle them
|
||||
// with the provided callback function. Consume can be configured using
|
||||
// PushConsumeOpt options:
|
||||
//
|
||||
// - Error handling and monitoring can be configured using ConsumeErrHandler.
|
||||
Consume(handler MessageHandler, opts ...PushConsumeOpt) (ConsumeContext, error)
|
||||
|
||||
// Info fetches current ConsumerInfo from the server.
|
||||
Info(context.Context) (*ConsumerInfo, error)
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this consumer.
|
||||
CachedInfo() *ConsumerInfo
|
||||
}
|
||||
|
||||
createConsumerRequest struct {
|
||||
Stream string `json:"stream_name"`
|
||||
Config *ConsumerConfig `json:"config"`
|
||||
Action string `json:"action"`
|
||||
}
|
||||
)
|
||||
|
||||
// Info fetches current ConsumerInfo from the server.
|
||||
func (p *pullConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
|
||||
ctx, cancel := p.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
infoSubject := fmt.Sprintf(apiConsumerInfoT, p.stream, p.name)
|
||||
var resp consumerInfoResponse
|
||||
|
||||
if _, err := p.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
if resp.Error == nil && resp.ConsumerInfo == nil {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
|
||||
p.info = resp.ConsumerInfo
|
||||
return resp.ConsumerInfo, nil
|
||||
}
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this consumer.
|
||||
// This method does not perform any network requests. The cached
|
||||
// ConsumerInfo is updated on every call to Info and Update.
|
||||
func (p *pullConsumer) CachedInfo() *ConsumerInfo {
|
||||
return p.info
|
||||
}
|
||||
|
||||
// Info fetches current ConsumerInfo from the server.
|
||||
func (p *pushConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
|
||||
ctx, cancel := p.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
infoSubject := fmt.Sprintf(apiConsumerInfoT, p.stream, p.name)
|
||||
var resp consumerInfoResponse
|
||||
|
||||
if _, err := p.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
if resp.Error == nil && resp.ConsumerInfo == nil {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
|
||||
p.info = resp.ConsumerInfo
|
||||
return resp.ConsumerInfo, nil
|
||||
}
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this consumer.
|
||||
// This method does not perform any network requests. The cached
|
||||
// ConsumerInfo is updated on every call to Info and Update.
|
||||
func (p *pushConsumer) CachedInfo() *ConsumerInfo {
|
||||
return p.info
|
||||
}
|
||||
|
||||
func upsertPullConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (Consumer, error) {
|
||||
resp, err := upsertConsumer(ctx, js, stream, cfg, action)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &pullConsumer{
|
||||
js: js,
|
||||
stream: stream,
|
||||
name: resp.Name,
|
||||
durable: cfg.Durable != "",
|
||||
info: resp.ConsumerInfo,
|
||||
subs: syncx.Map[string, *pullSubscription]{},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func upsertPushConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (PushConsumer, error) {
|
||||
if cfg.DeliverSubject == "" {
|
||||
return nil, ErrNotPushConsumer
|
||||
}
|
||||
|
||||
resp, err := upsertConsumer(ctx, js, stream, cfg, action)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &pushConsumer{
|
||||
js: js,
|
||||
stream: stream,
|
||||
name: resp.Name,
|
||||
info: resp.ConsumerInfo,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func upsertConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (*consumerInfoResponse, error) {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
req := createConsumerRequest{
|
||||
Stream: stream,
|
||||
Config: &cfg,
|
||||
Action: action,
|
||||
}
|
||||
reqJSON, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
consumerName := cfg.Name
|
||||
if consumerName == "" {
|
||||
if cfg.Durable != "" {
|
||||
consumerName = cfg.Durable
|
||||
} else {
|
||||
consumerName = generateConsName()
|
||||
}
|
||||
}
|
||||
if err := validateConsumerName(consumerName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var ccSubj string
|
||||
if cfg.FilterSubject != "" && len(cfg.FilterSubjects) == 0 {
|
||||
if err := validateSubject(cfg.FilterSubject); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ccSubj = fmt.Sprintf(apiConsumerCreateWithFilterSubjectT, stream, consumerName, cfg.FilterSubject)
|
||||
} else {
|
||||
ccSubj = fmt.Sprintf(apiConsumerCreateT, stream, consumerName)
|
||||
}
|
||||
var resp consumerInfoResponse
|
||||
|
||||
if _, err := js.apiRequestJSON(ctx, ccSubj, &resp, reqJSON); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeStreamNotFound {
|
||||
return nil, ErrStreamNotFound
|
||||
}
|
||||
if resp.Error.ErrorCode == JSErrCodeMaximumConsumersLimit {
|
||||
return nil, ErrMaximumConsumersLimit
|
||||
}
|
||||
|
||||
return nil, resp.Error
|
||||
}
|
||||
|
||||
if resp.Error == nil && resp.ConsumerInfo == nil {
|
||||
return nil, ErrConsumerCreationResponseEmpty
|
||||
}
|
||||
|
||||
// check whether multiple filter subjects (if used) are reflected in the returned ConsumerInfo
|
||||
if len(cfg.FilterSubjects) != 0 && len(resp.Config.FilterSubjects) == 0 {
|
||||
return nil, ErrConsumerMultipleFilterSubjectsNotSupported
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
const (
|
||||
consumerActionCreate = "create"
|
||||
consumerActionUpdate = "update"
|
||||
consumerActionCreateOrUpdate = ""
|
||||
)
|
||||
|
||||
func generateConsName() string {
|
||||
name := nuid.Next()
|
||||
sha := sha256.New()
|
||||
sha.Write([]byte(name))
|
||||
b := sha.Sum(nil)
|
||||
for i := 0; i < 8; i++ {
|
||||
b[i] = rdigits[int(b[i]%base)]
|
||||
}
|
||||
return string(b[:8])
|
||||
}
|
||||
|
||||
func getConsumer(ctx context.Context, js *jetStream, stream, name string) (Consumer, error) {
|
||||
info, err := fetchConsumerInfo(ctx, js, stream, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if info.Config.DeliverSubject != "" {
|
||||
return nil, ErrNotPullConsumer
|
||||
}
|
||||
|
||||
cons := &pullConsumer{
|
||||
js: js,
|
||||
stream: stream,
|
||||
name: name,
|
||||
durable: info.Config.Durable != "",
|
||||
info: info,
|
||||
subs: syncx.Map[string, *pullSubscription]{},
|
||||
}
|
||||
|
||||
return cons, nil
|
||||
}
|
||||
|
||||
func getPushConsumer(ctx context.Context, js *jetStream, stream, name string) (PushConsumer, error) {
|
||||
info, err := fetchConsumerInfo(ctx, js, stream, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if info.Config.DeliverSubject == "" {
|
||||
return nil, ErrNotPushConsumer
|
||||
}
|
||||
|
||||
cons := &pushConsumer{
|
||||
js: js,
|
||||
stream: stream,
|
||||
name: name,
|
||||
info: info,
|
||||
}
|
||||
|
||||
return cons, nil
|
||||
}
|
||||
|
||||
func fetchConsumerInfo(ctx context.Context, js *jetStream, stream, name string) (*ConsumerInfo, error) {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
if err := validateConsumerName(name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
infoSubject := fmt.Sprintf(apiConsumerInfoT, stream, name)
|
||||
|
||||
var resp consumerInfoResponse
|
||||
|
||||
if _, err := js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
if resp.Error == nil && resp.ConsumerInfo == nil {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
|
||||
return resp.ConsumerInfo, nil
|
||||
}
|
||||
|
||||
func deleteConsumer(ctx context.Context, js *jetStream, stream, consumer string) error {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
if err := validateConsumerName(consumer); err != nil {
|
||||
return err
|
||||
}
|
||||
deleteSubject := fmt.Sprintf(apiConsumerDeleteT, stream, consumer)
|
||||
|
||||
var resp consumerDeleteResponse
|
||||
|
||||
if _, err := js.apiRequestJSON(ctx, deleteSubject, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return ErrConsumerNotFound
|
||||
}
|
||||
return resp.Error
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func pauseConsumer(ctx context.Context, js *jetStream, stream, consumer string, pauseUntil *time.Time) (*ConsumerPauseResponse, error) {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
if err := validateConsumerName(consumer); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
subject := fmt.Sprintf(apiConsumerPauseT, stream, consumer)
|
||||
|
||||
var resp consumerPauseApiResponse
|
||||
req, err := json.Marshal(consumerPauseRequest{
|
||||
PauseUntil: pauseUntil,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := js.apiRequestJSON(ctx, subject, &resp, req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
return &ConsumerPauseResponse{
|
||||
Paused: resp.Paused,
|
||||
PauseUntil: resp.PauseUntil,
|
||||
PauseRemaining: resp.PauseRemaining,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func resumeConsumer(ctx context.Context, js *jetStream, stream, consumer string) (*ConsumerPauseResponse, error) {
|
||||
return pauseConsumer(ctx, js, stream, consumer, nil)
|
||||
}
|
||||
|
||||
func validateConsumerName(name string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("%w: name is required", ErrInvalidConsumerName)
|
||||
}
|
||||
if strings.ContainsAny(name, ">*. /\\\t\r\n") {
|
||||
return fmt.Errorf("%w: %q", ErrInvalidConsumerName, name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func unpinConsumer(ctx context.Context, js *jetStream, stream, consumer, group string) error {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
if err := validateConsumerName(consumer); err != nil {
|
||||
return err
|
||||
}
|
||||
unpinSubject := fmt.Sprintf(apiConsumerUnpinT, stream, consumer)
|
||||
|
||||
var req = consumerUnpinRequest{
|
||||
Group: group,
|
||||
}
|
||||
|
||||
reqJSON, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var resp apiResponse
|
||||
|
||||
if _, err := js.apiRequestJSON(ctx, unpinSubject, &resp, reqJSON); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return ErrConsumerNotFound
|
||||
}
|
||||
return resp.Error
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
+580
@@ -0,0 +1,580 @@
|
||||
// Copyright 2022-2024 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
type (
|
||||
// ConsumerInfo is the detailed information about a JetStream consumer.
|
||||
ConsumerInfo struct {
|
||||
// Stream specifies the name of the stream that the consumer is bound
|
||||
// to.
|
||||
Stream string `json:"stream_name"`
|
||||
|
||||
// Name represents the unique identifier for the consumer. This can be
|
||||
// either set explicitly by the client or generated automatically if not
|
||||
// set.
|
||||
Name string `json:"name"`
|
||||
|
||||
// Created is the timestamp when the consumer was created.
|
||||
Created time.Time `json:"created"`
|
||||
|
||||
// Config contains the configuration settings of the consumer, set when
|
||||
// creating or updating the consumer.
|
||||
Config ConsumerConfig `json:"config"`
|
||||
|
||||
// Delivered holds information about the most recently delivered
|
||||
// message, including its sequence numbers and timestamp.
|
||||
Delivered SequenceInfo `json:"delivered"`
|
||||
|
||||
// AckFloor indicates the message before the first unacknowledged
|
||||
// message.
|
||||
AckFloor SequenceInfo `json:"ack_floor"`
|
||||
|
||||
// NumAckPending is the number of messages that have been delivered but
|
||||
// not yet acknowledged.
|
||||
NumAckPending int `json:"num_ack_pending"`
|
||||
|
||||
// NumRedelivered counts the number of messages that have been
|
||||
// redelivered and not yet acknowledged. Each message is counted only
|
||||
// once, even if it has been redelivered multiple times. This count is
|
||||
// reset when the message is eventually acknowledged.
|
||||
NumRedelivered int `json:"num_redelivered"`
|
||||
|
||||
// NumWaiting is the count of active pull requests. It is only relevant
|
||||
// for pull-based consumers.
|
||||
NumWaiting int `json:"num_waiting"`
|
||||
|
||||
// NumPending is the number of messages that match the consumer's
|
||||
// filter, but have not been delivered yet.
|
||||
NumPending uint64 `json:"num_pending"`
|
||||
|
||||
// Cluster contains information about the cluster to which this consumer
|
||||
// belongs (if applicable).
|
||||
Cluster *ClusterInfo `json:"cluster,omitempty"`
|
||||
|
||||
// PushBound indicates whether at least one subscription exists for the
|
||||
// delivery subject of this consumer. This is only applicable to
|
||||
// push-based consumers.
|
||||
PushBound bool `json:"push_bound,omitempty"`
|
||||
|
||||
// TimeStamp indicates when the info was gathered by the server.
|
||||
TimeStamp time.Time `json:"ts"`
|
||||
|
||||
// PriorityGroups contains the information about the currently defined priority groups
|
||||
PriorityGroups []PriorityGroupState `json:"priority_groups,omitempty"`
|
||||
|
||||
// Paused indicates whether the consumer is paused.
|
||||
Paused bool `json:"paused,omitempty"`
|
||||
|
||||
// PauseRemaining contains the amount of time left until the consumer
|
||||
// unpauses. It will only be non-zero if the consumer is currently paused.
|
||||
PauseRemaining time.Duration `json:"pause_remaining,omitempty"`
|
||||
}
|
||||
|
||||
PriorityGroupState struct {
|
||||
// Group this status is for.
|
||||
Group string `json:"group"`
|
||||
|
||||
// PinnedClientID is the generated ID of the pinned client.
|
||||
PinnedClientID string `json:"pinned_client_id,omitempty"`
|
||||
|
||||
// PinnedTS is the timestamp when the client was pinned.
|
||||
PinnedTS time.Time `json:"pinned_ts,omitempty"`
|
||||
}
|
||||
|
||||
// ConsumerConfig represents the configuration of a JetStream consumer,
|
||||
// encompassing both push and pull consumer settings
|
||||
ConsumerConfig struct {
|
||||
// Name is an optional name for the consumer. If not set, one is
|
||||
// generated automatically.
|
||||
//
|
||||
// Name cannot contain whitespace, ., *, >, path separators (forward or
|
||||
// backwards slash), and non-printable characters.
|
||||
Name string `json:"name,omitempty"`
|
||||
|
||||
// Durable is an optional durable name for the consumer. If both Durable
|
||||
// and Name are set, they have to be equal. Unless InactiveThreshold is set, a
|
||||
// durable consumer will not be cleaned up automatically.
|
||||
//
|
||||
// Durable cannot contain whitespace, ., *, >, path separators (forward or
|
||||
// backwards slash), and non-printable characters.
|
||||
Durable string `json:"durable_name,omitempty"`
|
||||
|
||||
// Description provides an optional description of the consumer.
|
||||
Description string `json:"description,omitempty"`
|
||||
|
||||
// DeliverPolicy defines from which point to start delivering messages
|
||||
// from the stream. Defaults to DeliverAllPolicy.
|
||||
DeliverPolicy DeliverPolicy `json:"deliver_policy"`
|
||||
|
||||
// OptStartSeq is an optional sequence number from which to start
|
||||
// message delivery. Only applicable when DeliverPolicy is set to
|
||||
// DeliverByStartSequencePolicy.
|
||||
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
|
||||
|
||||
// OptStartTime is an optional time from which to start message
|
||||
// delivery. Only applicable when DeliverPolicy is set to
|
||||
// DeliverByStartTimePolicy.
|
||||
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
|
||||
|
||||
// AckPolicy defines the acknowledgement policy for the consumer.
|
||||
// Defaults to AckExplicitPolicy.
|
||||
AckPolicy AckPolicy `json:"ack_policy"`
|
||||
|
||||
// AckWait defines how long the server will wait for an acknowledgement
|
||||
// before resending a message. If not set, server default is 30 seconds.
|
||||
AckWait time.Duration `json:"ack_wait,omitempty"`
|
||||
|
||||
// MaxDeliver defines the maximum number of delivery attempts for a
|
||||
// message. Applies to any message that is re-sent due to ack policy.
|
||||
// If not set, server default is -1 (unlimited).
|
||||
MaxDeliver int `json:"max_deliver,omitempty"`
|
||||
|
||||
// BackOff specifies the optional back-off intervals for retrying
|
||||
// message delivery after a failed acknowledgement. It overrides
|
||||
// AckWait.
|
||||
//
|
||||
// BackOff only applies to messages not acknowledged in specified time,
|
||||
// not messages that were nack'ed.
|
||||
//
|
||||
// The number of intervals specified must be lower or equal to
|
||||
// MaxDeliver. If the number of intervals is lower, the last interval is
|
||||
// used for all remaining attempts.
|
||||
BackOff []time.Duration `json:"backoff,omitempty"`
|
||||
|
||||
// FilterSubject can be used to filter messages delivered from the
|
||||
// stream. FilterSubject is exclusive with FilterSubjects.
|
||||
FilterSubject string `json:"filter_subject,omitempty"`
|
||||
|
||||
// ReplayPolicy defines the rate at which messages are sent to the
|
||||
// consumer. If ReplayOriginalPolicy is set, messages are sent in the
|
||||
// same intervals in which they were stored on stream. This can be used
|
||||
// e.g. to simulate production traffic in development environments. If
|
||||
// ReplayInstantPolicy is set, messages are sent as fast as possible.
|
||||
// Defaults to ReplayInstantPolicy.
|
||||
ReplayPolicy ReplayPolicy `json:"replay_policy"`
|
||||
|
||||
// RateLimit specifies an optional maximum rate of message delivery in
|
||||
// bits per second.
|
||||
RateLimit uint64 `json:"rate_limit_bps,omitempty"`
|
||||
|
||||
// SampleFrequency is an optional frequency for sampling how often
|
||||
// acknowledgements are sampled for observability. See
|
||||
// https://docs.nats.io/running-a-nats-service/nats_admin/monitoring/monitoring_jetstream
|
||||
SampleFrequency string `json:"sample_freq,omitempty"`
|
||||
|
||||
// MaxWaiting is a maximum number of pull requests waiting to be
|
||||
// fulfilled. If not set, this will inherit settings from stream's
|
||||
// ConsumerLimits or (if those are not set) from account settings. If
|
||||
// neither are set, server default is 512.
|
||||
MaxWaiting int `json:"max_waiting,omitempty"`
|
||||
|
||||
// MaxAckPending is a maximum number of outstanding unacknowledged
|
||||
// messages. Once this limit is reached, the server will suspend sending
|
||||
// messages to the consumer. If not set, server default is 1000.
|
||||
// Set to -1 for unlimited.
|
||||
MaxAckPending int `json:"max_ack_pending,omitempty"`
|
||||
|
||||
// HeadersOnly indicates whether only headers of messages should be sent
|
||||
// (and no payload). Defaults to false.
|
||||
HeadersOnly bool `json:"headers_only,omitempty"`
|
||||
|
||||
// MaxRequestBatch is the optional maximum batch size a single pull
|
||||
// request can make. When set with MaxRequestMaxBytes, the batch size
|
||||
// will be constrained by whichever limit is hit first.
|
||||
MaxRequestBatch int `json:"max_batch,omitempty"`
|
||||
|
||||
// MaxRequestExpires is the maximum duration a single pull request will
|
||||
// wait for messages to be available to pull.
|
||||
MaxRequestExpires time.Duration `json:"max_expires,omitempty"`
|
||||
|
||||
// MaxRequestMaxBytes is the optional maximum total bytes that can be
|
||||
// requested in a given batch. When set with MaxRequestBatch, the batch
|
||||
// size will be constrained by whichever limit is hit first.
|
||||
MaxRequestMaxBytes int `json:"max_bytes,omitempty"`
|
||||
|
||||
// InactiveThreshold is a duration which instructs the server to clean
|
||||
// up the consumer if it has been inactive for the specified duration.
|
||||
// Durable consumers will not be cleaned up by default, but if
|
||||
// InactiveThreshold is set, they will be. If not set, this will inherit
|
||||
// settings from stream's ConsumerLimits. If neither are set, server
|
||||
// default is 5 seconds.
|
||||
//
|
||||
// A consumer is considered inactive if no pull requests are received by
|
||||
// the server (for pull consumers), or no interest is detected on the
|
||||
// deliver subject (for push consumers), not if there are no
|
||||
// messages to be delivered.
|
||||
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
|
||||
|
||||
// Replicas is the number of replicas for the consumer's state. By
|
||||
// default, consumers inherit the number of replicas from the stream.
|
||||
Replicas int `json:"num_replicas"`
|
||||
|
||||
// MemoryStorage is a flag to force the consumer to use memory storage
|
||||
// rather than inherit the storage type from the stream.
|
||||
MemoryStorage bool `json:"mem_storage,omitempty"`
|
||||
|
||||
// FilterSubjects allows filtering messages from a stream by subject.
|
||||
// This field is exclusive with FilterSubject. Requires nats-server
|
||||
// v2.10.0 or later.
|
||||
FilterSubjects []string `json:"filter_subjects,omitempty"`
|
||||
|
||||
// Metadata is a set of application-defined key-value pairs for
|
||||
// associating metadata on the consumer. This feature requires
|
||||
// nats-server v2.10.0 or later.
|
||||
Metadata map[string]string `json:"metadata,omitempty"`
|
||||
|
||||
// PauseUntil is for suspending the consumer until the deadline.
|
||||
PauseUntil *time.Time `json:"pause_until,omitempty"`
|
||||
|
||||
// PriorityPolicy represents the priority policy the consumer is set to.
|
||||
// Requires nats-server v2.11.0 or later.
|
||||
PriorityPolicy PriorityPolicy `json:"priority_policy,omitempty"`
|
||||
|
||||
// PinnedTTL represents the time after which the client will be unpinned
|
||||
// if no new pull requests are sent. Used with PriorityPolicyPinned.
|
||||
// Requires nats-server v2.11.0 or later.
|
||||
PinnedTTL time.Duration `json:"priority_timeout,omitempty"`
|
||||
|
||||
// PriorityGroups is a list of priority groups this consumer supports.
|
||||
PriorityGroups []string `json:"priority_groups,omitempty"`
|
||||
|
||||
// Fields specific to push consumers:
|
||||
|
||||
// DeliverSubject is the subject to deliver messages to for push consumers
|
||||
DeliverSubject string `json:"deliver_subject,omitempty"`
|
||||
|
||||
// DeliverGroup is the group name for push consumers
|
||||
DeliverGroup string `json:"deliver_group,omitempty"`
|
||||
|
||||
// FlowControl is a flag to enable flow control for the consumer.
|
||||
// When set, the server will regularly send an empty message with status
|
||||
// header 100 and a reply subject. Consumers must reply to these
|
||||
// messages to control the rate of message delivery.
|
||||
FlowControl bool `json:"flow_control,omitempty"`
|
||||
|
||||
// IdleHeartbeat enables push consumer idle heartbeat messages.
|
||||
// If the Consumer is idle for more than the set value, an empty message
|
||||
// with Status header 100 will be sent indicating the consumer is still
|
||||
// alive.
|
||||
IdleHeartbeat time.Duration `json:"idle_heartbeat,omitempty"`
|
||||
}
|
||||
|
||||
// OrderedConsumerConfig is the configuration of an ordered JetStream
|
||||
// consumer. For more information, see [Ordered Consumers] in README
|
||||
//
|
||||
// [Ordered Consumers]: https://github.com/nats-io/nats.go/blob/main/jetstream/README.md#ordered-consumers
|
||||
OrderedConsumerConfig struct {
|
||||
// FilterSubjects allows filtering messages from a stream by subject.
|
||||
// This field is exclusive with FilterSubject. Requires nats-server
|
||||
// v2.10.0 or later.
|
||||
FilterSubjects []string `json:"filter_subjects,omitempty"`
|
||||
|
||||
// DeliverPolicy defines from which point to start delivering messages
|
||||
// from the stream. Defaults to DeliverAllPolicy.
|
||||
DeliverPolicy DeliverPolicy `json:"deliver_policy"`
|
||||
|
||||
// OptStartSeq is an optional sequence number from which to start
|
||||
// message delivery. Only applicable when DeliverPolicy is set to
|
||||
// DeliverByStartSequencePolicy.
|
||||
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
|
||||
|
||||
// OptStartTime is an optional time from which to start message
|
||||
// delivery. Only applicable when DeliverPolicy is set to
|
||||
// DeliverByStartTimePolicy.
|
||||
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
|
||||
|
||||
// ReplayPolicy defines the rate at which messages are sent to the
|
||||
// consumer. If ReplayOriginalPolicy is set, messages are sent in the
|
||||
// same intervals in which they were stored on the stream. This can be
|
||||
// used e.g. to simulate production traffic in development environments.
|
||||
// If ReplayInstantPolicy is set, messages are sent as fast as possible.
|
||||
// Defaults to ReplayInstantPolicy.
|
||||
ReplayPolicy ReplayPolicy `json:"replay_policy"`
|
||||
|
||||
// InactiveThreshold is a duration which instructs the server to clean
|
||||
// up the consumer if it has been inactive for the specified duration.
|
||||
// Defaults to 5m.
|
||||
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
|
||||
|
||||
// HeadersOnly indicates whether only headers of messages should be sent
|
||||
// (and no payload). Defaults to false.
|
||||
HeadersOnly bool `json:"headers_only,omitempty"`
|
||||
|
||||
// MaxResetAttempts is the maximum number of attempts to recreate the
|
||||
// consumer in a single recovery cycle. Defaults to unlimited.
|
||||
MaxResetAttempts int
|
||||
|
||||
// Metadata is a set of application-defined key-value pairs for
|
||||
// associating metadata with the consumer. This feature requires
|
||||
// nats-server v2.10.0 or later.
|
||||
Metadata map[string]string `json:"metadata,omitempty"`
|
||||
|
||||
// NamePrefix is an optional custom prefix for the consumer name.
|
||||
// If provided, ordered consumer names will be generated as:
|
||||
// {NamePrefix}_{sequence_number} (e.g., "custom_1", "custom_2").
|
||||
// If not provided, a unique ID (NUID) will be used as the prefix.
|
||||
NamePrefix string `json:"-"`
|
||||
}
|
||||
|
||||
// DeliverPolicy determines from which point to start delivering messages.
|
||||
DeliverPolicy int
|
||||
|
||||
// AckPolicy determines how the consumer should acknowledge delivered
|
||||
// messages.
|
||||
AckPolicy int
|
||||
|
||||
// ReplayPolicy determines how the consumer should replay messages it
|
||||
// already has queued in the stream.
|
||||
ReplayPolicy int
|
||||
|
||||
// SequenceInfo has both the consumer and the stream sequence and last
|
||||
// activity.
|
||||
SequenceInfo struct {
|
||||
Consumer uint64 `json:"consumer_seq"`
|
||||
Stream uint64 `json:"stream_seq"`
|
||||
Last *time.Time `json:"last_active,omitempty"`
|
||||
}
|
||||
|
||||
// PriorityPolicy determines the priority policy the consumer is set to.
|
||||
PriorityPolicy int
|
||||
)
|
||||
|
||||
const (
|
||||
// PriorityPolicyNone is the default priority policy.
|
||||
PriorityPolicyNone PriorityPolicy = iota
|
||||
|
||||
// PriorityPolicyPinned is the priority policy that pins a consumer to a
|
||||
// specific client.
|
||||
PriorityPolicyPinned
|
||||
|
||||
// PriorityPolicyOverflow is the priority policy that allows for
|
||||
// restricting when a consumer will receive messages based on the number of
|
||||
// pending messages or acks.
|
||||
PriorityPolicyOverflow
|
||||
|
||||
// PriorityPolicyPrioritized is the priority policy that allows for the
|
||||
// server to deliver messages to clients based on their priority (instead
|
||||
// of round-robin). Requires nats-server v2.12.0 or later.
|
||||
PriorityPolicyPrioritized
|
||||
)
|
||||
|
||||
func (p *PriorityPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString(""):
|
||||
*p = PriorityPolicyNone
|
||||
case jsonString("pinned_client"):
|
||||
*p = PriorityPolicyPinned
|
||||
case jsonString("overflow"):
|
||||
*p = PriorityPolicyOverflow
|
||||
case jsonString("prioritized"):
|
||||
*p = PriorityPolicyPrioritized
|
||||
default:
|
||||
return fmt.Errorf("nats: cannot unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p PriorityPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch p {
|
||||
case PriorityPolicyNone:
|
||||
return json.Marshal("")
|
||||
case PriorityPolicyPinned:
|
||||
return json.Marshal("pinned_client")
|
||||
case PriorityPolicyOverflow:
|
||||
return json.Marshal("overflow")
|
||||
case PriorityPolicyPrioritized:
|
||||
return json.Marshal("prioritized")
|
||||
}
|
||||
return nil, fmt.Errorf("nats: unknown priority policy %v", p)
|
||||
}
|
||||
|
||||
const (
|
||||
// DeliverAllPolicy starts delivering messages from the very beginning of a
|
||||
// stream. This is the default.
|
||||
DeliverAllPolicy DeliverPolicy = iota
|
||||
|
||||
// DeliverLastPolicy will start the consumer with the last sequence
|
||||
// received.
|
||||
DeliverLastPolicy
|
||||
|
||||
// DeliverNewPolicy will only deliver new messages that are sent after the
|
||||
// consumer is created.
|
||||
DeliverNewPolicy
|
||||
|
||||
// DeliverByStartSequencePolicy will deliver messages starting from a given
|
||||
// sequence configured with OptStartSeq in ConsumerConfig.
|
||||
DeliverByStartSequencePolicy
|
||||
|
||||
// DeliverByStartTimePolicy will deliver messages starting from a given time
|
||||
// configured with OptStartTime in ConsumerConfig.
|
||||
DeliverByStartTimePolicy
|
||||
|
||||
// DeliverLastPerSubjectPolicy will start the consumer with the last message
|
||||
// for all subjects received.
|
||||
DeliverLastPerSubjectPolicy
|
||||
)
|
||||
|
||||
func (p *DeliverPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString("all"), jsonString("undefined"):
|
||||
*p = DeliverAllPolicy
|
||||
case jsonString("last"):
|
||||
*p = DeliverLastPolicy
|
||||
case jsonString("new"):
|
||||
*p = DeliverNewPolicy
|
||||
case jsonString("by_start_sequence"):
|
||||
*p = DeliverByStartSequencePolicy
|
||||
case jsonString("by_start_time"):
|
||||
*p = DeliverByStartTimePolicy
|
||||
case jsonString("last_per_subject"):
|
||||
*p = DeliverLastPerSubjectPolicy
|
||||
default:
|
||||
return fmt.Errorf("nats: cannot unmarshal %q", data)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p DeliverPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch p {
|
||||
case DeliverAllPolicy:
|
||||
return json.Marshal("all")
|
||||
case DeliverLastPolicy:
|
||||
return json.Marshal("last")
|
||||
case DeliverNewPolicy:
|
||||
return json.Marshal("new")
|
||||
case DeliverByStartSequencePolicy:
|
||||
return json.Marshal("by_start_sequence")
|
||||
case DeliverByStartTimePolicy:
|
||||
return json.Marshal("by_start_time")
|
||||
case DeliverLastPerSubjectPolicy:
|
||||
return json.Marshal("last_per_subject")
|
||||
}
|
||||
return nil, fmt.Errorf("nats: unknown deliver policy %v", p)
|
||||
}
|
||||
|
||||
func (p DeliverPolicy) String() string {
|
||||
switch p {
|
||||
case DeliverAllPolicy:
|
||||
return "all"
|
||||
case DeliverLastPolicy:
|
||||
return "last"
|
||||
case DeliverNewPolicy:
|
||||
return "new"
|
||||
case DeliverByStartSequencePolicy:
|
||||
return "by_start_sequence"
|
||||
case DeliverByStartTimePolicy:
|
||||
return "by_start_time"
|
||||
case DeliverLastPerSubjectPolicy:
|
||||
return "last_per_subject"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
const (
|
||||
// AckExplicitPolicy requires ack or nack for all messages.
|
||||
AckExplicitPolicy AckPolicy = iota
|
||||
|
||||
// AckAllPolicy when acking a sequence number, this implicitly acks all
|
||||
// sequences below this one as well.
|
||||
AckAllPolicy
|
||||
|
||||
// AckNonePolicy requires no acks for delivered messages.
|
||||
AckNonePolicy
|
||||
)
|
||||
|
||||
func (p *AckPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString("none"):
|
||||
*p = AckNonePolicy
|
||||
case jsonString("all"):
|
||||
*p = AckAllPolicy
|
||||
case jsonString("explicit"):
|
||||
*p = AckExplicitPolicy
|
||||
default:
|
||||
return fmt.Errorf("nats: cannot unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p AckPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch p {
|
||||
case AckNonePolicy:
|
||||
return json.Marshal("none")
|
||||
case AckAllPolicy:
|
||||
return json.Marshal("all")
|
||||
case AckExplicitPolicy:
|
||||
return json.Marshal("explicit")
|
||||
}
|
||||
return nil, fmt.Errorf("nats: unknown acknowledgement policy %v", p)
|
||||
}
|
||||
|
||||
func (p AckPolicy) String() string {
|
||||
switch p {
|
||||
case AckNonePolicy:
|
||||
return "AckNone"
|
||||
case AckAllPolicy:
|
||||
return "AckAll"
|
||||
case AckExplicitPolicy:
|
||||
return "AckExplicit"
|
||||
}
|
||||
return "Unknown AckPolicy"
|
||||
}
|
||||
|
||||
const (
|
||||
// ReplayInstantPolicy will replay messages as fast as possible.
|
||||
ReplayInstantPolicy ReplayPolicy = iota
|
||||
|
||||
// ReplayOriginalPolicy will maintain the same timing as the messages were
|
||||
// received.
|
||||
ReplayOriginalPolicy
|
||||
)
|
||||
|
||||
func (p *ReplayPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString("instant"):
|
||||
*p = ReplayInstantPolicy
|
||||
case jsonString("original"):
|
||||
*p = ReplayOriginalPolicy
|
||||
default:
|
||||
return fmt.Errorf("nats: cannot unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p ReplayPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch p {
|
||||
case ReplayOriginalPolicy:
|
||||
return json.Marshal("original")
|
||||
case ReplayInstantPolicy:
|
||||
return json.Marshal("instant")
|
||||
}
|
||||
return nil, fmt.Errorf("nats: unknown replay policy %v", p)
|
||||
}
|
||||
|
||||
func (p ReplayPolicy) String() string {
|
||||
switch p {
|
||||
case ReplayOriginalPolicy:
|
||||
return "original"
|
||||
case ReplayInstantPolicy:
|
||||
return "instant"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
+474
@@ -0,0 +1,474 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
type (
|
||||
// JetStreamError is an error result that happens when using JetStream.
|
||||
// In case of client-side error, [APIError] returns nil.
|
||||
JetStreamError interface {
|
||||
APIError() *APIError
|
||||
error
|
||||
}
|
||||
|
||||
jsError struct {
|
||||
apiErr *APIError
|
||||
message string
|
||||
}
|
||||
|
||||
// APIError is included in all API responses if there was an error.
|
||||
APIError struct {
|
||||
Code int `json:"code"`
|
||||
ErrorCode ErrorCode `json:"err_code"`
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ErrorCode represents error_code returned in response from JetStream API.
|
||||
ErrorCode uint16
|
||||
)
|
||||
|
||||
const (
|
||||
JSErrCodeBadRequest ErrorCode = 10003
|
||||
JSErrCodeConsumerCreate ErrorCode = 10012
|
||||
JSErrCodeConsumerNameExists ErrorCode = 10013
|
||||
JSErrCodeConsumerNotFound ErrorCode = 10014
|
||||
JSErrCodeMaximumConsumersLimit ErrorCode = 10026
|
||||
|
||||
JSErrCodeMessageNotFound ErrorCode = 10037
|
||||
JSErrCodeJetStreamNotEnabledForAccount ErrorCode = 10039
|
||||
|
||||
JSErrCodeStreamNameInUse ErrorCode = 10058
|
||||
JSErrCodeStreamNotFound ErrorCode = 10059
|
||||
|
||||
JSErrCodeStreamWrongLastSequence ErrorCode = 10071
|
||||
JSErrCodeJetStreamNotEnabled ErrorCode = 10076
|
||||
|
||||
JSErrCodeConsumerAlreadyExists ErrorCode = 10105
|
||||
|
||||
JSErrCodeDuplicateFilterSubjects ErrorCode = 10136
|
||||
JSErrCodeOverlappingFilterSubjects ErrorCode = 10138
|
||||
JSErrCodeConsumerEmptyFilter ErrorCode = 10139
|
||||
JSErrCodeConsumerExists ErrorCode = 10148
|
||||
JSErrCodeConsumerDoesNotExist ErrorCode = 10149
|
||||
)
|
||||
|
||||
var (
|
||||
// JetStream API errors
|
||||
|
||||
// ErrJetStreamNotEnabled is an error returned when JetStream is not
|
||||
// enabled.
|
||||
//
|
||||
// Note: This error will not be returned in clustered mode, even if each
|
||||
// server in the cluster does not have JetStream enabled. In clustered mode,
|
||||
// requests will time out instead.
|
||||
ErrJetStreamNotEnabled JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabled, Description: "jetstream not enabled", Code: 503}}
|
||||
|
||||
// ErrJetStreamNotEnabledForAccount is an error returned when JetStream is
|
||||
// not enabled for an account.
|
||||
ErrJetStreamNotEnabledForAccount JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabledForAccount, Description: "jetstream not enabled for account", Code: 503}}
|
||||
|
||||
// ErrStreamNotFound is an error returned when stream with given name does
|
||||
// not exist.
|
||||
ErrStreamNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNotFound, Description: "stream not found", Code: 404}}
|
||||
|
||||
// ErrStreamNameAlreadyInUse is returned when a stream with given name
|
||||
// already exists and has a different configuration.
|
||||
ErrStreamNameAlreadyInUse JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNameInUse, Description: "stream name already in use", Code: 400}}
|
||||
|
||||
// ErrStreamSubjectTransformNotSupported is returned when the connected
|
||||
// nats-server version does not support setting the stream subject
|
||||
// transform. If this error is returned when executing CreateStream(), the
|
||||
// stream with invalid configuration was already created in the server.
|
||||
ErrStreamSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
|
||||
|
||||
// ErrStreamSourceSubjectTransformNotSupported is returned when the
|
||||
// connected nats-server version does not support setting the stream source
|
||||
// subject transform. If this error is returned when executing
|
||||
// CreateStream(), the stream with invalid configuration was already created
|
||||
// in the server.
|
||||
ErrStreamSourceSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
|
||||
|
||||
// ErrStreamSourceNotSupported is returned when the connected nats-server
|
||||
// version does not support setting the stream sources. If this error is
|
||||
// returned when executing CreateStream(), the stream with invalid
|
||||
// configuration was already created in the server.
|
||||
ErrStreamSourceNotSupported JetStreamError = &jsError{message: "stream sourcing is not supported by nats-server"}
|
||||
|
||||
// ErrStreamSourceMultipleFilterSubjectsNotSupported is returned when the
|
||||
// connected nats-server version does not support setting the stream
|
||||
// sources. If this error is returned when executing CreateStream(), the
|
||||
// stream with invalid configuration was already created in the server.
|
||||
ErrStreamSourceMultipleFilterSubjectsNotSupported JetStreamError = &jsError{message: "stream sourcing with multiple subject filters not supported by nats-server"}
|
||||
|
||||
// ErrConsumerNotFound is an error returned when consumer with given name
|
||||
// does not exist.
|
||||
ErrConsumerNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerNotFound, Description: "consumer not found", Code: 404}}
|
||||
|
||||
// ErrConsumerCreationResponseEmpty is an error returned when the response from the server
|
||||
// when creating a consumer is empty. This means that the state of the consumer is unknown and
|
||||
// the consumer may not have been created successfully.
|
||||
ErrConsumerCreationResponseEmpty JetStreamError = &jsError{message: "consumer creation response is empty"}
|
||||
|
||||
// ErrConsumerExists is returned when attempting to create a consumer with
|
||||
// CreateConsumer but a consumer with given name already exists.
|
||||
ErrConsumerExists JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerExists, Description: "consumer already exists", Code: 400}}
|
||||
|
||||
// ErrConsumerNameExists is returned when attempting to update a consumer
|
||||
// with UpdateConsumer but a consumer with given name does not exist.
|
||||
ErrConsumerDoesNotExist JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerDoesNotExist, Description: "consumer does not exist", Code: 400}}
|
||||
|
||||
// ErrMsgNotFound is returned when message with provided sequence number
|
||||
// does not exist.
|
||||
ErrMsgNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMessageNotFound, Description: "message not found", Code: 404}}
|
||||
|
||||
// ErrBadRequest is returned when invalid request is sent to JetStream API.
|
||||
ErrBadRequest JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeBadRequest, Description: "bad request", Code: 400}}
|
||||
|
||||
// ErrConsumerCreate is returned when nats-server reports error when
|
||||
// creating consumer (e.g. illegal update).
|
||||
ErrConsumerCreate JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerCreate, Description: "could not create consumer", Code: 500}}
|
||||
|
||||
// ErrMaximumConsumersLimit is returned when user limit of allowed
|
||||
// consumers for stream is reached
|
||||
ErrMaximumConsumersLimit JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMaximumConsumersLimit, Description: "maximum consumers limit reached", Code: 400}}
|
||||
|
||||
// ErrDuplicateFilterSubjects is returned when both FilterSubject and
|
||||
// FilterSubjects are specified when creating consumer.
|
||||
ErrDuplicateFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeDuplicateFilterSubjects, Description: "consumer cannot have both FilterSubject and FilterSubjects specified", Code: 500}}
|
||||
|
||||
// ErrOverlappingFilterSubjects is returned when filter subjects overlap when
|
||||
// creating consumer.
|
||||
ErrOverlappingFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeOverlappingFilterSubjects, Description: "consumer subject filters cannot overlap", Code: 500}}
|
||||
|
||||
// ErrEmptyFilter is returned when a filter in FilterSubjects is empty.
|
||||
ErrEmptyFilter JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerEmptyFilter, Description: "consumer filter in FilterSubjects cannot be empty", Code: 500}}
|
||||
|
||||
// Client errors
|
||||
|
||||
// ErrConsumerMultipleFilterSubjectsNotSupported is returned when the
|
||||
// connected nats-server version does not support setting multiple filter
|
||||
// subjects with filter_subjects field. If this error is returned when
|
||||
// executing AddConsumer(), the consumer with invalid configuration was
|
||||
// already created in the server.
|
||||
ErrConsumerMultipleFilterSubjectsNotSupported JetStreamError = &jsError{message: "multiple consumer filter subjects not supported by nats-server"}
|
||||
|
||||
// ErrConsumerNameAlreadyInUse is an error returned when attempting to create
|
||||
// a consumer with a name that is already in use.
|
||||
ErrConsumerNameAlreadyInUse JetStreamError = &jsError{message: "consumer name already in use"}
|
||||
|
||||
// ErrNotPullConsumer is returned when attempting to fetch or create pull
|
||||
// consumer and the returned consumer is a push consumer.
|
||||
ErrNotPullConsumer JetStreamError = &jsError{message: "consumer is not a pull consumer"}
|
||||
|
||||
// ErrNotPushConsumer is returned when attempting to fetch or create push
|
||||
// consumer and the returned consumer is a pull consumer.
|
||||
ErrNotPushConsumer JetStreamError = &jsError{message: "consumer is not a push consumer"}
|
||||
|
||||
// ErrConsumerAlreadyConsuming is returned when attempting to consume from
|
||||
// the same push consumer more than once.
|
||||
ErrConsumerAlreadyConsuming JetStreamError = &jsError{message: "consumer is already consuming"}
|
||||
|
||||
// ErrInvalidJSAck is returned when JetStream ack from message publish is
|
||||
// invalid.
|
||||
ErrInvalidJSAck JetStreamError = &jsError{message: "invalid jetstream publish response"}
|
||||
|
||||
// ErrStreamNameRequired is returned when the provided stream name is empty.
|
||||
ErrStreamNameRequired JetStreamError = &jsError{message: "stream name is required"}
|
||||
|
||||
// ErrMsgAlreadyAckd is returned when attempting to acknowledge message more
|
||||
// than once.
|
||||
ErrMsgAlreadyAckd JetStreamError = &jsError{message: "message was already acknowledged"}
|
||||
|
||||
// ErrNoStreamResponse is returned when there is no response from stream
|
||||
// (e.g. no responders error).
|
||||
ErrNoStreamResponse JetStreamError = &jsError{message: "no response from stream"}
|
||||
|
||||
// ErrNotJSMessage is returned when attempting to get metadata from non
|
||||
// JetStream message.
|
||||
ErrNotJSMessage JetStreamError = &jsError{message: "not a jetstream message"}
|
||||
|
||||
// ErrInvalidStreamName is returned when the provided stream name is invalid
|
||||
// (contains '.').
|
||||
ErrInvalidStreamName JetStreamError = &jsError{message: "invalid stream name"}
|
||||
|
||||
// ErrInvalidSubject is returned when the provided subject name is invalid.
|
||||
ErrInvalidSubject JetStreamError = &jsError{message: "invalid subject name"}
|
||||
|
||||
// ErrInvalidConsumerName is returned when the provided consumer name is
|
||||
// invalid (contains '.').
|
||||
ErrInvalidConsumerName JetStreamError = &jsError{message: "invalid consumer name"}
|
||||
|
||||
// ErrNoMessages is returned when no messages are currently available for a
|
||||
// consumer.
|
||||
ErrNoMessages JetStreamError = &jsError{message: "no messages"}
|
||||
|
||||
// ErrPinIDMismatch is returned when Pin ID sent in the request does not match
|
||||
// the currently pinned consumer subscriber ID on the server.
|
||||
ErrPinIDMismatch JetStreamError = &jsError{message: "pin ID mismatch"}
|
||||
|
||||
// ErrMaxBytesExceeded is returned when a message would exceed MaxBytes set
|
||||
// on a pull request.
|
||||
ErrMaxBytesExceeded JetStreamError = &jsError{message: "message size exceeds max bytes"}
|
||||
|
||||
// ErrBatchCompleted is returned when a fetch request sent the whole batch,
|
||||
// but there are still bytes left. This is applicable only when MaxBytes is
|
||||
// set on a pull request.
|
||||
ErrBatchCompleted JetStreamError = &jsError{message: "batch completed"}
|
||||
|
||||
// ErrConsumerDeleted is returned when attempting to send pull request to a
|
||||
// consumer which does not exist.
|
||||
ErrConsumerDeleted JetStreamError = &jsError{message: "consumer deleted"}
|
||||
|
||||
// ErrConsumerLeadershipChanged is returned when pending requests are no
|
||||
// longer valid after leadership has changed.
|
||||
ErrConsumerLeadershipChanged JetStreamError = &jsError{message: "leadership change"}
|
||||
|
||||
// ErrHandlerRequired is returned when no handler func is provided in
|
||||
// Stream().
|
||||
ErrHandlerRequired JetStreamError = &jsError{message: "handler cannot be empty"}
|
||||
|
||||
// ErrEndOfData is returned when iterating over paged API from JetStream
|
||||
// reaches end of data.
|
||||
ErrEndOfData JetStreamError = &jsError{message: "end of data reached"}
|
||||
|
||||
// ErrNoHeartbeat is received when no message is received in IdleHeartbeat
|
||||
// time (if set).
|
||||
ErrNoHeartbeat JetStreamError = &jsError{message: "no heartbeat received"}
|
||||
|
||||
// ErrConsumerHasActiveSubscription is returned when a consumer is already
|
||||
// subscribed to a stream.
|
||||
ErrConsumerHasActiveSubscription JetStreamError = &jsError{message: "consumer has active subscription"}
|
||||
|
||||
// ErrMsgNotBound is returned when given message is not bound to any
|
||||
// subscription.
|
||||
ErrMsgNotBound JetStreamError = &jsError{message: "message is not bound to subscription/connection"}
|
||||
|
||||
// ErrMsgNoReply is returned when attempting to reply to a message without a
|
||||
// reply subject.
|
||||
ErrMsgNoReply JetStreamError = &jsError{message: "message does not have a reply"}
|
||||
|
||||
// ErrMsgDeleteUnsuccessful is returned when an attempt to delete a message
|
||||
// is unsuccessful.
|
||||
ErrMsgDeleteUnsuccessful JetStreamError = &jsError{message: "message deletion unsuccessful"}
|
||||
|
||||
// ErrAsyncPublishReplySubjectSet is returned when reply subject is set on
|
||||
// async message publish.
|
||||
ErrAsyncPublishReplySubjectSet JetStreamError = &jsError{message: "reply subject should be empty"}
|
||||
|
||||
// ErrTooManyStalledMsgs is returned when too many outstanding async
|
||||
// messages are waiting for ack.
|
||||
ErrTooManyStalledMsgs JetStreamError = &jsError{message: "stalled with too many outstanding async published messages"}
|
||||
|
||||
// ErrInvalidOption is returned when there is a collision between options.
|
||||
ErrInvalidOption JetStreamError = &jsError{message: "invalid jetstream option"}
|
||||
|
||||
// ErrMsgIteratorClosed is returned when attempting to get message from a
|
||||
// closed iterator.
|
||||
ErrMsgIteratorClosed JetStreamError = &jsError{message: "messages iterator closed"}
|
||||
|
||||
// ErrConnectionClosed is returned when JetStream operations fail due to
|
||||
// underlying connection being closed.
|
||||
ErrConnectionClosed JetStreamError = &jsError{message: "connection closed"}
|
||||
|
||||
// ErrServerShutdown is returned when pull request fails due to server
|
||||
// shutdown.
|
||||
ErrServerShutdown JetStreamError = &jsError{message: "server shutdown"}
|
||||
|
||||
// ErrOrderedConsumerReset indicates that the ordered consumer was
|
||||
// automatically reset and recreated to preserve message ordering.
|
||||
ErrOrderedConsumerReset JetStreamError = &jsError{message: "recreating ordered consumer"}
|
||||
|
||||
// ErrOrderConsumerUsedAsFetch is returned when ordered consumer was already
|
||||
// used to process messages using Fetch (or FetchBytes).
|
||||
ErrOrderConsumerUsedAsFetch JetStreamError = &jsError{message: "ordered consumer initialized as fetch"}
|
||||
|
||||
// ErrOrderConsumerUsedAsConsume is returned when ordered consumer was
|
||||
// already used to process messages using Consume or Messages.
|
||||
ErrOrderConsumerUsedAsConsume JetStreamError = &jsError{message: "ordered consumer initialized as consume"}
|
||||
|
||||
// ErrOrderedConsumerConcurrentRequests is returned when attempting to run
|
||||
// concurrent operations on ordered consumers.
|
||||
ErrOrderedConsumerConcurrentRequests JetStreamError = &jsError{message: "cannot run concurrent processing using ordered consumer"}
|
||||
|
||||
// ErrOrderedConsumerNotCreated is returned when trying to get consumer info
|
||||
// of an ordered consumer which was not yet created.
|
||||
ErrOrderedConsumerNotCreated JetStreamError = &jsError{message: "consumer instance not yet created"}
|
||||
|
||||
// ErrJetStreamPublisherClosed is returned for each unfinished ack future when JetStream.Cleanup is called.
|
||||
ErrJetStreamPublisherClosed JetStreamError = &jsError{message: "jetstream context closed"}
|
||||
|
||||
// ErrAsyncPublishTimeout is returned when waiting for ack on async publish
|
||||
ErrAsyncPublishTimeout JetStreamError = &jsError{message: "timeout waiting for ack"}
|
||||
|
||||
// KeyValue Errors
|
||||
|
||||
// ErrKeyExists is returned when attempting to create a key that already
|
||||
// exists.
|
||||
ErrKeyExists JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamWrongLastSequence, Code: 400}, message: "key exists"}
|
||||
|
||||
// ErrKeyValueConfigRequired is returned when attempting to create a bucket
|
||||
// without a config.
|
||||
ErrKeyValueConfigRequired JetStreamError = &jsError{message: "config required"}
|
||||
|
||||
// ErrInvalidBucketName is returned when attempting to create a bucket with
|
||||
// an invalid name.
|
||||
ErrInvalidBucketName JetStreamError = &jsError{message: "invalid bucket name"}
|
||||
|
||||
// ErrInvalidKey is returned when attempting to create a key with an invalid
|
||||
// name.
|
||||
ErrInvalidKey JetStreamError = &jsError{message: "invalid key"}
|
||||
|
||||
// ErrBucketExists is returned when attempting to create a bucket that
|
||||
// already exists and has a different configuration.
|
||||
ErrBucketExists JetStreamError = &jsError{message: "bucket name already in use"}
|
||||
|
||||
// ErrBucketNotFound is returned when attempting to access a bucket that
|
||||
// does not exist.
|
||||
ErrBucketNotFound JetStreamError = &jsError{message: "bucket not found"}
|
||||
|
||||
// ErrBadBucket is returned when attempting to access a bucket that is not a
|
||||
// key-value store.
|
||||
ErrBadBucket JetStreamError = &jsError{message: "bucket not valid key-value store"}
|
||||
|
||||
// ErrKeyNotFound is returned when attempting to access a key that does not
|
||||
// exist.
|
||||
ErrKeyNotFound JetStreamError = &jsError{message: "key not found"}
|
||||
|
||||
// ErrKeyDeleted is returned when attempting to access a key that was
|
||||
// deleted.
|
||||
ErrKeyDeleted JetStreamError = &jsError{message: "key was deleted"}
|
||||
|
||||
// ErrHistoryTooLarge is returned when provided history limit is larger than
|
||||
// 64.
|
||||
ErrHistoryTooLarge JetStreamError = &jsError{message: "history limited to a max of 64"}
|
||||
|
||||
// ErrNoKeysFound is returned when no keys are found.
|
||||
ErrNoKeysFound JetStreamError = &jsError{message: "no keys found"}
|
||||
|
||||
// ErrTTLOnDeleteNotSupported is returned when attempting to set a TTL
|
||||
// on a delete operation.
|
||||
ErrTTLOnDeleteNotSupported JetStreamError = &jsError{message: "TTL is not supported on delete"}
|
||||
|
||||
// ErrLimitMarkerTTLNotSupported is returned when the connected jetstream API
|
||||
// does not support setting the LimitMarkerTTL.
|
||||
ErrLimitMarkerTTLNotSupported JetStreamError = &jsError{message: "limit marker TTLs not supported by server"}
|
||||
|
||||
// ErrObjectConfigRequired is returned when attempting to create an object
|
||||
// without a config.
|
||||
ErrObjectConfigRequired JetStreamError = &jsError{message: "object-store config required"}
|
||||
|
||||
// ErrBadObjectMeta is returned when the meta information of an object is
|
||||
// invalid.
|
||||
ErrBadObjectMeta JetStreamError = &jsError{message: "object-store meta information invalid"}
|
||||
|
||||
// ErrObjectNotFound is returned when an object is not found.
|
||||
ErrObjectNotFound JetStreamError = &jsError{message: "object not found"}
|
||||
|
||||
// ErrInvalidStoreName is returned when the name of an object-store is
|
||||
// invalid.
|
||||
ErrInvalidStoreName JetStreamError = &jsError{message: "invalid object-store name"}
|
||||
|
||||
// ErrDigestMismatch is returned when the digests of an object do not match.
|
||||
ErrDigestMismatch JetStreamError = &jsError{message: "received a corrupt object, digests do not match"}
|
||||
|
||||
// ErrInvalidDigestFormat is returned when the digest hash of an object has
|
||||
// an invalid format.
|
||||
ErrInvalidDigestFormat JetStreamError = &jsError{message: "object digest hash has invalid format"}
|
||||
|
||||
// ErrNoObjectsFound is returned when no objects are found.
|
||||
ErrNoObjectsFound JetStreamError = &jsError{message: "no objects found"}
|
||||
|
||||
// ErrObjectAlreadyExists is returned when an object with the same name
|
||||
// already exists.
|
||||
ErrObjectAlreadyExists JetStreamError = &jsError{message: "an object already exists with that name"}
|
||||
|
||||
// ErrNameRequired is returned when a name is required.
|
||||
ErrNameRequired JetStreamError = &jsError{message: "name is required"}
|
||||
|
||||
// ErrLinkNotAllowed is returned when a link cannot be set when putting the
|
||||
// object in a bucket.
|
||||
ErrLinkNotAllowed JetStreamError = &jsError{message: "link cannot be set when putting the object in bucket"}
|
||||
|
||||
// ErrObjectRequired is returned when an object is required.
|
||||
ErrObjectRequired = &jsError{message: "object required"}
|
||||
|
||||
// ErrNoLinkToDeleted is returned when it is not allowed to link to a
|
||||
// deleted object.
|
||||
ErrNoLinkToDeleted JetStreamError = &jsError{message: "not allowed to link to a deleted object"}
|
||||
|
||||
// ErrNoLinkToLink is returned when it is not allowed to link to another
|
||||
// link.
|
||||
ErrNoLinkToLink JetStreamError = &jsError{message: "not allowed to link to another link"}
|
||||
|
||||
// ErrCantGetBucket is returned when an invalid Get is attempted on an
|
||||
// object that is a link to a bucket.
|
||||
ErrCantGetBucket JetStreamError = &jsError{message: "invalid Get, object is a link to a bucket"}
|
||||
|
||||
// ErrBucketRequired is returned when a bucket is required.
|
||||
ErrBucketRequired JetStreamError = &jsError{message: "bucket required"}
|
||||
|
||||
// ErrBucketMalformed is returned when a bucket is malformed.
|
||||
ErrBucketMalformed JetStreamError = &jsError{message: "bucket malformed"}
|
||||
|
||||
// ErrUpdateMetaDeleted is returned when the meta information of a deleted
|
||||
// object cannot be updated.
|
||||
ErrUpdateMetaDeleted JetStreamError = &jsError{message: "cannot update meta for a deleted object"}
|
||||
)
|
||||
|
||||
// Error prints the JetStream API error code and description.
|
||||
func (e *APIError) Error() string {
|
||||
return fmt.Sprintf("nats: API error: code=%d err_code=%d description=%s", e.Code, e.ErrorCode, e.Description)
|
||||
}
|
||||
|
||||
// APIError implements the JetStreamError interface.
|
||||
func (e *APIError) APIError() *APIError {
|
||||
return e
|
||||
}
|
||||
|
||||
// Is matches against an APIError.
|
||||
func (e *APIError) Is(err error) bool {
|
||||
if e == nil {
|
||||
return false
|
||||
}
|
||||
// Extract internal APIError to match against.
|
||||
var aerr *APIError
|
||||
ok := errors.As(err, &aerr)
|
||||
if !ok {
|
||||
return ok
|
||||
}
|
||||
return e.ErrorCode == aerr.ErrorCode
|
||||
}
|
||||
|
||||
func (err *jsError) APIError() *APIError {
|
||||
return err.apiErr
|
||||
}
|
||||
|
||||
func (err *jsError) Error() string {
|
||||
if err.apiErr != nil && err.apiErr.Description != "" {
|
||||
return err.apiErr.Error()
|
||||
}
|
||||
return fmt.Sprintf("nats: %s", err.message)
|
||||
}
|
||||
|
||||
func (err *jsError) Unwrap() error {
|
||||
// Allow matching to embedded APIError in case there is one.
|
||||
if err.apiErr == nil {
|
||||
return nil
|
||||
}
|
||||
return err.apiErr
|
||||
}
|
||||
+1288
File diff suppressed because it is too large
Load Diff
+732
@@ -0,0 +1,732 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
type pullOptFunc func(*consumeOpts) error
|
||||
|
||||
func (fn pullOptFunc) configureConsume(opts *consumeOpts) error {
|
||||
return fn(opts)
|
||||
}
|
||||
|
||||
func (fn pullOptFunc) configureMessages(opts *consumeOpts) error {
|
||||
return fn(opts)
|
||||
}
|
||||
|
||||
// WithClientTrace enables request/response API calls tracing.
|
||||
func WithClientTrace(ct *ClientTrace) JetStreamOpt {
|
||||
return func(opts *JetStreamOptions) error {
|
||||
opts.ClientTrace = ct
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPublishAsyncErrHandler sets error handler for async message publish.
|
||||
func WithPublishAsyncErrHandler(cb MsgErrHandler) JetStreamOpt {
|
||||
return func(opts *JetStreamOptions) error {
|
||||
opts.publisherOpts.aecb = cb
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPublishAsyncMaxPending sets the maximum outstanding async publishes that
|
||||
// can be inflight at one time.
|
||||
func WithPublishAsyncMaxPending(max int) JetStreamOpt {
|
||||
return func(opts *JetStreamOptions) error {
|
||||
if max < 1 {
|
||||
return fmt.Errorf("%w: max ack pending should be >= 1", ErrInvalidOption)
|
||||
}
|
||||
opts.publisherOpts.maxpa = max
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPublishAsyncTimeout sets the timeout for async message publish.
|
||||
// If not provided, timeout is disabled.
|
||||
func WithPublishAsyncTimeout(dur time.Duration) JetStreamOpt {
|
||||
return func(opts *JetStreamOptions) error {
|
||||
opts.publisherOpts.ackTimeout = dur
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithDefaultTimeout sets the default timeout for JetStream API requests.
|
||||
// It is used when context used for the request does not have a deadline set.
|
||||
// If not provided, a default of 5 seconds will be used.
|
||||
func WithDefaultTimeout(timeout time.Duration) JetStreamOpt {
|
||||
return func(opts *JetStreamOptions) error {
|
||||
if timeout <= 0 {
|
||||
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.DefaultTimeout = timeout
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPurgeSubject sets a specific subject for which messages on a stream will
|
||||
// be purged
|
||||
func WithPurgeSubject(subject string) StreamPurgeOpt {
|
||||
return func(req *StreamPurgeRequest) error {
|
||||
req.Subject = subject
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPurgeSequence is used to set a specific sequence number up to which (but
|
||||
// not including) messages will be purged from a stream Can be combined with
|
||||
// [WithPurgeSubject] option, but not with [WithPurgeKeep]
|
||||
func WithPurgeSequence(sequence uint64) StreamPurgeOpt {
|
||||
return func(req *StreamPurgeRequest) error {
|
||||
if req.Keep != 0 {
|
||||
return fmt.Errorf("%w: both 'keep' and 'sequence' cannot be provided in purge request", ErrInvalidOption)
|
||||
}
|
||||
req.Sequence = sequence
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPurgeKeep sets the number of messages to be kept in the stream after
|
||||
// purge. Can be combined with [WithPurgeSubject] option, but not with
|
||||
// [WithPurgeSequence]
|
||||
func WithPurgeKeep(keep uint64) StreamPurgeOpt {
|
||||
return func(req *StreamPurgeRequest) error {
|
||||
if req.Sequence != 0 {
|
||||
return fmt.Errorf("%w: both 'keep' and 'sequence' cannot be provided in purge request", ErrInvalidOption)
|
||||
}
|
||||
req.Keep = keep
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithGetMsgSubject sets the stream subject from which the message should be
|
||||
// retrieved. Server will return a first message with a seq >= to the input seq
|
||||
// that has the specified subject.
|
||||
func WithGetMsgSubject(subject string) GetMsgOpt {
|
||||
return func(req *apiMsgGetRequest) error {
|
||||
req.NextFor = subject
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// PullMaxMessages limits the number of messages to be buffered in the client.
|
||||
// If not provided, a default of 500 messages will be used.
|
||||
// This option is exclusive with PullMaxBytes.
|
||||
//
|
||||
// PullMaxMessages implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullMaxMessages int
|
||||
|
||||
func (max PullMaxMessages) configureConsume(opts *consumeOpts) error {
|
||||
if max <= 0 {
|
||||
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxMessages = int(max)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (max PullMaxMessages) configureMessages(opts *consumeOpts) error {
|
||||
if max <= 0 {
|
||||
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxMessages = int(max)
|
||||
return nil
|
||||
}
|
||||
|
||||
type pullMaxMessagesWithBytesLimit struct {
|
||||
maxMessages int
|
||||
maxBytes int
|
||||
}
|
||||
|
||||
// PullMaxMessagesWithBytesLimit limits the number of messages to be buffered
|
||||
// in the client. Additionally, it sets the maximum size a single fetch request
|
||||
// can have. Note that this will not limit the total size of messages buffered
|
||||
// in the client, but rather can serve as a way to limit what nats server will
|
||||
// have to internally buffer for a single fetch request.
|
||||
//
|
||||
// The byte limit should never be set to a value lower than the maximum message
|
||||
// size that can be expected from the server. If the byte limit is lower than
|
||||
// the maximum message size, the consumer will stall and not be able to consume
|
||||
// messages.
|
||||
//
|
||||
// This is an advanced option and should be used with caution. Most users should
|
||||
// use [PullMaxMessages] or [PullMaxBytes] instead.
|
||||
//
|
||||
// PullMaxMessagesWithBytesLimit implements both PullConsumeOpt and
|
||||
// PullMessagesOpt, allowing it to configure Consumer.Consume and Consumer.Messages.
|
||||
func PullMaxMessagesWithBytesLimit(maxMessages, byteLimit int) pullMaxMessagesWithBytesLimit {
|
||||
return pullMaxMessagesWithBytesLimit{maxMessages, byteLimit}
|
||||
}
|
||||
|
||||
func (m pullMaxMessagesWithBytesLimit) configureConsume(opts *consumeOpts) error {
|
||||
if m.maxMessages <= 0 {
|
||||
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
if m.maxBytes <= 0 {
|
||||
return fmt.Errorf("%w: maxBytes size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
if opts.MaxMessages > 0 {
|
||||
return fmt.Errorf("%w: maxMessages already set", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxMessages = m.maxMessages
|
||||
opts.MaxBytes = m.maxBytes
|
||||
opts.LimitSize = true
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m pullMaxMessagesWithBytesLimit) configureMessages(opts *consumeOpts) error {
|
||||
if m.maxMessages <= 0 {
|
||||
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
if m.maxBytes <= 0 {
|
||||
return fmt.Errorf("%w: maxBytes size must be at least 1", ErrInvalidOption)
|
||||
}
|
||||
if opts.MaxMessages > 0 {
|
||||
return fmt.Errorf("%w: maxMessages already set", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxMessages = m.maxMessages
|
||||
opts.MaxBytes = m.maxBytes
|
||||
opts.LimitSize = true
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullExpiry sets timeout on a single pull request, waiting until at least one
|
||||
// message is available.
|
||||
// If not provided, a default of 30 seconds will be used.
|
||||
//
|
||||
// PullExpiry implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullExpiry time.Duration
|
||||
|
||||
func (exp PullExpiry) configureConsume(opts *consumeOpts) error {
|
||||
expiry := time.Duration(exp)
|
||||
if expiry < time.Second {
|
||||
return fmt.Errorf("%w: expires value must be at least 1s", ErrInvalidOption)
|
||||
}
|
||||
opts.Expires = expiry
|
||||
return nil
|
||||
}
|
||||
|
||||
func (exp PullExpiry) configureMessages(opts *consumeOpts) error {
|
||||
expiry := time.Duration(exp)
|
||||
if expiry < time.Second {
|
||||
return fmt.Errorf("%w: expires value must be at least 1s", ErrInvalidOption)
|
||||
}
|
||||
opts.Expires = expiry
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullMaxBytes limits the number of bytes to be buffered in the client.
|
||||
// If not provided, the limit is not set (max messages will be used instead).
|
||||
// This option is exclusive with PullMaxMessages.
|
||||
//
|
||||
// The value should be set to a high enough value to accommodate the largest
|
||||
// message expected from the server. Note that it may not be sufficient to set
|
||||
// this value to the maximum message size, as this setting controls the client
|
||||
// buffer size, not the max bytes requested from the server within a single pull
|
||||
// request. If the value is set too low, the consumer will stall and not be able
|
||||
// to consume messages.
|
||||
//
|
||||
// PullMaxBytes implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullMaxBytes int
|
||||
|
||||
func (max PullMaxBytes) configureConsume(opts *consumeOpts) error {
|
||||
if max <= 0 {
|
||||
return fmt.Errorf("%w: max bytes must be greater than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxBytes = int(max)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (max PullMaxBytes) configureMessages(opts *consumeOpts) error {
|
||||
if max <= 0 {
|
||||
return fmt.Errorf("%w: max bytes must be greater than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MaxBytes = int(max)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullThresholdMessages sets the message count on which consuming will trigger
|
||||
// new pull request to the server. Defaults to 50% of MaxMessages.
|
||||
//
|
||||
// PullThresholdMessages implements both PullConsumeOpt and PullMessagesOpt,
|
||||
// allowing it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullThresholdMessages int
|
||||
|
||||
func (t PullThresholdMessages) configureConsume(opts *consumeOpts) error {
|
||||
opts.ThresholdMessages = int(t)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (t PullThresholdMessages) configureMessages(opts *consumeOpts) error {
|
||||
opts.ThresholdMessages = int(t)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullThresholdBytes sets the byte count on which consuming will trigger
|
||||
// new pull request to the server. Defaults to 50% of MaxBytes (if set).
|
||||
//
|
||||
// PullThresholdBytes implements both PullConsumeOpt and PullMessagesOpt,
|
||||
// allowing it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullThresholdBytes int
|
||||
|
||||
func (t PullThresholdBytes) configureConsume(opts *consumeOpts) error {
|
||||
opts.ThresholdBytes = int(t)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (t PullThresholdBytes) configureMessages(opts *consumeOpts) error {
|
||||
opts.ThresholdBytes = int(t)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullMinPending sets the minimum number of messages that should be pending for
|
||||
// a consumer with PriorityPolicyOverflow to be considered for delivery.
|
||||
// If provided, PullPriorityGroup must be set as well and the consumer has to have
|
||||
// PriorityPolicy set to PriorityPolicyOverflow.
|
||||
//
|
||||
// PullMinPending implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullMinPending int
|
||||
|
||||
func (min PullMinPending) configureConsume(opts *consumeOpts) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MinPending = int64(min)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (min PullMinPending) configureMessages(opts *consumeOpts) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MinPending = int64(min)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullMinAckPending sets the minimum number of pending acks that should be
|
||||
// present for a consumer with PriorityPolicyOverflow to be considered for
|
||||
// delivery. If provided, PullPriorityGroup must be set as well and the consumer
|
||||
// has to have PriorityPolicy set to PriorityPolicyOverflow.
|
||||
//
|
||||
// PullMinAckPending implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullMinAckPending int
|
||||
|
||||
func (min PullMinAckPending) configureConsume(opts *consumeOpts) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MinAckPending = int64(min)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (min PullMinAckPending) configureMessages(opts *consumeOpts) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.MinAckPending = int64(min)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullPrioritized sets the priority used when sending pull requests for consumer with
|
||||
// PriorityPolicyPrioritized. Lower values indicate higher priority (0 is the
|
||||
// highest priority). Maximum priority value is 9.
|
||||
//
|
||||
// If provided, PullPriorityGroup must be set as well and the consumer has to
|
||||
// have PriorityPolicy set to PriorityPolicyPrioritized.
|
||||
//
|
||||
// PullPrioritized implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullPrioritized uint8
|
||||
|
||||
func (p PullPrioritized) configureConsume(opts *consumeOpts) error {
|
||||
opts.Priority = uint8(p)
|
||||
return nil
|
||||
}
|
||||
func (p PullPrioritized) configureMessages(opts *consumeOpts) error {
|
||||
opts.Priority = uint8(p)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullPriorityGroup sets the priority group for a consumer.
|
||||
// It has to match one of the priority groups set on the consumer.
|
||||
//
|
||||
// PullPriorityGroup implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullPriorityGroup string
|
||||
|
||||
func (g PullPriorityGroup) configureConsume(opts *consumeOpts) error {
|
||||
opts.Group = string(g)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g PullPriorityGroup) configureMessages(opts *consumeOpts) error {
|
||||
opts.Group = string(g)
|
||||
return nil
|
||||
}
|
||||
|
||||
// PullHeartbeat sets the idle heartbeat duration for a pull subscription
|
||||
// If a client does not receive a heartbeat message from a stream for more
|
||||
// than the idle heartbeat setting, the subscription will be removed
|
||||
// and error will be passed to the message handler.
|
||||
// If not provided, a default PullExpiry / 2 will be used (capped at 30 seconds)
|
||||
//
|
||||
// PullHeartbeat implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type PullHeartbeat time.Duration
|
||||
|
||||
func (hb PullHeartbeat) configureConsume(opts *consumeOpts) error {
|
||||
hbTime := time.Duration(hb)
|
||||
if hbTime < 500*time.Millisecond || hbTime > 30*time.Second {
|
||||
return fmt.Errorf("%w: idle_heartbeat value must be within 500ms-30s range", ErrInvalidOption)
|
||||
}
|
||||
opts.Heartbeat = hbTime
|
||||
return nil
|
||||
}
|
||||
|
||||
func (hb PullHeartbeat) configureMessages(opts *consumeOpts) error {
|
||||
hbTime := time.Duration(hb)
|
||||
if hbTime < 500*time.Millisecond || hbTime > 30*time.Second {
|
||||
return fmt.Errorf("%w: idle_heartbeat value must be within 500ms-30s range", ErrInvalidOption)
|
||||
}
|
||||
opts.Heartbeat = hbTime
|
||||
return nil
|
||||
}
|
||||
|
||||
// StopAfter sets the number of messages after which the consumer is
|
||||
// automatically stopped and no more messages are pulled from the server.
|
||||
//
|
||||
// StopAfter implements both PullConsumeOpt and PullMessagesOpt, allowing
|
||||
// it to configure Consumer.Consume and Consumer.Messages.
|
||||
type StopAfter int
|
||||
|
||||
func (nMsgs StopAfter) configureConsume(opts *consumeOpts) error {
|
||||
if nMsgs <= 0 {
|
||||
return fmt.Errorf("%w: auto stop after value cannot be less than 1", ErrInvalidOption)
|
||||
}
|
||||
opts.StopAfter = int(nMsgs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (nMsgs StopAfter) configureMessages(opts *consumeOpts) error {
|
||||
if nMsgs <= 0 {
|
||||
return fmt.Errorf("%w: auto stop after value cannot be less than 1", ErrInvalidOption)
|
||||
}
|
||||
opts.StopAfter = int(nMsgs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// ConsumeErrHandler sets custom error handler invoked when an error was
|
||||
// encountered while consuming messages It will be invoked for both terminal
|
||||
// (Consumer Deleted, invalid request body) and non-terminal (e.g. missing
|
||||
// heartbeats) errors.
|
||||
type ConsumeErrHandler ConsumeErrHandlerFunc
|
||||
|
||||
func (c ConsumeErrHandler) configureConsume(opts *consumeOpts) error {
|
||||
opts.ErrHandler = c
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c ConsumeErrHandler) configurePushConsume(opts *pushConsumeOpts) error {
|
||||
opts.ErrHandler = c
|
||||
return nil
|
||||
}
|
||||
|
||||
// WithMessagesErrOnMissingHeartbeat sets whether a missing heartbeat error
|
||||
// should be reported when calling [MessagesContext.Next] (Default: true).
|
||||
func WithMessagesErrOnMissingHeartbeat(hbErr bool) PullMessagesOpt {
|
||||
return pullOptFunc(func(cfg *consumeOpts) error {
|
||||
cfg.ReportMissingHeartbeats = hbErr
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// FetchMinPending sets the minimum number of messages that should be pending for
|
||||
// a consumer with PriorityPolicyOverflow to be considered for delivery.
|
||||
// If provided, FetchPriorityGroup must be set as well and the consumer has to have
|
||||
// PriorityPolicy set to PriorityPolicyOverflow.
|
||||
func FetchMinPending(min int64) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
req.MinPending = min
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchMinAckPending sets the minimum number of pending acks that should be
|
||||
// present for a consumer with PriorityPolicyOverflow to be considered for
|
||||
// delivery. If provided, FetchPriorityGroup must be set as well and the consumer
|
||||
// has to have PriorityPolicy set to PriorityPolicyOverflow.
|
||||
func FetchMinAckPending(min int64) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
if min < 1 {
|
||||
return fmt.Errorf("%w: min ack pending should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
req.MinAckPending = min
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchPrioritized sets the priority used when sending fetch requests for consumer with
|
||||
// PriorityPolicyPrioritized. Lower values indicate higher priority (0 is the
|
||||
// highest priority). Maximum priority value is 9.
|
||||
//
|
||||
// If provided, FetchPriorityGroup must be set as well and the consumer has to
|
||||
// have PriorityPolicy set to PriorityPolicyPrioritized.
|
||||
func FetchPrioritized(priority uint8) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
req.Priority = priority
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchPriorityGroup sets the priority group for a consumer.
|
||||
// It has to match one of the priority groups set on the consumer.
|
||||
func FetchPriorityGroup(group string) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
req.Group = group
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchMaxWait sets custom timeout for fetching predefined batch of messages.
|
||||
//
|
||||
// If not provided, a default of 30 seconds will be used.
|
||||
func FetchMaxWait(timeout time.Duration) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
if timeout <= 0 {
|
||||
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
|
||||
}
|
||||
req.Expires = timeout
|
||||
req.maxWaitSet = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchHeartbeat sets custom heartbeat for individual fetch request. If a
|
||||
// client does not receive a heartbeat message from a stream for more than 2
|
||||
// times the idle heartbeat setting, Fetch will return [ErrNoHeartbeat].
|
||||
//
|
||||
// Heartbeat value has to be lower than FetchMaxWait / 2.
|
||||
//
|
||||
// If not provided, heartbeat will is set to 5s for requests with FetchMaxWait > 10s
|
||||
// and disabled otherwise.
|
||||
func FetchHeartbeat(hb time.Duration) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
if hb <= 0 {
|
||||
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
|
||||
}
|
||||
req.Heartbeat = hb
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FetchContext sets a context for the Fetch operation.
|
||||
// The Fetch operation will be canceled if the context is canceled.
|
||||
// If the context has a deadline, it will be used to set expiry on pull request.
|
||||
func FetchContext(ctx context.Context) FetchOpt {
|
||||
return func(req *pullRequest) error {
|
||||
req.ctx = ctx
|
||||
|
||||
// If context has a deadline, use it to set expiry
|
||||
if deadline, ok := ctx.Deadline(); ok {
|
||||
remaining := time.Until(deadline)
|
||||
if remaining <= 0 {
|
||||
return fmt.Errorf("%w: context deadline already exceeded", ErrInvalidOption)
|
||||
}
|
||||
// Use 90% of remaining time for server (capped at 1s)
|
||||
buffer := time.Duration(float64(remaining) * 0.1)
|
||||
if buffer > time.Second {
|
||||
buffer = time.Second
|
||||
}
|
||||
req.Expires = remaining - buffer
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithDeletedDetails can be used to display the information about messages
|
||||
// deleted from a stream on a stream info request
|
||||
func WithDeletedDetails(deletedDetails bool) StreamInfoOpt {
|
||||
return func(req *streamInfoRequest) error {
|
||||
req.DeletedDetails = deletedDetails
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithSubjectFilter can be used to display the information about messages
|
||||
// stored on given subjects.
|
||||
// NOTE: if the subject filter matches over 100k
|
||||
// subjects, this will result in multiple requests to the server to retrieve all
|
||||
// the information, and all of the returned subjects will be kept in memory.
|
||||
func WithSubjectFilter(subject string) StreamInfoOpt {
|
||||
return func(req *streamInfoRequest) error {
|
||||
req.SubjectFilter = subject
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithStreamListSubject can be used to filter results of ListStreams and
|
||||
// StreamNames requests to only streams that have given subject in their
|
||||
// configuration.
|
||||
func WithStreamListSubject(subject string) StreamListOpt {
|
||||
return func(req *streamsRequest) error {
|
||||
req.Subject = subject
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithMsgID sets the message ID used for deduplication.
|
||||
func WithMsgID(id string) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.id = id
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithMsgTTL sets per msg TTL.
|
||||
// Requires [StreamConfig.AllowMsgTTL] to be enabled.
|
||||
func WithMsgTTL(dur time.Duration) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.ttl = dur
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithExpectStream sets the expected stream the message should be published to.
|
||||
// If the message is published to a different stream server will reject the
|
||||
// message and publish will fail.
|
||||
func WithExpectStream(stream string) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.stream = stream
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithExpectLastSequence sets the expected sequence number the last message
|
||||
// on a stream should have. If the last message has a different sequence number
|
||||
// server will reject the message and publish will fail.
|
||||
func WithExpectLastSequence(seq uint64) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.lastSeq = &seq
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithExpectLastSequencePerSubject sets the expected sequence number the last
|
||||
// message on a subject the message is published to. If the last message on a
|
||||
// subject has a different sequence number server will reject the message and
|
||||
// publish will fail.
|
||||
func WithExpectLastSequencePerSubject(seq uint64) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.lastSubjectSeq = &seq
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithExpectLastSequenceForSubject sets the sequence and subject for which the
|
||||
// last sequence number should be checked. If the last message on a subject
|
||||
// has a different sequence number server will reject the message and publish
|
||||
// will fail.
|
||||
func WithExpectLastSequenceForSubject(seq uint64, subject string) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
if subject == "" {
|
||||
return fmt.Errorf("%w: subject cannot be empty", ErrInvalidOption)
|
||||
}
|
||||
opts.lastSubjectSeq = &seq
|
||||
opts.lastSubject = subject
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithExpectLastMsgID sets the expected message ID the last message on a stream
|
||||
// should have. If the last message has a different message ID server will
|
||||
// reject the message and publish will fail.
|
||||
func WithExpectLastMsgID(id string) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
opts.lastMsgID = id
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryWait sets the retry wait time when ErrNoResponders is encountered.
|
||||
// Defaults to 250ms.
|
||||
func WithRetryWait(dur time.Duration) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
if dur <= 0 {
|
||||
return fmt.Errorf("%w: retry wait should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.retryWait = dur
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryAttempts sets the retry number of attempts when ErrNoResponders is
|
||||
// encountered. Defaults to 2
|
||||
func WithRetryAttempts(num int) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
if num < 0 {
|
||||
return fmt.Errorf("%w: retry attempts cannot be negative", ErrInvalidOption)
|
||||
}
|
||||
opts.retryAttempts = num
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithStallWait sets the max wait when the producer becomes stall producing
|
||||
// messages. If a publish call is blocked for this long, ErrTooManyStalledMsgs
|
||||
// is returned.
|
||||
func WithStallWait(ttl time.Duration) PublishOpt {
|
||||
return func(opts *pubOpts) error {
|
||||
if ttl <= 0 {
|
||||
return fmt.Errorf("%w: stall wait should be more than 0", ErrInvalidOption)
|
||||
}
|
||||
opts.stallWait = ttl
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
type nextOptFunc func(*nextOpts)
|
||||
|
||||
func (fn nextOptFunc) configureNext(opts *nextOpts) {
|
||||
fn(opts)
|
||||
}
|
||||
|
||||
// NextMaxWait sets a timeout for the Next operation.
|
||||
// If the timeout is reached before a message is available, a timeout error is returned.
|
||||
func NextMaxWait(timeout time.Duration) NextOpt {
|
||||
return nextOptFunc(func(opts *nextOpts) {
|
||||
opts.timeout = timeout
|
||||
})
|
||||
}
|
||||
|
||||
// NextContext sets a context for the Next operation.
|
||||
// The Next operation will be canceled if the context is canceled.
|
||||
func NextContext(ctx context.Context) NextOpt {
|
||||
return nextOptFunc(func(opts *nextOpts) {
|
||||
opts.ctx = ctx
|
||||
})
|
||||
}
|
||||
+1593
File diff suppressed because it is too large
Load Diff
+131
@@ -0,0 +1,131 @@
|
||||
// Copyright 2024 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
type watchOptFn func(opts *watchOpts) error
|
||||
|
||||
func (opt watchOptFn) configureWatcher(opts *watchOpts) error {
|
||||
return opt(opts)
|
||||
}
|
||||
|
||||
// IncludeHistory instructs the key watcher to include historical values as
|
||||
// well (up to KeyValueMaxHistory).
|
||||
func IncludeHistory() WatchOpt {
|
||||
return watchOptFn(func(opts *watchOpts) error {
|
||||
if opts.updatesOnly {
|
||||
return fmt.Errorf("%w: include history cannot be used with updates only", ErrInvalidOption)
|
||||
}
|
||||
opts.includeHistory = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// UpdatesOnly instructs the key watcher to only include updates on values
|
||||
// (without latest values when started).
|
||||
func UpdatesOnly() WatchOpt {
|
||||
return watchOptFn(func(opts *watchOpts) error {
|
||||
if opts.includeHistory {
|
||||
return fmt.Errorf("%w: updates only cannot be used with include history", ErrInvalidOption)
|
||||
}
|
||||
opts.updatesOnly = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// IgnoreDeletes will prevent the key watcher from passing any deleted keys.
|
||||
func IgnoreDeletes() WatchOpt {
|
||||
return watchOptFn(func(opts *watchOpts) error {
|
||||
opts.ignoreDeletes = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// MetaOnly instructs the key watcher to retrieve only the entry metadata, not
|
||||
// the entry value.
|
||||
func MetaOnly() WatchOpt {
|
||||
return watchOptFn(func(opts *watchOpts) error {
|
||||
opts.metaOnly = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// ResumeFromRevision instructs the key watcher to resume from a specific
|
||||
// revision number.
|
||||
func ResumeFromRevision(revision uint64) WatchOpt {
|
||||
return watchOptFn(func(opts *watchOpts) error {
|
||||
opts.resumeFromRevision = revision
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// DeleteMarkersOlderThan indicates that delete or purge markers older than that
|
||||
// will be deleted as part of [KeyValue.PurgeDeletes] operation, otherwise, only the data
|
||||
// will be removed but markers that are recent will be kept.
|
||||
// Note that if no option is specified, the default is 30 minutes. You can set
|
||||
// this option to a negative value to instruct to always remove the markers,
|
||||
// regardless of their age.
|
||||
type DeleteMarkersOlderThan time.Duration
|
||||
|
||||
func (ttl DeleteMarkersOlderThan) configurePurge(opts *purgeOpts) error {
|
||||
opts.dmthr = time.Duration(ttl)
|
||||
return nil
|
||||
}
|
||||
|
||||
type deleteOptFn func(opts *deleteOpts) error
|
||||
|
||||
func (opt deleteOptFn) configureDelete(opts *deleteOpts) error {
|
||||
return opt(opts)
|
||||
}
|
||||
|
||||
// LastRevision deletes if the latest revision matches the provided one. If the
|
||||
// provided revision is not the latest, the delete will return an error.
|
||||
func LastRevision(revision uint64) KVDeleteOpt {
|
||||
return deleteOptFn(func(opts *deleteOpts) error {
|
||||
opts.revision = revision
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// PurgeTTL sets the TTL for the purge operation.
|
||||
// After the TTL expires, the delete markers will be removed.
|
||||
// This requires LimitMarkerTTL to be enabled on the bucket.
|
||||
// Note that this is not the same as the TTL for the key itself, which is set
|
||||
// using the KeyTTL option when creating the key.
|
||||
func PurgeTTL(ttl time.Duration) KVDeleteOpt {
|
||||
return deleteOptFn(func(opts *deleteOpts) error {
|
||||
opts.ttl = ttl
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
type createOptFn func(opts *createOpts) error
|
||||
|
||||
func (opt createOptFn) configureCreate(opts *createOpts) error {
|
||||
return opt(opts)
|
||||
}
|
||||
|
||||
// KeyTTL sets the TTL for the key. This is the time after which the key will be
|
||||
// automatically deleted. The TTL is set when the key is created and cannot be
|
||||
// changed later. This requires LimitMarkerTTL to be enabled on the bucket.
|
||||
func KeyTTL(ttl time.Duration) KVCreateOpt {
|
||||
return createOptFn(func(opts *createOpts) error {
|
||||
opts.ttl = ttl
|
||||
return nil
|
||||
})
|
||||
}
|
||||
+488
@@ -0,0 +1,488 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/internal/parser"
|
||||
)
|
||||
|
||||
type (
|
||||
// Msg contains methods to operate on a JetStream message. Metadata, Data,
|
||||
// Headers, Subject and Reply can be used to retrieve the specific parts of
|
||||
// the underlying message. Ack, DoubleAck, Nak, NakWithDelay, InProgress and
|
||||
// Term are various flavors of ack requests.
|
||||
Msg interface {
|
||||
// Metadata returns [MsgMetadata] for a JetStream message.
|
||||
Metadata() (*MsgMetadata, error)
|
||||
|
||||
// Data returns the message body.
|
||||
Data() []byte
|
||||
|
||||
// Headers returns a map of headers for a message.
|
||||
Headers() nats.Header
|
||||
|
||||
// Subject returns a subject on which a message was published/received.
|
||||
Subject() string
|
||||
|
||||
// Reply returns a reply subject for a message.
|
||||
Reply() string
|
||||
|
||||
// Ack acknowledges a message. This tells the server that the message was
|
||||
// successfully processed and it can move on to the next message.
|
||||
Ack() error
|
||||
|
||||
// DoubleAck acknowledges a message and waits for ack reply from the server.
|
||||
// While it impacts performance, it is useful for scenarios where
|
||||
// message loss is not acceptable.
|
||||
DoubleAck(context.Context) error
|
||||
|
||||
// Nak negatively acknowledges a message. This tells the server to
|
||||
// redeliver the message.
|
||||
//
|
||||
// Nak does not adhere to AckWait or Backoff configured on the consumer
|
||||
// and triggers instant redelivery. For a delayed redelivery, use
|
||||
// NakWithDelay.
|
||||
Nak() error
|
||||
|
||||
// NakWithDelay negatively acknowledges a message. This tells the server
|
||||
// to redeliver the message after the given delay.
|
||||
NakWithDelay(delay time.Duration) error
|
||||
|
||||
// InProgress tells the server that this message is being worked on. It
|
||||
// resets the redelivery timer on the server.
|
||||
InProgress() error
|
||||
|
||||
// Term tells the server to not redeliver this message, regardless of
|
||||
// the value of MaxDeliver.
|
||||
Term() error
|
||||
|
||||
// TermWithReason tells the server to not redeliver this message, regardless of
|
||||
// the value of MaxDeliver. The provided reason will be included in JetStream
|
||||
// advisory event sent by the server.
|
||||
//
|
||||
// Note: This will only work with JetStream servers >= 2.10.4.
|
||||
// For older servers, TermWithReason will be ignored by the server and the message
|
||||
// will not be terminated.
|
||||
TermWithReason(reason string) error
|
||||
}
|
||||
|
||||
// MsgMetadata is the JetStream metadata associated with received messages.
|
||||
MsgMetadata struct {
|
||||
// Sequence is the sequence information for the message.
|
||||
Sequence SequencePair
|
||||
|
||||
// NumDelivered is the number of times this message was delivered to the
|
||||
// consumer.
|
||||
NumDelivered uint64
|
||||
|
||||
// NumPending is the number of messages that match the consumer's
|
||||
// filter, but have not been delivered yet.
|
||||
NumPending uint64
|
||||
|
||||
// Timestamp is the time the message was originally stored on a stream.
|
||||
Timestamp time.Time
|
||||
|
||||
// Stream is the stream name this message is stored on.
|
||||
Stream string
|
||||
|
||||
// Consumer is the consumer name this message was delivered to.
|
||||
Consumer string
|
||||
|
||||
// Domain is the domain this message was received on.
|
||||
Domain string
|
||||
}
|
||||
|
||||
// SequencePair includes the consumer and stream sequence numbers for a
|
||||
// message.
|
||||
SequencePair struct {
|
||||
// Consumer is the consumer sequence number for message deliveries. This
|
||||
// is the total number of messages the consumer has seen (including
|
||||
// redeliveries).
|
||||
Consumer uint64 `json:"consumer_seq"`
|
||||
|
||||
// Stream is the stream sequence number for a message.
|
||||
Stream uint64 `json:"stream_seq"`
|
||||
}
|
||||
|
||||
jetStreamMsg struct {
|
||||
msg *nats.Msg
|
||||
ackd bool
|
||||
js *jetStream
|
||||
sync.Mutex
|
||||
}
|
||||
|
||||
ackOpts struct {
|
||||
nakDelay time.Duration
|
||||
termReason string
|
||||
}
|
||||
|
||||
ackType []byte
|
||||
)
|
||||
|
||||
const (
|
||||
statusControlMsg = "100"
|
||||
statusBadRequest = "400"
|
||||
statusNoMsgs = "404"
|
||||
statusTimeout = "408"
|
||||
statusConflict = "409"
|
||||
statusNoResponders = "503"
|
||||
statusPinIdMismatch = "423"
|
||||
|
||||
fcRequestDescr = "flowcontrol request"
|
||||
idleHeartbeatDescr = "idle heartbeat"
|
||||
consumerDeleted = "consumer deleted"
|
||||
leadershipChange = "leadership change"
|
||||
maxBytesExceeded = "message size exceeds maxbytes"
|
||||
batchCompleted = "batch completed"
|
||||
serverShutdown = "server shutdown"
|
||||
)
|
||||
|
||||
// Headers used when publishing messages.
|
||||
const (
|
||||
// MsgIdHeader is used to specify a user-defined message ID. It can be used
|
||||
// e.g. for deduplication in conjunction with the Duplicates duration on
|
||||
// ConsumerConfig or to provide optimistic concurrency safety together with
|
||||
// [ExpectedLastMsgIDHeader].
|
||||
//
|
||||
// This can be set when publishing messages using [WithMsgID] option.
|
||||
MsgIDHeader = "Nats-Msg-Id"
|
||||
|
||||
// ExpectedStreamHeader contains stream name and is used to assure that the
|
||||
// published message is received by expected stream. Server will reject the
|
||||
// message if it is not the case.
|
||||
//
|
||||
// This can be set when publishing messages using [WithExpectStream] option.
|
||||
ExpectedStreamHeader = "Nats-Expected-Stream"
|
||||
|
||||
// ExpectedLastSeqHeader contains the expected last sequence number of the
|
||||
// stream and can be used to apply optimistic concurrency control at stream
|
||||
// level. Server will reject the message if it is not the case.
|
||||
//
|
||||
// This can be set when publishing messages using [WithExpectLastSequence]
|
||||
// option.
|
||||
ExpectedLastSeqHeader = "Nats-Expected-Last-Sequence"
|
||||
|
||||
// ExpectedLastSubjSeqHeader contains the expected last sequence number on
|
||||
// the subject and can be used to apply optimistic concurrency control at
|
||||
// subject level. Server will reject the message if it is not the case.
|
||||
//
|
||||
// This can be set when publishing messages using
|
||||
// [WithExpectLastSequencePerSubject] option.
|
||||
ExpectedLastSubjSeqHeader = "Nats-Expected-Last-Subject-Sequence"
|
||||
|
||||
// ExpectedLastSubjSeqSubjHeader contains the subject for which the
|
||||
// expected last sequence number is set. This is used together with
|
||||
// [ExpectedLastSubjSeqHeader] to apply optimistic concurrency control at
|
||||
// subject level. Server will reject the message if it is not the case.
|
||||
ExpectedLastSubjSeqSubjHeader = "Nats-Expected-Last-Subject-Sequence-Subject"
|
||||
|
||||
// ExpectedLastMsgIDHeader contains the expected last message ID on the
|
||||
// subject and can be used to apply optimistic concurrency control at
|
||||
// stream level. Server will reject the message if it is not the case.
|
||||
//
|
||||
// This can be set when publishing messages using [WithExpectLastMsgID]
|
||||
// option.
|
||||
ExpectedLastMsgIDHeader = "Nats-Expected-Last-Msg-Id"
|
||||
|
||||
// MsgTTLHeader is used to specify the TTL for a specific message. This will
|
||||
// override the default TTL for the stream.
|
||||
MsgTTLHeader = "Nats-TTL"
|
||||
|
||||
// MsgRollup is used to apply a purge of all prior messages in the stream
|
||||
// ("all") or at the subject ("sub") before this message.
|
||||
MsgRollup = "Nats-Rollup"
|
||||
|
||||
// MarkerReasonHeader is used to specify a reason for message deletion.
|
||||
MarkerReasonHeader = "Nats-Marker-Reason"
|
||||
)
|
||||
|
||||
// Headers for republished messages and direct gets. Those headers are set by
|
||||
// the server and should not be set by the client.
|
||||
const (
|
||||
// StreamHeader contains the stream name the message was republished from or
|
||||
// the stream name the message was retrieved from using direct get.
|
||||
StreamHeader = "Nats-Stream"
|
||||
|
||||
// SequenceHeader contains the original sequence number of the message.
|
||||
SequenceHeader = "Nats-Sequence"
|
||||
|
||||
// TimeStampHeaer contains the original timestamp of the message.
|
||||
TimeStampHeaer = "Nats-Time-Stamp"
|
||||
|
||||
// SubjectHeader contains the original subject the message was published to.
|
||||
SubjectHeader = "Nats-Subject"
|
||||
|
||||
// LastSequenceHeader contains the last sequence of the message having the
|
||||
// same subject, otherwise zero if this is the first message for the
|
||||
// subject.
|
||||
LastSequenceHeader = "Nats-Last-Sequence"
|
||||
)
|
||||
|
||||
// Rollups, can be subject only or all messages.
|
||||
const (
|
||||
// MsgRollupSubject is used to purge all messages before this message on the
|
||||
// message subject.
|
||||
MsgRollupSubject = "sub"
|
||||
|
||||
// MsgRollupAll is used to purge all messages before this message on the
|
||||
// stream.
|
||||
MsgRollupAll = "all"
|
||||
)
|
||||
|
||||
var (
|
||||
ackAck ackType = []byte("+ACK")
|
||||
ackNak ackType = []byte("-NAK")
|
||||
ackProgress ackType = []byte("+WPI")
|
||||
ackTerm ackType = []byte("+TERM")
|
||||
)
|
||||
|
||||
// Metadata returns [MsgMetadata] for a JetStream message.
|
||||
func (m *jetStreamMsg) Metadata() (*MsgMetadata, error) {
|
||||
if err := m.checkReply(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tokens, err := parser.GetMetadataFields(m.msg.Reply)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", ErrNotJSMessage, err)
|
||||
}
|
||||
|
||||
meta := &MsgMetadata{
|
||||
Domain: tokens[parser.AckDomainTokenPos],
|
||||
NumDelivered: parser.ParseNum(tokens[parser.AckNumDeliveredTokenPos]),
|
||||
NumPending: parser.ParseNum(tokens[parser.AckNumPendingTokenPos]),
|
||||
Timestamp: time.Unix(0, int64(parser.ParseNum(tokens[parser.AckTimestampSeqTokenPos]))),
|
||||
Stream: tokens[parser.AckStreamTokenPos],
|
||||
Consumer: tokens[parser.AckConsumerTokenPos],
|
||||
}
|
||||
meta.Sequence.Stream = parser.ParseNum(tokens[parser.AckStreamSeqTokenPos])
|
||||
meta.Sequence.Consumer = parser.ParseNum(tokens[parser.AckConsumerSeqTokenPos])
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
// Data returns the message body.
|
||||
func (m *jetStreamMsg) Data() []byte {
|
||||
return m.msg.Data
|
||||
}
|
||||
|
||||
// Headers returns a map of headers for a message.
|
||||
func (m *jetStreamMsg) Headers() nats.Header {
|
||||
return m.msg.Header
|
||||
}
|
||||
|
||||
// Subject returns a subject on which a message was published/received.
|
||||
func (m *jetStreamMsg) Subject() string {
|
||||
return m.msg.Subject
|
||||
}
|
||||
|
||||
// Reply returns a reply subject for a JetStream message.
|
||||
func (m *jetStreamMsg) Reply() string {
|
||||
return m.msg.Reply
|
||||
}
|
||||
|
||||
// Ack acknowledges a message. This tells the server that the message was
|
||||
// successfully processed and it can move on to the next message.
|
||||
func (m *jetStreamMsg) Ack() error {
|
||||
return m.ackReply(context.Background(), ackAck, false, ackOpts{})
|
||||
}
|
||||
|
||||
// DoubleAck acknowledges a message and waits for ack reply from the server.
|
||||
// While it impacts performance, it is useful for scenarios where
|
||||
// message loss is not acceptable.
|
||||
func (m *jetStreamMsg) DoubleAck(ctx context.Context) error {
|
||||
return m.ackReply(ctx, ackAck, true, ackOpts{})
|
||||
}
|
||||
|
||||
// Nak negatively acknowledges a message. This tells the server to
|
||||
// redeliver the message.
|
||||
func (m *jetStreamMsg) Nak() error {
|
||||
return m.ackReply(context.Background(), ackNak, false, ackOpts{})
|
||||
}
|
||||
|
||||
// NakWithDelay negatively acknowledges a message. This tells the server
|
||||
// to redeliver the message after the given delay.
|
||||
func (m *jetStreamMsg) NakWithDelay(delay time.Duration) error {
|
||||
return m.ackReply(context.Background(), ackNak, false, ackOpts{nakDelay: delay})
|
||||
}
|
||||
|
||||
// InProgress tells the server that this message is being worked on. It
|
||||
// resets the redelivery timer on the server.
|
||||
func (m *jetStreamMsg) InProgress() error {
|
||||
return m.ackReply(context.Background(), ackProgress, false, ackOpts{})
|
||||
}
|
||||
|
||||
// Term tells the server to not redeliver this message, regardless of
|
||||
// the value of MaxDeliver.
|
||||
func (m *jetStreamMsg) Term() error {
|
||||
return m.ackReply(context.Background(), ackTerm, false, ackOpts{})
|
||||
}
|
||||
|
||||
// TermWithReason tells the server to not redeliver this message, regardless of
|
||||
// the value of MaxDeliver. The provided reason will be included in JetStream
|
||||
// advisory event sent by the server.
|
||||
//
|
||||
// Note: This will only work with JetStream servers >= 2.10.4.
|
||||
// For older servers, TermWithReason will be ignored by the server and the message
|
||||
// will not be terminated.
|
||||
func (m *jetStreamMsg) TermWithReason(reason string) error {
|
||||
return m.ackReply(context.Background(), ackTerm, false, ackOpts{termReason: reason})
|
||||
}
|
||||
|
||||
func (m *jetStreamMsg) ackReply(ctx context.Context, ackType ackType, sync bool, opts ackOpts) error {
|
||||
err := m.checkReply()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
m.Lock()
|
||||
if m.ackd {
|
||||
m.Unlock()
|
||||
return ErrMsgAlreadyAckd
|
||||
}
|
||||
m.Unlock()
|
||||
|
||||
if sync {
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = m.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
}
|
||||
|
||||
var body []byte
|
||||
if opts.nakDelay > 0 {
|
||||
body = []byte(fmt.Sprintf("%s {\"delay\": %d}", ackType, opts.nakDelay.Nanoseconds()))
|
||||
} else if opts.termReason != "" {
|
||||
body = []byte(fmt.Sprintf("%s %s", ackType, opts.termReason))
|
||||
} else {
|
||||
body = ackType
|
||||
}
|
||||
|
||||
if sync {
|
||||
_, err = m.js.conn.RequestWithContext(ctx, m.msg.Reply, body)
|
||||
} else {
|
||||
err = m.js.conn.Publish(m.msg.Reply, body)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Mark that the message has been acked unless it is ackProgress
|
||||
// which can be sent many times.
|
||||
if !bytes.Equal(ackType, ackProgress) {
|
||||
m.Lock()
|
||||
m.ackd = true
|
||||
m.Unlock()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *jetStreamMsg) checkReply() error {
|
||||
if m == nil || m.msg.Sub == nil {
|
||||
return ErrMsgNotBound
|
||||
}
|
||||
if m.msg.Reply == "" {
|
||||
return ErrMsgNoReply
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkMsg returns whether the given message is a user message or a control message.
|
||||
// If the status header is present, it returns an appropriate error based
|
||||
// on the status code (404, etc.)
|
||||
func checkMsg(msg *nats.Msg) (bool, error) {
|
||||
// If payload or no header, consider this a user message
|
||||
if len(msg.Data) > 0 || len(msg.Header) == 0 {
|
||||
return true, nil
|
||||
}
|
||||
// Look for status header
|
||||
val := msg.Header.Get("Status")
|
||||
descr := msg.Header.Get("Description")
|
||||
// If not present, then this is considered a user message
|
||||
if val == "" {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
switch val {
|
||||
case statusBadRequest:
|
||||
return false, ErrBadRequest
|
||||
case statusNoResponders:
|
||||
return false, nats.ErrNoResponders
|
||||
case statusNoMsgs:
|
||||
// 404 indicates that there are no messages.
|
||||
return false, ErrNoMessages
|
||||
case statusTimeout:
|
||||
return false, nats.ErrTimeout
|
||||
case statusControlMsg:
|
||||
return false, nil
|
||||
case statusPinIdMismatch:
|
||||
return false, ErrPinIDMismatch
|
||||
case statusConflict:
|
||||
if strings.Contains(strings.ToLower(descr), maxBytesExceeded) {
|
||||
return false, ErrMaxBytesExceeded
|
||||
}
|
||||
if strings.Contains(strings.ToLower(descr), batchCompleted) {
|
||||
return false, ErrBatchCompleted
|
||||
}
|
||||
if strings.Contains(strings.ToLower(descr), consumerDeleted) {
|
||||
return false, ErrConsumerDeleted
|
||||
}
|
||||
if strings.Contains(strings.ToLower(descr), leadershipChange) {
|
||||
return false, ErrConsumerLeadershipChanged
|
||||
}
|
||||
if strings.Contains(strings.ToLower(descr), serverShutdown) {
|
||||
return false, ErrServerShutdown
|
||||
}
|
||||
}
|
||||
return false, fmt.Errorf("nats: %s", msg.Header.Get("Description"))
|
||||
}
|
||||
|
||||
func parsePending(msg *nats.Msg) (int, int, error) {
|
||||
msgsLeftStr := msg.Header.Get("Nats-Pending-Messages")
|
||||
var msgsLeft int
|
||||
var err error
|
||||
if msgsLeftStr != "" {
|
||||
msgsLeft, err = strconv.Atoi(msgsLeftStr)
|
||||
if err != nil {
|
||||
return 0, 0, errors.New("nats: invalid format of Nats-Pending-Messages")
|
||||
}
|
||||
}
|
||||
bytesLeftStr := msg.Header.Get("Nats-Pending-Bytes")
|
||||
var bytesLeft int
|
||||
if bytesLeftStr != "" {
|
||||
bytesLeft, err = strconv.Atoi(bytesLeftStr)
|
||||
if err != nil {
|
||||
return 0, 0, errors.New("nats: invalid format of Nats-Pending-Bytes")
|
||||
}
|
||||
}
|
||||
return msgsLeft, bytesLeft, nil
|
||||
}
|
||||
|
||||
// toJSMsg converts core [nats.Msg] to [jetStreamMsg], exposing JetStream-specific operations
|
||||
func (js *jetStream) toJSMsg(msg *nats.Msg) *jetStreamMsg {
|
||||
return &jetStreamMsg{
|
||||
msg: msg,
|
||||
js: js,
|
||||
}
|
||||
}
|
||||
+1633
File diff suppressed because it is too large
Load Diff
+41
@@ -0,0 +1,41 @@
|
||||
// Copyright 2024 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
// GetObjectShowDeleted makes [ObjectStore.Get] return object even if it was
|
||||
// marked as deleted.
|
||||
func GetObjectShowDeleted() GetObjectOpt {
|
||||
return func(opts *getObjectOpts) error {
|
||||
opts.showDeleted = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// GetObjectInfoShowDeleted makes [ObjectStore.GetInfo] return object info event
|
||||
// if it was marked as deleted.
|
||||
func GetObjectInfoShowDeleted() GetObjectInfoOpt {
|
||||
return func(opts *getObjectInfoOpts) error {
|
||||
opts.showDeleted = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// ListObjectsShowDeleted makes [ObjectStore.ListObjects] also return deleted
|
||||
// objects.
|
||||
func ListObjectsShowDeleted() ListObjectsOpt {
|
||||
return func(opts *listObjectOpts) error {
|
||||
opts.showDeleted = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
+834
@@ -0,0 +1,834 @@
|
||||
// Copyright 2022-2024 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
type (
|
||||
orderedConsumer struct {
|
||||
js *jetStream
|
||||
cfg *OrderedConsumerConfig
|
||||
stream string
|
||||
currentConsumer *pullConsumer
|
||||
currentSub *pullSubscription
|
||||
cursor cursor
|
||||
namePrefix string
|
||||
serial int
|
||||
consumerType consumerType
|
||||
doReset chan struct{}
|
||||
resetInProgress atomic.Uint32
|
||||
userErrHandler ConsumeErrHandler
|
||||
stopAfter int
|
||||
stopAfterMsgsLeft chan int
|
||||
withStopAfter bool
|
||||
runningFetch *fetchResult
|
||||
subscription *orderedSubscription
|
||||
sync.Mutex
|
||||
}
|
||||
|
||||
orderedSubscription struct {
|
||||
consumer *orderedConsumer
|
||||
opts []PullMessagesOpt
|
||||
done chan struct{}
|
||||
closed atomic.Uint32
|
||||
}
|
||||
|
||||
cursor struct {
|
||||
streamSeq uint64
|
||||
deliverSeq uint64
|
||||
}
|
||||
|
||||
consumerType int
|
||||
)
|
||||
|
||||
const (
|
||||
consumerTypeNotSet consumerType = iota
|
||||
consumerTypeConsume
|
||||
consumerTypeFetch
|
||||
)
|
||||
|
||||
var (
|
||||
errOrderedSequenceMismatch = errors.New("sequence mismatch")
|
||||
errOrderedConsumerClosed = errors.New("ordered consumer closed")
|
||||
)
|
||||
|
||||
// Consume can be used to continuously receive messages and handle them
|
||||
// with the provided callback function. Consume cannot be used concurrently
|
||||
// when using ordered consumer.
|
||||
//
|
||||
// See [Consumer.Consume] for more details.
|
||||
func (c *orderedConsumer) Consume(handler MessageHandler, opts ...PullConsumeOpt) (ConsumeContext, error) {
|
||||
c.Lock()
|
||||
defer c.Unlock()
|
||||
if (c.consumerType == consumerTypeNotSet || c.consumerType == consumerTypeConsume) && c.currentConsumer == nil {
|
||||
err := c.reset()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if c.consumerType == consumerTypeConsume && c.currentConsumer != nil {
|
||||
return nil, ErrOrderedConsumerConcurrentRequests
|
||||
}
|
||||
if c.consumerType == consumerTypeFetch {
|
||||
return nil, ErrOrderConsumerUsedAsFetch
|
||||
}
|
||||
c.consumerType = consumerTypeConsume
|
||||
consumeOpts, err := parseConsumeOpts(true, opts...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", ErrInvalidOption, err)
|
||||
}
|
||||
c.userErrHandler = consumeOpts.ErrHandler
|
||||
opts = append(opts, consumeReconnectNotify(),
|
||||
ConsumeErrHandler(c.errHandler(c.serial)))
|
||||
if consumeOpts.StopAfter > 0 {
|
||||
c.withStopAfter = true
|
||||
c.stopAfter = consumeOpts.StopAfter
|
||||
}
|
||||
c.stopAfterMsgsLeft = make(chan int, 1)
|
||||
if c.stopAfter > 0 {
|
||||
opts = append(opts, consumeStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
|
||||
}
|
||||
sub := &orderedSubscription{
|
||||
consumer: c,
|
||||
done: make(chan struct{}, 1),
|
||||
}
|
||||
c.subscription = sub
|
||||
internalHandler := func(serial int) func(msg Msg) {
|
||||
return func(msg Msg) {
|
||||
c.Lock()
|
||||
// handler is a noop if message was delivered for a consumer with different serial
|
||||
if serial != c.serial {
|
||||
c.Unlock()
|
||||
return
|
||||
}
|
||||
meta, err := msg.Metadata()
|
||||
if err != nil {
|
||||
currentSub := c.currentSub
|
||||
c.Unlock()
|
||||
c.errHandler(serial)(currentSub, err)
|
||||
return
|
||||
}
|
||||
dseq := meta.Sequence.Consumer
|
||||
if dseq != c.cursor.deliverSeq+1 {
|
||||
c.Unlock()
|
||||
c.errHandler(serial)(sub, errOrderedSequenceMismatch)
|
||||
return
|
||||
}
|
||||
c.cursor.deliverSeq = dseq
|
||||
c.cursor.streamSeq = meta.Sequence.Stream
|
||||
c.Unlock()
|
||||
handler(msg)
|
||||
}
|
||||
}
|
||||
|
||||
cc, err := c.currentConsumer.Consume(internalHandler(c.serial), opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.currentSub = cc.(*pullSubscription)
|
||||
|
||||
go func() {
|
||||
for {
|
||||
select {
|
||||
case <-c.doReset:
|
||||
if err := c.reset(); err != nil {
|
||||
if errors.Is(err, errOrderedConsumerClosed) {
|
||||
continue
|
||||
}
|
||||
c.errHandler(c.serial)(c.currentSub, err)
|
||||
}
|
||||
if c.withStopAfter {
|
||||
select {
|
||||
case c.stopAfter = <-c.stopAfterMsgsLeft:
|
||||
default:
|
||||
}
|
||||
if c.stopAfter <= 0 {
|
||||
sub.Stop()
|
||||
return
|
||||
}
|
||||
}
|
||||
if c.stopAfter > 0 {
|
||||
opts = opts[:len(opts)-2]
|
||||
} else {
|
||||
opts = opts[:len(opts)-1]
|
||||
}
|
||||
|
||||
// overwrite the previous err handler to use the new serial
|
||||
opts = append(opts, ConsumeErrHandler(c.errHandler(c.serial)))
|
||||
if c.withStopAfter {
|
||||
opts = append(opts, consumeStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
|
||||
}
|
||||
if cc, err := c.currentConsumer.Consume(internalHandler(c.serial), opts...); err != nil {
|
||||
c.errHandler(c.serial)(cc, err)
|
||||
} else {
|
||||
c.Lock()
|
||||
c.currentSub = cc.(*pullSubscription)
|
||||
c.Unlock()
|
||||
}
|
||||
case <-sub.done:
|
||||
s := sub.consumer.currentSub
|
||||
if s != nil {
|
||||
sub.consumer.Lock()
|
||||
s.Stop()
|
||||
sub.consumer.Unlock()
|
||||
}
|
||||
return
|
||||
case msgsLeft, ok := <-c.stopAfterMsgsLeft:
|
||||
if !ok {
|
||||
close(sub.done)
|
||||
}
|
||||
c.stopAfter = msgsLeft
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
return sub, nil
|
||||
}
|
||||
|
||||
func (c *orderedConsumer) errHandler(serial int) func(cc ConsumeContext, err error) {
|
||||
return func(cc ConsumeContext, err error) {
|
||||
c.Lock()
|
||||
|
||||
if c.userErrHandler != nil && !errors.Is(err, errOrderedSequenceMismatch) && !errors.Is(err, errConnected) {
|
||||
c.userErrHandler(cc, err)
|
||||
}
|
||||
if errors.Is(err, ErrConnectionClosed) {
|
||||
if c.subscription != nil {
|
||||
c.Unlock()
|
||||
c.subscription.Stop()
|
||||
return
|
||||
}
|
||||
c.Unlock()
|
||||
return
|
||||
}
|
||||
|
||||
if errors.Is(err, ErrNoHeartbeat) ||
|
||||
errors.Is(err, errOrderedSequenceMismatch) ||
|
||||
errors.Is(err, ErrConsumerDeleted) ||
|
||||
errors.Is(err, errConnected) ||
|
||||
errors.Is(err, nats.ErrNoResponders) {
|
||||
// only reset if serial matches the current consumer serial and there is no reset in progress
|
||||
if serial == c.serial && c.resetInProgress.Load() == 0 {
|
||||
c.resetInProgress.Store(1)
|
||||
c.doReset <- struct{}{}
|
||||
}
|
||||
}
|
||||
c.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
// Messages returns MessagesContext, allowing continuously iterating
|
||||
// over messages on a stream. Messages cannot be used concurrently
|
||||
// when using ordered consumer.
|
||||
//
|
||||
// See [Consumer.Messages] for more details.
|
||||
func (c *orderedConsumer) Messages(opts ...PullMessagesOpt) (MessagesContext, error) {
|
||||
if (c.consumerType == consumerTypeNotSet || c.consumerType == consumerTypeConsume) && c.currentConsumer == nil {
|
||||
err := c.reset()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if c.consumerType == consumerTypeConsume && c.currentConsumer != nil {
|
||||
return nil, ErrOrderedConsumerConcurrentRequests
|
||||
}
|
||||
if c.consumerType == consumerTypeFetch {
|
||||
return nil, ErrOrderConsumerUsedAsFetch
|
||||
}
|
||||
c.consumerType = consumerTypeConsume
|
||||
consumeOpts, err := parseMessagesOpts(true, opts...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %s", ErrInvalidOption, err)
|
||||
}
|
||||
opts = append(opts,
|
||||
WithMessagesErrOnMissingHeartbeat(true),
|
||||
messagesReconnectNotify())
|
||||
c.stopAfterMsgsLeft = make(chan int, 1)
|
||||
if consumeOpts.StopAfter > 0 {
|
||||
c.withStopAfter = true
|
||||
c.stopAfter = consumeOpts.StopAfter
|
||||
}
|
||||
c.userErrHandler = consumeOpts.ErrHandler
|
||||
if c.stopAfter > 0 {
|
||||
opts = append(opts, messagesStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
|
||||
}
|
||||
cc, err := c.currentConsumer.Messages(opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.currentSub = cc.(*pullSubscription)
|
||||
|
||||
sub := &orderedSubscription{
|
||||
consumer: c,
|
||||
opts: opts,
|
||||
done: make(chan struct{}, 1),
|
||||
}
|
||||
c.subscription = sub
|
||||
|
||||
return sub, nil
|
||||
}
|
||||
|
||||
func (s *orderedSubscription) Next(opts ...NextOpt) (Msg, error) {
|
||||
for {
|
||||
msg, err := s.consumer.currentSub.Next(opts...)
|
||||
if err != nil {
|
||||
// Check for errors which should be returned directly
|
||||
// without resetting the consumer
|
||||
if errors.Is(err, ErrInvalidOption) {
|
||||
return nil, err
|
||||
}
|
||||
if errors.Is(err, nats.ErrTimeout) {
|
||||
return nil, err
|
||||
}
|
||||
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
|
||||
return nil, err
|
||||
}
|
||||
if errors.Is(err, ErrMsgIteratorClosed) {
|
||||
s.Stop()
|
||||
return nil, err
|
||||
}
|
||||
if s.consumer.withStopAfter {
|
||||
select {
|
||||
case s.consumer.stopAfter = <-s.consumer.stopAfterMsgsLeft:
|
||||
default:
|
||||
}
|
||||
if s.consumer.stopAfter <= 0 {
|
||||
s.Stop()
|
||||
return nil, ErrMsgIteratorClosed
|
||||
}
|
||||
s.opts[len(s.opts)-1] = StopAfter(s.consumer.stopAfter)
|
||||
}
|
||||
if err := s.consumer.reset(); err != nil {
|
||||
if errors.Is(err, errOrderedConsumerClosed) {
|
||||
return nil, ErrMsgIteratorClosed
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
cc, err := s.consumer.currentConsumer.Messages(s.opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.consumer.currentSub = cc.(*pullSubscription)
|
||||
continue
|
||||
}
|
||||
|
||||
meta, err := msg.Metadata()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
serial := serialNumberFromConsumer(meta.Consumer)
|
||||
if serial != s.consumer.serial {
|
||||
continue
|
||||
}
|
||||
dseq := meta.Sequence.Consumer
|
||||
if dseq != s.consumer.cursor.deliverSeq+1 {
|
||||
if err := s.consumer.reset(); err != nil {
|
||||
if errors.Is(err, errOrderedConsumerClosed) {
|
||||
return nil, ErrMsgIteratorClosed
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
cc, err := s.consumer.currentConsumer.Messages(s.opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.consumer.currentSub = cc.(*pullSubscription)
|
||||
continue
|
||||
}
|
||||
s.consumer.cursor.deliverSeq = dseq
|
||||
s.consumer.cursor.streamSeq = meta.Sequence.Stream
|
||||
return msg, nil
|
||||
}
|
||||
}
|
||||
|
||||
func (s *orderedSubscription) Stop() {
|
||||
if !s.closed.CompareAndSwap(0, 1) {
|
||||
return
|
||||
}
|
||||
s.consumer.Lock()
|
||||
defer s.consumer.Unlock()
|
||||
if s.consumer.currentSub != nil {
|
||||
s.consumer.currentSub.Stop()
|
||||
}
|
||||
close(s.done)
|
||||
}
|
||||
|
||||
func (s *orderedSubscription) Drain() {
|
||||
if !s.closed.CompareAndSwap(0, 1) {
|
||||
return
|
||||
}
|
||||
s.consumer.Lock()
|
||||
defer s.consumer.Unlock()
|
||||
if s.consumer.currentSub != nil {
|
||||
s.consumer.currentSub.Drain()
|
||||
}
|
||||
close(s.done)
|
||||
}
|
||||
|
||||
// Closed returns a channel that is closed when the consuming is
|
||||
// fully stopped/drained. When the channel is closed, no more messages
|
||||
// will be received and processing is complete.
|
||||
func (s *orderedSubscription) Closed() <-chan struct{} {
|
||||
closedCh := make(chan struct{})
|
||||
|
||||
go func() {
|
||||
// First wait for s.done to be closed
|
||||
<-s.done
|
||||
|
||||
// Then ensure underlying consumer is also closed (it may still be draining)
|
||||
s.consumer.Lock()
|
||||
if s.consumer.currentSub != nil {
|
||||
closed := s.consumer.currentSub.Closed()
|
||||
s.consumer.Unlock()
|
||||
<-closed
|
||||
} else {
|
||||
s.consumer.Unlock()
|
||||
}
|
||||
|
||||
close(closedCh)
|
||||
}()
|
||||
return closedCh
|
||||
}
|
||||
|
||||
// Fetch is used to retrieve up to a provided number of messages from a
|
||||
// stream. This method will always send a single request and wait until
|
||||
// either all messages are retrieved or request times out.
|
||||
//
|
||||
// It is not efficient to use Fetch with on an ordered consumer, as it will
|
||||
// reset the consumer for each subsequent Fetch call.
|
||||
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
|
||||
func (c *orderedConsumer) Fetch(batch int, opts ...FetchOpt) (MessageBatch, error) {
|
||||
c.Lock()
|
||||
if c.consumerType == consumerTypeConsume {
|
||||
c.Unlock()
|
||||
return nil, ErrOrderConsumerUsedAsConsume
|
||||
}
|
||||
if c.runningFetch != nil {
|
||||
if !c.runningFetch.closed() {
|
||||
return nil, ErrOrderedConsumerConcurrentRequests
|
||||
}
|
||||
if c.runningFetch.sseq != 0 {
|
||||
c.cursor.streamSeq = c.runningFetch.sseq
|
||||
}
|
||||
}
|
||||
c.consumerType = consumerTypeFetch
|
||||
sub := orderedSubscription{
|
||||
consumer: c,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
c.subscription = &sub
|
||||
c.Unlock()
|
||||
err := c.reset()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msgs, err := c.currentConsumer.Fetch(batch, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.runningFetch = msgs.(*fetchResult)
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// FetchBytes is used to retrieve up to a provided bytes from the
|
||||
// stream. This method will always send a single request and wait until
|
||||
// provided number of bytes is exceeded or request times out.
|
||||
//
|
||||
// It is not efficient to use FetchBytes with on an ordered consumer, as it will
|
||||
// reset the consumer for each subsequent Fetch call.
|
||||
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
|
||||
func (c *orderedConsumer) FetchBytes(maxBytes int, opts ...FetchOpt) (MessageBatch, error) {
|
||||
c.Lock()
|
||||
if c.consumerType == consumerTypeConsume {
|
||||
c.Unlock()
|
||||
return nil, ErrOrderConsumerUsedAsConsume
|
||||
}
|
||||
if c.runningFetch != nil {
|
||||
if !c.runningFetch.closed() {
|
||||
return nil, ErrOrderedConsumerConcurrentRequests
|
||||
}
|
||||
if c.runningFetch.sseq != 0 {
|
||||
c.cursor.streamSeq = c.runningFetch.sseq
|
||||
}
|
||||
}
|
||||
c.consumerType = consumerTypeFetch
|
||||
sub := orderedSubscription{
|
||||
consumer: c,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
c.subscription = &sub
|
||||
c.Unlock()
|
||||
err := c.reset()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msgs, err := c.currentConsumer.FetchBytes(maxBytes, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.runningFetch = msgs.(*fetchResult)
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// FetchNoWait is used to retrieve up to a provided number of messages
|
||||
// from a stream. This method will always send a single request and
|
||||
// immediately return up to a provided number of messages or wait until
|
||||
// at least one message is available or request times out.
|
||||
//
|
||||
// It is not efficient to use FetchNoWait with on an ordered consumer, as it will
|
||||
// reset the consumer for each subsequent Fetch call.
|
||||
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
|
||||
func (c *orderedConsumer) FetchNoWait(batch int) (MessageBatch, error) {
|
||||
if c.consumerType == consumerTypeConsume {
|
||||
return nil, ErrOrderConsumerUsedAsConsume
|
||||
}
|
||||
if c.runningFetch != nil && !c.runningFetch.done {
|
||||
return nil, ErrOrderedConsumerConcurrentRequests
|
||||
}
|
||||
c.consumerType = consumerTypeFetch
|
||||
sub := orderedSubscription{
|
||||
consumer: c,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
c.subscription = &sub
|
||||
err := c.reset()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.currentConsumer.FetchNoWait(batch)
|
||||
}
|
||||
|
||||
// Next is used to retrieve the next message from the stream. This
|
||||
// method will block until the message is retrieved or timeout is
|
||||
// reached.
|
||||
//
|
||||
// It is not efficient to use Next with on an ordered consumer, as it will
|
||||
// reset the consumer for each subsequent Fetch call.
|
||||
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
|
||||
func (c *orderedConsumer) Next(opts ...FetchOpt) (Msg, error) {
|
||||
res, err := c.Fetch(1, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msg := <-res.Messages()
|
||||
if msg != nil {
|
||||
return msg, nil
|
||||
}
|
||||
if res.Error() == nil {
|
||||
return nil, nats.ErrTimeout
|
||||
}
|
||||
return nil, res.Error()
|
||||
}
|
||||
|
||||
func serialNumberFromConsumer(name string) int {
|
||||
if len(name) == 0 {
|
||||
return 0
|
||||
}
|
||||
parts := strings.Split(name, "_")
|
||||
if len(parts) < 2 {
|
||||
return 0
|
||||
}
|
||||
serial, err := strconv.Atoi(parts[len(parts)-1])
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return serial
|
||||
}
|
||||
|
||||
func (c *orderedConsumer) reset() error {
|
||||
c.Lock()
|
||||
defer c.Unlock()
|
||||
defer c.resetInProgress.Store(0)
|
||||
if c.currentConsumer != nil {
|
||||
c.currentConsumer.Lock()
|
||||
if c.currentSub != nil {
|
||||
c.currentSub.Stop()
|
||||
}
|
||||
consName := c.currentConsumer.CachedInfo().Name
|
||||
c.currentConsumer.Unlock()
|
||||
go func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
_ = c.js.DeleteConsumer(ctx, c.stream, consName)
|
||||
cancel()
|
||||
}()
|
||||
}
|
||||
|
||||
c.cursor.deliverSeq = 0
|
||||
consumerConfig := c.getConsumerConfig()
|
||||
|
||||
var err error
|
||||
var cons Consumer
|
||||
|
||||
backoffOpts := backoffOpts{
|
||||
attempts: c.cfg.MaxResetAttempts,
|
||||
initialInterval: time.Second,
|
||||
factor: 2,
|
||||
maxInterval: 10 * time.Second,
|
||||
cancel: c.subscription.done,
|
||||
}
|
||||
err = retryWithBackoff(func(attempt int) (bool, error) {
|
||||
isClosed := c.subscription.closed.Load() == 1
|
||||
if isClosed {
|
||||
return false, errOrderedConsumerClosed
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
cons, err = c.js.CreateOrUpdateConsumer(ctx, c.stream, *consumerConfig)
|
||||
if err != nil {
|
||||
return true, err
|
||||
}
|
||||
return false, nil
|
||||
}, backoffOpts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.currentConsumer = cons.(*pullConsumer)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *orderedConsumer) getConsumerConfig() *ConsumerConfig {
|
||||
c.serial++
|
||||
var nextSeq uint64
|
||||
|
||||
// if stream sequence is not initialized, no message was consumed yet
|
||||
// therefore, start from the beginning (either from 1 or from the provided sequence)
|
||||
if c.cursor.streamSeq == 0 {
|
||||
if c.cfg.OptStartSeq != 0 {
|
||||
nextSeq = c.cfg.OptStartSeq
|
||||
} else {
|
||||
nextSeq = 1
|
||||
}
|
||||
} else {
|
||||
// otherwise, start from the next sequence
|
||||
nextSeq = c.cursor.streamSeq + 1
|
||||
}
|
||||
|
||||
if c.cfg.MaxResetAttempts == 0 {
|
||||
c.cfg.MaxResetAttempts = -1
|
||||
}
|
||||
name := fmt.Sprintf("%s_%d", c.namePrefix, c.serial)
|
||||
cfg := &ConsumerConfig{
|
||||
Name: name,
|
||||
DeliverPolicy: DeliverByStartSequencePolicy,
|
||||
OptStartSeq: nextSeq,
|
||||
AckPolicy: AckNonePolicy,
|
||||
InactiveThreshold: 5 * time.Minute,
|
||||
Replicas: 1,
|
||||
HeadersOnly: c.cfg.HeadersOnly,
|
||||
MemoryStorage: true,
|
||||
Metadata: c.cfg.Metadata,
|
||||
}
|
||||
if len(c.cfg.FilterSubjects) == 1 {
|
||||
cfg.FilterSubject = c.cfg.FilterSubjects[0]
|
||||
} else {
|
||||
cfg.FilterSubjects = c.cfg.FilterSubjects
|
||||
}
|
||||
if c.cfg.InactiveThreshold != 0 {
|
||||
cfg.InactiveThreshold = c.cfg.InactiveThreshold
|
||||
}
|
||||
|
||||
// if the cursor is not yet set, use the provided deliver policy
|
||||
if c.cursor.streamSeq != 0 {
|
||||
return cfg
|
||||
}
|
||||
|
||||
// initial request, some options may be modified at that point
|
||||
cfg.DeliverPolicy = c.cfg.DeliverPolicy
|
||||
if c.cfg.DeliverPolicy == DeliverLastPerSubjectPolicy ||
|
||||
c.cfg.DeliverPolicy == DeliverLastPolicy ||
|
||||
c.cfg.DeliverPolicy == DeliverNewPolicy ||
|
||||
c.cfg.DeliverPolicy == DeliverAllPolicy {
|
||||
|
||||
cfg.OptStartSeq = 0
|
||||
} else if c.cfg.DeliverPolicy == DeliverByStartTimePolicy {
|
||||
cfg.OptStartSeq = 0
|
||||
cfg.OptStartTime = c.cfg.OptStartTime
|
||||
} else {
|
||||
cfg.OptStartSeq = c.cfg.OptStartSeq
|
||||
}
|
||||
|
||||
if cfg.DeliverPolicy == DeliverLastPerSubjectPolicy && len(c.cfg.FilterSubjects) == 0 {
|
||||
cfg.FilterSubjects = []string{">"}
|
||||
}
|
||||
|
||||
return cfg
|
||||
}
|
||||
|
||||
func consumeStopAfterNotify(numMsgs int, msgsLeftAfterStop chan int) PullConsumeOpt {
|
||||
return pullOptFunc(func(opts *consumeOpts) error {
|
||||
opts.StopAfter = numMsgs
|
||||
opts.stopAfterMsgsLeft = msgsLeftAfterStop
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func messagesStopAfterNotify(numMsgs int, msgsLeftAfterStop chan int) PullMessagesOpt {
|
||||
return pullOptFunc(func(opts *consumeOpts) error {
|
||||
opts.StopAfter = numMsgs
|
||||
opts.stopAfterMsgsLeft = msgsLeftAfterStop
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func consumeReconnectNotify() PullConsumeOpt {
|
||||
return pullOptFunc(func(opts *consumeOpts) error {
|
||||
opts.notifyOnReconnect = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func messagesReconnectNotify() PullMessagesOpt {
|
||||
return pullOptFunc(func(opts *consumeOpts) error {
|
||||
opts.notifyOnReconnect = true
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// Info returns information about the ordered consumer.
|
||||
// Note that this method will fetch the latest instance of the
|
||||
// consumer from the server, which can be deleted by the library at any time.
|
||||
func (c *orderedConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
|
||||
c.Lock()
|
||||
defer c.Unlock()
|
||||
if c.currentConsumer == nil {
|
||||
return nil, ErrOrderedConsumerNotCreated
|
||||
}
|
||||
infoSubject := fmt.Sprintf(apiConsumerInfoT, c.stream, c.currentConsumer.name)
|
||||
var resp consumerInfoResponse
|
||||
|
||||
if _, err := c.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
if resp.Error == nil && resp.ConsumerInfo == nil {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
|
||||
c.currentConsumer.info = resp.ConsumerInfo
|
||||
return resp.ConsumerInfo, nil
|
||||
}
|
||||
|
||||
// CachedInfo returns cached information about the consumer currently
|
||||
// used by the ordered consumer. Cached info will be updated on every call
|
||||
// to [Consumer.Info] or on consumer reset.
|
||||
func (c *orderedConsumer) CachedInfo() *ConsumerInfo {
|
||||
c.Lock()
|
||||
defer c.Unlock()
|
||||
if c.currentConsumer == nil {
|
||||
return nil
|
||||
}
|
||||
return c.currentConsumer.info
|
||||
}
|
||||
|
||||
type backoffOpts struct {
|
||||
// total retry attempts
|
||||
// -1 for unlimited
|
||||
attempts int
|
||||
// initial interval after which first retry will be performed
|
||||
// defaults to 1s
|
||||
initialInterval time.Duration
|
||||
// determines whether first function execution should be performed immediately
|
||||
disableInitialExecution bool
|
||||
// multiplier on each attempt
|
||||
// defaults to 2
|
||||
factor float64
|
||||
// max interval between retries
|
||||
// after reaching this value, all subsequent
|
||||
// retries will be performed with this interval
|
||||
// defaults to 1 minute
|
||||
maxInterval time.Duration
|
||||
// custom backoff intervals
|
||||
// if set, overrides all other options except attempts
|
||||
// if attempts are set, then the last interval will be used
|
||||
// for all subsequent retries after reaching the limit
|
||||
customBackoff []time.Duration
|
||||
// cancel channel
|
||||
// if set, retry will be canceled when this channel is closed
|
||||
cancel <-chan struct{}
|
||||
}
|
||||
|
||||
func retryWithBackoff(f func(int) (bool, error), opts backoffOpts) error {
|
||||
var err error
|
||||
var shouldContinue bool
|
||||
// if custom backoff is set, use it instead of other options
|
||||
if len(opts.customBackoff) > 0 {
|
||||
if opts.attempts != 0 {
|
||||
return errors.New("cannot use custom backoff intervals when attempts are set")
|
||||
}
|
||||
for i, interval := range opts.customBackoff {
|
||||
select {
|
||||
case <-opts.cancel:
|
||||
return nil
|
||||
case <-time.After(interval):
|
||||
}
|
||||
shouldContinue, err = f(i)
|
||||
if !shouldContinue {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// set default options
|
||||
if opts.initialInterval == 0 {
|
||||
opts.initialInterval = 1 * time.Second
|
||||
}
|
||||
if opts.factor == 0 {
|
||||
opts.factor = 2
|
||||
}
|
||||
if opts.maxInterval == 0 {
|
||||
opts.maxInterval = 1 * time.Minute
|
||||
}
|
||||
if opts.attempts == 0 {
|
||||
return errors.New("retry attempts have to be set when not using custom backoff intervals")
|
||||
}
|
||||
interval := opts.initialInterval
|
||||
for i := 0; ; i++ {
|
||||
if i == 0 && opts.disableInitialExecution {
|
||||
time.Sleep(interval)
|
||||
continue
|
||||
}
|
||||
shouldContinue, err = f(i)
|
||||
if !shouldContinue {
|
||||
return err
|
||||
}
|
||||
if opts.attempts > 0 && i >= opts.attempts-1 {
|
||||
break
|
||||
}
|
||||
select {
|
||||
case <-opts.cancel:
|
||||
return nil
|
||||
case <-time.After(interval):
|
||||
}
|
||||
interval = time.Duration(float64(interval) * opts.factor)
|
||||
if interval >= opts.maxInterval {
|
||||
interval = opts.maxInterval
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
+672
@@ -0,0 +1,672 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nuid"
|
||||
)
|
||||
|
||||
type (
|
||||
asyncPublisherOpts struct {
|
||||
// For async publish error handling.
|
||||
aecb MsgErrHandler
|
||||
// Max async pub ack in flight
|
||||
maxpa int
|
||||
// ackTimeout is the max time to wait for an ack.
|
||||
ackTimeout time.Duration
|
||||
}
|
||||
|
||||
// PublishOpt are the options that can be passed to Publish methods.
|
||||
PublishOpt func(*pubOpts) error
|
||||
|
||||
pubOpts struct {
|
||||
id string
|
||||
lastMsgID string // Expected last msgId
|
||||
stream string // Expected stream name
|
||||
lastSeq *uint64 // Expected last sequence
|
||||
lastSubjectSeq *uint64 // Expected last sequence for subject
|
||||
lastSubject string // Expected subject for last sequence
|
||||
ttl time.Duration // Message TTL
|
||||
|
||||
// Publish retries for NoResponders err.
|
||||
retryWait time.Duration // Retry wait between attempts
|
||||
retryAttempts int // Retry attempts
|
||||
|
||||
// stallWait is the max wait of a async pub ack.
|
||||
stallWait time.Duration
|
||||
|
||||
// internal option to re-use existing paf in case of retry.
|
||||
pafRetry *pubAckFuture
|
||||
}
|
||||
|
||||
// PubAckFuture is a future for a PubAck.
|
||||
// It can be used to wait for a PubAck or an error after an async publish.
|
||||
PubAckFuture interface {
|
||||
// Ok returns a receive only channel that can be used to get a PubAck.
|
||||
Ok() <-chan *PubAck
|
||||
|
||||
// Err returns a receive only channel that can be used to get the error from an async publish.
|
||||
Err() <-chan error
|
||||
|
||||
// Msg returns the message that was sent to the server.
|
||||
Msg() *nats.Msg
|
||||
}
|
||||
|
||||
pubAckFuture struct {
|
||||
jsClient *jetStreamClient
|
||||
msg *nats.Msg
|
||||
retries int
|
||||
maxRetries int
|
||||
retryWait time.Duration
|
||||
ack *PubAck
|
||||
err error
|
||||
errCh chan error
|
||||
doneCh chan *PubAck
|
||||
reply string
|
||||
timeout *time.Timer
|
||||
}
|
||||
|
||||
jetStreamClient struct {
|
||||
asyncPublishContext
|
||||
asyncPublisherOpts
|
||||
}
|
||||
|
||||
// MsgErrHandler is used to process asynchronous errors from JetStream
|
||||
// PublishAsync. It will return the original message sent to the server for
|
||||
// possible retransmitting and the error encountered.
|
||||
MsgErrHandler func(JetStream, *nats.Msg, error)
|
||||
|
||||
asyncPublishContext struct {
|
||||
sync.RWMutex
|
||||
replyPrefix string
|
||||
replySub *nats.Subscription
|
||||
acks map[string]*pubAckFuture
|
||||
stallCh chan struct{}
|
||||
doneCh chan struct{}
|
||||
rr *rand.Rand
|
||||
// channel to signal when server is disconnected or conn is closed
|
||||
connStatusCh chan (nats.Status)
|
||||
}
|
||||
|
||||
pubAckResponse struct {
|
||||
apiResponse
|
||||
*PubAck
|
||||
}
|
||||
|
||||
// PubAck is an ack received after successfully publishing a message.
|
||||
PubAck struct {
|
||||
// Stream is the stream name the message was published to.
|
||||
Stream string `json:"stream"`
|
||||
|
||||
// Sequence is the stream sequence number of the message.
|
||||
Sequence uint64 `json:"seq"`
|
||||
|
||||
// Duplicate indicates whether the message was a duplicate.
|
||||
// Duplicate can be detected using the [MsgIDHeader] and [StreamConfig.Duplicates].
|
||||
Duplicate bool `json:"duplicate,omitempty"`
|
||||
|
||||
// Domain is the domain the message was published to.
|
||||
Domain string `json:"domain,omitempty"`
|
||||
|
||||
Value string `json:"val,omitempty"`
|
||||
}
|
||||
)
|
||||
|
||||
const (
|
||||
// Default time wait between retries on Publish if err is ErrNoResponders.
|
||||
DefaultPubRetryWait = 250 * time.Millisecond
|
||||
|
||||
// Default number of retries
|
||||
DefaultPubRetryAttempts = 2
|
||||
)
|
||||
|
||||
const (
|
||||
statusHdr = "Status"
|
||||
|
||||
rdigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
|
||||
base = 62
|
||||
)
|
||||
|
||||
// Publish performs a synchronous publish to a stream and waits for ack
|
||||
// from server. It accepts subject name (which must be bound to a stream)
|
||||
// and message payload.
|
||||
func (js *jetStream) Publish(ctx context.Context, subj string, data []byte, opts ...PublishOpt) (*PubAck, error) {
|
||||
return js.PublishMsg(ctx, &nats.Msg{Subject: subj, Data: data}, opts...)
|
||||
}
|
||||
|
||||
// PublishMsg performs a synchronous publish to a stream and waits for
|
||||
// ack from server. It accepts subject name (which must be bound to a
|
||||
// stream) and nats.Message.
|
||||
func (js *jetStream) PublishMsg(ctx context.Context, m *nats.Msg, opts ...PublishOpt) (*PubAck, error) {
|
||||
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
o := pubOpts{
|
||||
retryWait: DefaultPubRetryWait,
|
||||
retryAttempts: DefaultPubRetryAttempts,
|
||||
}
|
||||
if len(opts) > 0 {
|
||||
if m.Header == nil {
|
||||
m.Header = nats.Header{}
|
||||
}
|
||||
for _, opt := range opts {
|
||||
if err := opt(&o); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
if o.stallWait > 0 {
|
||||
return nil, fmt.Errorf("%w: stall wait cannot be set to sync publish", ErrInvalidOption)
|
||||
}
|
||||
|
||||
if o.id != "" {
|
||||
m.Header.Set(MsgIDHeader, o.id)
|
||||
}
|
||||
if o.lastMsgID != "" {
|
||||
m.Header.Set(ExpectedLastMsgIDHeader, o.lastMsgID)
|
||||
}
|
||||
if o.stream != "" {
|
||||
m.Header.Set(ExpectedStreamHeader, o.stream)
|
||||
}
|
||||
if o.lastSeq != nil {
|
||||
m.Header.Set(ExpectedLastSeqHeader, strconv.FormatUint(*o.lastSeq, 10))
|
||||
}
|
||||
if o.lastSubjectSeq != nil {
|
||||
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
|
||||
}
|
||||
if o.lastSubject != "" {
|
||||
m.Header.Set(ExpectedLastSubjSeqSubjHeader, o.lastSubject)
|
||||
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
|
||||
}
|
||||
if o.ttl > 0 {
|
||||
m.Header.Set(MsgTTLHeader, o.ttl.String())
|
||||
}
|
||||
|
||||
var resp *nats.Msg
|
||||
var err error
|
||||
|
||||
resp, err = js.conn.RequestMsgWithContext(ctx, m)
|
||||
|
||||
if err != nil {
|
||||
for r := 0; errors.Is(err, nats.ErrNoResponders) && (r < o.retryAttempts || o.retryAttempts < 0); r++ {
|
||||
// To protect against small blips in leadership changes etc, if we get a no responders here retry.
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
case <-time.After(o.retryWait):
|
||||
}
|
||||
resp, err = js.conn.RequestMsgWithContext(ctx, m)
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, nats.ErrNoResponders) {
|
||||
return nil, ErrNoStreamResponse
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
var ackResp pubAckResponse
|
||||
if err := json.Unmarshal(resp.Data, &ackResp); err != nil {
|
||||
return nil, ErrInvalidJSAck
|
||||
}
|
||||
if ackResp.Error != nil {
|
||||
return nil, fmt.Errorf("nats: %w", ackResp.Error)
|
||||
}
|
||||
if ackResp.PubAck == nil || ackResp.PubAck.Stream == "" {
|
||||
return nil, ErrInvalidJSAck
|
||||
}
|
||||
return ackResp.PubAck, nil
|
||||
}
|
||||
|
||||
// PublishAsync performs an asynchronous publish to a stream and returns
|
||||
// [PubAckFuture] interface. It accepts subject name (which must be bound
|
||||
// to a stream) and message payload.
|
||||
func (js *jetStream) PublishAsync(subj string, data []byte, opts ...PublishOpt) (PubAckFuture, error) {
|
||||
return js.PublishMsgAsync(&nats.Msg{Subject: subj, Data: data}, opts...)
|
||||
}
|
||||
|
||||
// PublishMsgAsync performs an asynchronous publish to a stream and
|
||||
// returns [PubAckFuture] interface. It accepts subject name (which must
|
||||
// be bound to a stream) and nats.Message.
|
||||
func (js *jetStream) PublishMsgAsync(m *nats.Msg, opts ...PublishOpt) (PubAckFuture, error) {
|
||||
o := pubOpts{
|
||||
retryWait: DefaultPubRetryWait,
|
||||
retryAttempts: DefaultPubRetryAttempts,
|
||||
}
|
||||
if len(opts) > 0 {
|
||||
if m.Header == nil {
|
||||
m.Header = nats.Header{}
|
||||
}
|
||||
for _, opt := range opts {
|
||||
if err := opt(&o); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
defaultStallWait := 200 * time.Millisecond
|
||||
|
||||
stallWait := defaultStallWait
|
||||
if o.stallWait > 0 {
|
||||
stallWait = o.stallWait
|
||||
}
|
||||
|
||||
if o.id != "" {
|
||||
m.Header.Set(MsgIDHeader, o.id)
|
||||
}
|
||||
if o.lastMsgID != "" {
|
||||
m.Header.Set(ExpectedLastMsgIDHeader, o.lastMsgID)
|
||||
}
|
||||
if o.stream != "" {
|
||||
m.Header.Set(ExpectedStreamHeader, o.stream)
|
||||
}
|
||||
if o.lastSeq != nil {
|
||||
m.Header.Set(ExpectedLastSeqHeader, strconv.FormatUint(*o.lastSeq, 10))
|
||||
}
|
||||
if o.lastSubjectSeq != nil {
|
||||
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
|
||||
}
|
||||
if o.lastSubject != "" {
|
||||
m.Header.Set(ExpectedLastSubjSeqSubjHeader, o.lastSubject)
|
||||
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
|
||||
}
|
||||
if o.ttl > 0 {
|
||||
m.Header.Set(MsgTTLHeader, o.ttl.String())
|
||||
}
|
||||
|
||||
paf := o.pafRetry
|
||||
if paf == nil && m.Reply != "" {
|
||||
return nil, ErrAsyncPublishReplySubjectSet
|
||||
}
|
||||
|
||||
var id string
|
||||
var reply string
|
||||
|
||||
// register new paf if not retrying
|
||||
if paf == nil {
|
||||
var err error
|
||||
reply, err = js.newAsyncReply()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nats: error creating async reply handler: %s", err)
|
||||
}
|
||||
id = reply[js.opts.replyPrefixLen:]
|
||||
paf = &pubAckFuture{msg: m, jsClient: js.publisher, maxRetries: o.retryAttempts, retryWait: o.retryWait, reply: reply}
|
||||
numPending, maxPending := js.registerPAF(id, paf)
|
||||
|
||||
if maxPending > 0 && numPending > maxPending {
|
||||
select {
|
||||
case <-js.asyncStall():
|
||||
case <-time.After(stallWait):
|
||||
js.clearPAF(id)
|
||||
return nil, ErrTooManyStalledMsgs
|
||||
}
|
||||
}
|
||||
if js.publisher.ackTimeout > 0 {
|
||||
paf.timeout = time.AfterFunc(js.publisher.ackTimeout, func() {
|
||||
js.publisher.Lock()
|
||||
defer js.publisher.Unlock()
|
||||
|
||||
if _, ok := js.publisher.acks[id]; !ok {
|
||||
// paf has already been resolved
|
||||
// while waiting for the lock
|
||||
return
|
||||
}
|
||||
|
||||
// ack timed out, remove from pending acks
|
||||
delete(js.publisher.acks, id)
|
||||
|
||||
// check on anyone stalled and waiting.
|
||||
if js.publisher.stallCh != nil && len(js.publisher.acks) < js.publisher.maxpa {
|
||||
close(js.publisher.stallCh)
|
||||
js.publisher.stallCh = nil
|
||||
}
|
||||
|
||||
// send error to user
|
||||
paf.err = ErrAsyncPublishTimeout
|
||||
if paf.errCh != nil {
|
||||
paf.errCh <- paf.err
|
||||
}
|
||||
|
||||
// call error callback if set
|
||||
if js.publisher.asyncPublisherOpts.aecb != nil {
|
||||
js.publisher.asyncPublisherOpts.aecb(js, paf.msg, ErrAsyncPublishTimeout)
|
||||
}
|
||||
|
||||
// check on anyone one waiting on done status.
|
||||
if js.publisher.doneCh != nil && len(js.publisher.acks) == 0 {
|
||||
close(js.publisher.doneCh)
|
||||
js.publisher.doneCh = nil
|
||||
}
|
||||
})
|
||||
}
|
||||
} else {
|
||||
// when retrying, get the ID from existing reply subject
|
||||
reply = paf.reply
|
||||
if paf.timeout != nil {
|
||||
paf.timeout.Reset(js.publisher.ackTimeout)
|
||||
}
|
||||
id = reply[js.opts.replyPrefixLen:]
|
||||
}
|
||||
|
||||
pubMsg := &nats.Msg{
|
||||
Subject: m.Subject,
|
||||
Reply: reply,
|
||||
Data: m.Data,
|
||||
Header: m.Header,
|
||||
}
|
||||
if err := js.conn.PublishMsg(pubMsg); err != nil {
|
||||
js.clearPAF(id)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paf, nil
|
||||
}
|
||||
|
||||
// For quick token lookup etc.
|
||||
const (
|
||||
aReplyTokensize = 6
|
||||
)
|
||||
|
||||
func (js *jetStream) newAsyncReply() (string, error) {
|
||||
js.publisher.Lock()
|
||||
if js.publisher.replySub == nil {
|
||||
// Create our wildcard reply subject.
|
||||
sha := sha256.New()
|
||||
sha.Write([]byte(nuid.Next()))
|
||||
b := sha.Sum(nil)
|
||||
for i := 0; i < aReplyTokensize; i++ {
|
||||
b[i] = rdigits[int(b[i]%base)]
|
||||
}
|
||||
js.publisher.replyPrefix = fmt.Sprintf("%s%s.", js.opts.replyPrefix, b[:aReplyTokensize])
|
||||
sub, err := js.conn.Subscribe(fmt.Sprintf("%s*", js.publisher.replyPrefix), js.handleAsyncReply)
|
||||
if err != nil {
|
||||
js.publisher.Unlock()
|
||||
return "", err
|
||||
}
|
||||
js.publisher.replySub = sub
|
||||
js.publisher.rr = rand.New(rand.NewSource(time.Now().UnixNano()))
|
||||
}
|
||||
if js.publisher.connStatusCh == nil {
|
||||
js.publisher.connStatusCh = js.conn.StatusChanged(nats.RECONNECTING, nats.CLOSED)
|
||||
go js.resetPendingAcksOnReconnect()
|
||||
}
|
||||
var sb strings.Builder
|
||||
sb.WriteString(js.publisher.replyPrefix)
|
||||
for {
|
||||
rn := js.publisher.rr.Int63()
|
||||
var b [aReplyTokensize]byte
|
||||
for i, l := 0, rn; i < len(b); i++ {
|
||||
b[i] = rdigits[l%base]
|
||||
l /= base
|
||||
}
|
||||
if _, ok := js.publisher.acks[string(b[:])]; ok {
|
||||
continue
|
||||
}
|
||||
sb.Write(b[:])
|
||||
break
|
||||
}
|
||||
|
||||
js.publisher.Unlock()
|
||||
return sb.String(), nil
|
||||
}
|
||||
|
||||
// Handle an async reply from PublishAsync.
|
||||
func (js *jetStream) handleAsyncReply(m *nats.Msg) {
|
||||
if len(m.Subject) <= js.opts.replyPrefixLen {
|
||||
return
|
||||
}
|
||||
id := m.Subject[js.opts.replyPrefixLen:]
|
||||
|
||||
js.publisher.Lock()
|
||||
|
||||
paf := js.getPAF(id)
|
||||
if paf == nil {
|
||||
js.publisher.Unlock()
|
||||
return
|
||||
}
|
||||
|
||||
closeStc := func() {
|
||||
// Check on anyone stalled and waiting.
|
||||
if js.publisher.stallCh != nil && len(js.publisher.acks) < js.publisher.maxpa {
|
||||
close(js.publisher.stallCh)
|
||||
js.publisher.stallCh = nil
|
||||
}
|
||||
}
|
||||
|
||||
closeDchFn := func() func() {
|
||||
var dch chan struct{}
|
||||
// Check on anyone one waiting on done status.
|
||||
if js.publisher.doneCh != nil && len(js.publisher.acks) == 0 {
|
||||
dch = js.publisher.doneCh
|
||||
js.publisher.doneCh = nil
|
||||
}
|
||||
// Return function to close done channel which
|
||||
// should be deferred so that error is processed and
|
||||
// can be checked.
|
||||
return func() {
|
||||
if dch != nil {
|
||||
close(dch)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
doErr := func(err error) {
|
||||
paf.err = err
|
||||
if paf.errCh != nil {
|
||||
paf.errCh <- paf.err
|
||||
}
|
||||
cb := js.publisher.asyncPublisherOpts.aecb
|
||||
js.publisher.Unlock()
|
||||
if cb != nil {
|
||||
cb(js, paf.msg, err)
|
||||
}
|
||||
}
|
||||
|
||||
if paf.timeout != nil {
|
||||
paf.timeout.Stop()
|
||||
}
|
||||
|
||||
// Process no responders etc.
|
||||
if len(m.Data) == 0 && m.Header.Get(statusHdr) == statusNoResponders {
|
||||
if paf.retries < paf.maxRetries {
|
||||
paf.retries++
|
||||
time.AfterFunc(paf.retryWait, func() {
|
||||
js.publisher.Lock()
|
||||
paf := js.getPAF(id)
|
||||
js.publisher.Unlock()
|
||||
if paf == nil {
|
||||
return
|
||||
}
|
||||
_, err := js.PublishMsgAsync(paf.msg, func(po *pubOpts) error {
|
||||
po.pafRetry = paf
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
js.publisher.Lock()
|
||||
doErr(err)
|
||||
}
|
||||
})
|
||||
js.publisher.Unlock()
|
||||
return
|
||||
}
|
||||
delete(js.publisher.acks, id)
|
||||
closeStc()
|
||||
defer closeDchFn()()
|
||||
doErr(ErrNoStreamResponse)
|
||||
return
|
||||
}
|
||||
|
||||
// Remove
|
||||
delete(js.publisher.acks, id)
|
||||
closeStc()
|
||||
defer closeDchFn()()
|
||||
|
||||
var pa pubAckResponse
|
||||
if err := json.Unmarshal(m.Data, &pa); err != nil {
|
||||
doErr(ErrInvalidJSAck)
|
||||
return
|
||||
}
|
||||
if pa.Error != nil {
|
||||
doErr(pa.Error)
|
||||
return
|
||||
}
|
||||
if pa.PubAck == nil || pa.PubAck.Stream == "" {
|
||||
doErr(ErrInvalidJSAck)
|
||||
return
|
||||
}
|
||||
|
||||
// So here we have received a proper puback.
|
||||
paf.ack = pa.PubAck
|
||||
if paf.doneCh != nil {
|
||||
paf.doneCh <- paf.ack
|
||||
}
|
||||
js.publisher.Unlock()
|
||||
}
|
||||
|
||||
func (js *jetStream) resetPendingAcksOnReconnect() {
|
||||
js.publisher.Lock()
|
||||
connStatusCh := js.publisher.connStatusCh
|
||||
js.publisher.Unlock()
|
||||
for {
|
||||
newStatus, ok := <-connStatusCh
|
||||
if !ok || newStatus == nats.CLOSED {
|
||||
return
|
||||
}
|
||||
js.publisher.Lock()
|
||||
errCb := js.publisher.asyncPublisherOpts.aecb
|
||||
for id, paf := range js.publisher.acks {
|
||||
paf.err = nats.ErrDisconnected
|
||||
if paf.errCh != nil {
|
||||
paf.errCh <- paf.err
|
||||
}
|
||||
if errCb != nil {
|
||||
defer errCb(js, paf.msg, nats.ErrDisconnected)
|
||||
}
|
||||
delete(js.publisher.acks, id)
|
||||
}
|
||||
if js.publisher.doneCh != nil {
|
||||
close(js.publisher.doneCh)
|
||||
js.publisher.doneCh = nil
|
||||
}
|
||||
js.publisher.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
// registerPAF will register for a PubAckFuture.
|
||||
func (js *jetStream) registerPAF(id string, paf *pubAckFuture) (int, int) {
|
||||
js.publisher.Lock()
|
||||
if js.publisher.acks == nil {
|
||||
js.publisher.acks = make(map[string]*pubAckFuture)
|
||||
}
|
||||
js.publisher.acks[id] = paf
|
||||
np := len(js.publisher.acks)
|
||||
maxpa := js.publisher.asyncPublisherOpts.maxpa
|
||||
js.publisher.Unlock()
|
||||
return np, maxpa
|
||||
}
|
||||
|
||||
// Lock should be held.
|
||||
func (js *jetStream) getPAF(id string) *pubAckFuture {
|
||||
if js.publisher.acks == nil {
|
||||
return nil
|
||||
}
|
||||
return js.publisher.acks[id]
|
||||
}
|
||||
|
||||
// clearPAF will remove a PubAckFuture that was registered.
|
||||
func (js *jetStream) clearPAF(id string) {
|
||||
js.publisher.Lock()
|
||||
delete(js.publisher.acks, id)
|
||||
js.publisher.Unlock()
|
||||
}
|
||||
|
||||
func (js *jetStream) asyncStall() <-chan struct{} {
|
||||
js.publisher.Lock()
|
||||
if js.publisher.stallCh == nil {
|
||||
js.publisher.stallCh = make(chan struct{})
|
||||
}
|
||||
stc := js.publisher.stallCh
|
||||
js.publisher.Unlock()
|
||||
return stc
|
||||
}
|
||||
|
||||
func (paf *pubAckFuture) Ok() <-chan *PubAck {
|
||||
paf.jsClient.Lock()
|
||||
defer paf.jsClient.Unlock()
|
||||
|
||||
if paf.doneCh == nil {
|
||||
paf.doneCh = make(chan *PubAck, 1)
|
||||
if paf.ack != nil {
|
||||
paf.doneCh <- paf.ack
|
||||
}
|
||||
}
|
||||
|
||||
return paf.doneCh
|
||||
}
|
||||
|
||||
func (paf *pubAckFuture) Err() <-chan error {
|
||||
paf.jsClient.Lock()
|
||||
defer paf.jsClient.Unlock()
|
||||
|
||||
if paf.errCh == nil {
|
||||
paf.errCh = make(chan error, 1)
|
||||
if paf.err != nil {
|
||||
paf.errCh <- paf.err
|
||||
}
|
||||
}
|
||||
|
||||
return paf.errCh
|
||||
}
|
||||
|
||||
func (paf *pubAckFuture) Msg() *nats.Msg {
|
||||
paf.jsClient.RLock()
|
||||
defer paf.jsClient.RUnlock()
|
||||
return paf.msg
|
||||
}
|
||||
|
||||
// PublishAsyncPending returns the number of async publishes outstanding
|
||||
// for this context.
|
||||
func (js *jetStream) PublishAsyncPending() int {
|
||||
js.publisher.RLock()
|
||||
defer js.publisher.RUnlock()
|
||||
return len(js.publisher.acks)
|
||||
}
|
||||
|
||||
// PublishAsyncComplete returns a channel that will be closed when all
|
||||
// outstanding asynchronously published messages are acknowledged by the
|
||||
// server.
|
||||
func (js *jetStream) PublishAsyncComplete() <-chan struct{} {
|
||||
js.publisher.Lock()
|
||||
defer js.publisher.Unlock()
|
||||
if js.publisher.doneCh == nil {
|
||||
js.publisher.doneCh = make(chan struct{})
|
||||
}
|
||||
dch := js.publisher.doneCh
|
||||
if len(js.publisher.acks) == 0 {
|
||||
close(js.publisher.doneCh)
|
||||
js.publisher.doneCh = nil
|
||||
}
|
||||
return dch
|
||||
}
|
||||
+1224
File diff suppressed because it is too large
Load Diff
+268
@@ -0,0 +1,268 @@
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nuid"
|
||||
)
|
||||
|
||||
type (
|
||||
pushConsumer struct {
|
||||
sync.Mutex
|
||||
js *jetStream
|
||||
stream string
|
||||
name string
|
||||
info *ConsumerInfo
|
||||
started atomic.Bool
|
||||
}
|
||||
|
||||
pushSubscription struct {
|
||||
sync.Mutex
|
||||
id string
|
||||
errs chan error
|
||||
subscription *nats.Subscription
|
||||
connStatusChanged chan nats.Status
|
||||
closedCh chan struct{}
|
||||
done chan struct{}
|
||||
closed atomic.Bool
|
||||
consumeOpts *pushConsumeOpts
|
||||
hbMonitor *hbMonitor
|
||||
idleHeartbeat time.Duration
|
||||
}
|
||||
|
||||
pushConsumeOpts struct {
|
||||
ErrHandler ConsumeErrHandler
|
||||
}
|
||||
|
||||
PushConsumeOpt interface {
|
||||
configurePushConsume(*pushConsumeOpts) error
|
||||
}
|
||||
)
|
||||
|
||||
func (p *pushConsumer) Consume(handler MessageHandler, opts ...PushConsumeOpt) (ConsumeContext, error) {
|
||||
if handler == nil {
|
||||
return nil, ErrHandlerRequired
|
||||
}
|
||||
consumeOpts := &pushConsumeOpts{}
|
||||
for _, opt := range opts {
|
||||
if err := opt.configurePushConsume(consumeOpts); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
p.Lock()
|
||||
defer p.Unlock()
|
||||
|
||||
if p.info == nil {
|
||||
return nil, ErrConsumerNotFound
|
||||
}
|
||||
|
||||
if p.started.Load() {
|
||||
return nil, ErrConsumerAlreadyConsuming
|
||||
}
|
||||
|
||||
consumeID := nuid.Next()
|
||||
sub := &pushSubscription{
|
||||
id: consumeID,
|
||||
errs: make(chan error, 1),
|
||||
done: make(chan struct{}, 1),
|
||||
consumeOpts: consumeOpts,
|
||||
connStatusChanged: p.js.conn.StatusChanged(nats.CONNECTED, nats.RECONNECTING),
|
||||
idleHeartbeat: p.info.Config.IdleHeartbeat,
|
||||
}
|
||||
|
||||
sub.hbMonitor = sub.scheduleHeartbeatCheck(sub.idleHeartbeat)
|
||||
internalHandler := func(msg *nats.Msg) {
|
||||
if sub.hbMonitor != nil {
|
||||
sub.hbMonitor.Stop()
|
||||
}
|
||||
defer func() {
|
||||
if sub.hbMonitor != nil {
|
||||
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
|
||||
}
|
||||
}()
|
||||
status, descr := msg.Header.Get("Status"), msg.Header.Get("Description")
|
||||
if status == "" {
|
||||
jsMsg := p.js.toJSMsg(msg)
|
||||
handler(jsMsg)
|
||||
return
|
||||
}
|
||||
sub.Lock()
|
||||
if err, terminate := sub.handleStatusMsg(msg, status, descr); err != nil {
|
||||
if sub.consumeOpts.ErrHandler != nil {
|
||||
sub.consumeOpts.ErrHandler(sub, err)
|
||||
}
|
||||
if terminate {
|
||||
sub.Stop()
|
||||
}
|
||||
}
|
||||
sub.Unlock()
|
||||
}
|
||||
|
||||
var err error
|
||||
if p.info.Config.DeliverGroup != "" {
|
||||
sub.subscription, err = p.js.conn.QueueSubscribe(p.info.Config.DeliverSubject, p.info.Config.DeliverGroup, internalHandler)
|
||||
} else {
|
||||
sub.subscription, err = p.js.conn.Subscribe(p.info.Config.DeliverSubject, internalHandler)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
sub.subscription.SetClosedHandler(func(sid string) func(string) {
|
||||
return func(subject string) {
|
||||
p.started.Store(false)
|
||||
sub.Lock()
|
||||
defer sub.Unlock()
|
||||
if sub.closedCh != nil {
|
||||
close(sub.closedCh)
|
||||
sub.closedCh = nil
|
||||
}
|
||||
}
|
||||
}(sub.id))
|
||||
|
||||
go func() {
|
||||
isConnected := true
|
||||
for {
|
||||
if sub.closed.Load() {
|
||||
return
|
||||
}
|
||||
select {
|
||||
case status, ok := <-sub.connStatusChanged:
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if status == nats.RECONNECTING {
|
||||
if sub.hbMonitor != nil {
|
||||
sub.hbMonitor.Stop()
|
||||
}
|
||||
isConnected = false
|
||||
}
|
||||
if status == nats.CONNECTED {
|
||||
sub.Lock()
|
||||
if !isConnected {
|
||||
isConnected = true
|
||||
|
||||
if sub.hbMonitor != nil {
|
||||
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
|
||||
}
|
||||
}
|
||||
sub.Unlock()
|
||||
}
|
||||
case err := <-sub.errs:
|
||||
sub.Lock()
|
||||
if sub.consumeOpts.ErrHandler != nil {
|
||||
sub.consumeOpts.ErrHandler(sub, err)
|
||||
}
|
||||
if errors.Is(err, ErrNoHeartbeat) {
|
||||
if sub.hbMonitor != nil {
|
||||
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
|
||||
}
|
||||
}
|
||||
sub.Unlock()
|
||||
case <-sub.done:
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
p.started.Store(true)
|
||||
|
||||
return sub, nil
|
||||
|
||||
}
|
||||
|
||||
func (s *pushSubscription) handleStatusMsg(msg *nats.Msg, status, description string) (error, bool) {
|
||||
switch status {
|
||||
case statusControlMsg:
|
||||
switch strings.ToLower(description) {
|
||||
case idleHeartbeatDescr:
|
||||
return nil, false
|
||||
case fcRequestDescr:
|
||||
if err := msg.Respond(nil); err != nil {
|
||||
if s.consumeOpts.ErrHandler != nil {
|
||||
s.consumeOpts.ErrHandler(s, err)
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
case statusConflict:
|
||||
if description == consumerDeleted {
|
||||
return ErrConsumerDeleted, true
|
||||
}
|
||||
if description == leadershipChange {
|
||||
if s.consumeOpts.ErrHandler != nil {
|
||||
s.consumeOpts.ErrHandler(s, ErrConsumerLeadershipChanged)
|
||||
return ErrConsumerLeadershipChanged, false
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// Stop unsubscribes from the stream and cancels subscription.
|
||||
// No more messages will be received after calling this method.
|
||||
// All messages that are already in the buffer are discarded.
|
||||
func (s *pushSubscription) Stop() {
|
||||
if !s.closed.CompareAndSwap(false, true) {
|
||||
return
|
||||
}
|
||||
s.Lock()
|
||||
defer s.Unlock()
|
||||
close(s.done)
|
||||
s.subscription.Unsubscribe()
|
||||
if s.hbMonitor != nil {
|
||||
s.hbMonitor.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
// Drain unsubscribes from the stream and cancels subscription.
|
||||
// All messages that are already in the buffer will be processed in callback function.
|
||||
func (s *pushSubscription) Drain() {
|
||||
if !s.closed.CompareAndSwap(false, true) {
|
||||
return
|
||||
}
|
||||
s.Lock()
|
||||
defer s.Unlock()
|
||||
close(s.done)
|
||||
s.subscription.Drain()
|
||||
if s.hbMonitor != nil {
|
||||
s.hbMonitor.Stop()
|
||||
}
|
||||
}
|
||||
|
||||
// Closed returns a channel that is closed when consuming is
|
||||
// fully stopped/drained. When the channel is closed, no more messages
|
||||
// will be received and processing is complete.
|
||||
func (s *pushSubscription) Closed() <-chan struct{} {
|
||||
s.Lock()
|
||||
defer s.Unlock()
|
||||
ch := s.closedCh
|
||||
if ch == nil {
|
||||
ch = make(chan struct{})
|
||||
s.closedCh = ch
|
||||
}
|
||||
if !s.subscription.IsValid() {
|
||||
close(s.closedCh)
|
||||
s.closedCh = nil
|
||||
}
|
||||
return ch
|
||||
}
|
||||
|
||||
func (s *pushSubscription) scheduleHeartbeatCheck(dur time.Duration) *hbMonitor {
|
||||
if dur == 0 {
|
||||
return nil
|
||||
}
|
||||
return &hbMonitor{
|
||||
timer: time.AfterFunc(2*dur, func() {
|
||||
s.Lock()
|
||||
defer s.Unlock()
|
||||
s.errs <- ErrNoHeartbeat
|
||||
}),
|
||||
}
|
||||
}
|
||||
+827
@@ -0,0 +1,827 @@
|
||||
// Copyright 2022-2025 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nuid"
|
||||
)
|
||||
|
||||
type (
|
||||
// Stream contains CRUD methods on a consumer via [ConsumerManager], as well
|
||||
// as operations on an existing stream. It allows fetching and removing
|
||||
// messages from a stream, as well as purging a stream.
|
||||
Stream interface {
|
||||
ConsumerManager
|
||||
|
||||
// Info returns StreamInfo from the server.
|
||||
Info(ctx context.Context, opts ...StreamInfoOpt) (*StreamInfo, error)
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this stream.
|
||||
// This method does not perform any network requests. The cached
|
||||
// StreamInfo is updated on every call to Info and Update.
|
||||
CachedInfo() *StreamInfo
|
||||
|
||||
// Purge removes messages from a stream. It is a destructive operation.
|
||||
// Use with caution. See StreamPurgeOpt for available options.
|
||||
Purge(ctx context.Context, opts ...StreamPurgeOpt) error
|
||||
|
||||
// GetMsg retrieves a raw stream message stored in JetStream by sequence number.
|
||||
GetMsg(ctx context.Context, seq uint64, opts ...GetMsgOpt) (*RawStreamMsg, error)
|
||||
|
||||
// GetLastMsgForSubject retrieves the last raw stream message stored in
|
||||
// JetStream on a given subject.
|
||||
GetLastMsgForSubject(ctx context.Context, subject string) (*RawStreamMsg, error)
|
||||
|
||||
// DeleteMsg deletes a message from a stream.
|
||||
// On the server, the message is marked as erased, but not overwritten.
|
||||
DeleteMsg(ctx context.Context, seq uint64) error
|
||||
|
||||
// SecureDeleteMsg deletes a message from a stream. The deleted message
|
||||
// is overwritten with random data. As a result, this operation is slower
|
||||
// than DeleteMsg.
|
||||
SecureDeleteMsg(ctx context.Context, seq uint64) error
|
||||
}
|
||||
|
||||
// ConsumerManager provides CRUD API for managing consumers. It is
|
||||
// available as a part of [Stream] interface. CreateConsumer,
|
||||
// UpdateConsumer, CreateOrUpdateConsumer and Consumer methods return a
|
||||
// [Consumer] interface, allowing to operate on a consumer (e.g. consume
|
||||
// messages).
|
||||
ConsumerManager interface {
|
||||
// CreateOrUpdateConsumer creates a pull consumer on a given stream with
|
||||
// given config. If consumer already exists, it will be updated (if
|
||||
// possible). Consumer interface is returned, allowing to operate on a
|
||||
// consumer (e.g. fetch messages).
|
||||
CreateOrUpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
|
||||
|
||||
// CreateConsumer creates a pull consumer on a given stream with given
|
||||
// config. If consumer already exists and the provided configuration
|
||||
// differs from its configuration, ErrConsumerExists is returned. If the
|
||||
// provided configuration is the same as the existing consumer, the
|
||||
// existing consumer is returned. Consumer interface is returned,
|
||||
// allowing to operate on a consumer (e.g. fetch messages).
|
||||
CreateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
|
||||
|
||||
// UpdateConsumer updates an existing pull consumer. If consumer does not
|
||||
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
|
||||
// returned, allowing to operate on a consumer (e.g. fetch messages).
|
||||
UpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
|
||||
|
||||
// OrderedConsumer returns an OrderedConsumer instance. OrderedConsumer
|
||||
// are managed by the library and provide a simple way to consume
|
||||
// messages from a stream. Ordered consumers are ephemeral in-memory
|
||||
// pull consumers and are resilient to deletes and restarts.
|
||||
OrderedConsumer(ctx context.Context, cfg OrderedConsumerConfig) (Consumer, error)
|
||||
|
||||
// Consumer returns an interface to an existing consumer, allowing processing
|
||||
// of messages. If consumer does not exist, ErrConsumerNotFound is
|
||||
// returned.
|
||||
//
|
||||
// It returns ErrNotPullConsumer if the consumer is not a pull consumer (deliver subject is not set).
|
||||
Consumer(ctx context.Context, consumer string) (Consumer, error)
|
||||
|
||||
// DeleteConsumer removes a consumer with given name from a stream.
|
||||
// If consumer does not exist, ErrConsumerNotFound is returned.
|
||||
DeleteConsumer(ctx context.Context, consumer string) error
|
||||
|
||||
// PauseConsumer pauses a consumer.
|
||||
PauseConsumer(ctx context.Context, consumer string, pauseUntil time.Time) (*ConsumerPauseResponse, error)
|
||||
|
||||
// ResumeConsumer resumes a consumer.
|
||||
ResumeConsumer(ctx context.Context, consumer string) (*ConsumerPauseResponse, error)
|
||||
|
||||
// ListConsumers returns ConsumerInfoLister enabling iterating over a
|
||||
// channel of consumer infos.
|
||||
ListConsumers(context.Context) ConsumerInfoLister
|
||||
|
||||
// ConsumerNames returns a ConsumerNameLister enabling iterating over a
|
||||
// channel of consumer names.
|
||||
ConsumerNames(context.Context) ConsumerNameLister
|
||||
|
||||
// UnpinConsumer unpins the currently pinned client for a consumer for the given group name.
|
||||
// If consumer does not exist, ErrConsumerNotFound is returned.
|
||||
UnpinConsumer(ctx context.Context, consumer string, group string) error
|
||||
|
||||
// CreateOrUpdatePushConsumer creates a push consumer on a given stream with
|
||||
// given config. If consumer already exists, it will be updated (if
|
||||
// possible). Consumer interface is returned, allowing to consume messages.
|
||||
CreateOrUpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
|
||||
|
||||
// CreatePushConsumer creates a push consumer on a given stream with given
|
||||
// config. If consumer already exists and the provided configuration
|
||||
// differs from its configuration, ErrConsumerExists is returned. If the
|
||||
// provided configuration is the same as the existing consumer, the
|
||||
// existing consumer is returned. Consumer interface is returned,
|
||||
// allowing to consume messages.
|
||||
CreatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
|
||||
|
||||
// UpdatePushConsumer updates an existing push consumer. If consumer does not
|
||||
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
|
||||
// returned, allowing to consume messages.
|
||||
UpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
|
||||
|
||||
// PushConsumer returns an interface to an existing push consumer, allowing processing
|
||||
// of messages. If consumer does not exist, ErrConsumerNotFound is
|
||||
// returned.
|
||||
//
|
||||
// It returns ErrNotPushConsumer if the consumer is not a push consumer (deliver subject is not set).
|
||||
PushConsumer(ctx context.Context, consumer string) (PushConsumer, error)
|
||||
}
|
||||
|
||||
RawStreamMsg struct {
|
||||
Subject string
|
||||
Sequence uint64
|
||||
Header nats.Header
|
||||
Data []byte
|
||||
Time time.Time
|
||||
}
|
||||
|
||||
stream struct {
|
||||
name string
|
||||
info *StreamInfo
|
||||
js *jetStream
|
||||
}
|
||||
|
||||
// StreamInfoOpt is a function setting options for [Stream.Info]
|
||||
StreamInfoOpt func(*streamInfoRequest) error
|
||||
|
||||
streamInfoRequest struct {
|
||||
apiPagedRequest
|
||||
DeletedDetails bool `json:"deleted_details,omitempty"`
|
||||
SubjectFilter string `json:"subjects_filter,omitempty"`
|
||||
}
|
||||
|
||||
consumerInfoResponse struct {
|
||||
apiResponse
|
||||
*ConsumerInfo
|
||||
}
|
||||
|
||||
// StreamPurgeOpt is a function setting options for [Stream.Purge]
|
||||
StreamPurgeOpt func(*StreamPurgeRequest) error
|
||||
|
||||
// StreamPurgeRequest is an API request body to purge a stream.
|
||||
|
||||
StreamPurgeRequest struct {
|
||||
// Purge up to but not including sequence.
|
||||
Sequence uint64 `json:"seq,omitempty"`
|
||||
// Subject to match against messages for the purge command.
|
||||
Subject string `json:"filter,omitempty"`
|
||||
// Number of messages to keep.
|
||||
Keep uint64 `json:"keep,omitempty"`
|
||||
}
|
||||
|
||||
streamPurgeResponse struct {
|
||||
apiResponse
|
||||
Success bool `json:"success,omitempty"`
|
||||
Purged uint64 `json:"purged"`
|
||||
}
|
||||
|
||||
consumerDeleteResponse struct {
|
||||
apiResponse
|
||||
Success bool `json:"success,omitempty"`
|
||||
}
|
||||
|
||||
consumerPauseRequest struct {
|
||||
PauseUntil *time.Time `json:"pause_until,omitempty"`
|
||||
}
|
||||
|
||||
ConsumerPauseResponse struct {
|
||||
// Paused is true if the consumer is paused.
|
||||
Paused bool `json:"paused"`
|
||||
// PauseUntil is the time until the consumer is paused.
|
||||
PauseUntil time.Time `json:"pause_until"`
|
||||
// PauseRemaining is the time remaining until the consumer is paused.
|
||||
PauseRemaining time.Duration `json:"pause_remaining,omitempty"`
|
||||
}
|
||||
|
||||
consumerPauseApiResponse struct {
|
||||
apiResponse
|
||||
ConsumerPauseResponse
|
||||
}
|
||||
|
||||
// GetMsgOpt is a function setting options for [Stream.GetMsg]
|
||||
GetMsgOpt func(*apiMsgGetRequest) error
|
||||
|
||||
apiMsgGetRequest struct {
|
||||
Seq uint64 `json:"seq,omitempty"`
|
||||
LastFor string `json:"last_by_subj,omitempty"`
|
||||
NextFor string `json:"next_by_subj,omitempty"`
|
||||
}
|
||||
|
||||
// apiMsgGetResponse is the response for a Stream get request.
|
||||
apiMsgGetResponse struct {
|
||||
apiResponse
|
||||
Message *storedMsg `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
// storedMsg is a raw message stored in JetStream.
|
||||
storedMsg struct {
|
||||
Subject string `json:"subject"`
|
||||
Sequence uint64 `json:"seq"`
|
||||
Header []byte `json:"hdrs,omitempty"`
|
||||
Data []byte `json:"data,omitempty"`
|
||||
Time time.Time `json:"time"`
|
||||
}
|
||||
|
||||
msgDeleteRequest struct {
|
||||
Seq uint64 `json:"seq"`
|
||||
NoErase bool `json:"no_erase,omitempty"`
|
||||
}
|
||||
|
||||
msgDeleteResponse struct {
|
||||
apiResponse
|
||||
Success bool `json:"success,omitempty"`
|
||||
}
|
||||
|
||||
// ConsumerInfoLister is used to iterate over a channel of consumer infos.
|
||||
// Err method can be used to check for errors encountered during iteration.
|
||||
// Info channel is always closed and therefore can be used in a range loop.
|
||||
ConsumerInfoLister interface {
|
||||
Info() <-chan *ConsumerInfo
|
||||
Err() error
|
||||
}
|
||||
|
||||
// ConsumerNameLister is used to iterate over a channel of consumer names.
|
||||
// Err method can be used to check for errors encountered during iteration.
|
||||
// Name channel is always closed and therefore can be used in a range loop.
|
||||
ConsumerNameLister interface {
|
||||
Name() <-chan string
|
||||
Err() error
|
||||
}
|
||||
|
||||
consumerLister struct {
|
||||
js *jetStream
|
||||
offset int
|
||||
pageInfo *apiPaged
|
||||
|
||||
consumers chan *ConsumerInfo
|
||||
names chan string
|
||||
err error
|
||||
}
|
||||
|
||||
consumerListResponse struct {
|
||||
apiResponse
|
||||
apiPaged
|
||||
Consumers []*ConsumerInfo `json:"consumers"`
|
||||
}
|
||||
|
||||
consumerNamesResponse struct {
|
||||
apiResponse
|
||||
apiPaged
|
||||
Consumers []string `json:"consumers"`
|
||||
}
|
||||
|
||||
consumerUnpinRequest struct {
|
||||
Group string `json:"group"`
|
||||
}
|
||||
)
|
||||
|
||||
// CreateOrUpdateConsumer creates a consumer on a given stream with
|
||||
// given config. If consumer already exists, it will be updated (if
|
||||
// possible). Consumer interface is returned, allowing to operate on a
|
||||
// consumer (e.g. fetch messages).
|
||||
func (s *stream) CreateOrUpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
|
||||
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionCreateOrUpdate)
|
||||
}
|
||||
|
||||
// CreateConsumer creates a consumer on a given stream with given
|
||||
// config. If consumer already exists and the provided configuration
|
||||
// differs from its configuration, ErrConsumerExists is returned. If the
|
||||
// provided configuration is the same as the existing consumer, the
|
||||
// existing consumer is returned. Consumer interface is returned,
|
||||
// allowing to operate on a consumer (e.g. fetch messages).
|
||||
func (s *stream) CreateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
|
||||
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionCreate)
|
||||
}
|
||||
|
||||
// UpdateConsumer updates an existing consumer. If consumer does not
|
||||
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
|
||||
// returned, allowing to operate on a consumer (e.g. fetch messages).
|
||||
func (s *stream) UpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
|
||||
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionUpdate)
|
||||
}
|
||||
|
||||
// CreateOrUpdatePushConsumer creates a consumer on a given stream with
|
||||
// given config. If consumer already exists, it will be updated (if
|
||||
// possible). Consumer interface is returned, allowing to consume messages.
|
||||
func (s *stream) CreateOrUpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
|
||||
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionCreateOrUpdate)
|
||||
}
|
||||
|
||||
// CreatePushConsumer creates a consumer on a given stream with given
|
||||
// config. If consumer already exists and the provided configuration
|
||||
// differs from its configuration, ErrConsumerExists is returned. If the
|
||||
// provided configuration is the same as the existing consumer, the
|
||||
// existing consumer is returned. Consumer interface is returned,
|
||||
// allowing to consume messages.
|
||||
func (s *stream) CreatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
|
||||
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionCreate)
|
||||
}
|
||||
|
||||
// UpdatePushConsumer updates an existing consumer. If consumer does not
|
||||
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
|
||||
// returned, allowing to consume messages.
|
||||
func (s *stream) UpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
|
||||
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionUpdate)
|
||||
}
|
||||
|
||||
// OrderedConsumer returns an OrderedConsumer instance. OrderedConsumer
|
||||
// are managed by the library and provide a simple way to consume
|
||||
// messages from a stream. Ordered consumers are ephemeral in-memory
|
||||
// pull consumers and are resilient to deletes and restarts.
|
||||
func (s *stream) OrderedConsumer(ctx context.Context, cfg OrderedConsumerConfig) (Consumer, error) {
|
||||
namePrefix := cfg.NamePrefix
|
||||
if namePrefix == "" {
|
||||
namePrefix = nuid.Next()
|
||||
}
|
||||
oc := &orderedConsumer{
|
||||
js: s.js,
|
||||
cfg: &cfg,
|
||||
stream: s.name,
|
||||
namePrefix: namePrefix,
|
||||
doReset: make(chan struct{}, 1),
|
||||
}
|
||||
consCfg := oc.getConsumerConfig()
|
||||
cons, err := s.CreateOrUpdateConsumer(ctx, *consCfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
oc.currentConsumer = cons.(*pullConsumer)
|
||||
|
||||
return oc, nil
|
||||
}
|
||||
|
||||
// Consumer returns an interface to an existing consumer, allowing processing
|
||||
// of messages. If consumer does not exist, ErrConsumerNotFound is
|
||||
// returned.
|
||||
func (s *stream) Consumer(ctx context.Context, name string) (Consumer, error) {
|
||||
return getConsumer(ctx, s.js, s.name, name)
|
||||
}
|
||||
|
||||
func (s *stream) PushConsumer(ctx context.Context, name string) (PushConsumer, error) {
|
||||
return getPushConsumer(ctx, s.js, s.name, name)
|
||||
}
|
||||
|
||||
// DeleteConsumer removes a consumer with given name from a stream.
|
||||
// If consumer does not exist, ErrConsumerNotFound is returned.
|
||||
func (s *stream) DeleteConsumer(ctx context.Context, name string) error {
|
||||
return deleteConsumer(ctx, s.js, s.name, name)
|
||||
}
|
||||
|
||||
// PauseConsumer pauses a consumer.
|
||||
func (s *stream) PauseConsumer(ctx context.Context, name string, pauseUntil time.Time) (*ConsumerPauseResponse, error) {
|
||||
return pauseConsumer(ctx, s.js, s.name, name, &pauseUntil)
|
||||
}
|
||||
|
||||
// ResumeConsumer resumes a consumer.
|
||||
func (s *stream) ResumeConsumer(ctx context.Context, name string) (*ConsumerPauseResponse, error) {
|
||||
return resumeConsumer(ctx, s.js, s.name, name)
|
||||
}
|
||||
|
||||
// Info returns StreamInfo from the server.
|
||||
func (s *stream) Info(ctx context.Context, opts ...StreamInfoOpt) (*StreamInfo, error) {
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
var infoReq *streamInfoRequest
|
||||
for _, opt := range opts {
|
||||
if infoReq == nil {
|
||||
infoReq = &streamInfoRequest{}
|
||||
}
|
||||
if err := opt(infoReq); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
var req []byte
|
||||
var err error
|
||||
var subjectMap map[string]uint64
|
||||
var offset int
|
||||
|
||||
infoSubject := fmt.Sprintf(apiStreamInfoT, s.name)
|
||||
var info *StreamInfo
|
||||
for {
|
||||
if infoReq != nil {
|
||||
if infoReq.SubjectFilter != "" {
|
||||
if subjectMap == nil {
|
||||
subjectMap = make(map[string]uint64)
|
||||
}
|
||||
infoReq.Offset = offset
|
||||
}
|
||||
req, err = json.Marshal(infoReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
var resp streamInfoResponse
|
||||
if _, err = s.js.apiRequestJSON(ctx, infoSubject, &resp, req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeStreamNotFound {
|
||||
return nil, ErrStreamNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
info = resp.StreamInfo
|
||||
var total int
|
||||
if resp.Total != 0 {
|
||||
total = resp.Total
|
||||
}
|
||||
if len(resp.StreamInfo.State.Subjects) > 0 {
|
||||
for subj, msgs := range resp.StreamInfo.State.Subjects {
|
||||
subjectMap[subj] = msgs
|
||||
}
|
||||
offset = len(subjectMap)
|
||||
}
|
||||
if total == 0 || total <= offset {
|
||||
info.State.Subjects = nil
|
||||
// we don't want to store subjects in cache
|
||||
cached := *info
|
||||
s.info = &cached
|
||||
info.State.Subjects = subjectMap
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return info, nil
|
||||
}
|
||||
|
||||
// CachedInfo returns ConsumerInfo currently cached on this stream.
|
||||
// This method does not perform any network requests. The cached
|
||||
// StreamInfo is updated on every call to Info and Update.
|
||||
func (s *stream) CachedInfo() *StreamInfo {
|
||||
return s.info
|
||||
}
|
||||
|
||||
// Purge removes messages from a stream. It is a destructive operation.
|
||||
// Use with caution. See StreamPurgeOpt for available options.
|
||||
func (s *stream) Purge(ctx context.Context, opts ...StreamPurgeOpt) error {
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
var purgeReq StreamPurgeRequest
|
||||
for _, opt := range opts {
|
||||
if err := opt(&purgeReq); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
var req []byte
|
||||
var err error
|
||||
req, err = json.Marshal(purgeReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
purgeSubject := fmt.Sprintf(apiStreamPurgeT, s.name)
|
||||
|
||||
var resp streamPurgeResponse
|
||||
if _, err = s.js.apiRequestJSON(ctx, purgeSubject, &resp, req); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return resp.Error
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetMsg retrieves a raw stream message stored in JetStream by sequence number.
|
||||
func (s *stream) GetMsg(ctx context.Context, seq uint64, opts ...GetMsgOpt) (*RawStreamMsg, error) {
|
||||
req := &apiMsgGetRequest{Seq: seq}
|
||||
for _, opt := range opts {
|
||||
if err := opt(req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return s.getMsg(ctx, req)
|
||||
}
|
||||
|
||||
// GetLastMsgForSubject retrieves the last raw stream message stored in
|
||||
// JetStream on a given subject subject.
|
||||
func (s *stream) GetLastMsgForSubject(ctx context.Context, subject string) (*RawStreamMsg, error) {
|
||||
return s.getMsg(ctx, &apiMsgGetRequest{LastFor: subject})
|
||||
}
|
||||
|
||||
func (s *stream) getMsg(ctx context.Context, mreq *apiMsgGetRequest) (*RawStreamMsg, error) {
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
req, err := json.Marshal(mreq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var gmSubj string
|
||||
|
||||
// handle direct gets
|
||||
if s.info.Config.AllowDirect {
|
||||
if mreq.LastFor != "" {
|
||||
gmSubj = fmt.Sprintf(apiDirectMsgGetLastBySubjectT, s.name, mreq.LastFor)
|
||||
r, err := s.js.apiRequest(ctx, gmSubj, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return convertDirectGetMsgResponseToMsg(r.msg)
|
||||
}
|
||||
gmSubj = fmt.Sprintf(apiDirectMsgGetT, s.name)
|
||||
r, err := s.js.apiRequest(ctx, gmSubj, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return convertDirectGetMsgResponseToMsg(r.msg)
|
||||
}
|
||||
|
||||
var resp apiMsgGetResponse
|
||||
dsSubj := fmt.Sprintf(apiMsgGetT, s.name)
|
||||
_, err = s.js.apiRequestJSON(ctx, dsSubj, &resp, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if resp.Error != nil {
|
||||
if resp.Error.ErrorCode == JSErrCodeMessageNotFound {
|
||||
return nil, ErrMsgNotFound
|
||||
}
|
||||
return nil, resp.Error
|
||||
}
|
||||
|
||||
msg := resp.Message
|
||||
|
||||
var hdr nats.Header
|
||||
if len(msg.Header) > 0 {
|
||||
hdr, err = nats.DecodeHeadersMsg(msg.Header)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
return &RawStreamMsg{
|
||||
Subject: msg.Subject,
|
||||
Sequence: msg.Sequence,
|
||||
Header: hdr,
|
||||
Data: msg.Data,
|
||||
Time: msg.Time,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func convertDirectGetMsgResponseToMsg(r *nats.Msg) (*RawStreamMsg, error) {
|
||||
// Check for 404/408. We would get a no-payload message and a "Status" header
|
||||
if len(r.Data) == 0 {
|
||||
val := r.Header.Get(statusHdr)
|
||||
if val != "" {
|
||||
switch val {
|
||||
case statusNoMsgs:
|
||||
return nil, ErrMsgNotFound
|
||||
default:
|
||||
desc := r.Header.Get("Description")
|
||||
if desc == "" {
|
||||
desc = "unable to get message"
|
||||
}
|
||||
return nil, fmt.Errorf("nats: %s", desc)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check for headers that give us the required information to
|
||||
// reconstruct the message.
|
||||
if len(r.Header) == 0 {
|
||||
return nil, errors.New("nats: response should have headers")
|
||||
}
|
||||
stream := r.Header.Get(StreamHeader)
|
||||
if stream == "" {
|
||||
return nil, errors.New("nats: missing stream header")
|
||||
}
|
||||
|
||||
seqStr := r.Header.Get(SequenceHeader)
|
||||
if seqStr == "" {
|
||||
return nil, errors.New("nats: missing sequence header")
|
||||
}
|
||||
seq, err := strconv.ParseUint(seqStr, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nats: invalid sequence header '%s': %v", seqStr, err)
|
||||
}
|
||||
timeStr := r.Header.Get(TimeStampHeaer)
|
||||
if timeStr == "" {
|
||||
return nil, errors.New("nats: missing timestamp header")
|
||||
}
|
||||
|
||||
tm, err := time.Parse(time.RFC3339Nano, timeStr)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nats: invalid timestamp header '%s': %v", timeStr, err)
|
||||
}
|
||||
subj := r.Header.Get(SubjectHeader)
|
||||
if subj == "" {
|
||||
return nil, errors.New("nats: missing subject header")
|
||||
}
|
||||
return &RawStreamMsg{
|
||||
Subject: subj,
|
||||
Sequence: seq,
|
||||
Header: r.Header,
|
||||
Data: r.Data,
|
||||
Time: tm,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// DeleteMsg deletes a message from a stream.
|
||||
// On the server, the message is marked as erased, but not overwritten.
|
||||
func (s *stream) DeleteMsg(ctx context.Context, seq uint64) error {
|
||||
return s.deleteMsg(ctx, &msgDeleteRequest{Seq: seq, NoErase: true})
|
||||
}
|
||||
|
||||
// SecureDeleteMsg deletes a message from a stream. The deleted message
|
||||
// is overwritten with random data. As a result, this operation is slower
|
||||
// than DeleteMsg.
|
||||
func (s *stream) SecureDeleteMsg(ctx context.Context, seq uint64) error {
|
||||
return s.deleteMsg(ctx, &msgDeleteRequest{Seq: seq})
|
||||
}
|
||||
|
||||
func (s *stream) deleteMsg(ctx context.Context, req *msgDeleteRequest) error {
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
r, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
subj := fmt.Sprintf(apiMsgDeleteT, s.name)
|
||||
var resp msgDeleteResponse
|
||||
if _, err = s.js.apiRequestJSON(ctx, subj, &resp, r); err != nil {
|
||||
return err
|
||||
}
|
||||
if !resp.Success {
|
||||
return fmt.Errorf("%w: %s", ErrMsgDeleteUnsuccessful, resp.Error.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListConsumers returns ConsumerInfoLister enabling iterating over a
|
||||
// channel of consumer infos.
|
||||
func (s *stream) ListConsumers(ctx context.Context) ConsumerInfoLister {
|
||||
l := &consumerLister{
|
||||
js: s.js,
|
||||
consumers: make(chan *ConsumerInfo),
|
||||
}
|
||||
go func() {
|
||||
defer close(l.consumers)
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
for {
|
||||
page, err := l.consumerInfos(ctx, s.name)
|
||||
if err != nil && !errors.Is(err, ErrEndOfData) {
|
||||
l.err = err
|
||||
return
|
||||
}
|
||||
for _, info := range page {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
l.err = ctx.Err()
|
||||
return
|
||||
default:
|
||||
}
|
||||
if info != nil {
|
||||
l.consumers <- info
|
||||
}
|
||||
}
|
||||
if errors.Is(err, ErrEndOfData) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
return l
|
||||
}
|
||||
|
||||
func (s *consumerLister) Info() <-chan *ConsumerInfo {
|
||||
return s.consumers
|
||||
}
|
||||
|
||||
func (s *consumerLister) Err() error {
|
||||
return s.err
|
||||
}
|
||||
|
||||
// ConsumerNames returns a ConsumerNameLister enabling iterating over a
|
||||
// channel of consumer names.
|
||||
func (s *stream) ConsumerNames(ctx context.Context) ConsumerNameLister {
|
||||
l := &consumerLister{
|
||||
js: s.js,
|
||||
names: make(chan string),
|
||||
}
|
||||
go func() {
|
||||
defer close(l.names)
|
||||
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
|
||||
if cancel != nil {
|
||||
defer cancel()
|
||||
}
|
||||
for {
|
||||
page, err := l.consumerNames(ctx, s.name)
|
||||
if err != nil && !errors.Is(err, ErrEndOfData) {
|
||||
l.err = err
|
||||
return
|
||||
}
|
||||
for _, info := range page {
|
||||
select {
|
||||
case l.names <- info:
|
||||
case <-ctx.Done():
|
||||
l.err = ctx.Err()
|
||||
return
|
||||
}
|
||||
}
|
||||
if errors.Is(err, ErrEndOfData) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
return l
|
||||
}
|
||||
|
||||
func (s *consumerLister) Name() <-chan string {
|
||||
return s.names
|
||||
}
|
||||
|
||||
// consumerInfos fetches the next ConsumerInfo page
|
||||
func (s *consumerLister) consumerInfos(ctx context.Context, stream string) ([]*ConsumerInfo, error) {
|
||||
if s.pageInfo != nil && s.offset >= s.pageInfo.Total {
|
||||
return nil, ErrEndOfData
|
||||
}
|
||||
|
||||
req, err := json.Marshal(
|
||||
apiPagedRequest{Offset: s.offset},
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
slSubj := fmt.Sprintf(apiConsumerListT, stream)
|
||||
var resp consumerListResponse
|
||||
_, err = s.js.apiRequestJSON(ctx, slSubj, &resp, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return nil, resp.Error
|
||||
}
|
||||
|
||||
s.pageInfo = &resp.apiPaged
|
||||
s.offset += len(resp.Consumers)
|
||||
return resp.Consumers, nil
|
||||
}
|
||||
|
||||
// consumerNames fetches the next consumer names page
|
||||
func (s *consumerLister) consumerNames(ctx context.Context, stream string) ([]string, error) {
|
||||
if s.pageInfo != nil && s.offset >= s.pageInfo.Total {
|
||||
return nil, ErrEndOfData
|
||||
}
|
||||
|
||||
req, err := json.Marshal(
|
||||
apiPagedRequest{Offset: s.offset},
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
slSubj := fmt.Sprintf(apiConsumerNamesT, stream)
|
||||
var resp consumerNamesResponse
|
||||
_, err = s.js.apiRequestJSON(ctx, slSubj, &resp, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Error != nil {
|
||||
return nil, resp.Error
|
||||
}
|
||||
|
||||
s.pageInfo = &resp.apiPaged
|
||||
s.offset += len(resp.Consumers)
|
||||
return resp.Consumers, nil
|
||||
}
|
||||
|
||||
// UnpinConsumer unpins the currently pinned client for a consumer for the given group name.
|
||||
// If consumer does not exist, ErrConsumerNotFound is returned.
|
||||
func (s *stream) UnpinConsumer(ctx context.Context, consumer string, group string) error {
|
||||
return unpinConsumer(ctx, s.js, s.name, consumer, group)
|
||||
}
|
||||
+685
@@ -0,0 +1,685 @@
|
||||
// Copyright 2022-2024 The NATS Authors
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package jetstream
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type (
|
||||
// StreamInfo shows config and current state for this stream.
|
||||
StreamInfo struct {
|
||||
// Config contains the configuration settings of the stream, set when
|
||||
// creating or updating the stream.
|
||||
Config StreamConfig `json:"config"`
|
||||
|
||||
// Created is the timestamp when the stream was created.
|
||||
Created time.Time `json:"created"`
|
||||
|
||||
// State provides the state of the stream at the time of request,
|
||||
// including metrics like the number of messages in the stream, total
|
||||
// bytes, etc.
|
||||
State StreamState `json:"state"`
|
||||
|
||||
// Cluster contains information about the cluster to which this stream
|
||||
// belongs (if applicable).
|
||||
Cluster *ClusterInfo `json:"cluster,omitempty"`
|
||||
|
||||
// Mirror contains information about another stream this one is
|
||||
// mirroring. Mirroring is used to create replicas of another stream's
|
||||
// data. This field is omitted if the stream is not mirroring another
|
||||
// stream.
|
||||
Mirror *StreamSourceInfo `json:"mirror,omitempty"`
|
||||
|
||||
// Sources is a list of source streams from which this stream collects
|
||||
// data.
|
||||
Sources []*StreamSourceInfo `json:"sources,omitempty"`
|
||||
|
||||
// TimeStamp indicates when the info was gathered by the server.
|
||||
TimeStamp time.Time `json:"ts"`
|
||||
}
|
||||
|
||||
// StreamConfig is the configuration of a JetStream stream.
|
||||
StreamConfig struct {
|
||||
// Name is the name of the stream. It is required and must be unique
|
||||
// across the JetStream account.
|
||||
//
|
||||
// Name Names cannot contain whitespace, ., *, >, path separators
|
||||
// (forward or backwards slash), and non-printable characters.
|
||||
Name string `json:"name"`
|
||||
|
||||
// Description is an optional description of the stream.
|
||||
Description string `json:"description,omitempty"`
|
||||
|
||||
// Subjects is a list of subjects that the stream is listening on.
|
||||
// Wildcards are supported. Subjects cannot be set if the stream is
|
||||
// created as a mirror.
|
||||
Subjects []string `json:"subjects,omitempty"`
|
||||
|
||||
// Retention defines the message retention policy for the stream.
|
||||
// Defaults to LimitsPolicy.
|
||||
Retention RetentionPolicy `json:"retention"`
|
||||
|
||||
// MaxConsumers specifies the maximum number of consumers allowed for
|
||||
// the stream.
|
||||
MaxConsumers int `json:"max_consumers"`
|
||||
|
||||
// MaxMsgs is the maximum number of messages the stream will store.
|
||||
// After reaching the limit, stream adheres to the discard policy.
|
||||
// If not set, server default is -1 (unlimited).
|
||||
MaxMsgs int64 `json:"max_msgs"`
|
||||
|
||||
// MaxBytes is the maximum total size of messages the stream will store.
|
||||
// After reaching the limit, stream adheres to the discard policy.
|
||||
// If not set, server default is -1 (unlimited).
|
||||
MaxBytes int64 `json:"max_bytes"`
|
||||
|
||||
// Discard defines the policy for handling messages when the stream
|
||||
// reaches its limits in terms of number of messages or total bytes.
|
||||
Discard DiscardPolicy `json:"discard"`
|
||||
|
||||
// DiscardNewPerSubject is a flag to enable discarding new messages per
|
||||
// subject when limits are reached. Requires DiscardPolicy to be
|
||||
// DiscardNew and the MaxMsgsPerSubject to be set.
|
||||
DiscardNewPerSubject bool `json:"discard_new_per_subject,omitempty"`
|
||||
|
||||
// MaxAge is the maximum age of messages that the stream will retain.
|
||||
MaxAge time.Duration `json:"max_age"`
|
||||
|
||||
// MaxMsgsPerSubject is the maximum number of messages per subject that
|
||||
// the stream will retain.
|
||||
MaxMsgsPerSubject int64 `json:"max_msgs_per_subject"`
|
||||
|
||||
// MaxMsgSize is the maximum size of any single message in the stream.
|
||||
MaxMsgSize int32 `json:"max_msg_size,omitempty"`
|
||||
|
||||
// Storage specifies the type of storage backend used for the stream
|
||||
// (file or memory).
|
||||
Storage StorageType `json:"storage"`
|
||||
|
||||
// Replicas is the number of stream replicas in clustered JetStream.
|
||||
// Defaults to 1, maximum is 5.
|
||||
Replicas int `json:"num_replicas"`
|
||||
|
||||
// NoAck is a flag to disable acknowledging messages received by this
|
||||
// stream.
|
||||
//
|
||||
// If set to true, publish methods from the JetStream client will not
|
||||
// work as expected, since they rely on acknowledgements. Core NATS
|
||||
// publish methods should be used instead. Note that this will make
|
||||
// message delivery less reliable.
|
||||
NoAck bool `json:"no_ack,omitempty"`
|
||||
|
||||
// Duplicates is the window within which to track duplicate messages.
|
||||
// If not set, server default is 2 minutes.
|
||||
Duplicates time.Duration `json:"duplicate_window,omitempty"`
|
||||
|
||||
// Placement is used to declare where the stream should be placed via
|
||||
// tags and/or an explicit cluster name.
|
||||
Placement *Placement `json:"placement,omitempty"`
|
||||
|
||||
// Mirror defines the configuration for mirroring another stream.
|
||||
Mirror *StreamSource `json:"mirror,omitempty"`
|
||||
|
||||
// Sources is a list of other streams this stream sources messages from.
|
||||
Sources []*StreamSource `json:"sources,omitempty"`
|
||||
|
||||
// Sealed streams do not allow messages to be published or deleted via limits or API,
|
||||
// sealed streams can not be unsealed via configuration update. Can only
|
||||
// be set on already created streams via the Update API.
|
||||
Sealed bool `json:"sealed,omitempty"`
|
||||
|
||||
// DenyDelete restricts the ability to delete messages from a stream via
|
||||
// the API. Defaults to false.
|
||||
DenyDelete bool `json:"deny_delete,omitempty"`
|
||||
|
||||
// DenyPurge restricts the ability to purge messages from a stream via
|
||||
// the API. Defaults to false.
|
||||
DenyPurge bool `json:"deny_purge,omitempty"`
|
||||
|
||||
// AllowRollup allows the use of the Nats-Rollup header to replace all
|
||||
// contents of a stream, or subject in a stream, with a single new
|
||||
// message.
|
||||
AllowRollup bool `json:"allow_rollup_hdrs,omitempty"`
|
||||
|
||||
// Compression specifies the message storage compression algorithm.
|
||||
// Defaults to NoCompression.
|
||||
Compression StoreCompression `json:"compression"`
|
||||
|
||||
// FirstSeq is the initial sequence number of the first message in the
|
||||
// stream.
|
||||
FirstSeq uint64 `json:"first_seq,omitempty"`
|
||||
|
||||
// SubjectTransform allows applying a transformation to matching
|
||||
// messages' subjects.
|
||||
SubjectTransform *SubjectTransformConfig `json:"subject_transform,omitempty"`
|
||||
|
||||
// RePublish allows immediate republishing a message to the configured
|
||||
// subject after it's stored.
|
||||
RePublish *RePublish `json:"republish,omitempty"`
|
||||
|
||||
// AllowDirect enables direct access to individual messages using direct
|
||||
// get API. Defaults to false.
|
||||
AllowDirect bool `json:"allow_direct"`
|
||||
|
||||
// MirrorDirect enables direct access to individual messages from the
|
||||
// origin stream using direct get API. Defaults to false.
|
||||
MirrorDirect bool `json:"mirror_direct"`
|
||||
|
||||
// ConsumerLimits defines limits of certain values that consumers can
|
||||
// set, defaults for those who don't set these settings
|
||||
ConsumerLimits StreamConsumerLimits `json:"consumer_limits,omitempty"`
|
||||
|
||||
// Metadata is a set of application-defined key-value pairs for
|
||||
// associating metadata on the stream. This feature requires nats-server
|
||||
// v2.10.0 or later.
|
||||
Metadata map[string]string `json:"metadata,omitempty"`
|
||||
|
||||
// Template identifies the template that manages the Stream.
|
||||
// Deprecated: This feature is no longer supported.
|
||||
Template string `json:"template_owner,omitempty"`
|
||||
|
||||
// AllowMsgTTL allows header initiated per-message TTLs.
|
||||
// This feature requires nats-server v2.11.0 or later.
|
||||
AllowMsgTTL bool `json:"allow_msg_ttl,omitempty"`
|
||||
|
||||
// Enables and sets a duration for adding server markers for delete, purge and max age limits.
|
||||
// This feature requires nats-server v2.11.0 or later.
|
||||
SubjectDeleteMarkerTTL time.Duration `json:"subject_delete_marker_ttl,omitempty"`
|
||||
|
||||
// AllowMsgCounter enables the feature
|
||||
AllowMsgCounter bool `json:"allow_msg_counter,omitempty"`
|
||||
|
||||
// AllowAtomicPublish allows atomic batch publishing into the stream.
|
||||
AllowAtomicPublish bool `json:"allow_atomic,omitempty"`
|
||||
|
||||
// AllowMsgSchedules enables the scheduling of messages
|
||||
AllowMsgSchedules bool `json:"allow_msg_schedules,omitempty"`
|
||||
|
||||
// PersistMode allows to opt-in to different persistence mode settings.
|
||||
PersistMode PersistModeType `json:"persist_mode,omitempty"`
|
||||
}
|
||||
|
||||
// StreamSourceInfo shows information about an upstream stream
|
||||
// source/mirror.
|
||||
StreamSourceInfo struct {
|
||||
// Name is the name of the stream that is being replicated.
|
||||
Name string `json:"name"`
|
||||
|
||||
// Lag informs how many messages behind the source/mirror operation is.
|
||||
// This will only show correctly if there is active communication
|
||||
// with stream/mirror.
|
||||
Lag uint64 `json:"lag"`
|
||||
|
||||
// Active informs when last the mirror or sourced stream had activity.
|
||||
// Value will be -1 when there has been no activity.
|
||||
Active time.Duration `json:"active"`
|
||||
|
||||
// FilterSubject is the subject filter defined for this source/mirror.
|
||||
FilterSubject string `json:"filter_subject,omitempty"`
|
||||
|
||||
// SubjectTransforms is a list of subject transforms defined for this
|
||||
// source/mirror.
|
||||
SubjectTransforms []SubjectTransformConfig `json:"subject_transforms,omitempty"`
|
||||
}
|
||||
|
||||
// StreamState is the state of a JetStream stream at the time of request.
|
||||
StreamState struct {
|
||||
// Msgs is the number of messages stored in the stream.
|
||||
Msgs uint64 `json:"messages"`
|
||||
|
||||
// Bytes is the number of bytes stored in the stream.
|
||||
Bytes uint64 `json:"bytes"`
|
||||
|
||||
// FirstSeq is the sequence number of the first message in the stream.
|
||||
FirstSeq uint64 `json:"first_seq"`
|
||||
|
||||
// FirstTime is the timestamp of the first message in the stream.
|
||||
FirstTime time.Time `json:"first_ts"`
|
||||
|
||||
// LastSeq is the sequence number of the last message in the stream.
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
|
||||
// LastTime is the timestamp of the last message in the stream.
|
||||
LastTime time.Time `json:"last_ts"`
|
||||
|
||||
// Consumers is the number of consumers on the stream.
|
||||
Consumers int `json:"consumer_count"`
|
||||
|
||||
// Deleted is a list of sequence numbers that have been removed from the
|
||||
// stream. This field will only be returned if the stream has been
|
||||
// fetched with the DeletedDetails option.
|
||||
Deleted []uint64 `json:"deleted"`
|
||||
|
||||
// NumDeleted is the number of messages that have been removed from the
|
||||
// stream. Only deleted messages causing a gap in stream sequence numbers
|
||||
// are counted. Messages deleted at the beginning or end of the stream
|
||||
// are not counted.
|
||||
NumDeleted int `json:"num_deleted"`
|
||||
|
||||
// NumSubjects is the number of unique subjects the stream has received
|
||||
// messages on.
|
||||
NumSubjects uint64 `json:"num_subjects"`
|
||||
|
||||
// Subjects is a map of subjects the stream has received messages on
|
||||
// with message count per subject. This field will only be returned if
|
||||
// the stream has been fetched with the SubjectFilter option.
|
||||
Subjects map[string]uint64 `json:"subjects"`
|
||||
}
|
||||
|
||||
// ClusterInfo shows information about the underlying set of servers that
|
||||
// make up the stream or consumer.
|
||||
ClusterInfo struct {
|
||||
// Name is the name of the cluster.
|
||||
Name string `json:"name,omitempty"`
|
||||
|
||||
// RaftGroup is the name of the Raft group managing the asset (in
|
||||
// clustered environments).
|
||||
RaftGroup string `json:"raft_group,omitempty"`
|
||||
|
||||
// Leader is the server name of the RAFT leader.
|
||||
Leader string `json:"leader,omitempty"`
|
||||
|
||||
// LeaderSince is the time that it was elected as leader in RFC3339
|
||||
// format, absent when not the leader.
|
||||
LeaderSince *time.Time `json:"leader_since,omitempty"`
|
||||
|
||||
// SystemAcc indicates if the traffic_account is the system account.
|
||||
// When true, replication traffic goes over the system account.
|
||||
SystemAcc bool `json:"system_account,omitempty"`
|
||||
|
||||
// TrafficAcc is the account where the replication traffic goes over.
|
||||
TrafficAcc string `json:"traffic_account,omitempty"`
|
||||
|
||||
// Replicas is the list of members of the RAFT cluster.
|
||||
Replicas []*PeerInfo `json:"replicas,omitempty"`
|
||||
}
|
||||
|
||||
// PeerInfo shows information about the peers in the cluster that are
|
||||
// supporting the stream or consumer.
|
||||
PeerInfo struct {
|
||||
// Name is the server name of the peer.
|
||||
Name string `json:"name"`
|
||||
|
||||
// Current indicates if the peer is up to date and synchronized with the
|
||||
// leader.
|
||||
Current bool `json:"current"`
|
||||
|
||||
// Offline indicates if the peer is considered offline by the group.
|
||||
Offline bool `json:"offline,omitempty"`
|
||||
|
||||
// Active it the duration since this peer was last seen.
|
||||
Active time.Duration `json:"active"`
|
||||
|
||||
// Lag is the number of uncommitted operations this peer is behind the
|
||||
// leader.
|
||||
Lag uint64 `json:"lag,omitempty"`
|
||||
}
|
||||
|
||||
// SubjectTransformConfig is for applying a subject transform (to matching
|
||||
// messages) before doing anything else when a new message is received.
|
||||
SubjectTransformConfig struct {
|
||||
// Source is the subject pattern to match incoming messages against.
|
||||
Source string `json:"src"`
|
||||
|
||||
// Destination is the subject pattern to remap the subject to.
|
||||
Destination string `json:"dest"`
|
||||
}
|
||||
|
||||
// RePublish is for republishing messages once committed to a stream. The
|
||||
// original subject is remapped from the subject pattern to the destination
|
||||
// pattern.
|
||||
RePublish struct {
|
||||
// Source is the subject pattern to match incoming messages against.
|
||||
Source string `json:"src,omitempty"`
|
||||
|
||||
// Destination is the subject pattern to republish the subject to.
|
||||
Destination string `json:"dest"`
|
||||
|
||||
// HeadersOnly is a flag to indicate that only the headers should be
|
||||
// republished.
|
||||
HeadersOnly bool `json:"headers_only,omitempty"`
|
||||
}
|
||||
|
||||
// Placement is used to guide placement of streams in clustered JetStream.
|
||||
Placement struct {
|
||||
// Cluster is the name of the cluster to which the stream should be
|
||||
// assigned.
|
||||
Cluster string `json:"cluster"`
|
||||
|
||||
// Tags are used to match streams to servers in the cluster. A stream
|
||||
// will be assigned to a server with a matching tag.
|
||||
Tags []string `json:"tags,omitempty"`
|
||||
}
|
||||
|
||||
// StreamSource dictates how streams can source from other streams.
|
||||
StreamSource struct {
|
||||
// Name is the name of the stream to source from.
|
||||
Name string `json:"name"`
|
||||
|
||||
// OptStartSeq is the sequence number to start sourcing from.
|
||||
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
|
||||
|
||||
// OptStartTime is the timestamp of messages to start sourcing from.
|
||||
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
|
||||
|
||||
// FilterSubject is the subject filter used to only replicate messages
|
||||
// with matching subjects.
|
||||
FilterSubject string `json:"filter_subject,omitempty"`
|
||||
|
||||
// SubjectTransforms is a list of subject transforms to apply to
|
||||
// matching messages.
|
||||
//
|
||||
// Subject transforms on sources and mirrors are also used as subject
|
||||
// filters with optional transformations.
|
||||
SubjectTransforms []SubjectTransformConfig `json:"subject_transforms,omitempty"`
|
||||
|
||||
// External is a configuration referencing a stream source in another
|
||||
// account or JetStream domain.
|
||||
External *ExternalStream `json:"external,omitempty"`
|
||||
|
||||
// Domain is used to configure a stream source in another JetStream
|
||||
// domain. This setting will set the External field with the appropriate
|
||||
// APIPrefix.
|
||||
Domain string `json:"-"`
|
||||
}
|
||||
|
||||
// ExternalStream allows you to qualify access to a stream source in another
|
||||
// account.
|
||||
ExternalStream struct {
|
||||
// APIPrefix is the subject prefix that imports the other account/domain
|
||||
// $JS.API.CONSUMER.> subjects.
|
||||
APIPrefix string `json:"api"`
|
||||
|
||||
// DeliverPrefix is the delivery subject to use for the push consumer.
|
||||
DeliverPrefix string `json:"deliver"`
|
||||
}
|
||||
|
||||
// StreamConsumerLimits are the limits for a consumer on a stream. These can
|
||||
// be overridden on a per consumer basis.
|
||||
StreamConsumerLimits struct {
|
||||
// InactiveThreshold is a duration which instructs the server to clean
|
||||
// up the consumer if it has been inactive for the specified duration.
|
||||
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
|
||||
|
||||
// MaxAckPending is a maximum number of outstanding unacknowledged
|
||||
// messages for a consumer.
|
||||
MaxAckPending int `json:"max_ack_pending,omitempty"`
|
||||
}
|
||||
|
||||
// DiscardPolicy determines how to proceed when limits of messages or bytes
|
||||
// are reached.
|
||||
DiscardPolicy int
|
||||
|
||||
// RetentionPolicy determines how messages in a stream are retained.
|
||||
RetentionPolicy int
|
||||
|
||||
// StorageType determines how messages are stored for retention.
|
||||
StorageType int
|
||||
|
||||
// StoreCompression determines how messages are compressed.
|
||||
StoreCompression uint8
|
||||
|
||||
// PersistModeType determines what persistence mode the stream uses.
|
||||
PersistModeType int
|
||||
)
|
||||
|
||||
const (
|
||||
// LimitsPolicy (default) means that messages are retained until any given
|
||||
// limit is reached. This could be one of MaxMsgs, MaxBytes, or MaxAge.
|
||||
LimitsPolicy RetentionPolicy = iota
|
||||
|
||||
// InterestPolicy specifies that when all known observables have
|
||||
// acknowledged a message it can be removed.
|
||||
InterestPolicy
|
||||
|
||||
// WorkQueuePolicy specifies that when the first worker or subscriber
|
||||
// acknowledges the message it can be removed.
|
||||
WorkQueuePolicy
|
||||
)
|
||||
|
||||
const (
|
||||
// DiscardOld will remove older messages to return to the limits. This is
|
||||
// the default.
|
||||
DiscardOld DiscardPolicy = iota
|
||||
|
||||
// DiscardNew will fail to store new messages once the limits are reached.
|
||||
DiscardNew
|
||||
)
|
||||
|
||||
const (
|
||||
limitsPolicyString = "limits"
|
||||
interestPolicyString = "interest"
|
||||
workQueuePolicyString = "workqueue"
|
||||
)
|
||||
|
||||
const (
|
||||
// DefaultPersistMode specifies the default persist mode. Writes to the stream will immediately be flushed.
|
||||
// The publish acknowledgement will be sent after the persisting completes.
|
||||
DefaultPersistMode = PersistModeType(iota)
|
||||
// AsyncPersistMode specifies writes to the stream will be flushed asynchronously.
|
||||
// The publish acknowledgement may be sent before the persisting completes.
|
||||
// This means writes could be lost if they weren't flushed prior to a hard kill of the server.
|
||||
AsyncPersistMode
|
||||
)
|
||||
|
||||
func (rp RetentionPolicy) String() string {
|
||||
switch rp {
|
||||
case LimitsPolicy:
|
||||
return "Limits"
|
||||
case InterestPolicy:
|
||||
return "Interest"
|
||||
case WorkQueuePolicy:
|
||||
return "WorkQueue"
|
||||
default:
|
||||
return "Unknown Retention Policy"
|
||||
}
|
||||
}
|
||||
|
||||
func (rp RetentionPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch rp {
|
||||
case LimitsPolicy:
|
||||
return json.Marshal(limitsPolicyString)
|
||||
case InterestPolicy:
|
||||
return json.Marshal(interestPolicyString)
|
||||
case WorkQueuePolicy:
|
||||
return json.Marshal(workQueuePolicyString)
|
||||
default:
|
||||
return nil, fmt.Errorf("nats: can not marshal %v", rp)
|
||||
}
|
||||
}
|
||||
|
||||
func (rp *RetentionPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString(limitsPolicyString):
|
||||
*rp = LimitsPolicy
|
||||
case jsonString(interestPolicyString):
|
||||
*rp = InterestPolicy
|
||||
case jsonString(workQueuePolicyString):
|
||||
*rp = WorkQueuePolicy
|
||||
default:
|
||||
return fmt.Errorf("nats: can not unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dp DiscardPolicy) String() string {
|
||||
switch dp {
|
||||
case DiscardOld:
|
||||
return "DiscardOld"
|
||||
case DiscardNew:
|
||||
return "DiscardNew"
|
||||
default:
|
||||
return "Unknown Discard Policy"
|
||||
}
|
||||
}
|
||||
|
||||
func (dp DiscardPolicy) MarshalJSON() ([]byte, error) {
|
||||
switch dp {
|
||||
case DiscardOld:
|
||||
return json.Marshal("old")
|
||||
case DiscardNew:
|
||||
return json.Marshal("new")
|
||||
default:
|
||||
return nil, fmt.Errorf("nats: can not marshal %v", dp)
|
||||
}
|
||||
}
|
||||
|
||||
func (dp *DiscardPolicy) UnmarshalJSON(data []byte) error {
|
||||
switch strings.ToLower(string(data)) {
|
||||
case jsonString("old"):
|
||||
*dp = DiscardOld
|
||||
case jsonString("new"):
|
||||
*dp = DiscardNew
|
||||
default:
|
||||
return fmt.Errorf("nats: can not unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (pm PersistModeType) String() string {
|
||||
switch pm {
|
||||
case DefaultPersistMode:
|
||||
return "Default"
|
||||
case AsyncPersistMode:
|
||||
return "Async"
|
||||
default:
|
||||
return "Unknown Persist Mode"
|
||||
}
|
||||
}
|
||||
|
||||
func (pm PersistModeType) MarshalJSON() ([]byte, error) {
|
||||
switch pm {
|
||||
case DefaultPersistMode:
|
||||
return json.Marshal("default")
|
||||
case AsyncPersistMode:
|
||||
return json.Marshal("async")
|
||||
default:
|
||||
return nil, fmt.Errorf("nats: can not marshal %v", pm)
|
||||
}
|
||||
}
|
||||
|
||||
func (pm *PersistModeType) UnmarshalJSON(data []byte) error {
|
||||
switch strings.ToLower(string(data)) {
|
||||
case jsonString("default"):
|
||||
*pm = DefaultPersistMode
|
||||
case jsonString("async"):
|
||||
*pm = AsyncPersistMode
|
||||
default:
|
||||
return fmt.Errorf("nats: can not unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const (
|
||||
// FileStorage specifies on disk storage. It's the default.
|
||||
FileStorage StorageType = iota
|
||||
// MemoryStorage specifies in memory only.
|
||||
MemoryStorage
|
||||
)
|
||||
|
||||
const (
|
||||
memoryStorageString = "memory"
|
||||
fileStorageString = "file"
|
||||
)
|
||||
|
||||
func (st StorageType) String() string {
|
||||
switch st {
|
||||
case MemoryStorage:
|
||||
return "Memory"
|
||||
case FileStorage:
|
||||
return "File"
|
||||
default:
|
||||
return "Unknown Storage Type"
|
||||
}
|
||||
}
|
||||
|
||||
func (st StorageType) MarshalJSON() ([]byte, error) {
|
||||
switch st {
|
||||
case MemoryStorage:
|
||||
return json.Marshal(memoryStorageString)
|
||||
case FileStorage:
|
||||
return json.Marshal(fileStorageString)
|
||||
default:
|
||||
return nil, fmt.Errorf("nats: can not marshal %v", st)
|
||||
}
|
||||
}
|
||||
|
||||
func (st *StorageType) UnmarshalJSON(data []byte) error {
|
||||
switch string(data) {
|
||||
case jsonString(memoryStorageString):
|
||||
*st = MemoryStorage
|
||||
case jsonString(fileStorageString):
|
||||
*st = FileStorage
|
||||
default:
|
||||
return fmt.Errorf("nats: can not unmarshal %q", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func jsonString(s string) string {
|
||||
return "\"" + s + "\""
|
||||
}
|
||||
|
||||
const (
|
||||
// NoCompression disables compression on the stream. This is the default.
|
||||
NoCompression StoreCompression = iota
|
||||
|
||||
// S2Compression enables S2 compression on the stream.
|
||||
S2Compression
|
||||
)
|
||||
|
||||
func (alg StoreCompression) String() string {
|
||||
switch alg {
|
||||
case NoCompression:
|
||||
return "None"
|
||||
case S2Compression:
|
||||
return "S2"
|
||||
default:
|
||||
return "Unknown StoreCompression"
|
||||
}
|
||||
}
|
||||
|
||||
func (alg StoreCompression) MarshalJSON() ([]byte, error) {
|
||||
var str string
|
||||
switch alg {
|
||||
case S2Compression:
|
||||
str = "s2"
|
||||
case NoCompression:
|
||||
str = "none"
|
||||
default:
|
||||
return nil, errors.New("unknown compression algorithm")
|
||||
}
|
||||
return json.Marshal(str)
|
||||
}
|
||||
|
||||
func (alg *StoreCompression) UnmarshalJSON(b []byte) error {
|
||||
var str string
|
||||
if err := json.Unmarshal(b, &str); err != nil {
|
||||
return err
|
||||
}
|
||||
switch str {
|
||||
case "s2":
|
||||
*alg = S2Compression
|
||||
case "none":
|
||||
*alg = NoCompression
|
||||
default:
|
||||
return errors.New("unknown compression algorithm")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user