# `Chunx.Chunker.Recursive`
[🔗](https://github.com/preciz/chunx/blob/v0.2.0/lib/chunx/chunker/recursive.ex#L1)

Splits text through an ordered sequence of structural boundaries.

Text is split using each configured level in order. Segments that still exceed
`:chunk_size` are passed to the next level, while adjacent segments are merged
whenever they fit. The default hierarchy tries paragraphs, sentences,
punctuation, whitespace, and finally token boundaries.

# `chunk_opts`

```elixir
@type chunk_opts() :: [chunk_size: pos_integer(), levels: [level(), ...]]
```

# `level`

```elixir
@type level() :: [String.t(), ...] | :whitespace | :tokens
```

# `chunk`

```elixir
@spec chunk(binary(), Chunx.Tokenizer.t(), chunk_opts()) ::
  {:ok, [Chunx.Chunk.t()]} | {:error, term()}
```

Recursively splits text toward the `:chunk_size` target.

## Options

  * `:chunk_size` - Target maximum content-token count (default: 512). An
    indivisible grapheme may exceed the target.
  * `:levels` - Ordered splitting levels. Each level is a non-empty list of
    delimiters, `:whitespace`, or `:tokens`. Token splitting is always used as
    a final fallback when custom levels are exhausted.

## Examples

    iex> {:ok, tokenizer} = Tokenizers.Tokenizer.from_pretrained("gpt2")
    iex> text = "First paragraph." <> <<10, 10>> <> "Second paragraph."
    iex> {:ok, chunks} = Chunx.Chunker.Recursive.chunk(text, tokenizer, chunk_size: 4)
    iex> Enum.map(chunks, & &1.text)
    ["First paragraph." <> <<10, 10>>, "Second paragraph."]

---

*Consult [api-reference.md](api-reference.md) for complete listing*