Collection Operators

Overview

rill provides five collection operators for transforming, filtering, and reducing data:

All five operators are host-callable functions. They share similar invocation patterns but differ in execution model and output.

Important: Loop bodies cannot modify outer-scope variables (see Variables). Use fold or acc with accumulators instead.

# Sequential: results in order, one at a time
[1, 2, 3] -> seq({ $ * 2 })
# Result: [2, 4, 6]

# Parallel: results in order, concurrent execution
[1, 2, 3] -> fan({ $ * 2 })
# Result: [2, 4, 6]

# Parallel filter: keep matching elements
[1, 2, 3, 4, 5] -> filter({ $ > 2 })
# Result: [3, 4, 5]

# Reduction: accumulates to single value
[1, 2, 3] -> fold(0, { $@ + $ })
# Result: 6

# Scan: accumulates, returns all intermediate values
[1, 2, 3] -> acc(0, { $@ + $ })
# Result: [1, 3, 6]

seq — Sequential Iteration

seq iterates over a collection in order. Each iteration completes before the next begins. Returns a list of all body results.

# Double each number
[1, 2, 3] -> seq({ $ * 2 })
# Result: [2, 4, 6]

# Transform strings
["a", "b", "c"] -> seq({ "{$}!" })
# Result: ["a!", "b!", "c!"]

# Iterate string characters
"hello" -> seq({ $ })
# Result: ["h", "e", "l", "l", "o"]

Dict Iteration

When iterating over a dict, $ contains key and value fields.

[name: "alice", age: 30] -> seq({ "{$.key}: {$.value}" })
# Result: ["name: alice", "age: 30"]

[a: 1, b: 2, c: 3] -> seq({ $.value * 2 })
# Result: [2, 4, 6]

Variable Closure

Pass a pre-defined closure by reference.

|x| ($x * 2) => $double
[1, 2, 3] -> seq($double)
# Result: [2, 4, 6]

Early Termination

seq catches break. Returns partial results collected before the break.

[1, 2, 3, 4, 5] -> seq({
  ($ == 3) ? break
  $ * 2
})
# Result: [2, 4]

Empty Collections

seq returns [] for empty collections. The body never executes.

[] -> seq({ $ * 2 })
# Result: []

Streams with seq

use<ext:app> => $app

# Stream: each chunk is one call
$app.lines("file.txt") -> seq({ $ -> .upper })
# Returns list of uppercased lines

An empty stream returns [] without executing the body.

fan — Parallel Iteration

fan iterates concurrently. Order is preserved despite parallel execution. Does not catch break.

# Map with closure parameter
["a", "b", "c"] -> fan({ "{$}!" })
# Result: ["a!", "b!", "c!"]

# Block expression (implicit $)
[1, 2, 3] -> fan({ $ * 2 })
# Result: [2, 4, 6]

Concurrency Limit

Pass an options dict as the second argument to cap concurrent operations.

[1, 2, 3, 4, 5] -> fan({ $ * $ }, [concurrency: 2])
# Result: [1, 4, 9, 16, 25]

Key Differences from seq

1. No accumulator: Parallel execution has no “previous” value
2. No break: break in a fan body bubbles up as a halt signal
3. Concurrent execution: All iterations start immediately

When to Use fan

Use fan when:

• Operations are independent (no shared state)
• Order of execution does not matter (results still ordered)
• I/O-bound operations benefit from concurrency

# CPU-bound: same result as seq, but runs in parallel
[1, 2, 3, 4, 5] -> fan({ $ * $ })
# Result: [1, 4, 9, 16, 25]

Empty Collections

fan returns [] for empty collections. The body never executes.

[] -> fan({ $ * 2 })
# Result: []

Streams with fan

use<ext:app> => $app

# Each stream chunk is transformed; result is a list
$app.stream_numbers() -> fan({ $ * 2 })
# Returns list[...] — not a stream

An empty stream returns [].

filter — Parallel Filtering

filter keeps elements where the predicate returns true. Predicates must return boolean values. Executes concurrently. Does not catch break.

# Keep numbers greater than 2
[1, 2, 3, 4, 5] -> filter({ $ > 2 })
# Result: [3, 4, 5]

# Keep non-empty strings
["hello", "", "world", ""] -> filter({ !.empty })
# Result: ["hello", "world"]

# Keep even numbers
[1, 2, 3, 4, 5, 6] -> filter({ ($ % 2) == 0 })
# Result: [2, 4, 6]

Variable Closure

Pass a pre-defined closure by reference.

|x| ($x > 2) => $gtTwo
[1, 2, 3, 4, 5] -> filter($gtTwo)
# Result: [3, 4, 5]

Dict Filtering

When filtering a dict, $ contains key and value fields. Returns list of matching entries.

[a: 1, b: 5, c: 3] -> filter({ $.value > 2 })
# Result: [[key: "b", value: 5], [key: "c", value: 3]]

String Filtering

Filters characters in a string.

"hello" -> filter({ $ != "l" })
# Result: ["h", "e", "o"]

Chaining with Other Operators

# Filter then transform
[1, 2, 3, 4, 5] -> filter({ $ > 2 }) -> fan({ $ * 2 })
# Result: [6, 8, 10]

# Transform then filter
[1, 2, 3, 4, 5] -> fan({ $ * 2 }) -> filter({ $ > 5 })
# Result: [6, 8, 10]

# Filter, transform, reduce
[1, 2, 3, 4, 5] -> filter({ $ > 2 }) -> fan({ $ * 2 }) -> fold(0, { $@ + $ })
# Result: 24

Empty Collections

filter returns [] for empty collections or when nothing matches.

[] -> filter({ $ > 0 })
# Result: []

[1, 2, 3] -> filter({ $ > 10 })
# Result: []

Streams with filter

use<ext:app> => $app

# Each stream chunk is tested; result is a list
$app.stream_numbers() -> filter({ $ > 0 })
# Returns list[...] of matching chunks — not a stream

An empty stream returns [].

fold — Sequential Reduction

fold reduces a collection to a single value. Requires a seed and a body callable. Does not catch break.

# Sum numbers
[1, 2, 3] -> fold(0, { $@ + $ })
# Result: 6

# Same with inline closure
[1, 2, 3] -> fold(0, |item|($@ + $item))
# Result: 6

Common Patterns

Sum

[1, 2, 3, 4, 5] -> fold(0, { $@ + $ })
# Result: 15

Product

[1, 2, 3, 4] -> fold(1, { $@ * $ })
# Result: 24

Maximum

[3, 1, 4, 1, 5, 9] -> fold(0, {
  ($@ > $) ? $@ ! $
})
# Result: 9

Count

[1, 2, 3, 4, 5] -> fold(0, { $@ + 1 })
# Result: 5

String Join

["a", "b", "c"] -> fold("", { "{$@}{$}" })
# Result: "abc"

Typed Accumulator Closure

Use a two-type anonymous closure to declare element and accumulator types.

[1, 2, 3] -> fold(0, |number, number|{ $@ + $ })
# Result: 6

Dict Reduction

When folding over a dict, $ contains key and value fields.

[a: 1, b: 2, c: 3] -> fold(0, { $@ + $.value })
# Result: 6

Reusable Reducers

Define closures for common reductions.

|x| ($@ + $x) => $summer
|x| (($x > $@) ? $x ! $@) => $maxer

[1, 2, 3] -> fold(0, $summer) => $r1
[3, 7, 2] -> fold(0, $maxer) => $r2
[9, 1, 5] -> fold(0, $maxer) => $r3
$r1
# Result: 6

Empty Collections

fold returns the initial value for empty collections. The body never executes.

[] -> fold(0, { $@ + $ })
# Result: 0

[] -> fold(42, { $@ + $ })
# Result: 42

Streams with fold

use<ext:app> => $app

# Reduce all stream chunks to a single value
$app.stream_numbers() -> fold(0, { $@ + $ })
# Returns the sum of all chunks

An empty stream returns the initial value without executing the body.

acc — Sequential Scan

acc iterates sequentially with a running accumulator. Unlike fold, it returns a list of all intermediate accumulator values, not just the final one. Catches break.

# Running sum (scan pattern)
[1, 2, 3] -> acc(0, { $@ + $ })
# Result: [1, 3, 6]

# String concatenation scan
["a", "b", "c"] -> acc("", { "{$@}{$}" })
# Result: ["a", "ab", "abc"]

Typed Accumulator Closure

Use a two-type anonymous closure to declare element and accumulator types.

[1, 2, 3] -> acc(0, |number, number|{ $@ + $ })
# Result: [1, 3, 6]

Early Termination

acc catches break. Returns partial intermediate results collected before the break.

[1, 2, 3, 4, 5] -> acc(0, {
  ($ == 3) ? break
  $@ + $
})
# Result: [1, 3]

Empty Collections

acc returns [] for empty collections. The body never executes.

[] -> acc(0, { $@ + $ })
# Result: []

Streams with acc

use<ext:app> => $app

# Accumulator persists across stream chunks
$app.stream_numbers() -> acc(0, { $@ + $ })
# Returns running totals across all chunks

An empty stream returns [].

Comparison: acc vs fold

Both acc and fold support accumulators. The difference is what they return.

Side-by-Side Example

# acc: returns every intermediate result
[1, 2, 3] -> acc(0, { $@ + $ })
# Result: [1, 3, 6]

# fold: returns only the final result
[1, 2, 3] -> fold(0, { $@ + $ })
# Result: 6

When to Choose

Use acc when you need intermediate states (scan pattern):

# Running balance
[100, -50, 200, -75] -> acc(0, { $@ + $ })
# Result: [100, 50, 250, 175]

Use fold when you only need the final result:

# Final balance
[100, -50, 200, -75] -> fold(0, { $@ + $ })
# Result: 175

Stream Iteration

Streams produce chunks over time. Collection operators consume all chunks before returning. All stream examples use text fences because stream host functions are unavailable in the test harness.

break in Stream Operators

break stops iteration immediately. The host cleanup function (dispose) runs to release stream resources.

use<ext:app> => $app

# Stop after the first matching chunk; host disposes the stream
$app.log_stream() -> seq({
  ($ -> .contains("ERROR")) ? break
  $
})

Stream Chunk Limits

The ^(limit: N) annotation is not valid inside collection operator calls. The parser rejects it with RILL-R081.

use<ext:app> => $app

# Rejected — RILL-R081: ^(limit: N) is not accepted inside seq()
$app.events() -> seq(^(limit: 100) { $ })

To stop early, use break inside the operator body. Inner scopes cannot reassign outer variables, so use the incoming value $ or an acc/fold accumulator for stateful conditions:

use<ext:app> => $app

$app.events() -> seq({
  ($ > 100) ? break
  $
})

Iteration Ceiling

Exactly 10,000 chunks complete without error. The 10,001st chunk triggers RILL-R010 and halts execution. Use break inside the body to consume at most N chunks and stay within bounds.

use<ext:app> => $app

# Error: exceeds iteration ceiling
$app.infinite_stream() -> seq({ $ })
# RILL-R010: halts on the 10,001st chunk

Re-iteration Halt

A consumed stream cannot be re-iterated. Passing a consumed stream to a second operator halts execution with an error.

use<ext:app> => $app

# Error: stream already consumed
$app.stream_numbers() => $s
$s -> seq({ $ })
$s -> seq({ $ })   # Halts: stream is consumed

Chaining Operators

Combine operators for multi-stage transformations.

# Double each element, then sum
[1, 2, 3] -> fan({ $ * 2 }) -> fold(0, { $@ + $ })
# Result: 12

# Filter even numbers (using parallel filter)
[1, 2, 3, 4, 5] -> filter({ ($ % 2) == 0 })
# Result: [2, 4]

# Complex pipeline: filter, then transform
[1, 2, 3, 4, 5] -> filter({ $ > 2 }) -> fan({ $ * 10 })
# Result: [30, 40, 50]

Error Cases

Iterating Different Types

Lists

[1, 2, 3] -> seq({ $ * 2 })
# Result: [2, 4, 6]

Strings

Iterates over characters.

"abc" -> seq({ "{$}!" })
# Result: ["a!", "b!", "c!"]

Dicts

Iterates over entries with key and value fields.

[a: 1, b: 2] -> seq({ "{$.key}={$.value}" })
# Result: ["a=1", "b=2"]

Nested Collections

Process nested structures with nested operators.

# Double nested values
[list[1, 2], list[3, 4]] -> fan({ $ -> seq({ $ * 2 }) })
# Result: [[2, 4], [6, 8]]

# Sum all nested values
[list[1, 2], list[3, 4]] -> fold(0, { $@ + ($ -> fold(0, { $@ + $ })) })
# Result: 10

Performance Considerations

Sequential vs Parallel

Memory

• seq, fan, acc allocate result lists proportional to input size
• fold maintains constant memory (accumulator only)

Quick Reference

# seq - sequential, all results
[1, 2, 3] -> seq({ $ * 2 })            # [2, 4, 6]

# fan - parallel, all results
[1, 2, 3] -> fan({ $ * 2 })            # [2, 4, 6]
[1, 2, 3] -> fan({ $ * 2 }, [concurrency: 2])   # concurrency cap

# filter - parallel, matching elements
[1, 2, 3, 4, 5] -> filter({ $ > 2 })   # [3, 4, 5]

# fold - sequential, final result only
[1, 2, 3] -> fold(0, { $@ + $ })       # 6

# acc - sequential, all intermediate results
[1, 2, 3] -> acc(0, { $@ + $ })        # [1, 3, 6]

# Dict iteration
[a: 1, b: 2] -> seq({ $.key })         # ["a", "b"]
[a: 1, b: 2] -> seq({ $.value })       # [1, 2]

# Break (seq and acc only)
[1, 2, 3] -> seq({ ($ > 2) ? break ! $ })  # [1, 2]

# Empty collections
[] -> seq({ $ })       # []
[] -> fan({ $ })       # []
[] -> filter({ $ })    # []
[] -> fold(42, { $ })  # 42
[] -> acc(0, { $ })    # []

See Also

• Iterators — Lazy sequences with range, repeat, .first()
• Reference — Complete language specification
• Closures — Closure semantics and patterns
• Guide — Beginner introduction
• Examples — Workflow examples