Pyjo.Iterator - Lazy iterator

from Pyjo.Iterator import Iterator

# Elements of an iterable, one by one, through a chain
for word in Iterator(['just', 'works']).map(lambda e: e.upper()):
    print(word)

# Or import the module, as in Google Python Style Guide

import Pyjo.Iterator

# Three requests at a time, and their titles in the order of the URLs
titles = Pyjo.Iterator.new(urls).map(ua.get_p, concurrency=3).map(lambda tx: tx.result.dom().at('title').text)
for title in titles:
    print(title)

# The same in a coroutine
async for title in titles:
    print(title)

# A collection, lazily
from Pyjo.Collection import c
c(1, 2, 3).iterate().map(lambda e: e * 2).join(', ').say()

Pyjo.Iterator is a lazy iterator of the elements of an iterable or of an asynchronous iterable, with the methods of Pyjo.Collection. Each step of a chain passes an element on to the next one as soon as it has it, so that the first element is processed before the last one is taken from the source. It is iterated with for and with async for alike, and each iteration takes the elements of the source anew, which a source that is iterated once, as a generator or a stream, gives once.

An iteration that ends before the end of the source, as of first_p(), of head() or of an exception, closes the asynchronous iterators of the chain with their aclose, as contextlib.aclosing() does, down to the one of the source, as an async generator or a stream, so that the garbage collector does not finalize them later.

A callback may return a Pyjo.Promise or another awaitable, or be a coroutine function, and the iterator passes on the value that it waits for. async for awaits it, and for waits for it with wait(), which runs the event loop, so that for cannot wait in an event loop that runs already, as in an action of a server, where it raises a RuntimeError, and where async for or a method whose name ends with _p waits instead. for takes the elements of an asynchronous iterable, as a stream, the same way.

# The messages of a stream as JSON, in a coroutine
async for request in messages.iterate().map(Pyjo.JSON.decode_json):
    ...

See also

Pyjoyment, https://mojolicious.org.

It has no counterpart in Mojolicious; it is inspired by the iterator helpers of JavaScript, with the methods of Pyjo.Collection.

class Pyjo.Iterator.Pyjo_Iterator(iterable: Iterable | AsyncIterable = ())[source]

Pyjo.Iterator inherits all methods from Pyjo.Base and implements the following new ones.

__aiter__() → AsyncIterator[source]
async for e in iterator:
    ...

Take the elements in a coroutine, and await the values of the callbacks.

__iter__() → Iterator[source]
for e in iterator:
    ...

Take the elements, and wait for the values of the callbacks with the event loop.

compact() → Self[source]
new = iterator.compact()

Pass on all elements that are not None and not an empty string.

# "0, 1, 2, 3"
Pyjo.Iterator.new([0, 1, None, 2, '', 3]).compact().join(', ')
each(cb: Callable[[T, int], Any]) → Self[source]
iterator = iterator.each(lambda e, num: ...)

Evaluate callback for each element, one after another, and wait for its value. The element will be the first argument passed to the callback, and its number the second.

# Make a numbered list
iterator.each(lambda e, num: print(f'{num}: {e}'))
each_p(cb: Callable[[T, int], Any]) → Pyjo_Promise[source]
promise = iterator.each_p(lambda e, num: ...)

Same as each(), but returns a Pyjo.Promise object, which is fulfilled with the iterator once the callback has been evaluated for each element.

# Process the messages one after another
await messages.iterate().each_p(handle)
first(cb: Callable[[T], Anything] | None = None, /) → T | None[source]
first(cb: Callable[[...], Any] | str | Pattern[str], /, *args: Any, **kwargs: Any) → T | None
first = iterator.first()
first = iterator.first(re.compile(r'foo'))
first = iterator.first(lambda e: ...)
first = iterator.first('some_method')
first = iterator.first('some_method', *args)

Evaluate regular expression/callback for, or call method on, each element and return the first one that matched the regular expression, or for which the callback/method returned true, without taking the elements after it.

# Find first value that is greater than 5
greater = iterator.first(lambda e: e > 5)
first_p(cb: Callable[[...], Any] | str | Pattern[str] | None = None, /, *args: Any, **kwargs: Any) → Pyjo_Promise[source]
promise = iterator.first_p(lambda e: ...)

Same as first(), but returns a Pyjo.Promise object, which is fulfilled with the element, or None.

flatten() → Pyjo_Iterator[Any][source]
new = iterator.flatten()

Flatten nested collections/lists/tuples recursively and pass on all elements.

# "1, 2, 3, 4, 5, 6, 7"
Pyjo.Iterator.new([1, [2, [3, 4], 5, [6]], 7]).flatten().join(', ')
grep(cb: Callable[[T], Anything], /) → Self[source]
grep(cb: Callable[[...], Any] | str | Pattern[str], /, *args: Any, **kwargs: Any) → Self
new = iterator.grep(re.compile(r'foo'))
new = iterator.grep(lambda e: ...)
new = iterator.grep('some_method')
new = iterator.grep('some_method', *args)

Evaluate regular expression/callback for, or call method on, each element and pass on all elements that matched the regular expression, or for which the callback/method returned true.

# Find all values that are greater than 5
greater = iterator.grep(lambda e: e > 5)
head(size: int) → Self[source]
new = iterator.head(4)

Pass on up to the specified number of elements from the beginning, and take no more of them.

# "A B C"
Pyjo.Iterator.new(['A', 'B', 'C', 'D', 'E']).head(3).join(' ')
join(string: str | None = None) → Pyjo_String_Unicode[source]
string = iterator.join()
string = iterator.join("\n")

Turn elements into Pyjo.String.Unicode.

# Join all values with commas
iterator.join(', ').say()
join_p(string: str | None = None) → Pyjo_Promise[source]
promise = iterator.join_p(', ')

Same as join(), but returns a Pyjo.Promise object, which is fulfilled with the Pyjo.String.Unicode.

map(cb: Callable[[T], Awaitable[U]], /, *, concurrency: int = 1) → Pyjo_Iterator[U][source]
map(cb: Callable[[T], U], /, *, concurrency: int = 1) → Pyjo_Iterator[U]
map(cb: Callable[[...], Any] | str, /, *args: Any, concurrency: int = 1, **kwargs: Any) → Pyjo_Iterator[Any]
new = iterator.map(lambda e: ...)
new = iterator.map(lambda e: ..., concurrency=3)
new = iterator.map('some_method')
new = iterator.map('some_method', *args)

Evaluate callback for, or call method on, each element and pass on its value. The element will be the first argument passed to the callback.

# Append the word "pyjo" to all values
pyjoified = iterator.map(lambda e: e + 'pyjo')

A callback that returns a Pyjo.Promise or another awaitable, or a coroutine function, is evaluated ahead for up to concurrency elements, so that as many promises are in progress at the same time, while the values are passed on in the order of the elements.

# Perform 3 requests at a time concurrently
iterator.map(ua.get_p, concurrency=3).each(lambda tx, num: print(tx.result.dom().at('title').text))

These options are currently available:

concurrency
concurrency=3

The maximum number of elements whose callbacks are in progress at the same time, 1 by default, as a source may have no end.

classmethod new(iterable: Iterable | AsyncIterable = ()) → Self[source]
iterator = Pyjo.Iterator.new([1, 2, 3])
iterator = Pyjo.Iterator.new(n for n in range(3))
iterator = Pyjo.Iterator.new(stream)

Construct a new Pyjo.Iterator object of an iterable or of an asynchronous iterable.

reduce(cb: Callable[[Any, T], Any], *initial: Any) → Any[source]
result = iterator.reduce(lambda a, b: ...)
result = iterator.reduce(lambda a, b: ..., initial)

Reduce elements with a callback and return its final result, passing a and b each time the callback is executed. The first time a will be an optional initial value or the first element. And from then on a will be the value of the callback, while b will always be the next element.

# Calculate the sum of all values
total = iterator.reduce(lambda a, b: a + b)
reduce_p(cb: Callable[[Any, T], Any], *initial: Any) → Pyjo_Promise[source]
promise = iterator.reduce_p(lambda a, b: ...)
promise = iterator.reduce_p(lambda a, b: ..., initial)

Same as reduce(), but returns a Pyjo.Promise object, which is fulfilled with the final result.

to_list() → Pyjo.Collection.object[T][source]
collection = iterator.to_list()

Take all elements and turn them into a Pyjo.Collection, which is a list.

to_list_p() → Pyjo_Promise[source]
promise = iterator.to_list_p()

Same as to_list(), but returns a Pyjo.Promise object, which is fulfilled with the Pyjo.Collection.

# The titles of all pages, 3 at a time
titles = await Pyjo.Iterator.new(urls).map(ua.get_p, concurrency=3).map(title).to_list_p()
uniq(cb: Callable[[...], Any] | str | None = None, *args: Any, **kwargs: Any) → Self[source]
new = iterator.uniq()
new = iterator.uniq(lambda e: ...)
new = iterator.uniq('some_method')
new = iterator.uniq('some_method', *args)

Pass on the elements without duplicates, using the string representation of either the elements or the value of the callback/method to decide uniqueness. Note that None and empty string are treated the same.

# "foo bar baz"
Pyjo.Iterator.new(['foo', 'bar', 'bar', 'baz']).uniq().join(' ')
Pyjo.Iterator.new(iterable: Iterable | AsyncIterable = ()) → Self[source]
iterator = Pyjo.Iterator.new([1, 2, 3])
iterator = Pyjo.Iterator.new(n for n in range(3))
iterator = Pyjo.Iterator.new(stream)

Construct a new Pyjo.Iterator object of an iterable or of an asynchronous iterable.