Lazy Stream is a library for lazy evaluation in Python, inspired by the Scala Stream class.
Lazy Stream allows you to store and chain operations on a finite or infinite sequence of values. The final result will only be computed when needed, for better memory and performance over conventional Python lists.
Lazy Stream can be installed via pip:
pip install lazystreamLazy Stream supports basic stream operations and parallelism. Below is a quick overview of the current features. Feel free to request additional features.
Lazy Stream is very easy to use. You can create a stream from any function or iterator and obtain the results via evaluate. Finite and infinite streams are supported, but care must be taken when using infinite streams to avoid infinite operation.
from lazystream import LazyStream
# Finite stream from iterator
incremental_stream = LazyStream.from_iterator(iter(range(5)))
incremental_stream.evaluate()
# [0, 1, 2, 3, 4]
def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
# Infinite stream from function-defined iterator
fibo_stream = LazyStream.from_iterator(fibonacci())
fibo_stream.evaluate(5)
# [0, 1, 1, 2, 3]
fibo_stream.evaluate()
# Unsafe, will never terminateYou can also create a stream from a function, which will be evaluated lazily:
import random
from lazystream import LazyStream
# Infinite streams from function
random_stream = LazyStream.from_lambda(lambda: random.randint(0, 100))You can chain operations on streams, which will be evaluated lazily. Classic stream-compatible operations are supported, such as map and filter. Non-stream-compatible operations are not supported and should be done after evaluation.
from lazystream import LazyStream
stream = LazyStream.from_iterator(iter(range(10)))
stream.filter(lambda x: x % 2 == 0).map(lambda x: x * 2).evaluate()
# [0, 4, 8, 12, 16]You can obtain the results of a stream as a list via evaluate, which allows you to optionally set a limit on the number of elements to evaluate. This is useful/required for infinite streams.
from lazystream import LazyStream
stream = LazyStream.from_lambda(lambda: 1)
stream.evaluate(5)
# [1, 1, 1, 1, 1]You can also use the stream as an iterator itself. Note that proper termination conditions must be set for infinite streams.
import random
from lazystream import LazyStream
# Iterate on a finite stream
finite_stream = LazyStream.from_iterator(iter(range(10)))
for x in finite_stream:
print(x)
# Iterate on an infinite stream
infinite_stream = LazyStream.from_lambda(lambda: random.randint(0, 1))
for x in infinite_stream:
print(x)
if x == 1:
breakIn addition, the reduce operation is supported, which allows you to obtain a single value from a stream.
from lazystream import LazyStream
# Reduce on a finite stream
stream = LazyStream.from_iterator(iter(range(10)))
stream.reduce(lambda x, y: x + y, accum=0)
# 45
# Reduce on an infinite stream
stream = LazyStream.from_lambda(lambda: 1)
stream.reduce(lambda x, y: x + y, accum=0, limit=5)
# 5You can add parallelism to the stream via functional mapping par_map and via results evaluation par_evaluate. Due to Python's parallelism implementation, this is only useful if your mapping function is IO-bound.
Note that Lazy Stream does not check for thread safety.
from concurrent.futures import ThreadPoolExecutor
from lazystream import LazyStream
def io_bound_function(x):
# Do some IO-bound operation
return x
stream = LazyStream.from_iterator(iter(range(10)))
stream.par_map(
io_bound_function, executor=ThreadPoolExecutor(4)
).evaluate()
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent