Stream Module

This module provides a class for streaming directory tarballs. This is used by browsepy.file.Directory.download() method.

TarFileStream

class browsepy.stream.TarFileStream(path, buffsize=10240, exclude=None)

Tarfile which compresses while reading for streaming.

Buffsize can be provided, it must be 512 multiple (the tar block size) for compression.

Note on corroutines: this class uses threading by default, but corroutine-based applications can change this behavior overriding the event_class and thread_class values.

event_class

alias of Event

fill()

Writes data on internal tarfile instance, which writes to current object, using write().

As this method is blocking, it is used inside a thread.

This method is called automatically, on a thread, on initialization, so there is little need to call it manually.

read(want=0)

Read method, gets data from internal buffer while releasing write() locks when needed.

The lock usage means it must ran on a different thread than fill(), ie. the main thread, otherwise will deadlock.

The combination of both write and this method running on different threads makes tarfile being streamed on-the-fly, with data chunks being processed and retrieved on demand.

Parameters:want (int) – number bytes to read, defaults to 0 (all available) Returns:tarfile data as bytes Return type:bytes

tarfile_class(name=None, mode='r', fileobj=None, bufsize=10240, **kwargs)

Open a tar archive for reading, writing or appending. Return an appropriate TarFile class.

mode: ‘r’ or ‘r:*’ open for reading with transparent compression ‘r:’ open for reading exclusively uncompressed ‘r:gz’ open for reading with gzip compression ‘r:bz2’ open for reading with bzip2 compression ‘r:xz’ open for reading with lzma compression ‘a’ or ‘a:’ open for appending, creating the file if necessary ‘w’ or ‘w:’ open for writing without compression ‘w:gz’ open for writing with gzip compression ‘w:bz2’ open for writing with bzip2 compression ‘w:xz’ open for writing with lzma compression

‘x’ or ‘x:’ create a tarfile exclusively without compression, raise

an exception if the file is already created

‘x:gz’ create a gzip compressed tarfile, raise an exception

if the file is already created

‘x:bz2’ create a bzip2 compressed tarfile, raise an exception

if the file is already created

‘x:xz’ create an lzma compressed tarfile, raise an exception

if the file is already created

‘r|*’ open a stream of tar blocks with transparent compression ‘r|’ open an uncompressed stream of tar blocks for reading ‘r|gz’ open a gzip compressed stream of tar blocks ‘r|bz2’ open a bzip2 compressed stream of tar blocks ‘r|xz’ open an lzma compressed stream of tar blocks ‘w|’ open an uncompressed stream for writing ‘w|gz’ open a gzip compressed stream for writing ‘w|bz2’ open a bzip2 compressed stream for writing ‘w|xz’ open an lzma compressed stream for writing

thread_class

alias of Thread

write(data)

Write method used by internal tarfile instance to output data. This method blocks tarfile execution once internal buffer is full.

As this method is blocking, it is used inside the same thread of fill().

Parameters:data (bytes) – bytes to write to internal buffer Returns:number of bytes written Return type:int