Utilities

Tokens

TexSoup.utils.CC

alias of CategoryCodes

TexSoup.utils.TC

alias of TokenCode

class TexSoup.utils.Token

Enhanced string object with knowledge of global position.

lstrip(*args, **kwargs)

Strip leading whitespace for text.

>>> t = Token('  asdf  ', 2)
>>> t.lstrip()
'asdf  '

rstrip(*args, **kwargs)

Strip trailing whitespace for text.

>>> t = Token('  asdf  ', 2)
>>> t.rstrip()
'  asdf'

class TexSoup.utils.CharToLineOffset(src)

Utility to convert absolute position in the source file to line_no:char_no_in_line. This can be very useful if we want to parse LaTeX and navigate to some elements in the generated DVI/PDF via SyncTeX.

>>> clo = CharToLineOffset('''hello
... world
... I scream for ice cream!''')
>>> clo(3)
(0, 3)
>>> clo(6)
(1, 0)
>>> clo(12)
(2, 0)

Buffer

class TexSoup.utils.Buffer(iterator, join=<bound method Token.join of <class 'TexSoup.utils.Token'>>, empty=<function Buffer.<lambda>>, init=<function Buffer.<lambda>>)

Converts string or iterable into a navigable iterator of strings.

>>> b1 = Buffer("012345")
>>> next(b1)
'0'
>>> b1.forward()
'1'
>>> b1.endswith('1')
True
>>> b1.backward(2)
'01'
>>> b1.peek()
'0'
>>> b1.peek(2)
'2'
>>> b1.peek((0, 2))
'01'
>>> b1.startswith('01')
True
>>> b1[2:4]
'23'
>>> Buffer('asdf')[:10]
'asdf'
>>> def gen():
...     for i in range(10):
...         yield i
>>> list(gen())
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(Buffer(gen()))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

backward(j=1)

Move backward by j steps.

>>> b = Buffer('abcdef')
>>> b.backward(-3)
'abc'
>>> b.backward(2)
'bc'

endswith(s)

Check if iterator ends with s, ending at current position.

forward(j=1)

Move forward by j steps.

>>> b = Buffer('abcdef')
>>> b.forward(3)
'abc'
>>> b.forward(-2)
'bc'

forward_until(condition, peek=True)

Forward until one of the provided matches is found.

The returned string contains all characters found before the condition was met. In other words, the condition will be true for the remainder of the buffer.

Parameters:condition (Callable) – lambda condition for the token to stop at

>>> buf = Buffer(map(str, range(9)))
>>> _ = buf.forward_until(lambda x: int(x) > 3)
>>> c = buf.forward_until(lambda x: int(x) > 6)
>>> c
'456'
>>> c.position
4

hasNext(n=1)

Returns whether or not there is another element.

num_forward_until(condition)

Forward until one of the provided matches is found.

Parameters:condition – set of valid strings

peek(j=0)

Peek at the next value(s), without advancing the Buffer.

Return None if index is out of range.

startswith(s)

Check if iterator starts with s, beginning from the current position.

TexSoup.utils.to_buffer(convert_in=True, convert_out=True, Buffer=<class 'TexSoup.utils.Buffer'>)

Decorator converting all strings and iterators/iterables into Buffers.

Parameters:

• convert_in (bool) – Convert inputs where applicable to Buffers
• convert_out (bool) – Convert output to a Buffer
• Buffer (type) – Type of Buffer to convert into