Tutorial

Status and compatibility information

The original implementation is done in python 3.5 and it is tested under python 3.5 and 3.9. It should work in all python 3 environments.

The actual development status is released.

Using the itertree package

To understand the full functionality of itertree the user should have a look on the related examples which can be found in the example folder of itertree.

In this chapter we try to dive in the functions of itertree in a clear structured way. The user might look in the class description of the modules too. But the huge number of methods in the iTree class might be very confusing. And we hope this chapter orders the things in a much better way.

Construction of an itertree

The first step in the construction of a itertree is to instance the itertree `iTree class`.

class itertree.iTree(tag, data=None, subtree=None)[source]

This is the main class related to iTrees.

This object is the parent of a sub-tree (children, sub-children, etc.). The iTree object itself can also be a child of a parent iTree object. If this is not the case the iTree object is the root of the tree.

A iTree object can only be integrated in one tree (one parent only)!

Each iTree object contains a tag. In case your tags are stings it’s recommended to use tag strings without wildcards “*”,”?” and without the standard separators “/” and “#”. If you use these characters you might get confusing results in find, filter and match operations.

In general we allow all hashable objects to be used as a tag in the iTree objects (only search operation might be limited in this case). But we have two exceptions: We do not allow integers and TagIdx objects as tags because those objects used for direct item access.

Different than in dictionaries it is allowed to put multiple times the same tag inside the iTree. The items with the same tag are placed and ordered (enumerated) in the related tag-family. They can be reached via TagIdx objects by giving the tag, index pair (tag_idx).

Linked iTree objects will behave different. They have a read only structure (children) and they contain the children (tree) of the linked iTree. The “local” attributes like tag, data, … can be set independent from the linked item (local properties). To change the tree structure of such an object you must manipulated the source object and reload the link.

Additionally a iTree object can contain:

  • data - a iTData object to store any kind of python objects

  • couple - you can couple the object to another one by giving a pointer

  • is_temporary - you can mark it as temporary. Those iTree items behave like normal ones. But they will not be considered during encoding for storage, etc.

There are different ways to access the children and sub-children in the tree of a iTree object.

The standard access for single items is via itree_obj[] (__getitem__()) call.

More complex access is available via find() and findall() methods. Have a look in the documentation related to each method.

The delivery of access related operations in the iTree objects is for unique targets an iTree object and for multi target operations an iterator over the matching items. We don’t deliver something like a list.

If really needed an iterator can be easily converted into a list by list() method but this may take a long time for huge iterators. The iterator should only be used in the final step of the operation. It’s recommended to have a look into itertools for better usage of the delivered iterators.

The design of the object is made to have best possible performance even that it is pure python. For more details you may run the performance tests in the test section (But you might have to install additional packages run the comparisons and to get the full picture.)

The function related to iterations iter; iter_children and find_all can be used with an item_filter. By this mechanism you can create queries regarding any property in an iTree.

To initialize the class the following parameters are available

Parameters
  • tag – tag string or hashable object used for the iTree identification

  • data – data dict or item to be stored in the node

  • subtree

    The subtree is a iterable structure that contains sub-items (iTree objects) that should be the children of this iTree.

    Warning

    subtree: In case the given iTree objects have already a parent an implicit copy will be made.

__init__(tag, data=None, subtree=None)[source]

Instance the iTree object:

>>> item1=iTree('item1') # itertree item with the tag 'item1'
>>> item2=iTree('item2', data={'mykey':1}) # instance a iTree-object with data content (defined as a dict)
>>> item3=iTreeTemporary('temp_item') # instance a temporary iTree-object
>>> # instance a iTree-object containing a link:
>>> item4=iTreeLink('linked_item', data={'mykey':2}, link_file_path='dt.itz', link_key_path=iTreeTagIdx('child',0), load_links=True)

iTreeTemporary objects can be filtered out and when dumping the whole iTree into a file the iTreeTemporary items are ignored and not stored.

In case a link is set by using the iTreeLink class will integrate the childs of the linked iTree-objects as it’s own childs into the tree. The iTree object can have own properties like temporary or own data. But it can also contain own, local children ( see iTree linked sub-trees ).

To add or manipulate the children of an item we have several possibilities. The following direct operations are recommended for structural manipulations in the tree:

>>> root=iTree('root')
>>> root.append(iTree('child')) # append a child
>>> root[0]=iTree('newchild') # replace the child with index 0
>>> del root[iTreeTagIdx('newchild',0)] # deletes the child with matching iTreeTagIdx

Additionally a huge set of methods is available for structural manipulations related to the children of a item.

itertree.iTree.append(self, item)

Append the given iTree object to the tree (new last child)

Except

raise TypeError in case iTree object has already a parent

Parameters

item – iTree object to be appended

Returns

True in case append was successful

itertree.iTree.__iadd__(self, other)

Implement self+=value.

itertree.iTree.appendleft(self, item)

Append the given iTree object to the left of the the tree (new first child)

Except

raise TypeError in case iTree object has already a parent

Parameters

item – iTree object to be appended

itertree.itree_main.iTree.extend(self, extend_items)

We extend the iTree with given items (multi append)

Note

In case the extend items have already a parent an implicit copy will be made. We do this because we might get an iTree-object as extend_items parameter and then the children will have automatically a parent even that the parent object might be a temporary one.

Parameters

extend_items – iterable object that contains iTree objects as items

Returns

True

itertree.itree_main.iTree.extendleft(self, extend_items)

We extend the iTree with given items in the beginning (multi appendleft)

Note

In case the extend items have already a parent an implicit copy will be made. We do this because we might get an iTree-object as extend_items parameter and then the children will have automatically a parent even that the parent object might be a temporary one.

Note

The extendleft() operation is a lot slower then the normal extend operation

Parameters

extend_items – iterable object that contains iTree objects as items

itertree.itree_main.iTree.insert(self, insert_key, item)

Insert an item before a specific position

Parameters
  • insert_key – position key (integer index or TagIdx)

  • item – item that should be inserted in the tree (new child)

itertree.itree_main.iTree.move(self, insert_key)

move the item in another position

Parameters

insert_key – item will be insert before this key

itertree.itree_main.iTree.rename(self, new_tag)

give the item a new tag

Parameters

new_tag – new tag object string or hashable object

itertree.itree_main.iTree.pop(self, key=- 1)

pop the item out of the tree, if no key is given the last item will be popped out

Parameters

key – specific identification key for an item (integer index, TagIdx)

Returns

popped out item (parent will be set to None)

itertree.itree_main.iTree.popleft(self)

pop the first item out of the tree

Returns

popped out item (parent will be set to None)

itertree.iTree.clear(self)

deletes all children and data! All flags stay unchanged!

The addition of iTrees is possible the result contains always the properties of the first added item and the children of the second added item are appended by creating a copy.

>>> a=iTree('a',data={'mykey':1},subtree=[iTree('a1'),iTree('a2')])
>>> b=iTree('b',subtree=[iTree('b1'),iTree('b2')])
>>> c=a+b
>>> c
iTree("'a'", data="{'mykey': 1}", subtree=[iTree("'a1'"), iTree("'a2'"), iTree("'b1'"), iTree("'b2'")])

Multiplication of a iTree is possible too the result is a list of iTree copies of the original one.

>>> itree_list=iTree('a')*1000 # creates a list of 1000 copies of the original iTree
>>> root=iTree('root')
>>> root.extend(itree_list) # we can extend an existing `iTree` with the list (add 1000 identical children)
True

item access

The items in the iTree can be accessed via __getitem__() method:

itertree.iTree.__getitem__()

Main getter for items

If given key targets to only one item we will deliver an iTree. If no matching item is found an IndexError or KeyError exception will be raised.

If the given key targets to multiple items (tag family, slice, iterable of single target keys) and iterator will be delivered.

Parameters

key – single target: index, TagIdx or tuple (tag, index) (not recommended) multi target: TagIdx_s; iMatch; slice or an iterable (like list) of these keys

Returns

iTree item or iterator (multi target)

>>> root=iTree('root')
>>> root+=iTree('child',data=0)
>>> root+=iTree('child',data=1)
>>> root+=iTree('child',data=2)
>>> root+=iTree('child',data=3)
>>> root+=iTree('child',data=4)
>>> root[1] # index access
iTree("'child'", data=1)
>>> root[TagIdx('child',1)] # TagIdx access (index targets the index in the tag family!)
iTree("'child'", data=1)
>>> root[TagIdx('child',-1)] # TagIdx access with negative index
iTree("'child'", data=4)
>>> root[TagIdx('child',[0,2])] # TagIdx give index list -> result is an iterator!
<list_iterator object at 0x0000029E12F69B00>
>>> list(root[TagIdx('child',[0,2])]) # make ietartor content visible by casting into a list
[iTree("'child'", data=0), iTree("'child'", data=2)]
>>> list(root[[0,2]]) # index list access (absolute indexes)
[iTree("'child'", data=0), iTree("'child'", data=2)]
>>> list(root[1:3]) # slices are allowed too
[iTree("'child'", data=1), iTree("'child'", data=2)]

The TagIdx class is used to address items that contains the same tag. The second argument of the TagIdx is the index that the item has in the related tag-family. But we can also give multiple indexes or a slice. As the given example shows is the result of not unique targets always an iterator object.

itertree.iTree.get_deep()

deep key access the function is a replacement for self[key_list[0]][key_list[1]]…[key_list[-1]] but you can also feed with an iterator

dives into the tree key_list=[1,0,2] -> second element level 1 -> first element level 2 -> third element level 3 -> same as self[1][0][2]

Note

Each key in the key list must target to a single item only! E.g. do not use tags here they deliver always a family iterator not a single item (the method will raise an exception). Use index integers or TagIdx objects instead

Parameters

key_list – list or iterator of keys (indexes,TagIdx, tuple(tag,index) -> only in case no tuple tags!

Returns

iTree object the key list targets

itertree.iTree.find()

The find function targets over multiple levels of the iTree, it returns single items only! This means in case the key_path targets to multiple items the default_return will be given. If the key_path targets to a family with only one item inside or the item_filter extracts only one item in a family the item will be given back as result. For multiple result utilize the find_all() method (which is slower).

Note

The method will deliver a default_return when ever in the whole key_path a match is not unique. This means iteration is stopped here and even that a deeper iteration with the defined filtering might deliver at least a unique result. To ensure to find this deeper results you must utilize the slower find_all() method.

The key_path parameter given is normally a list. This can be a list of keys or TagIdx objects. The function will search for the first item in the first level, fo next item in the next level and so on…

Absolut and relative key_paths:

If the first item is the separator (default: ‘/’) the find search is like an absolute path and we start at the root of the iTree. For compatibility reasons with find_all we accept a leading “./” (or to be exact: “.%s”#str_path_separator) as absolute path indicator. If the first item is different, the key_path is relative and we start from the actual item and search the children and sub-children.

Single string key_path: If the user searches for string type tags he can use a string with a separator (default: ‘/’) in between the tags (Those key_paths will be implicit translated in a list). An index separator (default = ‘#’) in between the tag and the index can also be used in this case. If the argument is already a list the single keys will not be parsed regarding the str_path_separator.

Note

If iTree contains tags with characters that used for separators or the all match ‘*’ character the find() result might contain that tagged item instead of the expected separated or wildcard match.

Note

Quickest find operations can be performed by giving a list containing index integers or TagIdx objects

The parameters in detail:

Parameters
  • key_path

    single key or list of keys identification path for the item/items to be searched. Possible keys: integer - behaves like normal __getitem__() -> itree_item[key] TagIdx- behaves like normal __getitem__() -> itree_item[key] iTreeTagSlice - select a tag sliced group of sub-elements iTMatch - search pattern can be used too, but keep in mind it must deliver a unique result! Slice - a slice of indexes (like a special index list) string - will be parsed by the separators, special string ‘*” is as interpreted as any match iterable list/tuple/deque,… -

    run over single items

  • item_filter – filters the item content regarding NORMAL, TEMPORARY and LINKED flag or a given filtering method

  • default_return – object will be return in case of no match (default = None)

  • str_path_separator – separator character in case of strings for the search levels (default: “/”)

  • str_index_separator – separator character for given tag indexes (default: “#”)

Returns

iTree single item

iTree compare items

itertree.iTree.__eq__()

A iTree object is always unique we test therefore just for matching object IDs This is needed for quick index findings! ..node:: To check if properties content is equal use equal() instead :param other: iTree object to compare with :return:

itertree.iTree.equal()

compares if the data content of another item matches with this item

Parameters
  • other – other iTree

  • check_parent – check if item has same parent object too? (Default False)

  • check_coupled – check the couple object too? (Default False)

Returns

boolean match result (True match/False no match)

Because the __eq__() method (== operator) is internally used for same item object findings. We compare here based on the python object id. Therefore for the comparison of two non possibly not identical objects the equal() method should be used.

itertree.iTree.__contains__()

checks if an iTree object is part of the iTree :param item: iTree object we searching for :return:

itertree.iTree.__hash__()

The hash operation is available but not a quick operation! ..node::: We do here not consider, parent and coupled item :return: integer hash

itertree.iTree.__len__()

Return len(self).

Based on the iTree length the comparison operators <; <=; >; >= are available too.

itertree.iTree.count()

count the number of children that match to the given filter :: note: The operation is not very quick on huge iTrees and complicate filters!

Parameters

item_filter

Returns

integer number of children matching to the filter

iTree properties

As we will see later on some properties of the iTree object can be modified by the related methods. Warning:: The user should NEVER modify any of the given properties directly. Especially the not discussed private properties (marked with the beginning underline). Direct modifications will normally lead into inconsistencies of the iTree object!

The iTree object contains the following general properties:

itertree.iTree.root()

property delivers the root item of the tree

Returns

iTree root item

itertree.iTree.is_root()

is this item a root item (has no parent)

Returns

True/False

itertree.iTree.parent()

property contains the parent item

Returns

iTree parent object (or None in case no parent exists)

itertree.iTree.pre_item()

delivers the pre item (predecessor) of this object

Returns

iTree predecessor or None (no match)

itertree.iTree.post_item()

delivers the post item (successor)

Returns

iTree successor or None (no match)

itertree.iTree.depth_up()

delivers the distance (number of levels) to the root element of the tree

Returns

integer

itertree.iTree.max_depth_down()

delivers the max_depth in the direction of the children

Returns

integer maximal children depth

itertree.iTree.is_temporary()

In contrast to iTreeTemporary class this is False

Returns

False

itertree.iTree.is_read_only()

In contrast to iTreeReadOnly class this is False

Returns

False

itertree.iTree.is_linked()

In contrast to iTreeLinked class this is False

Returns

False

Item identification properties:

itertree.iTree.idx()

Index of this object in the iTree

Returns

integer index

itertree.iTree.tag_idx()

Get the TagIdx object related to this object (contains the tag and the index of the object in the tag-family)

Returns

TagIdx

itertree.iTree.idx_path()

delivers the a list of indexes from the root to this item

Returns

list of index integers (here we do not deliver an iterator)

itertree.iTree.tag_idx_path()

delivers the a list of TagIdx objects from the root to this item

Returns

list of TagIdx (here we do not deliver an iterator)

>>> root=iTree('root')
>>> root+=iTree('child',data=0)
>>> root+=iTree((1,2),data='tuple_child0')
>>> root+=iTree('child',data=1)
>>> root+=iTree('child',data=2)
>>> root+=iTree((1,2),data='tuple_child1')
>>> root[0]+=iTree('subchild')
>>> root.render()
iTree('root')
 └──iTree('child', data=0)
    └──iTree('subchild')
 └──iTree((1, 2), data='tuple_child0')
 └──iTree('child', data=1)
 └──iTree('child', data=2)
 └──iTree((1, 2), data='tuple_child1')
>>> root[0][0].root
iTree("'root'", subtree=[iTree("'child'", data=0, subtree=[iTree("'subchild'")]), iTree("(1, 2)", data='tuple_child0'), iTree("'child'", data=1), iTree("'child'", data=2), iTree("(1, 2)", data='tuple_child1')])
>>> root[0][0].idx
0
>>> root[0][0].tag_idx
TagIdx('subchild', 0)
>>> root[0][0].idx_path
[0, 0]
>>> root[0][0].tag_idx_path
[TagIdx('child', 0), TagIdx('subchild', 0)]
>>> root[1].tag_idx
TagIdx((1, 2), 0)
>>> root[-1].tag_idx
TagIdx((1, 2), 1)

As shown in the last example hashable objects can be used as tags for the itertree items to be stored in the iTree object. Even for those kind of tag objects it is possible to store multiple items with the same tag. In the example the enumeration inside the tag family can be seen in the index enumeration in the TagIdx object.

Beside those structural properties the iTree objects contains some more properties that might be modified by the related methods.

itertree.iTree.coupled_object()

The iTree object can be couple with another python object. The pointer to the object is stored and can be reached via this property. (E.g. this can be helpful when connecting the iTree with a visual element (tree-list item) in a GUI)

Returns

pointer to coupled object

itertree.iTree.set_coupled_object()

User can couple this object with others with the help of this attribute .. note:: E.g. this might be an object in a GUI that are related to this item

Parameters

couple_object – object pointer to the object that should be coupled with this iTree item

Different than the data the coupled_obj is just a pointer to another python object. E.g. by this you might couple the iTree to a graphical user interface object e.g. an item in a hypertreelist or it can be used to couple the iTree object to an item in a mapping dictionary. The property couple_obj is not managed by the iTree object it’s just a place to store a pointer. In file exports or string exports this information will not be considered.

iTree iterators and queries

The standard iterator for iTrees delivers all children of the opbject. Beside this we have some special iterators that contain specific filter possibilities.

itertree.iTree.__iter__()

standard iterator over all items in the iTree :param item_filter: ALL = default :return:

itertree.iTree.iter_children()

main iterator in children level

Parameters

item_filter – the items can be filtered by giving a filter constants or giving a filter method or iTFilter object

Returns

iterator

itertree.iTree.iter_all()

main iterator for whole tree runs in top-> down order e.g.

iTree('child')
 └──iTree('sub0')
     └──iTree('sub0_0')
     └──iTree('sub0_1')
     └──iTree('sub0_2')
     └──iTree('sub0_3')
 └──iTree('sub1')
     └──iTree('sub1_0')

will be iterated like:

iTree('child')
iTree('sub0')
iTree('sub0_0')
iTree('sub0_1')
iTree('sub0_2')
iTree('sub0_3')
iTree('sub1')
iTree('sub1_0')
Parameters
  • item_filter – filter for filter the items you can give a filter constant or a method for filtering (should return True/False)

  • filter_or

    • True - we combine the filtering with or this means even if we have no match in the higher levels of the tree we will go deeper to find matches

    • False - filters are combined with and which means children will only be parsed in case the parent matches also to the filter condition

Returns

iterator

itertree.iTree.iter_all_bottom_up()

main iterator for whole tree runs in down-> top order (We start at the children and afterwards the parents: e.g.:

iTree('child')
 └──iTree('sub0')
     └──iTree('sub0_0')
     └──iTree('sub0_1')
     └──iTree('sub0_2')
     └──iTree('sub0_3')
 └──iTree('sub1')
     └──iTree('sub1_0')

Will be iterated:

iTree('sub0_0')
iTree('sub0_1')
iTree('sub0_2')
iTree('sub0_3')
iTree('sub0')
iTree('sub1_0')
iTree('sub1')
iTree('child')
Parameters
  • item_filter – filter method for filtering (should return True/False when fet with an item) or iTFilter object

  • filter_or

    • True - we combine the filtering with or this means even if we have no match in the higher levels of the tree we will go deeper to find matches

    • False - filters are combined with and which means children will only be parsed in case the parent matches also to the filter condition

Returns

iterator

itertree.iTree.iter_tag_idxs()

iter over all children and deliver the children TagIdx

Parameters

item_filter – the items can be filtered by giving a filter constants or giving a filter method or iTFilter object

Returns

iterator over the TagIdx of the children

itertree.iTree.index()

The index method allows to search for the index of the item in a parent object This is especially useful if you must use a item_filter. The delivered index is delivered relative to the given item filter!

For the item index of the item in the unfiltered tree (ALL) it’s recommended to use the idx property instead: (parent.index(item,ALL) == item.idx)

Parameters
  • item – item index should be delivered for

  • item_filter – filter integer; method can not handle filter methods yet!

Returns

index integer of the item relative to the given filter

Beside the classical iterators we have the more query related find methods:

itertree.iTree.find_all()

The find all function works on all levels of the tree. The key_path given (e.g. a list of indexes) addresses the items into the depth first item first level, second item second level,….

The method returns always an iterator or in case of no match an empty list! If you target to unique objects you will get anyway an iterator containing this unique element.

Warning

It’s possible to create invalid recursions when constructing the key_path. In these cases the recursion depth exceeded exception will be raised by the interpreter

In case the target in the upper keys is not unique, all matches will be delivered! e.g. The operation my_tree.find_all([‘child’,’sub_child’]) takes first all items in the “child” family:

TagIdx(‘child’,0),TagIdx(‘child’,1),…TagIdx(‘child’,n) in an iterator and in the next step the function will go one level deeper and will cumulate all the ‘sub_child’ families in these items as the result:

This means we have something like this:

my_tree[TagIdx(‘child’,0)][TagIdx(‘sub_child’,0)],my_tree[TagIdx(‘child’,0)][TagIdx(‘sub_child’,1)],…,

my_tree[TagIdx(‘child’,1)][TagIdx(‘sub_child’,0)],my_tree[TagIdx(‘child’,0)][TagIdx(‘sub_child’,1)],…,

and in case of no match in the keys items are skipped.

Note

It’s not at all the same as: my_tree[‘child’][‘sub_child’] -> this operation will raise an exception!

Note

When addressing a single item it’s quicker (~10x faster depending on tree depth) to use the get_deep() method instead of the find_all() method.

The key_path parameter is very flexible in case of the objects you put in. We have several possibilities:

  1. Special keys: We have the following special keys that might be used in the key_path:

    • “/” default path separator (might be changed by str_path_separator parameter) If this is the first key the find_all() search will be started in the root element not in the element the method is called.

      Note

      Be careful with “//” or “/” placed not in the beginning of the path this will rollback the find_all() to the root which means anything in the key_path before this key will be ignored.

    • “*”-wildcard will iterate over all children of the item

    • “**”-wildcard will iterate over all items of the item. The item itself is the first element of the iterator delivered

      Note

      find_all(‘**’) creates an different iterator then iter_all() list(my_tree.find_all(‘**’)) = [my_tree] + list(my_tree.iter_all())

    Warning

    It’s always recommended to avoid the usage of string tags containing functional characters like “**”,”*”,”/”,”#”,”?”. E.g. In case the iTree contains a family with the tag “/” or “*” or “**” the related family will be delivered. The special functionality is blocked in this moment (for “/” you might use the str_path_separator parameter to keep the functionality). Also filtering via iTMatch objects is limited in this case.

  2. Give normal keys like in __getitem__() method: normal keys can be:

    • index integers

    • tag strings

    • TagIdx,TagIdxStr,TagIdxBytes

    • TagMultiIdx,slices

    • for index lists you must give[[1,2,3,4]] because first level will be interpreted as

    • a list targeting into the depth of the tree

    e.g. by index

    • my_tree.find_all(1) is same as my_tree[1]

    • my_tree.find_all(‘child’) is same as my_tree[‘child’]

    • my_tree.find_all(TagIdx(‘child’,1))`is same as `my_tree[TagIdx(‘child’,1)]

  3. Give a list of normal keys:

    e.g. by index

    • `my_tree.find_all([1,2])`is same as my_tree[1][2]

    • my_tree.find_all([‘child’,’sub_child’]) delivers an iterator over all “sub_child” families found

      in all “child” families

    • my_tree.find_all([TagIdx(‘child’,1),TagIdx(‘sub_child’,1)])`is same as `my_tree[TagIdx(‘child’,1)][TagIdx(‘sub_child’,1)]

  4. Give iTMatch() object or list of iTMatch() objects:

    An iterator of all matching tags will be created the matches will be combined with the and operation. You can also use an item_filter containing the Filter.iTFilterItemTagMatch to have the same functionality. In case a list is given the find_all() function is again going one level deeper for each element in the list.

Parameters
  • key_path – iterable/iterator that addresses items in the tree (see above explanations and examples)

  • item_filter – item_filter method

  • str_path_separator – In case of string tags the user can give also strings that are internally casted into a list by using the str_path_separator (default=”/”) e.g.: “/child_tag/sub_child_tag” -> [“child_tag”,”sub_child_tag”]

  • str_index_separator – In case of string tags the user can give TgaIdx also by string definition this is the separator used to separate the index number from the tag (default=”#”) e.g. “child_tag#89” -> TagIdx(“child_tag”,89)

Returns

iterator over the matches or in case of no match found an empty list -> []

For filter creation we have some helper classes (itree_filter.py)

itertree.Filter.iTFilterTrue()

This filter might be useless but it delivers True for all items (or False if inverted).

Parameters
  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterItemType()

Filter for iTree types (we have iTree,ITreeReadOnly,iTreeTemporary,iTreeLink types)

Parameters
  • item_type – target type class

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterItemTagMatch()

Filter using the iTMatch object (have a look on th iTMatch for more details). In generalyou can use wild cards, etc. to find matching item tags

Parameters
  • match – iTMatch object that checks the item for a match

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterData()

This is the main data filter that allows a large number of different filtering based on iTree.data content. It’s the recommended filter for this proposes because different than the simpler data filters in this module we can filter based on combinations (key/value) related to the iTree.data items

Parameters
  • data_key – Checks if the given data key exists in item.data in case iTMatch is given matching keys will be considered None - all keys will be considered

  • data_value – Checks if the given data value exists in item.data in case iTMatch is given matching values will be considered, if iTInterval is given numerical values matching to interval will be considered. None - all values will be considered

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterDataKey()

Filters in all items for the data key given. Delivers all items that have the given key in there data

Parameters
  • data_key – Checks if the given data key exists in item.data

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterDataKeyMatch()

Filters in all items for the data key which matches to the given pattern (fnmatch search is used) you can use wildcards here. This filter works only on string or byte keys in the item.data (not on other objects)

Parameters
  • match_pattern – string/bytes that contains a match pattern

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

itertree.Filter.iTFilterDataValueMatch()

Filters in all items for containing a matching data value to given pattern. (Works only on string and byte values

Parameters
  • match_pattern – pattern fnmatch will search for (you can use wildcards here)

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

Depending on the data stored in the iTree.data object the user might create own filters. In general just a method must be created that takes the item as an argument and that delivers True in case of a match and False in case of no match. We have also a base class (super-class) of the given filters available which might be used for own filters too.

itertree.Filter.iTFilterBase()

Base/Super class for all itertree filter classes might be used for user defined filters too

Parameters
  • filter_method – method that is fet with an iTree item and that delivers True/False

  • pre_item_filter – Additional filter to combine with this filter (will always be calculated before this filter)

  • invert – True - invert the result of the filter (not) False (default) - result of filter is kept unchanged

  • use_and – True (default) - combine this filter with item_filter via and operator False - use or operator instead of and

The fitering in iTree is very effective and quick. As an example one might execute the example script itree_usage_example1.py where the itertree.Filter.iTFilterData object is used.

iTree formatted output

itertree.iTree.__repr__()

create representation string from which the object can be reconstructed via eval (might not work in case of data that do not have a working repr method) :return: representation string

itertree.iTree.renders()

render the iTree into a string

Parameters

item_filter – the items can be filtered by giving a filter constants or giving a filter method or iTFilter object

Returns

Tree representation as string

itertree.iTree.render()

print the rendered the iTree string to the terminal

Parameters

item_filter – the items can be filtered by giving a filter constants or giving a filter method or iTFilter object

iTree file storage

itertree.iTree.dump()

serializes the iTree object to JSON (default serializer) and store it in a file

Parameters
  • target_path – target path of the file where the iTree should be stored in

  • pack – True - data will be packed via gzip before storage

  • calc_hash – True - create the hash information of iTree and store it in the header

  • overwrite – True - overwrite an existing file

Returns

True if file is stored successful

itertree.iTree.dumps()

serializes the iTree object to JSON (default serializer)

Parameters

calc_hash – Tell if the hash should be calculated and stored in the header of string

Returns

serialized string (JSON in case of default serializer)

itertree.iTree.load()

create an iTree object by loading from a file

If not overloaded or reinitialized the iTree Standard Serializer will be used. In this case we expect a matching JSON representation.

Parameters
  • file_path – file path to the file that contains the iTree information

  • check_hash – True the hash of the file will be checked and the loading will be stopped if it doesn’t match False - do not check the iTree hash

  • load_links – True - linked iTree objects will be loaded

Returns

iTree object loaded from file

itertree.iTree.loads()

create an iTree object by loading from a string

If not overloaded or reinitialized the iTree Standard Serializer will be used. In this case we expect a matching JSON representation.

Parameters
  • data_str – source string that contains the iTree information

  • check_hash – True the hash of the file will be checked and the loading will be stopped if it doesn’t match False - do not check the iTree hash

  • load_links – True - linked iTree objects will be loaded

Returns

iTree object loaded from file

The file storage methods and the rendering methods are initialized by:

itertree.iTree.init_serializer()

Method sets the exchange environment that should be used. If you leave the parameters as default, the standard objects will be used.

Note

The method logic is called only one time the first time serializing is needed.

Parameters
  • force – False (Default) - do not reload in case we have already loaded the items

  • exporter – exporter object for file export of iTree (dump, dumps)

  • importer – importer object in ces a file import is done (load, loads)

  • serializer – Object serializer (especially needed for data objects!)

  • renderer – A renderer for pretty print output of the iTree object

Returns

None

This method is implicit executed and set to the default serializing functions of itertree. The user might load his own functionalities explicit by using this method or he might overload the iTree class and the init_serializer() method with his own functionality (e.g. an xml export/import might be realized by this).

iTree linked sub-trees

The iTree objects can be merged to one main tree from different source files by using the iTreeLink class. The result is a merged iTree that contains all the linked subtrees. Beside the linking from different files links inside a iTree structure (internal links) can be defined too.

Additionally the user can manipulate the linked items by making them local (covering) or by appending local items. The functionalities given here are limited to operations that do not imply a reordering of the elements in the tree. The reason for this is that the linked items cannot be reordered furthermore they gave the tree a fixed, static structure. E.g. mainly we have append() and make_self_local() functions and we cannot appendleft() or insert() because this would mean we have to reorder the other elements. A change of a linked structure can only be made by manipulating the original source structure. We allow only the localization of items that are a child of the linked root element, in deeper levels this is not possible.

The local items in a linked iTree are integrated in the tree during the load process of the linked elements. The identification is always made via the TagIdx of the item. The local storage of the tree contains iTreePlaceholder elements which will be replaced by the linked in elements during the load process. Those placeholders are needed to create the matching tag-idx combination for the real elements that should be kept after reload. In case the loaded structure is changed and and no matching item is found the iTreePlaceHolder items will remain in the iTree. All appended local items which are outside of the linked structure will be found at the end of the itertree.

Local items can be manipuplated as normal iTree items with one exception. In case a local item is deleted and a matching linked item is available (was covered by the local item) the linked item will replace the local element after deletion. This means in this case a delete of an item will not reduce the numbers of the elements. If the local item has no corresponding linked item the number of children will decrease as usual.

The linked items must be loaded by an explizit operation. They are not loaded automatically. The links must be loaded via the load_links() method which can be executed at any level of the tree and it will start loading all links in the subtree (use load_links() on the iTree root to be sure to load all links). The behavior in case of load erros can be switched between Exceptions or deleting invalid elements (delete_invalid_items parameter). In case of exceptions the iTree is in an incomplete load state and if the exception is kept this must be handled (e.g. copy original tree before loading and replace back). The commands for loading iTree files can be influenced by the load_links parameter (to activate or deactivate the link loading) during file load.

Warning

The user must be aware that changing the source structure and local items in parallel might lead to unexpected results. The identification of local items is always done via the TagIdx. If we miss items during load placeholders are used to keep the TagIdx of the “real” local items. Normally those artefacts will be replaced during the load with linked items (if found) but in case of missmatches they will stay in the tree. Using wild linking in between different iTree elements can lead into very confusing situations especially if the user removes local items. We recommend to use the feature only in special cases where the source architecture is clearly defined and remains structural relative stable. For stability reasons we have also functional limitations in iTreeLink objects (e.g. we do allow only linking on not already linked elements (protection for circular definitions); local items cannot be linked items or temporary items).

This class is used to define linked subtrees in a iTree object. The target source can be a subtree in another iTree related file (external links) or internal links to a subtree of the already loaded subtree.

Linking has some functional limitations so is it not allowed to link to already linked objects (we must protect iTree from circular definitions).

The iTreeLink objects supports local items which can be added additional to the linked items. Furthermore there is also a mechanism so that local items can overlay the linked items in the tree. This is done by localizing the linked items with the make_child_local() or make_self_local() method. Afterwards the item can be manipulated as a normal iTree object. Only exception is that after deleting such a overlaying item the linked item will come back into the iTree.

load all linked items

Parameters
  • force – False (default) - load only if not already loaded True - load even if already loaded (update)

  • delete_invalid_items – False (default) - in case of invalid items we will raise an exception! True - invalid items will be removed from parent no exception raised

  • _items – internal list parameter used for recursive calls of the function

Returns

  • True - success

  • False - load failed

Beside this the following specific functions are available on linked items:

itertree.iTreeLink.make_self_local()

make the current linked object a local object This is only possible if the parent parent is a normal iTree object -> only the first level children in a linked iTree can be made local The operation raises an SyntaxError in case it is used on a deeper level of the linked tree

Returns

None

itertree.iTreeLink.make_child_local()

make the item related to the given key a local object This is only possible if the parent of self is a normal iTree object -> only the first level children in a linked iTree can be made local The operation raises an SyntaxError in case it is used on a deeper level of the linked tree

Parameters

key – identification key for the child item that should be converted in a local item

Returns

None

itertree.iTreeLink.iter_locals()

iterator that iterates only over the local elements

Parameters

add_placeholders – If this flag is set the (normally ignored) placeholder items are included in the iteration

Returns

iterator over local items

For a better understanding please have a look in the example file examples/itree_link_example1.py in the package. That contains the following examples too.

Special functionalities related to linking of iTrees:

To link a subtree in the current tree the iTreeLinked class is used.

>>> #We create a small iTree:
>>> root = iTree('root')
>>> root += iTree('A')
>>> root += iTree('B')
>>> # we add "B" tag two times to get an enumerated tag family
>>> B=iTree('B')
>>> B +=iTree('Ba')
>>> #we create multiple 'Bb' elements to show how the placeholders are used during save and load
>>> B +=iTree('Bb')
>>> B +=iTree('Bb')
>>> B +=iTree('Bc')
>>> root += B
>>> #Now we create a internal link (but we disable the loading -> load_links=False):
>>> linked_element=iTreeLink('internal_link',link_key_path='/B',load_links=False)
>>> root.append(linked_element)
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
    └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
>>> # now we load the links:
>>> root.load_links()
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
     └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
         └──iTreeLink('Ba')
         └──iTreeLink('Bb')
         └──iTreeLink('Bb')
         └──iTreeLink('Bc')

As shown in the example the internal linked element contains now the same subtree as the element “B”. But they are integrated as iTreeLink objects which protects the items from changes (readonly). If we change the elements in the “B” item the changes are only considered if we reload the links in the tree!

>>> B +=iTree('B_post_append')
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
         └──iTree('B_post_append')
     └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
         └──iTreeLink('Ba')
         └──iTreeLink('Bb')
         └──iTreeLink('Bb')
         └──iTreeLink('Bc')
>>> root.load_links(force=True) # we must force the reloading, if not forced already loaded trees will not be updated
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
         └──iTree('B_post_append')
     └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
         └──iTreeLink('Ba')
         └──iTreeLink('Bb')
         └──iTreeLink('Bb')
         └──iTreeLink('Bc')
         └──iTreeLink('B_post_append')

The toplevel iTreeLink object allows manipulations of the subtree. We can append elements and we can change existing subitems to a local item taht covers the linked item and that can contain diffrent data and different children.

>>> #get the linked element
>>> il=root[TagIdx('internal_link',0)]
>>> #append an item
>>> il.append(iTree('new'))
>>> #we make second element local and append a item in the subtree
>>> local=il.make_child_local(2)
>>> local+=iTree('sublocal')
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
         └──iTree('B_post_append')
     └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
         └──iTreeLink('Ba')
         └──iTreeLink('Bb')
         └──iTree('Bb')
             └──iTree('sublocal')
         └──iTreeLink('Bc')
         └──iTreeLink('B_post_append')
         └──iTree('new')

The element ‘Bb’ in the linked subtree is now no more an iTreeLink object, its a normal iTree object. The identification of the covering item is internally always done via the TagIdx of the item. We can do all iTree related operations on this object. But there is one exception: if we delete the object the linked object will come back into the tree!

>>> del il[TagIdx('Bb',1)]
>>> print(root.render())
iTree('root')
     └──iTree('A')
     └──iTree('B')
     └──iTree('B')
         └──iTree('Ba')
         └──iTree('Bb')
         └──iTree('Bb')
         └──iTree('Bc')
         └──iTree('B_post_append')
     └──iTreeLink('internal_link', link=iTreeLink(file_path=None, key_path=['/', TagIdx(tag='B', idx=1)]))
         └──iTreeLink('Ba')
         └──iTreeLink('Bb')
         └──iTreeLink('Bb')
         └──iTreeLink('Bc')
         └──iTreeLink('B_post_append')
         └──iTree('new')

The link functionality in iTrees can be understood like the overloading mechanism of classes. By linking a subtree in the tree this is like defining a superclass for a specific tree section. By making a subitem local this part of the linked iTree is covered (overloaded). But we should not stress this analogy to much because the functionalities in this covered data structures are much less then we have it for the class concept.

iTree helpers classes

In the itertree helper module we have some helper classes that can be used to construct specific iTree objects.

We have the following helper classes available:

itertree.itree_helpers.iTInterval()[source]

helper class that defines an interval for range definitions in Data Models or Filters

the class contains a check if a given value is in the defined interval or not

The class might be a little bit under estimated in all the itertree functionalities but its a short but very powerful implementation of an Interval class for python.

The class contains anything you might need in case of a Interval functionality. You can given open/closed interval definitions including infinite limits. The intervals can be combined to a mathematical set via the pre_interval parameter. And the check method allows to give other limits as defined. This is especially useful for dynamically calculated limits.

The interval definition is also possible via a mathematical string like: “(1,2)” or “[10,+inf)”.

If you need a more advanced implementation you might have a look on the intervals/portion python package.

Note

For equal just set upper_limit to None (upper_open, lower_open parameter will be ignored in this case)

itertree.itree_helpers.iTInterval.__init__()

helper class that defines an interval for range definitions

the class contains a check if a given value is in the defined interval or not

Note

For equal you give lower_limit and set upper_limit to None (lower_open,upper_open parameters will be

ignored in this case). The math representation in this case is “== %s”%lower_limit

Note

The not_in=True can be given to invert the interval check result (match is anything outside the interval) in the math representation we add in this case a “!” before the interval

Note

Cascade interval definitions can be created the pre_interval definition e.g. math_repr= “(([1,5]) and [9,12]) and [100,200]’ valid values: 1…5,9…12,100..200

Parameters
  • lower_limit – lower limit of the interval

  • upper_limit – upper limit of the interval

  • lower_open – True - open interval (x>lower_limit) False - closed interval (x>=lower_limit)

  • upper_open – True - open interval (x<upper_limit) False - closed interval (x<=upper_limit)

  • not_in – False - check for in interval True - check for not in interval (outside)

  • pre_interval – Interval object to be checked before this interval

  • pre_and – True - combine the result of pre check with and this Interval check with the and operator False - combine the result of pre check with and this Interval check with the or operator

  • str_def – instance the object from given math_repr string (other parameters will be ignored in this case)

itertree.itree_helpers.iTMatch()[source]

The match object is used to defined match to elements in the DtaTree used in iterations over the DataTree The defined iMatch object can be used for checks against iTree objects (mainly for checks against the tag and also for string matches e.g. for finding iTree.data.keys() or .values() in filters.

itertree.itree_helpers.iTMatch.__init__()

Create a match pattern for different proposes. Depending on the type we have following functions:

  • int - check for an index

  • TagIdx - check for a TagIdx

  • str - string pattern using fnmatch

  • iterable like list, tuple, … combine the given patterns with the combine key

Parameters
  • pattern – give pattern

  • combine_or – True - or ; False - and; combination of matches/match patterns

itertree.itree_helpers.TagIdx()

TagIdx(tag, idx)

itertree.itree_helpers.TagIdxStr()[source]

Define a TagIdx by a sting with an index separator (default=’#’)

Example: “mytag#1” will be translated in the TagIdx(“mytag”,1)

Note

This makes only sense and can only be used if the tag is a string (not for other objects)

Parameters

tag_idx_str – string containing the definition

itertree.itree_helpers.TagIdxBytes()[source]

Define a TagIdx by bytes with an index separator (default=b’#’)

Example: b”mytag#1” will be translated in the TagIdx(b”mytag”,1)

Note

This makes only sense and can only be used if the tag is a byte (not for other objects)

Parameters

tag_idx_bytes – bytes containing the definition

The other classes in itree_helpers are used internally in the iTree object and might be less interesting for the user.

Addinionally the user might have also a look in the other itertree modules like itertree_data.py or itertree_filter.py. Especially the class iTDataModel might be a good starting point for own data model definitions as it is also shown in examples/itertree_data_model.py.

itertree.itree_data.iTDataModel()[source]

The default iTree data model class This the interface definition for specific data model classes that might be created using this superclass

The data model checks the given value for a specific data item. So that we can ensure that the given value matches to the expectations. We can check for types, shapes (length), limits, or matching patterns.

Besides the check we can also define a default formatter for the value that is used when it is translated into a string.

(see examples/itree_data_examples.py)