Getting Information on Datasets¶
This second User Guide tutorial will introduce you to getting information on Pymicro’s datasets, and how to visualize datasets content. You will find a short summary of all methods reviewed in this tutorial at the end of this page.
Warning
This Notebook review the methods to get information on SampleData HDF5 datsets content. Some of the methods detailed here produce very long outputs, that have been conserved in the documentation version. Reading completely the content of this output is absolutely not necessary to learn what is detailed on this page, they are just provided here as examples. So do not be afraid free to scroll down quickly when you see large prints !
[1]:
from pymicro.core.samples import SampleData as SD
I - The SampleData Naming system¶
SampleData datasets are composed of a set of organized data items. When handling datasets, you will need to specify which item you want to interact with or create. The SampleData class provides 4 different ways to refer to datasets. The first type of data item identificator is :
the Path of the data item in the HDF5 file.
Like a file within a filesystem has a path, HDF5 data items have a Path within the dataset. Each data item is the children of a HDF5 Group (analogous to a file contained in a directory), and each Group may also be children of a Group (analogous to a directory contained in a directory). The origin directory, that we already encountered above, is called the root group, and has the path '/'. The Path offers a completely non-ambiguous way to designate a data item within the dataset, as it is
unique. A typical path of a dataitem will look like that : /Parent_Group1/Parent_Group2/ItemName. However, pathes can become very long strings, and are usually not a convenient way to name data items. For that reason, you also can refer to them in SampleData methods using:
the Name of the data item.
It is the last element of its Path, that comes after the last / character. For a dataset that has the path /Parent_Group1/Parent_Group2/ItemName, the dataset Name is ItemName. It allows to refer quickly to the data item without writing its whole Path.
However, note that two different datasets may have the same Name (but different pathes), and thus it may be necessary to use additional names to refer to them with no ambiguity without having to write their full path. In addition, it may be convenient to be able to use, in addition to its storage name, one or more additional and meaningfull names to designate a data item. For these reasons, two additional identificators can be used:
the Indexname of the data item
the Alias or aliases of the data item
Those two types of indentificators are strings that can be used as additional data item Names. They play completely similar roles. The Indexname is also used in the dataset Index (see below), that gather the data item indexnames together with their pathes within the dataset. All data items must have an Indexname, that can be identical to their Name. If additionnal names are given to a dataset, they are stored as an Alias.
Many SampleData methods have a nodename or name argument. Everytime you will encounter it, you may use one of the 4 identificators presented in this section, to provide the name of the dataset you want to create or interact with. Many examples will follow in the rest of this Notebook, and of this User Guide.
Let us now move on to discover the methods that allow to explore the datasets content.
II- Interactively get information on datasets content¶
The goal of this section is to review the various way to get interactive information on your SampleData dataset (interactive in the sens that you can get them by executing SampleData class methods calls into a Python interpreter console).
For this purpose, we will use a pre-existing dataset that already has some data stored, and look into its content. This dataset is a reference SampleData dataset used for the core package unit tests.
[2]:
from pymicro import get_examples_data_dir # import file directory path
PYMICRO_EXAMPLES_DATA_DIR = get_examples_data_dir() # get the file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'test_sampledata_ref') # test dataset file path
data = SD(filename=dataset_file)
1- The Dataset Index¶
As explained in the previous section, all data items have a Path, and an Indexname. The collection of Indexname/Path pairs forms the Index of the dataset. For each SampleData dataset, an Index Group is stored in the root Group, and the collection of those pairs is stored as attributes of this Index Group. Additionnaly, a class attribute content_index stores them as a dictionary in the clas instance, and allows to access them easily. The dictionary and the Index
Group attributes are automatically synchronized by the class.
Let us see if we can see the dictionary content:
[3]:
data.content_index
[3]:
{'array': '/test_group/test_array',
'group': '/test_group',
'image': '/test_image',
'image_Field_index': '/test_image/Field_index',
'mesh': '/test_mesh',
'mesh_ElTagsList': '/test_mesh/Geometry/Elem_tags_list',
'mesh_ElTagsTypeList': '/test_mesh/Geometry/Elem_tag_type_list',
'mesh_ElemTags': '/test_mesh/Geometry/ElementsTags',
'mesh_Elements': '/test_mesh/Geometry/Elements',
'mesh_Field_index': '/test_mesh/Field_index',
'mesh_Geometry': '/test_mesh/Geometry',
'mesh_NodeTags': '/test_mesh/Geometry/NodeTags',
'mesh_NodeTagsList': '/test_mesh/Geometry/Node_tags_list',
'mesh_Nodes': '/test_mesh/Geometry/Nodes',
'mesh_Nodes_ID': '/test_mesh/Geometry/Nodes_ID',
'mesh_Test_field1': '/test_mesh/Test_field1',
'mesh_Test_field2': '/test_mesh/Test_field2',
'mesh_Test_field3': '/test_mesh/Test_field3',
'mesh_Test_field4': '/test_mesh/Test_field4',
'tensor9_T1_0': '/test_image/test_tensor_T1_0',
'tensor9_T2_0': '/test_image/test_tensor_T2_0',
'tensor9_T3_0': '/test_image/test_tensor_T3_0',
'test_image_field': '/test_image/test_image_field',
'test_table': '/test_group/test_table',
'vector_T1_0': '/test_mesh/test_vector_T1_0',
'vector_T2_0': '/test_mesh/test_vector_T2_0'}
You should see the dictionary keys that are names of data items, and associated values, that are hdf5 pathes. You can see also data item Names at the end of their Pathes. The data item aliases are also stored in a dictionary, that is an attribute of the class, named aliases:
[4]:
data.aliases
[4]:
{}
You can see that this dictionary contains keys only for data item that have additional names, and also that those keys are the data item indexnames.
The dataset index can be plotted together with the aliases, with a prettier aspect, by calling the method print_index:
[5]:
data.print_index()
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
Name : test_table H5_Path : /test_group/test_table
Name : vector_T1_0 H5_Path : /test_mesh/test_vector_T1_0
Name : vector_T2_0 H5_Path : /test_mesh/test_vector_T2_0
This method prints the content of the dataset Index, with a given depth and from a specific root. The depth is the number of parents that a data item has. The root Group has thus a depth of 0, its childrens a depth of 1, the childrens of its childrens a depth of 2, and so on… The local root argument can be changed, to print only the Index for data items that are children of a specific group. When used without arguments, print_index uses a depth of 3 and the dataset root as default
settings.
As you can see, our dataset already contains some data items. We can already identify 3 HDF5 Groups (test_group, test_image, test_group), as they have childrens, and a lot of other data items.
Let us try different to print Indexes with different parameters. To start, Let us try to print the Index from a different local root, for instance the group with the path /test_image. The way to do it is to use the local_root argument. We will hence give it the value of the /test_image path.
[6]:
data.print_index(local_root="/test_image")
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/test_image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
The print_index method local root arguments needs the name of the Group whose children Index must be printed. As explained in section II, you may use for this other identificators than its Path. Let us try its Name (last part of its path), which is test_image, or its Indexname, which is image:
[7]:
data.print_index(local_root="test_image")
data.print_index(local_root="image")
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `test_image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
As you can see, the result is the same in the 3 cases.
Let us now try to print the dataset Index with a maximal data item depth of 2, using the max_depth argument:
[8]:
data.print_index(max_depth=2)
Dataset Content Index :
------------------------:
index printed with max depth `2` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : mesh H5_Path : /test_mesh
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
Name : test_table H5_Path : /test_group/test_table
Name : vector_T1_0 H5_Path : /test_mesh/test_vector_T1_0
Name : vector_T2_0 H5_Path : /test_mesh/test_vector_T2_0
As you can see, the information about data items stored into the /test_mesh/Geometry group, that have a depth of 3, has not been printed.
Of course, you can combine those two arguments:
[9]:
data.print_index(max_depth=2, local_root='mesh')
Dataset Content Index :
------------------------:
index printed with max depth `2` and under local root `mesh`
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Name : vector_T1_0 H5_Path : /test_mesh/test_vector_T1_0
Name : vector_T2_0 H5_Path : /test_mesh/test_vector_T2_0
The print_index method is usefull to get a glimpse of the content and organization of the whole dataset, and to quickly see the short indexnames or aliases that you can use to refer to data items. In practical applications, datasets can become complex, so be sure to use the local_root and max_depth option to limit the printed information and quickly find the data items you are looking for.
To add aliases to data items or Groups, you can use the add_alias method.
The Index allows to quickly see the internal structure of your dataset, however, it does not provide detailed information on the data items. We will now see how to retrieve it with the SampleData class.
2- The Dataset content¶
The SampleData class provides a method to print an organized and detailed overview of the data items in the dataset, the print_dataset_content method. Let us see what the methods prints when called with no arguments:
[10]:
data.print_dataset_content()
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: test_sampledata_ref.h5
-- Size: 1.670 Mb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : None -- Root Group
-- Group attributes :
* description :
This is a test dataset created by the SampleData class unit tests.
* sample_name : test_sample
-- Childrens : Index, test_group, test_image, test_mesh,
----------------
************************************************
GROUP test_group
=====================
-- Parent Group : /
-- Group attributes :
* group_type : Group
* test_attribute : test
-- Childrens : test_array, test_table,
----------------
****** Group /test_group CONTENT ******
NODE: /test_group/test_array
====================
-- Parent Group : test_group
-- Node name : test_array
-- test_array attributes :
* empty : False
* node_type : data_array
-- content : /test_group/test_array (CArray(51,)) 'array'
-- Compression options for node `/test_group/test_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_group/test_table
====================
-- Parent Group : test_group
-- Node name : test_table
-- test_table attributes :
* node_type : structured_array
-- content : /test_group/test_table (Table(2,)) ''
-- table description :
{
"density": Float32Col(shape=(), dflt=0.0, pos=0),
"melting_Pt": Float32Col(shape=(), dflt=0.0, pos=1),
"Chemical_comp": StringCol(itemsize=30, shape=(), dflt=b'', pos=2)}
-- Compression options for node `/test_group/test_table`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1724,)
-- Node memory size : 63.977 Kb
----------------
************************************************
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* time_list : [1.0, 2.0, 3.0]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field, test_tensor_T1_0, test_tensor_T2_0, test_tensor_T3_0,
----------------
****** Group /test_image CONTENT ******
NODE: /test_image/Field_index
====================
-- Parent Group : test_image
-- Node name : Field_index
-- Field_index attributes :
* empty : True
* node_type : string_array
-- content : /test_image/Field_index (EArray(4,)) ''
-- Compression options for node `/test_image/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_image/test_image_field
====================
-- Parent Group : test_image
-- Node name : test_image_field
-- test_image_field attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : test_image_field
* xdmf_gridname : test_image
-- content : /test_image/test_image_field (CArray(10, 10, 10)) 'test_image_field'
-- Compression options for node `/test_image/test_image_field`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (327, 10, 10)
-- Node memory size : 63.867 Kb
----------------
NODE: /test_image/test_tensor_T1_0
====================
-- Parent Group : test_image
-- Node name : test_tensor_T1_0
-- test_tensor_T1_0 attributes :
* empty : False
* field_dimensionality : Tensor
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* time : 1.0
* time_serie_name : test_tensor
* transpose_components : [0, 4, 8, 1, 5, 6, 3, 7, 2]
* transpose_indices : [2, 1, 0, 3]
* xdmf_fieldname : test_tensor
* xdmf_gridname : test_image
-- content : /test_image/test_tensor_T1_0 (CArray(10, 10, 10, 9)) 'tensor9_T1_0'
-- Compression options for node `/test_image/test_tensor_T1_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (18, 10, 10, 9)
-- Node memory size : 63.281 Kb
----------------
NODE: /test_image/test_tensor_T2_0
====================
-- Parent Group : test_image
-- Node name : test_tensor_T2_0
-- test_tensor_T2_0 attributes :
* empty : False
* field_dimensionality : Tensor
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* time : 2.0
* time_serie_name : test_tensor
* transpose_components : [0, 4, 8, 1, 5, 6, 3, 7, 2]
* transpose_indices : [2, 1, 0, 3]
* xdmf_fieldname : test_tensor
* xdmf_gridname : test_image
-- content : /test_image/test_tensor_T2_0 (CArray(10, 10, 10, 9)) 'tensor9_T2_0'
-- Compression options for node `/test_image/test_tensor_T2_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (18, 10, 10, 9)
-- Node memory size : 63.281 Kb
----------------
NODE: /test_image/test_tensor_T3_0
====================
-- Parent Group : test_image
-- Node name : test_tensor_T3_0
-- test_tensor_T3_0 attributes :
* empty : False
* field_dimensionality : Tensor
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* time : 3.0
* time_serie_name : test_tensor
* transpose_components : [0, 4, 8, 1, 5, 6, 3, 7, 2]
* transpose_indices : [2, 1, 0, 3]
* xdmf_fieldname : test_tensor
* xdmf_gridname : test_image
-- content : /test_image/test_tensor_T3_0 (CArray(10, 10, 10, 9)) 'tensor9_T3_0'
-- Compression options for node `/test_image/test_tensor_T3_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (18, 10, 10, 9)
-- Node memory size : 63.281 Kb
----------------
************************************************
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* time_list : [1.0, 2.0]
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4, test_vector_T1_0, test_vector_T2_0,
----------------
****** Group /test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* empty : True
* node_type : string_array
-- content : /test_mesh/Field_index (EArray(11,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
****** Group /test_mesh/Geometry CONTENT ******
NODE: /test_mesh/Geometry/Elem_tag_type_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tag_type_list
-- Elem_tag_type_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Elem_tag_type_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tag_type_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elem_tags_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tags_list
-- Elem_tags_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Elem_tags_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elements
====================
-- Parent Group : Geometry
-- Node name : Elements
-- Elements attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Elements (CArray(24,)) 'mesh_Elements'
-- Compression options for node `/test_mesh/Geometry/Elements`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
GROUP ElementsTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : ET_2D, ET_Bottom, ET_Top, field_2D, field_Bottom, field_Top,
---------------- -- Use "get_mesh_elem_tags_names" methods to print content.
GROUP NodeTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : NT_Z0_plane, NT_out_of_plane, field_Z0_plane, field_out_of_plane,
---------------- -- Use "get_mesh_node_tags_names" methods to print content.
NODE: /test_mesh/Geometry/Node_tags_list
====================
-- Parent Group : Geometry
-- Node name : Node_tags_list
-- Node_tags_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Node_tags_list (EArray(2,)) ''
-- Compression options for node `/test_mesh/Geometry/Node_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Nodes
====================
-- Parent Group : Geometry
-- Node name : Nodes
-- Nodes attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes (CArray(6, 3)) 'mesh_Nodes'
-- Compression options for node `/test_mesh/Geometry/Nodes`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/Geometry/Nodes_ID
====================
-- Parent Group : Geometry
-- Node name : Nodes_ID
-- Nodes_ID attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes_ID (CArray(6,)) 'mesh_Nodes_ID'
-- Compression options for node `/test_mesh/Geometry/Nodes_ID`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/test_vector_T1_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T1_0
-- test_vector_T1_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 1.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T1_0 (CArray(6, 3)) 'vector_T1_0'
-- Compression options for node `/test_mesh/test_vector_T1_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/test_vector_T2_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T2_0
-- test_vector_T2_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 2.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T2_0 (CArray(6, 3)) 'vector_T2_0'
-- Compression options for node `/test_mesh/test_vector_T2_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
************************************************
As you can see, this method prints by increasing depth, detailed information on each Group and each data item of the dataset, with a maximum depth that can be specified with a max_depth argument (like the method print_index, that has a default value of 3). The printed output is structured by groups: each Group node is described first, followed by a Group CONTENT string that provide decripstion for all of its childrens.
For each data item, or Group, the method prints their name, path, type, attributes, content, compression settings and memory size if it is an array, children names if it is a Group. Hence, when calling this method, you can see the content and organization of the dataset, all the metadata attached to all data items, and the disk size occupied by each data item. As you progress through this tutorial, you will learn the meaning of those informations for all types of SampleData data items.
The print_dataset_content method has a short boolean argument, that allows to plot a condensed string representation of the dataset:
[11]:
data.print_dataset_content(short=True)
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
--NODE test_table: /test_group/test_table (structured_array) ( 63.977 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
--NODE test_tensor_T1_0: /test_image/test_tensor_T1_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T2_0: /test_image/test_tensor_T2_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T3_0: /test_image/test_tensor_T3_0 (field_array) ( 63.281 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
-- Use "get_mesh_elem_tags_names" methods to print content.
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
-- Use "get_mesh_node_tags_names" methods to print content.
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
--NODE test_vector_T1_0: /test_mesh/test_vector_T1_0 (field_array) ( 63.984 Kb)
--NODE test_vector_T2_0: /test_mesh/test_vector_T2_0 (field_array) ( 63.984 Kb)
This shorter print, provide a complete and visual overview of the dataset organization, and indicate the memory size and type of each data item or Group in the dataset. The printed output distinguishes Group data items, from Nodes data item. The later regroups all types of arrays that may be stored in the HDF5 file.
Both short and long version of the print_dataset_content output can be written into a text file, if a filename is provided as value for the to_file method argument:
[12]:
data.print_dataset_content(short=True, to_file='dataset_information.txt')
[13]:
# Let us open the content of the created file, to see if the dataset information has been written in it:
%cat dataset_information.txt
os.remove('dataset_information.txt')
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
--NODE test_table: /test_group/test_table (structured_array) ( 63.977 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
--NODE test_tensor_T1_0: /test_image/test_tensor_T1_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T2_0: /test_image/test_tensor_T2_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T3_0: /test_image/test_tensor_T3_0 (field_array) ( 63.281 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
-- Use "get_mesh_elem_tags_names" methods to print content.
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
-- Use "get_mesh_node_tags_names" methods to print content.
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
--NODE test_vector_T1_0: /test_mesh/test_vector_T1_0 (field_array) ( 63.984 Kb)
--NODE test_vector_T2_0: /test_mesh/test_vector_T2_0 (field_array) ( 63.984 Kb)
Note
The string representation of the SampleData class is composed of a first part, which is the output of the print_index method, and a second part, that is the output of the print_datase_content method (short output).
[14]:
# SampleData string representation :
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Name : tensor9_T1_0 H5_Path : /test_image/test_tensor_T1_0
Name : tensor9_T2_0 H5_Path : /test_image/test_tensor_T2_0
Name : tensor9_T3_0 H5_Path : /test_image/test_tensor_T3_0
Name : test_image_field H5_Path : /test_image/test_image_field
Name : test_table H5_Path : /test_group/test_table
Name : vector_T1_0 H5_Path : /test_mesh/test_vector_T1_0
Name : vector_T2_0 H5_Path : /test_mesh/test_vector_T2_0
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
--NODE test_table: /test_group/test_table (structured_array) ( 63.977 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
--NODE test_tensor_T1_0: /test_image/test_tensor_T1_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T2_0: /test_image/test_tensor_T2_0 (field_array) ( 63.281 Kb)
--NODE test_tensor_T3_0: /test_image/test_tensor_T3_0 (field_array) ( 63.281 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
-- Use "get_mesh_elem_tags_names" methods to print content.
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
-- Use "get_mesh_node_tags_names" methods to print content.
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
--NODE test_vector_T1_0: /test_mesh/test_vector_T1_0 (field_array) ( 63.984 Kb)
--NODE test_vector_T2_0: /test_mesh/test_vector_T2_0 (field_array) ( 63.984 Kb)
Now you know how to get a detailed overview of the dataset content. However, with large datasets, that may have a complex internal organization (many Groups, lot of data items and metadata…), the print_dataset_content return string can become very large. In this case, it becomes cumbersome to look for a specific information on a Group or on a particular data item. For this reason, the SampleData class provides methods to only print information on one or several data items of the dataset.
They are presented in the next subsections.
3- Get information on data items¶
To get information on a specific data item (including Groups), you may use the print_node_info method. This method has 2 arguments: the name argument, and the short argument. As explain in section II, the name argument can be one of the 4 possible identifier that the target node can have (name, path, indexname or alias). The short argument has the same effect on the printed output as for the print_dataset_content method. Let us look at some examples. Its default value is
False, i.e. the detailed output.
First, we will for instance want to have information on the Image Group that is stored in the dataset. The print_index and short print_dataset_content allowed us to see that this group has the name test_image, the indexname image, and the path /test_image. We will call the method with two of those identificators, and with the two possible values of the short argument.
[15]:
# Method called with data item indexname, and short output
data.print_node_info(nodename='image', short=True)
|--GROUP test_image: /test_image (3DImage)
[16]:
# Method called with data item Path and long output
data.print_node_info(nodename='/test_image', short=False)
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* time_list : [1.0, 2.0, 3.0]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field, test_tensor_T1_0, test_tensor_T2_0, test_tensor_T3_0,
----------------
You can observe that this method prints the same block of information that the one that appeared in the print_dataset_content method output, for the description of the test_image group. With this block, we can learn that this Group is a children of the root Group (‘/’), that it has five childrens that are the data items named Field_index, test_image_field, and three items test_tensor_TX_X. We can also see its attributes names and values. Here they provide information on the nature of
the Group, that is a 3D image group, and on the topology of this image (for instance, that it is a 9x9x9 voxel image, of size 0.2). Finally, we can see that grid data can be associated to a specific time. Indeed, three time values are provided for the image in the time_list attribute. They correspond to the time values associated to the three children arrays named test_tensor_TX_X.
Let us now apply this method on a data item that is not a group, the test_array data item. The print_index function instructed us that this node has an alias name, that is test_alias. We will use it here to get information on this node, to illustrate the use of the only type of node indicator that has not been used throughout this notebook:
[17]:
data.print_node_info('test_alias')
No group named test_alias
Here, we can learn which is the Node parent, what is the node Name, see that it has no attributes, see that it is an array of shape (51,), that it is not stored with data compression (compresion level to 0), and that it occupies a disk space of 64 Kb.
The print_node_info method is usefull to get information on a specific target, and avoir dealing with the sometimes too large output returned by the print_dataset_content method.
4- Get information on Groups content¶
The previous subsection showed that the print_node_info method applied on Groups returns only information about the group name, metadata and children names. The SampleData class offers a method that allows to print this information, with in addition, the detailed content of each children of the target group: the print_group_content method.
Let us try it on the Mesh group of our test dataset:
[18]:
data.print_group_content(groupname='test_mesh')
****** Group test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* empty : True
* node_type : string_array
-- content : /test_mesh/Field_index (EArray(11,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/test_vector_T1_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T1_0
-- test_vector_T1_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 1.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T1_0 (CArray(6, 3)) 'vector_T1_0'
-- Compression options for node `/test_mesh/test_vector_T1_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/test_vector_T2_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T2_0
-- test_vector_T2_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 2.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T2_0 (CArray(6, 3)) 'vector_T2_0'
-- Compression options for node `/test_mesh/test_vector_T2_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
Obviously, this methods is identical to the print_dataset_content method, but restricted to one Group. As the first one, it has a to_file, a short and a max_depth arguments. These arguments work just as for print_dataset_content method, hence there use is not detailed here. the However, you may see one difference here. In the output printed above, we see that the test_mesh group has a Geometry children which is a group, but whose content is not printed. The
print_group_content has indeed, by default, a non-recursive behavior. To get a recursive print of the group content, you must set the recursive argument to True:
[19]:
data.print_group_content('test_mesh', recursive=True)
****** Group test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* empty : True
* node_type : string_array
-- content : /test_mesh/Field_index (EArray(11,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
****** Group /test_mesh/Geometry CONTENT ******
NODE: /test_mesh/Geometry/Elem_tag_type_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tag_type_list
-- Elem_tag_type_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Elem_tag_type_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tag_type_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elem_tags_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tags_list
-- Elem_tags_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Elem_tags_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elements
====================
-- Parent Group : Geometry
-- Node name : Elements
-- Elements attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Elements (CArray(24,)) 'mesh_Elements'
-- Compression options for node `/test_mesh/Geometry/Elements`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
GROUP ElementsTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : ET_2D, ET_Bottom, ET_Top, field_2D, field_Bottom, field_Top,
---------------- -- Use "get_mesh_elem_tags_names" methods to print content.
GROUP NodeTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : NT_Z0_plane, NT_out_of_plane, field_Z0_plane, field_out_of_plane,
---------------- -- Use "get_mesh_node_tags_names" methods to print content.
NODE: /test_mesh/Geometry/Node_tags_list
====================
-- Parent Group : Geometry
-- Node name : Node_tags_list
-- Node_tags_list attributes :
* empty : False
* node_type : string_array
-- content : /test_mesh/Geometry/Node_tags_list (EArray(2,)) ''
-- Compression options for node `/test_mesh/Geometry/Node_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Nodes
====================
-- Parent Group : Geometry
-- Node name : Nodes
-- Nodes attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes (CArray(6, 3)) 'mesh_Nodes'
-- Compression options for node `/test_mesh/Geometry/Nodes`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/Geometry/Nodes_ID
====================
-- Parent Group : Geometry
-- Node name : Nodes_ID
-- Nodes_ID attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes_ID (CArray(6,)) 'mesh_Nodes_ID'
-- Compression options for node `/test_mesh/Geometry/Nodes_ID`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/test_vector_T1_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T1_0
-- test_vector_T1_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 1.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T1_0 (CArray(6, 3)) 'vector_T1_0'
-- Compression options for node `/test_mesh/test_vector_T1_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/test_vector_T2_0
====================
-- Parent Group : test_mesh
-- Node name : test_vector_T2_0
-- test_vector_T2_0 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* time : 2.0
* time_serie_name : test_vector
* xdmf_fieldname : test_vector
* xdmf_gridname : test_mesh
-- content : /test_mesh/test_vector_T2_0 (CArray(6, 3)) 'vector_T2_0'
-- Compression options for node `/test_mesh/test_vector_T2_0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
As you can see, the information on the childrens of the Geometry group have been printed. Note that the max_depth argument is considered by this method as an absolute depth, meaning that you have to specify a depth that is at least the depth of the target group to see some output printed for the group content. The default maximum depth for this method is set to a very high value of 1000. Hence, print_group_content prints by defaults group contents with a recursive behavior. Note also
that print_group_content with the recursive option, is equivalent to print_dataset_content but prints the dataset content as if the target group was the root.
5- Get information on grids¶
One of the SampleData class main functionalities is the manipulation and storage of spatially organized data, which is handled by Grid groups in the data model. Because they are usually key data for mechanical sample datasets, the SampleData class provides a method to print Grouo informations only for Grid groups, the print_grids_info method:
[20]:
data.print_grids_info()
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* time_list : [1.0, 2.0, 3.0]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field, test_tensor_T1_0, test_tensor_T2_0, test_tensor_T3_0,
----------------
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* time_list : [1.0, 2.0]
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4, test_vector_T1_0, test_vector_T2_0,
----------------
This method also has the to_file and short arguments of the print_dataset_content method:
[21]:
data.print_grids_info(short=True, to_file='dataset_information.txt')
%cat dataset_information.txt
os.remove('dataset_information.txt')
|--GROUP test_image: /test_image (3DImage)
|--GROUP test_mesh: /test_mesh (3DMesh)
6- Get xdmf tree¶
As explained in the data management information page the SampleData class can automatically produce a XDMF file containing metadata, describing Grid groups topology, data types and fields, contained in the HDF5 dataset. The class handles internally the construction of a XDMF tree structure that matches the data stored in the dataset, with the help of the lxml package. However, you may want to look at the content of the XDMF tree while you are interactively using
your SampleData instance. In this case, you can use the print_xdmf method:
[22]:
data.print_xdmf()
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_image" GridType="Collection" CollectionType="Temporal">
<Grid Name="test_image_T0" GridType="Uniform">
<Time Value="1.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_image_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10" NumberType="Int" Precision="16">test_sampledata_ref.h5:/test_image/test_image_field</DataItem>
</Attribute>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T1_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image_T1" GridType="Uniform">
<Time Value="2.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T2_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image_T2" GridType="Uniform">
<Time Value="3.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T3_0</DataItem>
</Attribute>
</Grid>
</Grid>
<Grid Name="test_mesh" GridType="Collection" CollectionType="Temporal">
<Grid Name="test_mesh_T0" GridType="Uniform">
<Time Value="1.0"/>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
<Attribute Name="test_vector" AttributeType="Vector" Center="Node">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/test_vector_T1_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_mesh_T1" GridType="Uniform">
<Time Value="2.0"/>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="test_vector" AttributeType="Vector" Center="Node">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/test_vector_T2_0</DataItem>
</Attribute>
</Grid>
</Grid>
</Domain>
</Xdmf>
As you can observe, you will get a print of the content of the XDMF file that would be written if you would close the file right now. You can observe that the XDMF file provides information on the grids that match those given by the Groups and Nodes attributes printed above with the previously studied method: the test image is a regular grid of 10x10x10 nodes, i.e. a 9x9x9 voxels grid. Only one field is defined on test_image, test_image_field, whereas two are defined on test_mesh.
This XDMF file can directly be opened in Paraview, if both file are closed. If any syntax or formatting issue is encountered when Paraview reads the XDMF file, it will return an error message and the data visualization will not be rendered. The print_xdmf method allows you to verify your XDMF data and syntax, to make sure that the data formatting is correct.
To generate the XDMF file associated to this tree, you can use the following method:
[23]:
data.write_xdmf(filename='test.xdmf')
%cat test.xdmf # print XDMF file content
os.remove('test.xdmf')
<?xml version='1.0' ?>
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd"[]>
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_image" GridType="Collection" CollectionType="Temporal">
<Grid Name="test_image_T0" GridType="Uniform">
<Time Value="1.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_image_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10" NumberType="Int" Precision="16">test_sampledata_ref.h5:/test_image/test_image_field</DataItem>
</Attribute>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T1_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image_T1" GridType="Uniform">
<Time Value="2.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T2_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image_T2" GridType="Uniform">
<Time Value="3.0"/>
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_tensor" AttributeType="Tensor" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10 9" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_image/test_tensor_T3_0</DataItem>
</Attribute>
</Grid>
</Grid>
<Grid Name="test_mesh" GridType="Collection" CollectionType="Temporal">
<Grid Name="test_mesh_T0" GridType="Uniform">
<Time Value="1.0"/>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
<Attribute Name="test_vector" AttributeType="Vector" Center="Node">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/test_vector_T1_0</DataItem>
</Attribute>
</Grid>
<Grid Name="test_mesh_T1" GridType="Uniform">
<Time Value="2.0"/>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="test_vector" AttributeType="Vector" Center="Node">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/test_vector_T2_0</DataItem>
</Attribute>
</Grid>
</Grid>
</Domain>
</Xdmf>
This method has a filename argument, that can be used to specify the name of the written XDMF file. If it is not specified, the file will be written with the same basename as the HDF5 dataset file. This XDMF can be opened with Paraview to visualize the 3D data that the dataset contains. For that, the two files must be in the same directory.
Note
The XDMF file associated to the dataset is automatically written when you close a dataset.
7- Get memory size of file and data items¶
SampleData is designed to create large datasets, with data items that can reprensent tens Gb of data or more. Being able to easily see and identify which data items use the most disk space is a crucial aspect for data management. Until now, with the method we have reviewed, we only have been able to print the Nodes disk sizes together with a lot of other information. In order to speed up this process, the SampleData class has one method that allow to directly query and print only the memory
size of a Node, the get_node_disk_size method:
[24]:
data.get_node_disk_size(nodename='test_array')
Node test_array size on disk is 64.000 Kb
[24]:
(64.0, 'Kb')
As you can see, the default behavior of this method is to print a message indicating the Node disk size, but also to return a tuple containing the value of the disk size and its unit. If you want to print data in bytes, you may call this method with the convert argument set to False:
[25]:
data.get_node_disk_size(nodename='test_array', convert=False)
Node test_array size on disk is 65536.000 bytes
[25]:
(65536, 'bytes')
If you want to use this method to get a numerical value within a script, but do not want the class to print anything, you can use the print_flag argument:
[26]:
size, unit = data.get_node_disk_size(nodename='test_array', print_flag=False)
print(f'node size is {size} {unit}')
size, unit = data.get_node_disk_size(nodename='test_array', print_flag=False, convert=False)
print(f'node size is {size} {unit}')
node size is 64.0 Kb
node size is 65536 bytes
The disk size of the whole HDF5 file can also be printed/returned, using the get_file_disk_size method, that has the same print_flag and convert arguments:
[27]:
data.get_file_disk_size()
size, unit = data.get_file_disk_size(convert=False, print_flag=False)
print(f'\nPrinted by script: file size is {size} {unit}')
File size is 1.670 Mb for file
test_sampledata_ref.h5
Printed by script: file size is 1750685 bytes
8- Get nodes/groups attributes (metadata)¶
Another central aspect of the SampleData class is the management of metadata, that can be attached to all Groups or Nodes of the dataset. Metadata comes in the form of HDF5 attributes, that are Name/Value pairs, and that we already encountered when exploring the outputs of methods like print_dataset_content, print_node_info…
Those methods print the Group/Node attributes together with other information. To only print the attributes of a given data item, you can use the print_node_attributes method:
[28]:
data.print_node_attributes(nodename='test_mesh')
-- test_mesh attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* time_list : [1.0, 2.0]
* xdmf_gridname : test_mesh
As you can see, this method prints a list of all data item attributes, with the format * Name : Value \n. It allows you to quickly see what attributes are stored together with a given data item, and their values.
If you want to get the value of a specific attribute, you can use the get_attribute method. It takes two arguments, the name of the attribute you want to retrieve, and the name of the data item where it is stored:
[29]:
Nnodes = data.get_attribute(attrname='number_of_nodes', nodename='test_mesh')
print(f'The mesh test_mesh has {Nnodes} nodes')
The mesh test_mesh has 6 nodes
You can also get all attributes of a data item as a dictionary. In this case, you just need to specify the name of the data item from which you want attributes, and use the get_dic_from_attributes method:
[30]:
mesh_attrs = data.get_dic_from_attributes(nodename='test_mesh')
for name, value in mesh_attrs.items():
print(f' Attribute {name} is {value}')
Attribute Elements_offset is [0]
Attribute Number_of_boundary_elements is [0]
Attribute Number_of_bulk_elements is [8]
Attribute Number_of_elements is [8]
Attribute Topology is Uniform
Attribute Xdmf_elements_code is ['Triangle']
Attribute description is
Attribute element_type is ['tri3']
Attribute elements_path is /test_mesh/Geometry/Elements
Attribute empty is False
Attribute group_type is 3DMesh
Attribute nodesID_path is /test_mesh/Geometry/Nodes_ID
Attribute nodes_path is /test_mesh/Geometry/Nodes
Attribute number_of_nodes is 6
Attribute time_list is [1.0, 2.0]
Attribute xdmf_gridname is test_mesh
We have now seen how to explore all of types of information that a SampleData dataset may contain, individually or all together, interactively, from a Python console. Let us review now how to explore the content of SampleData datasets with external softwares.
III - Visualize dataset contents with Vitables¶
All the information that you can get with all the methods presented in the previous section can also be accessed externally by opening the HDF5 dataset file with the Vitables software. This software is usually part of the Pytables package, that is a dependency of pymicro. You should be able to use it in a Python environement compatible with pymicro. If needed, you may refer to the Vitables website to find download and installations instructions for PyPi or conda: https://vitables.org/.
Vitables provide a graphical interface that allows you to browse through all your dataset data items, and access or modify their stored data and metadata values. You may either open Vitables and then open your HDF5 dataset file from the Vitables interface, or you can directly open Vitables to read a specfic file from command line, by running: vitables my_dataset_path.h5.
This command will work only if your dataset file is closed (if the SampleData instance still exists in your Python console, this will not work, you first need to delete your instance to close the files).
However, the SampleData class has a specific method to allowing to open your dataset with Vitables interactively, directly from your Python console: the method pause_for_visualization. As explained just above, this method closes the HDF5 dataset, and runs in your shell the command vitables my_dataset_path.h5. Then, it freezes the interactive Python console and keep the dataset files closed, for as long as the Vitables software is running. When Vitables is shutdown, the
SampleData class will reopen the HDF5 file, synchronize with them and resume the interactive Python console.
Warning
When calling the pause_for_visualization method from a python console (ipython, Jupyter…), you may face environment issue leading to your shell not finding the proper Vitables software executable. To ensure that the right Vitables is found, the method can take an optional argument Vitables_path. If this argument is passed, the method will run, after closing the HDF5 dataset, the command Vitables_path my_dataset_path.h5
Note
The method is not called here to allow automatic execution of the Notebook when building the documentation on a platform that do not have Vitables available.
[31]:
# uncomment to test
# data.pause_for_visualization(Vitables=True, Vitables_path='Path_to_Vitables_executable')
Please refer to the Vitables documentation, that can be downloaded here https://sourceforge.net/projects/vitables/files/ViTables-3.0.0/, to learn how to browse through your HDF5 file. The Vitables software is very intuitive, you will see that it is provides a usefull and convenient tool to explore your SampleData datasets outside of your interactive Python consoles.
IV - Visualize datasets grids and fields with Paraview¶
As for Vitables, the pause_for_visualization method allows you to open your dataset with Paraview, interactively from a Python console.
Paraview will provide you with a very powerfull visualization tool to render your spatially organized data (grids) stored in your datasets. Unlike Vitables, Paraview must read a XDMF file to be able to render 3D data. Hence, if you want to open your dataset with Paraview, outside of a Python console, make sure to have Paraview in your PATH, that you have generated the XDMF file associated to your HDF5 dataset and that both are in the same directory (see above), and run in your shell the
command: paraview my_dataset_path.xdmf.
As you may have guessed, the pause_for_visualization method, when called interactively with the Paraview argument set to True, will execute those steps automatically, just like for the Vitables option. The dataset will remained closed and the Python console freezed for as long as you will keep the Paraview software running. When you shutdown Paraview, the SampleData class will reopen the HDF5 file, synchronize with them and resume the interactive Python console.
Warning
When calling the pause_for_visualization method from a python console (ipython, Jupyter…), you may face environment issues, leading to the interactive console not finding the proper Paraview executable file. To ensure that the right Paraview is found, the method can take an optional argument Paraview_path. If this argument is passed, the method will run, after closing the HDF5 and XDMF files, the command Paraview_path my_dataset_path.xdmf
[32]:
# Like for Vitables --> uncomment to test
# data.pause_for_visualization(Paraview=True, Paraview_path='Path_to_Paraview_executable')
Note
It is recommended to use a recent version of the Paraview software to visualize SampleData datasets (>= 5.0). When opening the XDMF file, Paraview may ask you to choose a specific file reader. It is recommended to choose the XDMF_reader, and not the Xdmf3ReaderT, or Xdmf3ReaderS.
V - Using command line tools¶
You can also examine the content of your HDF5 datasets with generoc HDF5 command line tools, such as h5ls or h5dump:
Warning
In the following, executable programs that come with the HDF5 library and the Pytables package are used. If you are executing this notebook with Jupyter, you may not be able to have those executable in your path, if your environment is not suitably set. A workaround consist in finding the absolute path of the executable, and replacing the executable name in the following cells by its full path. For instance, replace
`ptdump file.h5`
with
`/full/path/to/ptdump file.h5`
To find this full path, you can run in your shell the command which ptdump. Of course, the same applies for h5ls and h5dump.
Note
Most code lines below are commented as they produce very large outputs, that otherwise pollute the documentation if they are included in the automatic build process. Uncomment them to test them if you are using interactively these notebooks !
For that, you must first close your dataset. If you don’t, this tools will not be able to open the HDF5 file as it is opened by the SampleData class in the Python interpretor.
[33]:
del data
[34]:
# raw output of H5ls --> prints the childrens of the file root group
!h5ls ../data/test_sampledata_ref.h5
/usr/bin/sh: 1: h5ls: not found
[35]:
# recursive output of h5ls (-r option) --> prints all data items
!h5ls -r ../data/test_sampledata_ref.h5
/usr/bin/sh: 1: h5ls: not found
[36]:
# recursive (-r) and detailed (-d) output of h5ls --> also print the content of the data arrays
# !h5ls -rd ../data/test_sampledata_ref.h5
[37]:
# output of h5dump:
# !h5dump ../data/test_sampledata_ref.h5
As you can see if you uncommented and executed this cell, h5dump prints a a fully detailed description of your dataset: organization, data types, item names and path, and item content (value stored in arrays). As it produces a very large output, it may be convenient to write its output in a file:
[38]:
# !h5dump ../data/test_sampledata_ref.h5 > test_dump.txt
[39]:
# !cat test_dump.txt
You can also use the command line tool of the Pytables software ptdump, that also takes as argument the HDF5 file, and has two command options, the verbose mode -v, and the detailed mode -d:
[40]:
# uncomment to test !
# !ptdump ../data/test_sampledata_ref.h5
[41]:
# uncomment to test!
# !ptdump -v ../data/test_sampledata_ref.h5
[42]:
# uncomment to test !
# !ptdump -d ../data/test_sampledata_ref.h5
This second tutorial on Data Management with Pymicro User Guide is now finished. You should now know how to retrieve all the information you need in your datasets, and visualize 3D data with Paraview from your datasets.