Pymicro is an open-source Python package to work with material microstructures and 3d data sets.
Introduction¶
pymicro is a python library to load, study and vizualize three dimensional material data with a particular focus on crystalline microstructures. It combine the generic power of the main scientific python libraries (namely numpy, scipy and matplotlib) with the 3D visualisation capabilities of a state of the art library as VTK. The goal of pymicro is to make it easy (or easier) to process your 3D datasets especially when it comes to automated processing needed for in situ data analysis.
Contents:
Overview¶
This page provides a quick overview of the pymicro code base.
Pymicro is organised around 8 packages:
pymicro.appscontains a simple application to visualise and interact with 2D images and 3D volumes;pymicro.coregather the samples.py module at the root of the data management in Pymicro and some related utilities.pymicro.crystalcontains various modules to handle crystal lattices, crystallographic grains and to organise them into microstructures;pymicro.externalregroup external utilities used for instance to read TIF or CIF files;pymicro.fecontains a module to manipulate files for finite-elements analysis;pymicro.filecontains many helpers methods to handle typical file reading and writting found at ESRF, ANKA and Soleil;pymicro.viewcontains the two principal modules for 3D visualisation: vtk_utils.py for rendering still 3D images and vtk_anim.py to create animations;pymicro.xrayregroup modules to simulate and analyse xray experiments like tomography and diffraction..
Installation¶
You can grab a copy of the latest version on the git repository at Centre des Materiaux, open a terminal and type:
git clone http://github.com/heprom/pymicro
This will create a pymicro folder containing the source code (another pymicro folder), the documentation and the examples. Then you must add this folder to the PYTHONPATH environement variable. To do this you can add this line to your .cshrc file:
setenv PYTHONPATH /path/to/source/folder/pymicro
Then you will be able to use pymicro (you may have to source the .cshrc or restart the terminal). For instance after starting a ipython shell:
from pymicro.file.file_utils import HST_read
If you want to use pymicro interactively and import all modules at ipython startup you may apply the following recipe: Using pymicro interactively
Dependencies¶
Python 3.7+ required, please report any problem on the github page.
numpy - For array, matrix and other numerical manipulations. Used extensively by all modules.
scipy, mainly used for ndimage filters.
skimage for additional image analysis (http://scikit-image.org).
matplotlib 1.1+ for plotting (e.g. pole figures or 3D image slices).
VTK with Python bindings (http://www.vtk.org/) for visualization of 3D data using the
pymicro.viewpackage. Version should be > 5.8.0, 9.0.1 recommended.h5py and pytables to deal with HDF5 files.
basictools. Starting with version 0.5, we rely on this library to support mesh data; basictools is open source and can be installed from conda using:
`conda install -c conda-forge basictools`.
External¶
Crystal lattices can be created using CIF files usig the
pymicro.crystal.lattice.from_cifmethod. We use PyCifRW to read and parse CIF files.reading and writing 3d Tiff files is supported via the TiffFile module.
API documentation¶
For detailed documentation of all modules and classes, please refer to the API docs.
Pymicro Data Platform: The Sample Data and Microstructure Classes User Guide¶
The pymicro package is based on a data format designed to create, organize and manage efficiently complex multi-modal datasets, used in material science, with a particular focus on the mechanics of microstructures. These data sets are said to be multi-modal because they bring together data from various measurement and numerical simulation techniques, originally produced in very different formats.
The pymicro.core package implements a data platform class allowing to meet the numerous challenges raised by the complexity and size of these datasets. Pymicro’s main class Microstructure is now based on this data format. The data platform is based on a central class, SampleData, that implements the backend and user interface with the datasets.
This User Guide objective is to serve as a documentation of the classe, and as a guide to learn step by step the SampleData file format, data model, and the use of the class to manipulate complex and multimodal datasets. It is composed of a serie of Jupyter notebooks, that have been integrated into the Pymicro package documentation.
The first documentation page/notebook is an introduction, and does not contain Python code. It is dedicated to the presentation of the challenges that motivated the implementation of this data platform, and of its main features.
The introduction is not yet part of the documentation, it will be as soons as the presenting publication of the code will be released.
The other pages/notebooks are tutorials that will introduce you to the various functionnalites of the platform, the different data item types that can be stored into dataset, and their data model. They are introduced from the simplest to the most complex. Hence, it is adviced to read through them in the indicated order. You will find at the end of each page/notebook a short summary of the functionalities and code lines presented. All these summary have been gathered in a complete quick reference sheet, in the last page/notebook of this guide.
If you are reading through this on the on-line documentation, please note that the associated notebooks can be found in the pymicro package directories, under the path pymicro_package_path/examples/SampleDataUserGuide/. You can open these files with the jupyter-notebook program so that you can run the code as you read this documentation, and get familiar with the use of SampleData. If you do not have jupyter installed in your python environement, you may consult the dedicated webpage of the Jupyter project to help you get it: https://jupyter.org/install.
Even without jupyter-notebooks, you may reproduce the code content of this guide within any Python interactive console (like ipython). We strongly encourage you to do so while reading through this user guide !
1 - Introduction to the SampleData data platform¶
This first User Guide page/notebook will:
present the main features of the data format associated with the platform
list the prerequisite knowledge that we recommend to master before starting to use SampleData
Note: This page/notebook does not include Python code to learn how to manipulate the SampleData class. However, it provides a complete overview of the scientific challenges that motivated the implementation of the class, which surely helps gaining insight in using the code and understand relevant applications.
I - The SampleData platform¶
The pymicro package is used on a daily basis by material scientists teams involved in the development of the data paradigm presented in section I. Its original design, that relied on classical Python and Numpy data objects to handle material data, dit not allow to efficiently handle the multi-modal 4D material science data challenges. Therefore, an new subpackage has been introduced within pymicro, the pymicro.core subpackage, to serve as pymicro’s data platform.
This pymicro.core package has been designed to be independent of the rest of the pymicro code. Its aim is to serve as a prototype implementation of what could constitute a large-scale data platform the material microstructure science community. In the future, it is likely that it will become a fully independent package, and hence an external pymicro dependency. In that case, it will most likely be name SampleData, as its main class object.
SampleData is a Python class designed to implement two essential functions for the management of multimodal material data sets within the Pymicro code: 1. A unified file format and data model able to handle all data modalities presented below, within the same dataset. 2. A high-level interface allowing interactions with datasets for user and external numerical tools while hiding all the data management technical complexity
As its name suggests, it is designed to associate a dataset file for each material sample, whose data must be collected, stored and/or processed. The class then behave as a high-level interface allowing to interact with the data. Its main functionalities are: * Add, remove, and organize easily data of various types, formats, shapes into the same dataset * Add, remove and organize easily light metadata for all elements of a dataset * get simple or detailed information on the dataset content *
allow easy visualization of spatially organized data (images, maps, measured or simulated fields) * allow a flexible and efficient compression of each element of a dataset * a framework to automate interface between the datasets and data processing or numerical simulation tools * a framework to derive classes from SampleData that are specialized to a specific type of material samples, such as the Microstructure class of the pymicro.crystal.microstructure module, dedicated to
polycrystalline samples * ensure convergence of spatially organized data
Regarding this last point, the “convergence” of data refers to the conversion into a unified data model of all data representing microstructure geometries, of mechanical fields (measured or simulated), of different origins. This allows to directly visualize and process them together with the same tools.
The SampleData class is thus the interface that the data platform user has to manipulate in order to access those functionalities, and interact with the dataset. Its detailed presentation is the subject of the series of notebooks of which this document is only the first. The rest of the section is dedicated to the presentation of the file format and data model that is implement by the SampleData class.
The SampleData file format¶
Even if its implementation does not require it, the SampleData class has been designed to be used with the following convention: 1 dataset per material sample. Each dataset is associated with a pair of files:
a HDF5 file
a XDMF file
HDF5 is a library and file format that supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. It is a hierarchical file format in the sense that it allows to gather multiple data within a file and create links to organize them in relation to each other, creating an internal file hierarchy, replicating a file system with directories and files within one file. HDF5 stores data with a binary encoding, and is compatible with many of the most powerfull compression algorithms available. It also allows to add text metadata to all data elements in the file, called HDF5 attributes. The HDF5 file, within the SampleData file pair, is the actual data set: it contains all data and metadata related to the material sample.
XMDF is an extensible Data Model to exchange scientific data between High Performance Computing codes and numerical tools. It has been practically designed for spatially organized data, i.e. fields, used as inputs and outputs by simulation tools. Practically, it is an XML file containing the description, for each grid supporting the fields, of its topology, its geometry, its dimensions, as well as the data arrays, types and dimensions of field defined on the grid. Rather than writing the heavy data describing gris and fields in to the XML file, the XDMF syntax allow to refer to arrays stored in HDF5 data files. In that case, and in the case of the ``SampleData`` file pair, the XDMF file can be seen as an external metadata file allowing to interpret raw binary data stored in a HDF5 file as spatially organized data.
One important feature of thos HDF5/XDMF file pairs, is that they are a rather standard format. In particular, a reader for this format is implemented in the powerfull and popular 3D visualization software Paraview. The ``SampleData`` class ensures that both HDF5 and XML are always synchronized together and with the content of the dataset. This means that whenever users add spatially organized data into a SampleData HDF5 dataset, they can simply visualize their dataset content by opening
with Paraview the XMDF file.
The SampleData Data Model¶
In addition to the HDF5/XDMF file pair, SampleData datasets comply with a specific data model. This section provides an quick overview of the different elements of this data model. Each of these elements will be extensively detailed in the next Notebooks of this User Guide.
This data model is built on top of the HDF5 data model, whose primary objects are:
Groups: a data structure that can be linked to other groups or datasets, and is used to organized data objects. They can been seen as the ‘directories’ of a HDF5 dataset. Every HDF5 file contains a root group that can contain other groups.
Data arrays or Nodes: arrays of data that can have different types/shapes, and are attached to a Group.
Attributes: Name/Value pairs that can take any form as long as they remain small. It is the official way to store metadata into HDF5 files. Both Groups and Datasets can hold as many Attributes as required.
The SampleData data model introduces two types of particular HDF5 Groups. They are dedicated to the representation of spatially organized data. Such data consist in geometrical grid supporting fields. These grids can be regular gris, or have more complex topologies. Those two situations correspond to the 2 Group types in the data model, that are:
Image Groups are HDF5 groups designed to store data describing 2D or 3D images, i.e. regular grids supporting scalar or tensorial fields. They are used for instance, to store data coming from SEM/EBSD or X-ray tomography imaging, or FFT simulation results. They are actually 3 types of image groups:
2DImagegroups: represent two dimensional images, i.e. grids of \((N_x,N_y)\) pixels3DImagegroups: represent three dimensional images, i.e. grids of \((N_x,N_y, N_z)\) voxelsemptyImagegroups: represent image groups that do not yet support any data and topology yet
Mesh Groups are HDF5 groups designed to store data describing 2D or 3D meshes, i.e. grids described by node coordinates and elements connectivities, supporting scalar or tensorial fields. They can be used to store data coming from finite element simulations, CAD designs etc…They are actually 3 types of image groups:
2DMeshgroups: represent two dimensional meshes, i.e. grid of nodes defined by their \((X_i,Y_i)\) coordinate pairs3DMeshgroups: represent three dimensional meshes, i.e. grid of nodes defined by their \((X_i,Y_i,Z_i)\) coordinate pairsemptyMeshgroups: represent mesh groups that do not yet support any data and topology yet
In addition, the data model introduces two additional types of HDF5 Nodes:
Structured Tables are heterogeneous and bidimensional data arrays, i.e. that may contain data of different types (integers, floats, strings, sub arrays…) within the same row, all rows having the same format. Each column of those arrays can have a specific name. Those arrays are the in-memory equivalent of Numpy structured arrays
Fields are specific data arrays that must belong to a grid group (Image or Mesh Group), and whose shape and dimensions must comply with the grid topology. They are used to store and manipulate spatially organized arrays, that represent mechanical fields (for instance, displacement or temperature fields, microstructure phase maps, EBSD orientation maps….)
Those specific data objects introduce by SampleData all have a specific data model, associated metadata, and a specific interface. They are all reviewed in details by a specific tutorial Notebook of this user guide.
Data Model example¶
To illustrate what SampleData datasets may look like, a virtual example of a polycrystalline material sample dataset is represented in the schematic diagram below:
Groups are used to organize data into coherent categories. Metadata are represented in green, and datasets in red. As it appears on the diagram, Groups can be dedicated only to organize metadata, for instance to document material nature, composition, elaboration process, or the experimental set-up used for imaging and mechanical tests. The Macro Data and Statistics Groups will typically contain simple arrays datasets or structured tables, to store mechanical tests macroscopic outputs (loading curves) or some statistics on the microstructure geometry or mechanical state. The Heavy Data group will typically contain the spatially organized data, coming from 3D or 2D in-situ imaging techniques, and from numerical simulation softwares. It will most likely contain Image or Mesh Groups, filled with Fields datasets.
II - SampleData dependencies and pre-requisites¶
As SampleData is a rather complex class, relying on several libraries and packages, it is strongly adviced to take a look at and learn the basics of the following tools/packages:
Obviously, basic notions of Python (Python syntax, data types, namespaces, imports, classes, working with class methods, Python interpreters)
SampleData HDF5 interface is based on the Python package Pytables. It handles the management of the HDF5 file and the compression of datasets. Knowing its basics can be a valuable help when using SampleData.
Warning: In the series of notebooks that make up this SampleData user guide, a minimal knowledge of these elements will be assumed.
Now that the relevant context and description of the data platform has been set, you can start to learn how to use it with the second Notebook of this user guide !
2 - Getting started with SampleData : Exploring dataset contents¶
This second User Guide tutorial will introduce you to:
create and open datasets with the SampleData class
the SampleData Naming System
how to get informations on a dataset content interactively
how to use the external software Vitables to visualize the content and organization of a dataset
how to use the Paraview software to visualize the spatially organized data stored in datasets
how to use generic HDF5 command line tools to print the content of your dataset
You will find a short summary of all methods reviewed in this tutorial at the end of this page.
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the previous notebook of this User Guide.
Warning
This Notebook review the methods to get information on SampleData HDF5 datsets content. Some of the methods detailed here produce very long outputs, that have been conserved in the documentation version. Reading completely the content of this output is absolutely not necessary to learn what is detailed on this page, they are just provided here as examples. So do not be afraid and fill free to scroll down quickly when you see large prints !
I - Create and Open datasets with the SampleData class¶
In this first section, we will see how to create SampleData datasets, or open pre-existing ones. These two operations are performed by instantiating a SampleData class object.
Before that, you will need to import the SampleData class. We will import it with the alias name SD, by executing:
Import SampleData and get help¶
[1]:
from pymicro.core.samples import SampleData as SD
Before starting to create our datasets, we will take a look at the SampleData class documenation, to discover the arguments of the class constructor. You can read it on the pymicro.core package API doc page, or print interactively by executing:
>>> help(SD)
or, if you are working with a Jupyter notebook, by executing the magic command:
>>> ?SD
Do not hesitate to systematically use the ``help`` function or the ``”?”`` magic command to get information on methods when you encounter a new one. All SampleData methods are documented with explicative docstrings, that detail the method arguments and returns.
Dataset creation¶
The class docstring is divided in multiple rubrics, one of them giving the list of the class constructor arguments. Let us review them one by one.
filename: basename of the HDF5/XDMF pair of file of the dataset
This is the first and only mandatory argument of the class constructor. If this string corresponds to an existing file, the SampleData class will open these file, and create a file instance to interact with this already existing dataset. If the filename do not correspond to an existing file, the class will create a new dataset, which is what we want to do here.
Let us create a SampleData dataset:
[2]:
data = SD(filename='my_first_dataset')
That is it. The class has created a new HDF5/XDMF pair of files, and associated the interface with this dataset to the variable data. No message has been returned by the code, how can we know that the dataset has been created ?
When the name of the file is not an absolute path, the default behavior of the class is to create the dataset in the current work directory. Let us print the content of this directory then !
[3]:
import os # load python module to interact with operating system
cwd = os.getcwd() # get current directory
file_list = os.listdir(cwd) # get content of current work directory
print(file_list,'\n')
# now print only files that start with our dataset basename
print('Our dataset files:')
for file in file_list:
if file.startswith('my_first_dataset'):
print(file)
['3_SampleData_Image_groups.ipynb', 'test_dump.txt', '5_SampleData_data_compression.ipynb', '6_SampleData_inheritance_and_Microstructure_class.ipynb', 'Introduction_backup.md', 'test_crop_data.h5', 'my_first_dataset.h5', '4_SampleData_Mesh_groups.ipynb', 'SampleData_Introduction.ipynb', 'dataset_information.txt', 'SampleDataUserGuide.rst', '1_Getting_Information_from_SampleData_datasets.ipynb', '2_SampleData_basic_data_items.ipynb', 'my_first_dataset.xdmf', 'Images', '.ipynb_checkpoints', 'test_crop_data.xdmf', 'SampleData_Quick_Reference_Sheet.ipynb']
Our dataset files:
my_first_dataset.h5
my_first_dataset.xdmf
The two files my_first_dataset.h5 and my_first_dataset.xdmf have indeed been created.
If you want interactive prints about the dataset creation, you can set the verbose argument to True. This will set the activate the verbose mode of the class. When it is, the class instance prints a lot of information about what it is doing. This flag can be set by using the set_verbosity method:
[4]:
data.set_verbosity(True)
Let us now close our dataset, and see if the class instance prints information about it:
[5]:
del data
Deleting DataSample object
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
Note
It is a good practice to always delete your SampleData instances once you are done working with a dataset, or if you want to re-open it. As the class instance handles opened files as long as it exists, deleting it ensures that the files are properly closed. Otherwise, file may close at some random times or stay opened, and you may encounter undesired behavior of your datasets.
The class indeed returns some prints during the instance destruction. As you can see, the class instance wrights data into the pair of files, and then closes the dataset instance and the files.
Dataset opening and verbose mode¶
Let us now try to create a new SD instance for the same dataset file "my_first_dataset". As the dataset files (HDF5, XDMF) already exist, this new *SampleData* instance will open the dataset files and synchronize with them. with the verbose mode on. When activated, SampleData class instances will display messages about the actions performed by the class (creating, deleting data items for instance)
[6]:
data = SD(filename='my_first_dataset', verbose=True)
-- Opening file "my_first_dataset.h5"
Minimal data model initialization....
Minimal data model initialization done
**** FILE CONTENT ****
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Printing dataset content with max depth 3
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
You can see that the printed information states that the dataset file my_first_dataset.h5 has been opened, and not created. This second instantiation of the class has not created a new dataset, but instead, has opened the one that we have just closed. Indeed, in that case, we provided a filename that already existed.
Some information about the dataset content are also printed by the class. This information can be retrived with specific methods that will be detailed in the next section of this Notebook. Let us focus for now on one part of it.
The printed info reveals that our dataset content is composed only of one object, a Group data object named /. This group is the Root Group of the dataset. Each dataset has necessarily a Root Group, automatically created along with the dataset. You can see that this Group already have a Child, named Index. This particular data object will be presented in the third section of this Notebook. You can also observe that the Root Group already has attributes (recall from introduction
Notebook that they are Name/Value pairs used to store metadata in datasets). Two of those attributes match arguments of the SampleData class constructor:
the description attribute
the sample_name attribute
The description and sample_name are not modified in the dataset when reading a dataset. These SD constructor arguments are only used when creating a dataset. They are string metadata whose role is to give a general name/title to the dataset, and a general description. However, they can be set or changed after the dataset creation with the methods set_sample_name and set_description, used a little further in this Notebook.
Now we know how to open a dataset previously created with SampleData. We could want to open a new dataset, with the name of an already existing data, but overwrite it. The SampleData constructor allows to do that, and we will see it in the next subsection. But first, we will close our dataset again:
[7]:
del data
Deleting DataSample object
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
Overwriting datasets¶
The overwrite_hdf5 argument of the class constructor, if it is set to True, will remove the filename dataset and create a new empty one, if this dataset already exists:
[8]:
data = SD(filename='my_first_dataset', verbose=True, overwrite_hdf5=True)
-- File "/home/amarano/Codes/pymicro/examples/SampleDataUserGuide/my_first_dataset.h5" exists and will be overwritten
-- File "my_first_dataset.h5" not found : file created
-- File "my_first_dataset.xdmf" not found : file created
.... writing xdmf file : my_first_dataset.xdmf
Minimal data model initialization....
Minimal data model initialization done
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
As you can see, the dataset files have been overwritten, as requested. We will now close our dataset again and continue to see the possibilities offered by the class constructor.
[9]:
del data
Deleting DataSample object
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
Copying dataset¶
One last thing that may be interesting to do with already existing dataset files, is to create a new dataset that is a copy of them, associated with a new class instance. This is usefull for instance when you have to try new processing on a set of valuable data, without risking to damage the data.
To do this, you may use the copy_sample method of the SampleData class. Its main arguments are:
src_sample_file: basename of the dataset files to copy (source file)dst_sample_file: basename of the dataset to create as a copy of the source (desctination file)get_object: ifFalse, the method will just create the new dataset files and close them. IfTrue, the method will leave the files open and return a SampleData instance that you may use to interact with your new dataset.
Let us try to create a copy of our first dataset:
[10]:
data2 = SD.copy_sample(src_sample_file='my_first_dataset', dst_sample_file='dataset_copy', get_object=True)
[11]:
cwd = os.getcwd() # get current directory
file_list = os.listdir(cwd) # get content of current work directory
print(file_list,'\n')
# now print only files that start with our dataset basename
print('Our dataset files:')
for file in file_list:
if file.startswith('dataset_copy'):
print(file)
['3_SampleData_Image_groups.ipynb', 'dataset_copy.xdmf', 'test_dump.txt', '5_SampleData_data_compression.ipynb', '6_SampleData_inheritance_and_Microstructure_class.ipynb', 'Introduction_backup.md', 'test_crop_data.h5', 'my_first_dataset.h5', '4_SampleData_Mesh_groups.ipynb', 'SampleData_Introduction.ipynb', 'dataset_information.txt', 'SampleDataUserGuide.rst', '1_Getting_Information_from_SampleData_datasets.ipynb', '2_SampleData_basic_data_items.ipynb', 'my_first_dataset.xdmf', 'Images', '.ipynb_checkpoints', 'dataset_copy.h5', 'test_crop_data.xdmf', 'SampleData_Quick_Reference_Sheet.ipynb']
Our dataset files:
dataset_copy.xdmf
dataset_copy.h5
The copy_dataset HDF5 and XDMF files have indeed been created, and are a copy of the my_first_dataset HDF5 and XDMF files.
Note that the copy_sample is a static method, that can be called even without SampleData instance. Note also that it has a overwrite argument, that allows to overwrite an already existing dst_sample_file. It also has, like the class constructor, a autodelete argument, that we will discover in the next subsection.
Automatically removing dataset files¶
In some occasions, we may want to remove our dataset files after using our SampleData class instance. This can be the case for instance if you are trying some new data processing, or using the class for visualization purposes, and are not interested in keeping your test data.
The class has a autodelete attribute for this purpose. IF it is set to True, the class destructor will remove the dataset file pair in addition to deleting the class instance. The class constructor and the copy_sample method also have a autodelete argument, which, if True, will automatically set the class instance autodelete attribute to True.
To illustrate this feature, we will try to change the autodelete attribute of our copied dataset to True, and remove it.
[12]:
# set the autodelete argument to True
data2.autodelete = True
# Set the verbose mode on for copied dataset
data2.set_verbosity(True)
[13]:
# Close copied dataset
del data2
Deleting DataSample object
.... Storing content index in dataset_copy.h5:/Index attributes
.... writing xdmf file : dataset_copy.xdmf
.... flushing data in file dataset_copy.h5
File dataset_copy.h5 synchronized with in memory data tree
Dataset and Datafiles closed
SampleData Autodelete:
Removing hdf5 file dataset_copy.h5 and xdmf file dataset_copy.xdmf
The class destructor ends by priting a confirmation message of the dataset files removal in verbose mode, as you can see in the cell above. Let us verify that it has been effectively deleted:
[14]:
file_list = os.listdir(cwd) # get content of current work directory
print(file_list,'\n')
# now print only files that start with our dataset basename
print('Our copied dataset files:')
for file in file_list:
if file.startswith('dataset_copy'):
print(file)
['3_SampleData_Image_groups.ipynb', 'test_dump.txt', '5_SampleData_data_compression.ipynb', '6_SampleData_inheritance_and_Microstructure_class.ipynb', 'Introduction_backup.md', 'test_crop_data.h5', 'my_first_dataset.h5', '4_SampleData_Mesh_groups.ipynb', 'SampleData_Introduction.ipynb', 'dataset_information.txt', 'SampleDataUserGuide.rst', '1_Getting_Information_from_SampleData_datasets.ipynb', '2_SampleData_basic_data_items.ipynb', 'my_first_dataset.xdmf', 'Images', '.ipynb_checkpoints', 'test_crop_data.xdmf', 'SampleData_Quick_Reference_Sheet.ipynb']
Our copied dataset files:
As you can see, the dataset files have been suppressed. Now we can also open and remove our first created dataset using the class constructor autodelete option:
[15]:
data = SD(filename='my_first_dataset', verbose=True, autodelete=True)
print(f'Is autodelete mode on ? {data.autodelete}')
del data
-- Opening file "my_first_dataset.h5"
Minimal data model initialization....
Minimal data model initialization done
**** FILE CONTENT ****
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Printing dataset content with max depth 3
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
Is autodelete mode on ? True
Deleting DataSample object
.... Storing content index in my_first_dataset.h5:/Index attributes
.... writing xdmf file : my_first_dataset.xdmf
.... flushing data in file my_first_dataset.h5
File my_first_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
SampleData Autodelete:
Removing hdf5 file my_first_dataset.h5 and xdmf file my_first_dataset.xdmf
[16]:
file_list = os.listdir(cwd) # get content of current work directory
print(file_list,'\n')
# now print only files that start with our dataset basename
print('Our dataset files:')
for file in file_list:
if file.startswith('my_first_dataset'):
print(file)
['3_SampleData_Image_groups.ipynb', 'test_dump.txt', '5_SampleData_data_compression.ipynb', '6_SampleData_inheritance_and_Microstructure_class.ipynb', 'Introduction_backup.md', 'test_crop_data.h5', '4_SampleData_Mesh_groups.ipynb', 'SampleData_Introduction.ipynb', 'dataset_information.txt', 'SampleDataUserGuide.rst', '1_Getting_Information_from_SampleData_datasets.ipynb', '2_SampleData_basic_data_items.ipynb', 'Images', '.ipynb_checkpoints', 'test_crop_data.xdmf', 'SampleData_Quick_Reference_Sheet.ipynb']
Our dataset files:
Now, you now how to create or open SampleData datasets. Before starting to explore their content in detail, a last feature of the SampleData class must be introduced: the naming system and conventions used to create or access data items in datasets.
Note
Using the autodelete option is usefull when you want are using the class for tries, or tests, and do not want to keep the dataset files on your computer. It is also a proper way to remove a SampleData dataset, as it allows to remove both files in one time.
II - The SampleData Naming system¶
SampleData datasets are composed of a set of organized data items. When handling datasets, you will need to specify which item you want to interact with or create. The SampleData class provides 4 different ways to refer to datasets. The first type of data item identificator is :
the Path of the data item in the HDF5 file.
Like a file within a filesystem has a path, HDF5 data items have a Path within the dataset. Each data item is the children of a HDF5 Group (analogous to a file contained in a directory), and each Group may also be children of a Group (analogous to a directory contained in a directory). The origin directory is called the root group, and has the path '/'. The Path offers a completely non-ambiguous way to designate a data item within the dataset, as it is unique. A typical path of a dataitem
will look like that : /Parent_Group1/Parent_Group2/ItemName. However, pathes can become very long strings, and are usually not a convenient way to name data items. For that reason, you also can refer to them in SampleData methods using:
the Name of the data item.
It is the last element of its Path, that comes after the last / character. For a dataset that has the path /Parent_Group1/Parent_Group2/ItemName, the dataset Name is ItemName. It allows to refer quickly to the data item without writing its whole Path.
However, note that two different datasets may have the same Name (but different pathes), and thus it may be necessary to use additional names to refer to them with no ambiguity without having to write their full path. In addition, it may be convenient to be able to use, in addition to its storage name, one or more additional and meaningfull names to designate a data item. For these reasons, two additional identificators can be used:
the Indexname of the data item
the Alias or aliases of the data item
Those two types of indentificators are strings that can be used as additional data item Names. They play completely similar roles. The Indexname is also used in the dataset Index (see below), that gather the data item indexnames together with their pathes within the dataset. All data items must have an Indexname, that can be identical to their Name. If additionnal names are given to a dataset, they are stored as an Alias.
Many SampleData methods have a nodename or name argument. Everytime you will encounter it, you may use one of the 4 identificators presented in this section, to provide the name of the dataset you want to create or interact with. Many examples will follow in the rest of this Notebook, and of this User Guide.
Let us now move on to discover the methods that allow to explore the datasets content.
III- Interactively get information on datasets content¶
The goal of this section is to review the various way to get interactive information on your SampleData dataset (interactive in the sens that you can get them by executing SampleData class methods calls into a Python interpreter console).
For this purpose, we will use a pre-existing dataset that already has some data stored, and look into its content. This dataset is a reference SampleData dataset used for the core package unit tests.
[17]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'test_sampledata_ref') # test dataset file path
data = SD(filename=dataset_file)
1- The Dataset Index¶
As explained in the previous section, all data items have a Path, and an Indexname. The collection of Indexname/Path pairs forms the Index of the dataset. For each SampleData dataset, an Index Group is stored in the root Group, and the collection of those pairs is stored as attributes of this Index Group. Additionnaly, a class attribute content_index stores them as a dictionary in the clas instance, and allows to access them easily. The dictionary and the Index
Group attributes are automatically synchronized by the class.
Let us see if we can see the dictionary content:
[18]:
data.content_index
[18]:
{'array': '/test_group/test_array',
'group': '/test_group',
'image': '/test_image',
'image_Field_index': '/test_image/Field_index',
'image_test_image_field': '/test_image/test_image_field',
'mesh': '/test_mesh',
'mesh_ElTagsList': '/test_mesh/Geometry/Elem_tags_list',
'mesh_ElTagsTypeList': '/test_mesh/Geometry/Elem_tag_type_list',
'mesh_ElemTags': '/test_mesh/Geometry/ElementsTags',
'mesh_Elements': '/test_mesh/Geometry/Elements',
'mesh_Field_index': '/test_mesh/Field_index',
'mesh_Geometry': '/test_mesh/Geometry',
'mesh_NodeTags': '/test_mesh/Geometry/NodeTags',
'mesh_NodeTagsList': '/test_mesh/Geometry/Node_tags_list',
'mesh_Nodes': '/test_mesh/Geometry/Nodes',
'mesh_Nodes_ID': '/test_mesh/Geometry/Nodes_ID',
'mesh_Test_field1': '/test_mesh/Test_field1',
'mesh_Test_field2': '/test_mesh/Test_field2',
'mesh_Test_field3': '/test_mesh/Test_field3',
'mesh_Test_field4': '/test_mesh/Test_field4'}
You should see the dictionary keys that are names of data items, and associated values, that are hdf5 pathes. You can see also data item Names at the end of their Pathes. The data item aliases are also stored in a dictionary, that is an attribute of the class, named aliases:
[19]:
data.aliases
[19]:
{}
You can see that this dictionary contains keys only for data item that have additional names, and also that those keys are the data item indexnames.
The dataset index can be plotted together with the aliases, with a prettier aspect, by calling the method print_index:
[20]:
data.print_index()
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
This method prints the content of the dataset Index, with a given depth and from a specific root. The depth is the number of parents that a data item has. The root Group has thus a depth of 0, its children a depth of 1, the children of its children a depth of 2, and so on… The local root argument can be changed, to print only the Index for data items that are children of a specific group. When used without arguments, print_index uses a depth of 3 and the dataset root as default settings.
As you can see, our dataset already contains some data items. We can already identify at least 3 HDF5 Groups (test_group, test_image, test_group), as they have childrens, and a lot of other data items.
Let us try different to print Indexes with different parameters. To start, Let us try to print the Index from a different local root, for instance the group with the path /test_image. The way to do it is to use the local_root argument. We will hence give it the value of the /test_image path.
[21]:
data.print_index(local_root="/test_image")
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/test_image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
The print_index method local root arguments needs the name of the Group whose children Index must be printed. As explained in section II, you may use for this other identificators than its Path. Let us try its Name (last part of its path), which is test_image, or its Indexname, which is image:
[22]:
data.print_index(local_root="test_image")
data.print_index(local_root="image")
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `test_image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `image`
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
As you can see, the result is the same in the 3 cases.
Let us now try to print the dataset Index with a maximal data item depth of 2, using the max_depth argument:
[23]:
data.print_index(max_depth=2)
Dataset Content Index :
------------------------:
index printed with max depth `2` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Of course, you can combine those two arguments:
[24]:
data.print_index(max_depth=2, local_root='mesh')
Dataset Content Index :
------------------------:
index printed with max depth `2` and under local root `mesh`
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
The print_index method is usefull to get a glimpse of the content and organization of the whole dataset, or some part of it, and to quickly see the short indexnames or aliases that you can use to refer to data items.
To add aliases to data items or Groups, you can use the add_alias method.
The Index allows to quickly see the internal structure of your dataset, however, it does not provide detailed information on the data items. We will now see how to retrieve it with the SampleData class.
2- The Dataset content¶
The SampleData class provides a method to print an organized and detailed overview of the data items in the dataset, the print_dataset_content method. Let us see what the methods prints when called with no arguments:
[25]:
data.print_dataset_content()
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: test_sampledata_ref.h5
-- Size: 1.271 Mb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
This is a test dataset created by the SampleData class unit tests.
* sample_name : test_sample
-- Childrens : Index, test_group, test_image, test_mesh,
----------------
************************************************
GROUP test_group
=====================
-- Parent Group : /
-- Group attributes :
* group_type : Group
-- Childrens : test_array,
----------------
****** Group /test_group CONTENT ******
NODE: /test_group/test_array
====================
-- Parent Group : test_group
-- Node name : test_array
-- test_array attributes :
* empty : False
* node_type : data_array
-- content : /test_group/test_array (CArray(51,)) 'array'
-- Compression options for node `/test_group/test_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
************************************************
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field,
----------------
****** Group /test_image CONTENT ******
NODE: /test_image/Field_index
====================
-- Parent Group : test_image
-- Node name : Field_index
-- Field_index attributes :
* node_type : string array
-- content : /test_image/Field_index (EArray(1,)) ''
-- Compression options for node `/test_image/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_image/test_image_field
====================
-- Parent Group : test_image
-- Node name : test_image_field
-- test_image_field attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : test_image_field
* xdmf_gridname : test_image
-- content : /test_image/test_image_field (CArray(10, 10, 10)) 'image_test_image_field'
-- Compression options for node `/test_image/test_image_field`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (327, 10, 10)
-- Node memory size : 63.867 Kb
----------------
************************************************
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4,
----------------
****** Group /test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* node_type : string array
-- content : /test_mesh/Field_index (EArray(9,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
****** Group /test_mesh/Geometry CONTENT ******
NODE: /test_mesh/Geometry/Elem_tag_type_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tag_type_list
-- Elem_tag_type_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Elem_tag_type_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tag_type_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elem_tags_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tags_list
-- Elem_tags_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Elem_tags_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elements
====================
-- Parent Group : Geometry
-- Node name : Elements
-- Elements attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Elements (CArray(24,)) 'mesh_Elements'
-- Compression options for node `/test_mesh/Geometry/Elements`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
GROUP ElementsTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : ET_2D, ET_Bottom, ET_Top, field_2D, field_Bottom, field_Top,
----------------
GROUP NodeTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : NT_Z0_plane, NT_out_of_plane, field_Z0_plane, field_out_of_plane,
----------------
NODE: /test_mesh/Geometry/Node_tags_list
====================
-- Parent Group : Geometry
-- Node name : Node_tags_list
-- Node_tags_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Node_tags_list (EArray(2,)) ''
-- Compression options for node `/test_mesh/Geometry/Node_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Nodes
====================
-- Parent Group : Geometry
-- Node name : Nodes
-- Nodes attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes (CArray(6, 3)) 'mesh_Nodes'
-- Compression options for node `/test_mesh/Geometry/Nodes`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/Geometry/Nodes_ID
====================
-- Parent Group : Geometry
-- Node name : Nodes_ID
-- Nodes_ID attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes_ID (CArray(6,)) 'mesh_Nodes_ID'
-- Compression options for node `/test_mesh/Geometry/Nodes_ID`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
************************************************
As you can see, this method prints by increasing depth, detailed information on each Group and each data item of the dataset, with a maximum depth that can be specified with a max_depth argument (like the method print_index, that has a default value of 3). The printed output is structured by groups: each Group that has childrenis described by a first set of information, followed by a Group CONTENT string that describes all of its childrens.
For each data item, or Group, the method prints their name, path, type, attributes, content, compression settings and memory size if it is an array, children names if it is a Group. Hence, when calling this method, you can see the content and organization of the dataset, all the metadata attached to all data items, and the disk size occupied by each data item. As you progress through this tutorial, you will learn the meaning of those informations for all types of SampleData data items.
The print_dataset_content method has a short boolean argument, that allows to plot a condensed string representation of the dataset:
[26]:
data.print_dataset_content(short=True)
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
This shorter print can be read easily, provide a complete and visual overview of the dataset organization, and indicate the memory size and type of each data item or Group in the dataset. The printed output distinguishes Group data items, from Nodes data item. The later regroups all types of arrays that may be stored in the HDF5 file.
Both short and long version of the print_dataset_content output can be written into a text file, if a filename is provided as value for the to_file method argument:
[27]:
data.print_dataset_content(short=True, to_file='dataset_information.txt')
[28]:
# Let us open the content of the created file, to see if the dataset information has been written in it:
%cat dataset_information.txt
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
Note
The string representation of the SampleData class is composed of a first part, which is the output of the print_index method, and a second part, that is the output of the print_datase_content method (short output).
[29]:
# SampleData string representation :
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
Now you now how to get a detailed overview of the dataset content. However, with large datasets, that may have a complex internal organization (many Groups, lot of data items and metadata…), the print_dataset_content return string can become very large. In this case, it becomes cumbersome to look for a specific information on a Group or on a particular data item. For this reason, the SampleData class provides methods to only print information on one or several data items of the dataset.
They are presented in the next subsections.
3- Get information on data items¶
To get information on a specific data item (including Groups), you may use the print_node_info method. This method has 2 arguments: the name argument, and the short argument. As explain in section II, the name argument can be one of the 4 possible identifier that the target node can have (name, path, indexname or alias). The short argument has the same effect on the printed output as for the print_dataset_content method. Let us look at some examples. Its default value is
False, i.e. the detailed output.
First, we will for instance want to have information on the Image Group that is stored in the dataset. The print_index and short print_dataset_content allowed us to see that this group has the name test_image, the indexname image, and the path /test_image. We will call the method with two of those identificators, and with the two possible values of the short argument.
[30]:
# Method called with data item indexname, and short output
data.print_node_info(nodename='image', short=True)
|--GROUP test_image: /test_image (3DImage)
[31]:
# Method called with data item Path and long output
data.print_node_info(nodename='/test_image', short=False)
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field,
----------------
You can observe that this method prints the same block of information that the one that appeared in the print_dataset_content method output, for the description of the test_image group. With this block, we can learn that this Group is a children of the root Group (‘/’), that it has two children that are the data items named Field_index and test_image_field. We can also see its attributes names and values. Here they provide information on the nature of the Group, that is a 3D image
group, and on the topology of this image (for instance, that it is a 9x9x9 voxel image, of size 0.2).
Let us now apply this method on a data item that is not a group, the test_array data item. The print_index function instructed us that this node has an alias name, that is test_alias. We will use it here to get information on this node, to illustrate the use of the only type of node indicator that has not been used throughout this notebook:
[32]:
data.print_node_info('test_alias')
No group named test_alias
Here, we can learn which is the Node parent, what is the node Name, see that it has no attributes, see that it is an array of shape (51,), that it is not stored with data compression (compresion level to 0), and that it occupies a disk space of 64 Kb.
The print_node_info method is usefull to get information on a specific target, and avoir dealing with the sometimes too large output returned by the print_dataset_content method.
4- Get information on Groups content¶
The previous subsection showed that the print_node_info method applied on Groups returns only information about the group name, metadata and children names. The SampleData class offers a method that allows to print this information, with in addition, the detailed content of each children of the target group: the print_group_content method.
Let us try it on the Mesh group of our test dataset:
[33]:
data.print_group_content(groupname='test_mesh')
****** Group test_mesh CONTENT ******
****** Group test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* node_type : string array
-- content : /test_mesh/Field_index (EArray(9,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
Obviously, this methods is identical to the print_dataset_content method, but restricted to one Group. As the first one, it has a to_file, a short and a max_depth arguments. These arguments work just as for print_dataset_content method, hence there use is not detailed here. the However, you may see one difference here. In the output printed above, we see that the test_mesh group has a Geometry children which is a group, but whose content is not printed. The
print_group_content has indeed, by default, a non-recursive behavior. To get a recursive print of the group content, you must set the recursive argument to True:
[34]:
data.print_group_content('test_mesh', recursive=True)
****** Group test_mesh CONTENT ******
****** Group test_mesh CONTENT ******
NODE: /test_mesh/Field_index
====================
-- Parent Group : test_mesh
-- Node name : Field_index
-- Field_index attributes :
* node_type : string array
-- content : /test_mesh/Field_index (EArray(9,)) ''
-- Compression options for node `/test_mesh/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
****** Group /test_mesh/Geometry CONTENT ******
NODE: /test_mesh/Geometry/Elem_tag_type_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tag_type_list
-- Elem_tag_type_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Elem_tag_type_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tag_type_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elem_tags_list
====================
-- Parent Group : Geometry
-- Node name : Elem_tags_list
-- Elem_tags_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Elem_tags_list (EArray(3,)) ''
-- Compression options for node `/test_mesh/Geometry/Elem_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Elements
====================
-- Parent Group : Geometry
-- Node name : Elements
-- Elements attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Elements (CArray(24,)) 'mesh_Elements'
-- Compression options for node `/test_mesh/Geometry/Elements`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
GROUP ElementsTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : ET_2D, ET_Bottom, ET_Top, field_2D, field_Bottom, field_Top,
----------------
GROUP NodeTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : NT_Z0_plane, NT_out_of_plane, field_Z0_plane, field_out_of_plane,
----------------
NODE: /test_mesh/Geometry/Node_tags_list
====================
-- Parent Group : Geometry
-- Node name : Node_tags_list
-- Node_tags_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Node_tags_list (EArray(2,)) ''
-- Compression options for node `/test_mesh/Geometry/Node_tags_list`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_mesh/Geometry/Nodes
====================
-- Parent Group : Geometry
-- Node name : Nodes
-- Nodes attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes (CArray(6, 3)) 'mesh_Nodes'
-- Compression options for node `/test_mesh/Geometry/Nodes`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/Geometry/Nodes_ID
====================
-- Parent Group : Geometry
-- Node name : Nodes_ID
-- Nodes_ID attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes_ID (CArray(6,)) 'mesh_Nodes_ID'
-- Compression options for node `/test_mesh/Geometry/Nodes_ID`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `/test_mesh/Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field2
====================
-- Parent Group : test_mesh
-- Node name : Test_field2
-- Test_field2 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field2
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field2 (CArray(6, 1)) 'mesh_Test_field2'
-- Compression options for node `/test_mesh/Test_field2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `/test_mesh/Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field4
====================
-- Parent Group : test_mesh
-- Node name : Test_field4
-- Test_field4 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field4
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field4 (CArray(8, 1)) 'mesh_Test_field4'
-- Compression options for node `/test_mesh/Test_field4`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
As you can see, the information on the childrens of the Geometry group have been printed. Note that the max_depth argument is considered by this method as an absolute depth, meaning that you have to specify a depth that is at least the depth of the target group to see some output printed for the group content. The default maximum depth for this method is set to a very high value of 1000. Hence, print_group_content prints by defaults group contents with a recursive behavior. Note also
that print_group_content with the recursive option, is equivalent to print_dataset_content but prints the dataset content as if the target group was the root.
5- Get information on grids¶
One of the SampleData class main functionalities is the manipulation and storage of spatially organized data, which is handled by Grid groups in the data model. Because they are usually key data for mechanical sample datasets, the SampleData class provides a method to print Grouo informations only for Grid groups, the print_grids_info method:
[35]:
data.print_grids_info()
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field,
----------------
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4,
----------------
This method also has the to_file and short arguments of the print_dataset_content method:
[36]:
data.print_grids_info(short=True, to_file='dataset_information.txt')
%cat dataset_information.txt
|--GROUP test_image: /test_image (3DImage)
|--GROUP test_mesh: /test_mesh (3DMesh)
6- Get xdmf tree content¶
As explained in the first Notebook of this User Guide, these grid Groups and associated data are stored in a dual format by the SampleData class. This dual format is composed of the dataset HDF5 file, and an associated XDMF file containing metadata, describing Grid groups topology, data types and fields.
The XDMF file is handled in the SampleData class by the xdmf_tree attribute, which is an instance of the lxml.etree class of the lxml package:
[37]:
data.xdmf_tree
[37]:
<lxml.etree._ElementTree at 0x7f4d3917c640>
The XDMF file is synchronized with the in-memory xdmf_tree argument when calling the sync method, or when deleting the SampleData instance. However, you may want to look at the content of the XDMF tree while you are interactively using your SampleData instance. In this case, you can use the print_xdmf method:
[38]:
data.print_xdmf()
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd">
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_mesh" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_image_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10" NumberType="Int" Precision="16">test_sampledata_ref.h5:/test_image/test_image_field</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
As you can observe, you will get a print of the content of the XDMF file that would be written if you would close the file right now. You can observe that the XDMF file provides information on the grids that match those given by the Groups and Nodes attributes printed above with the previously studied method: the test image is a regular grid of 10x10x10 nodes, i.e. a 9x9x9 voxels grid. Only one field is defined on test_image, test_image_field, whereas two are defined on test_mesh.
This XDMF file can directly be opened in Paraview, if both file are closed. If any syntax or formatting issue is encountered when Paraview reads the XDMF file, it will return an error message and the data visualization will not be rendered. The print_xdmf method allows you to verify your XDMF data and syntax, to make sure that the data formatting is correct.
7- Get memory size of file and data items¶
SampleData is designed to create large datasets, with data items that can reprensent tens Gb of data or more. Being able to easily see and identify which data items use the most disk space is a crucial aspect for data management. Until now, with the method we have reviewed, we only have been able to print the Nodes disk sizes together with a lot of other information. In order to speed up this process, the SampleData class has one method that allow to directly query and print only the memory
size of a Node, the get_node_disk_size method:
[39]:
data.get_node_disk_size(nodename='test_array')
Node test_array size on disk is 64.000 Kb
[39]:
(64.0, 'Kb')
As you can see, the default behavior of this method is to print a message indicating the Node disk size, but also to return a tuple containing the value of the disk size and its unit. If you want to print data in bytes, you may call this method with the convert argument set to False:
[40]:
data.get_node_disk_size(nodename='test_array', convert=False)
Node test_array size on disk is 65536.000 bytes
[40]:
(65536, 'bytes')
If you want to use this method to get a numerical value within a script, but do not want the class to print anything, you can use the print_flag argument:
[41]:
size, unit = data.get_node_disk_size(nodename='test_array', print_flag=False)
print(f'Printed by script: node size is {size} {unit}')
size, unit = data.get_node_disk_size(nodename='test_array', print_flag=False, convert=False)
print(f'Printed by script: node size is {size} {unit}')
Printed by script: node size is 64.0 Kb
Printed by script: node size is 65536 bytes
The disk size of the whole HDF5 file can also be printed/returned, using the get_file_disk_size method, that has the same print_flag and convert arguments:
[42]:
data.get_file_disk_size()
size, unit = data.get_file_disk_size(convert=False, print_flag=False)
print(f'\nPrinted by script: file size is {size} {unit}')
File size is 1.271 Mb for file
test_sampledata_ref.h5
Printed by script: file size is 1333013 bytes
8- Get nodes/groups attributes (metadata)¶
Another central aspect of the SampleData class is the management of metadata, that can be attached to all Groups or Nodes of the dataset. Metadata comes in the form of HDF5 attributes, that are Name/Value pairs, and that we already encountered when exploring the outputs of methods like print_dataset_content, print_node_info…
Those methods print the Group/Node attributes together with other information. To only print the attributes of a given data item, you can use the print_node_attributes method:
[43]:
data.print_node_attributes(nodename='test_mesh')
-- test_mesh attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* xdmf_gridname : test_mesh
As you can see, this method prints a list of all data item attributes, with the format * Name : Value \n. It allows you to quickly see what attributes are stored together with a given data item, and their values.
If you want to get the value of a specific attribute, you can use the get_attribute method. It takes two arguments, the name of the attribute you want to retrieve, and the name of the data item where it is stored:
[44]:
Nnodes = data.get_attribute(attrname='number_of_nodes', nodename='test_mesh')
print(f'The mesh test_mesh has {Nnodes} nodes')
The mesh test_mesh has 6 nodes
You can also get all attributes of a data item as a dictionary. In this case, you just need to specify the name of the data item from which you want attributes, and use the get_dic_from_attributes method:
[45]:
mesh_attrs = data.get_dic_from_attributes(nodename='test_mesh')
for name, value in mesh_attrs.items():
print(f' Attribute {name} is {value}')
Attribute Elements_offset is [0]
Attribute Number_of_boundary_elements is [0]
Attribute Number_of_bulk_elements is [8]
Attribute Number_of_elements is [8]
Attribute Topology is Uniform
Attribute Xdmf_elements_code is ['Triangle']
Attribute description is
Attribute element_type is ['tri3']
Attribute elements_path is /test_mesh/Geometry/Elements
Attribute empty is False
Attribute group_type is 3DMesh
Attribute nodesID_path is /test_mesh/Geometry/Nodes_ID
Attribute nodes_path is /test_mesh/Geometry/Nodes
Attribute number_of_nodes is 6
Attribute xdmf_gridname is test_mesh
We have now seen how to explore all of types of information that a SampleData dataset may contain, individually or all together, interactively, from a Python console. Let us review now how to explore the content of SampleData datasets with external softwares.
IV - Visualize dataset contents with Vitables¶
All the information that you can get with all the methods presented in the previous section can also be accessed externally by opening the HDF5 dataset file with the Vitables software. This software is usually part of the Pytables package, that is a dependency of pymicro. You should be able to use it in a Python environement compatible with pymicro. If needed, you may refer to the Vitables website to find download and installations instructions for PyPi or conda: https://vitables.org/.
Vitables provide a graphical interface that allows you to browse through all your dataset data items, and access or modify their stored data and metadata values. You may either open Vitables and then open your HDF5 dataset file from the Vitables interface, or you can directly open Vitables to read a specfic file from command line, by running: vitables my_dataset_path.h5.
This command will work only if your dataset file is closed (if the SampleData instance still exists in your Python console, this will not work, you first need to delete your instance to close the files).
However, the SampleData class has a specific method to allowing to open your dataset with Vitables interactively, directly from your Python console: the method pause_for_visualization. As explained just above, this method closes the XDMF and HDF5 datasets, and runs in your shell the command vitables my_dataset_path.h5. Then, it freezes the interactive Python console and keep the dataset files closed, for as long as the Vitables software is running. When Vitables is shutdown, the
SampleData class will reopen the HDF5 and XDMF files, synchronize with them and resume the interactive Python console.
Warning
When calling the pause_for_visualization method from a python console (ipython, Jupyter…), you may face environment issue leading to your shell not finding the proper Vitables software executable. To ensure that the right Vitables is found, the method can take an optional argument Vitables_path, which must be the path of the Vitables executable. If this argument is passed, the method will run, after closing the HDF5 and XDMF files, the command
Vitables_path my_dataset_path.hdf5
Note
The method is not called here to allow automatic execution of the Notebook when building the documentation on a platform that do not have Vitables available.
[46]:
# uncomment to test
# data.pause_for_visualization(Vitables=True, Vitables_path='Path_to_Vitables_executable')
Please refer to the Vitables documentation, that can be downloaded here https://sourceforge.net/projects/vitables/files/ViTables-3.0.0/, to learn how to browse through your HDF5 file. The Vitables software is very intuitive, you will see that it is provides a usefull and convenient tool to explore your SampleData datasets outside of your interactive Python consoles.
V - Visualize datasets grids and fields with Paraview¶
As for Vitables, the pause_for_visualization method allows you to open your dataset with Paraview, interactively from a Python console.
Paraview will provide you with a very powerfull visualization tool to render your spatially organized data (grids) stored in your datasets. Unlike Vitables, Paraview must can read the XDMF format. Hence, if you want to open your dataset with Paraview, outside of a Python console, make sure that the HFD5 and XDMF file are not opened by another program, and run in your shell the command: paraview my_dataset_path.xdmf.
As you may have guessed, the pause_for_visualization method, when called interactively with the Paraview argument set to True, will close both files, and run this command, just like for the Vitables option. The datasets will remained closed and the Python console freezed for as long as you will keep the Paraview software running. When you will shutdown Paraview, the SampleData class will reopen the HDF5 and XDMF files, synchronize with them and resume the interactive Python console.
Warning
When calling the pause_for_visualization method from a python console (ipython, Jupyter…), you may face environment issue leading to your shell not finding the proper Paraview software executable. To ensure that the right Paraview is found, the method can take an optional argument Paraview_path, which must be the path of the Vitables executable. If this argument is passed, the method will run, after closing the HDF5 and XDMF files, the command Paraview_path my_dataset_path.xdmf
[47]:
# Like for Vitables --> uncomment to test
# data.pause_for_visualization(Paraview=True, Paraview_path='Path_to_Paraview_executable')
Note
It is recommended to use a recent version of the Paraview software to visualize SampleData datasets (>= 5.0). When opening the XDMF file, Paraview may ask you to choose a specific file reader. It is recommended to choose the XDMF_reader, and not the Xdmf3ReaderT, or Xdmf3ReaderS.
VI - Using command line tools¶
You can also examine the content of your HDF5 datasets with generoc HDF5 command line tools, such as h5ls or h5dump:
Warning
In the following, executable programs that come with the HDF5 library and the Pytables package are used. If you are executing this notebook with Jupyter, you may not be able to have those executable in your path, if your environment is not suitably set. A workaround consist in finding the absolute path of the executable, and replacing the executable name in the following cells by its full path. For instance, replace
`ptdump file.h5`
with
`/full/path/to/ptdump file.h5`
To find this full path, you can run in your shell the command which ptdump. Of course, the same applies for h5ls and h5dump.
Note
Most code lines below are commented as they produce very large outputs, that otherwise pollute the documentation if they are included in the automatic build process. Uncomment them to test them if you are using interactively these notebooks !
For that, you must first close your dataset. If you don’t, this tools will not be able to open the HDF5 file as it is opened by the SampleData class in the Python interpretor.
[48]:
del data
[49]:
# raw output of H5ls --> prints the childrens of the file root group
!h5ls ../data/test_sampledata_ref.h5
Index Group
test_group Group
test_image Group
test_mesh Group
[50]:
# recursive output of h5ls (-r option) --> prints all data items
!h5ls -r ../data/test_sampledata_ref.h5
/ Group
/Index Group
/Index/Aliases Group
/test_group Group
/test_group/test_array Dataset {51/8192}
/test_image Group
/test_image/Field_index Dataset {1/Inf}
/test_image/test_image_field Dataset {10/327, 10, 10}
/test_mesh Group
/test_mesh/Field_index Dataset {9/Inf}
/test_mesh/Geometry Group
/test_mesh/Geometry/Elem_tag_type_list Dataset {3/Inf}
/test_mesh/Geometry/Elem_tags_list Dataset {3/Inf}
/test_mesh/Geometry/Elements Dataset {24/8192}
/test_mesh/Geometry/ElementsTags Group
/test_mesh/Geometry/ElementsTags/ET_2D Dataset {8/8192}
/test_mesh/Geometry/ElementsTags/ET_Bottom Dataset {4/8192}
/test_mesh/Geometry/ElementsTags/ET_Top Dataset {4/8192}
/test_mesh/Geometry/ElementsTags/field_2D Dataset {8/8192, 1}
/test_mesh/Geometry/ElementsTags/field_Bottom Dataset {8/8192, 1}
/test_mesh/Geometry/ElementsTags/field_Top Dataset {8/8192, 1}
/test_mesh/Geometry/NodeTags Group
/test_mesh/Geometry/NodeTags/NT_Z0_plane Dataset {4/8192}
/test_mesh/Geometry/NodeTags/NT_out_of_plane Dataset {2/8192}
/test_mesh/Geometry/NodeTags/field_Z0_plane Dataset {6/65536, 1}
/test_mesh/Geometry/NodeTags/field_out_of_plane Dataset {6/65536, 1}
/test_mesh/Geometry/Node_tags_list Dataset {2/Inf}
/test_mesh/Geometry/Nodes Dataset {6/2730, 3}
/test_mesh/Geometry/Nodes_ID Dataset {6/8192}
/test_mesh/Test_field1 Dataset {6/8192, 1}
/test_mesh/Test_field2 Dataset {6/8192, 1}
/test_mesh/Test_field3 Dataset {8/8192, 1}
/test_mesh/Test_field4 Dataset {8/8192, 1}
[51]:
# recursive (-r) and detailed (-d) output of h5ls --> also print the content of the data arrays
# !h5ls -rd ../data/test_sampledata_ref.h5
[52]:
# output of h5dump:
# !h5dump ../data/test_sampledata_ref.h5
As you can see if you uncommented and executed this cell, h5dump prints a a fully detailed description of your dataset: organization, data types, item names and path, and item content (value stored in arrays). As it produces a very large output, it may be convenient to write its output in a file:
[53]:
# !h5dump ../data/test_sampledata_ref.h5 > test_dump.txt
[54]:
# !cat test_dump.txt
You can also use the command line tool of the Pytables software ptdump, that also takes as argument the HDF5 file, and has two command options, the verbose mode -v, and the detailed mode -d:
[55]:
# uncomment to test !
# !ptdump ../data/test_sampledata_ref.h5
[56]:
# uncomment to test!
# !ptdump -v ../data/test_sampledata_ref.h5
[57]:
# uncomment to test !
# !ptdump -d ../data/test_sampledata_ref.h5
This second tutorial of the SampleData User Guide is now finished. You should now be able to easily find all the information you are interested in from a SampleData dataset !
3 - Creating and retrieving basic data items¶
This second Notebook will introduce you to:
interactively get data stored in a data item
creating and getting Group data items
creating and getting HDF5 attributes (metadata)
creating and getting data arrays
creating and getting string arrays
creating and getting structured arrays
removing data items from datasets
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the first notebook of this User Guide of this User Guide.
I - Interactively get data from data items¶
This first section will present the generic ways to get the data contained into a data item in a SampleData dataset. Like in the previous tutorial, we will use the reference dataset used for the pymicro.core package unit tests:
[1]:
from pymicro.core.samples import SampleData as SD
[2]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'test_sampledata_ref') # test dataset file path
data = SD(filename=dataset_file)
We will start by printing the content of the dataset and its Index (see previous tutorial, section III), to see which data we could load from the dataset:
[3]:
data.print_dataset_content(short=True)
data.print_index()
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
SampleData datasets can contain many types of data items, with different formats, shapes and contents. For this reason, the class provides specific methods to get each type of data item. They will be presented, for each type of data, in the next sections of this Notebook.
In addition, the SampleData class provides two generic mechanisms to retrieve data, that only require the name of the targeted data item. They automatically try to identify which type of data matches the provided name, and call the adapted specific “get” method. They are usefull to quickly and easily get data, but do not allow to access all options offered by specific methods. We will start by reviewing these generic data access mechanisms.
Dictionary like access to data¶
The first way to get a data item is to use the SampleData class instance as if ot was a dictionary whose keys and values were respectively the data item Names and content. For a given data item, as dictionary key, you can use one of its 4 possible identificators, Name, Path, Indexname or Aliases (see tutorial 1, section II).
Let us see an example, by trying to get the array test_array of our dataset:
[4]:
# get array in a variable, using the data item Name
array = data['test_array']
print(array.shape,'\n', array)
print(type(array))
# directly print array, getting it with its Indexname
print('\n',data['array'])
(51,)
[-1.00000000e+00 -9.39062506e-01 -8.81618592e-01 -8.27271946e-01
-7.75679511e-01 -7.26542528e-01 -6.79599298e-01 -6.34619298e-01
-5.91398351e-01 -5.49754652e-01 -5.09525449e-01 -4.70564281e-01
-4.32738642e-01 -3.95928009e-01 -3.60022153e-01 -3.24919696e-01
-2.90526857e-01 -2.56756360e-01 -2.23526483e-01 -1.90760202e-01
-1.58384440e-01 -1.26329378e-01 -9.45278312e-02 -6.29146673e-02
-3.14262660e-02 1.11022302e-16 3.14262660e-02 6.29146673e-02
9.45278312e-02 1.26329378e-01 1.58384440e-01 1.90760202e-01
2.23526483e-01 2.56756360e-01 2.90526857e-01 3.24919696e-01
3.60022153e-01 3.95928009e-01 4.32738642e-01 4.70564281e-01
5.09525449e-01 5.49754652e-01 5.91398351e-01 6.34619298e-01
6.79599298e-01 7.26542528e-01 7.75679511e-01 8.27271946e-01
8.81618592e-01 9.39062506e-01 1.00000000e+00]
<class 'numpy.ndarray'>
[-1.00000000e+00 -9.39062506e-01 -8.81618592e-01 -8.27271946e-01
-7.75679511e-01 -7.26542528e-01 -6.79599298e-01 -6.34619298e-01
-5.91398351e-01 -5.49754652e-01 -5.09525449e-01 -4.70564281e-01
-4.32738642e-01 -3.95928009e-01 -3.60022153e-01 -3.24919696e-01
-2.90526857e-01 -2.56756360e-01 -2.23526483e-01 -1.90760202e-01
-1.58384440e-01 -1.26329378e-01 -9.45278312e-02 -6.29146673e-02
-3.14262660e-02 1.11022302e-16 3.14262660e-02 6.29146673e-02
9.45278312e-02 1.26329378e-01 1.58384440e-01 1.90760202e-01
2.23526483e-01 2.56756360e-01 2.90526857e-01 3.24919696e-01
3.60022153e-01 3.95928009e-01 4.32738642e-01 4.70564281e-01
5.09525449e-01 5.49754652e-01 5.91398351e-01 6.34619298e-01
6.79599298e-01 7.26542528e-01 7.75679511e-01 8.27271946e-01
8.81618592e-01 9.39062506e-01 1.00000000e+00]
As you can see, when used as a dictionary, the class returned the content of the test_array data item as a numpy array.
Attribute like access to data¶
In addition to the dictionary like access, you can also get data items as if they were attributes of the class, using their Name, Indexname or Alias:
[5]:
print(data.array)
[-1.00000000e+00 -9.39062506e-01 -8.81618592e-01 -8.27271946e-01
-7.75679511e-01 -7.26542528e-01 -6.79599298e-01 -6.34619298e-01
-5.91398351e-01 -5.49754652e-01 -5.09525449e-01 -4.70564281e-01
-4.32738642e-01 -3.95928009e-01 -3.60022153e-01 -3.24919696e-01
-2.90526857e-01 -2.56756360e-01 -2.23526483e-01 -1.90760202e-01
-1.58384440e-01 -1.26329378e-01 -9.45278312e-02 -6.29146673e-02
-3.14262660e-02 1.11022302e-16 3.14262660e-02 6.29146673e-02
9.45278312e-02 1.26329378e-01 1.58384440e-01 1.90760202e-01
2.23526483e-01 2.56756360e-01 2.90526857e-01 3.24919696e-01
3.60022153e-01 3.95928009e-01 4.32738642e-01 4.70564281e-01
5.09525449e-01 5.49754652e-01 5.91398351e-01 6.34619298e-01
6.79599298e-01 7.26542528e-01 7.75679511e-01 8.27271946e-01
8.81618592e-01 9.39062506e-01 1.00000000e+00]
[6]:
# get the test array in a variable with the attribute like access
array2 = data.test_array
# Test if both array are equal
import numpy as np
np.all(array == array2)
[6]:
True
Now that these two generic mechanisms have been presented, we will review the basic data item types that can compose your datasets, and how to create or retrieve them with specific class methods. The more complex data types, representing grids and fields, will not be presented here. Dedicated tutorials follow this one to introduce you to these more advanced features of the class.
To do that, we will create our own dataset. So first, we have to close the test dataset:
[7]:
data.set_verbosity(True)
del data
Deleting DataSample object
.... Storing content index in test_sampledata_ref.h5:/Index attributes
.... writing xdmf file : test_sampledata_ref.xdmf
.... flushing data in file test_sampledata_ref.h5
File test_sampledata_ref.h5 synchronized with in memory data tree
Dataset and Datafiles closed
II - HDF5 Groups¶
We will start by HDF5 Groups, are they are the most simple type of data item in the data model. Groups have 2 functions within SampleData datasets: 1. organize data by containing other data items 2. organizing metadata by containing attributes
First, we will start by creating a dataset, with the verbose and autodelete options, so that we get information on the actions performed by the class, and so that our dataset is removed once we end this tutorial. We also set the overwrite_hdf5 option to True, in case tutorial_dataset.h5/xdmf exist in the current work directory (created and not removed by another tutorial file for instance).
[8]:
data = SD(filename='tutorial_dataset', sample_name='test_sample', verbose = True, autodelete=True, overwrite_hdf5=True)
-- File "tutorial_dataset.h5" not found : file created
-- File "tutorial_dataset.xdmf" not found : file created
.... writing xdmf file : tutorial_dataset.xdmf
Minimal data model initialization....
Minimal data model initialization done
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Note that the verbose mode of the class informed us that dataset files with the required name already existed and where hence deleted.
For now, our dataset is empty and thus contains only the Root group '/'. To create a group, you must use the add_group method. It has 4 arguments: * groupname: the group to create will have this Name * location: indicate the parent group that will contain the created group. By defaults it’s value is '/', the root group * indexname: the group to create will have this Indexname. If none is provided, the indexname will be duplicated from the Name * replace: if a
group with the same Name exists, the SampleData class will remove it to create the new one only if this argument is set to True. If not, the group will not be created. By default, it is set to False
Let us create a test group, from the root group. We will call it test_group, and give it a short indexname, for instance testG:
[9]:
data.add_group(groupname='test_group', location='/', indexname='testG')
Creating Group group `test_group` in file tutorial_dataset.h5 at /
[9]:
/test_group (Group) 'test_group'
children := []
As you can see, the verbose mode prints the confirmation that the group has been created. You can observe also that the method return a Group object, that is an instance of a class from the Pytables package. In practice, you do not need to use Pytables object when working with the SampleData class. However, if you want to use them, you can find the documentation of the group class here.
Let us now look at the content of our dataset:
[10]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : testG H5_Path : /test_group
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
The group has indeed been created, with the right Path, Name and Indexname.
Let us try to create a new group, with the same path and name, but a different indexname:
[11]:
import tables
# We run the command in a try structure as it will raise an exception
try:
data.add_group(groupname='test_group', location='/', indexname='Gtest')
except tables.NodeError as NodeError:
print(NodeError)
Group test_group already exists. Set arg. `replace=True` to replace it by a new Group.
We got an error, more specifically a NodeError linked to the HDF5 dataset structure, as the group already exists. As explained earlier, if the replace argument is set to False (the default value), the class protects the pre-existing data and do not create the new data item. As we are sure that we want to overwrite the Group, we must set the correct argument value:
[12]:
group = data.add_group(groupname='test_group', location='/', indexname='Gtest', replace=True)
Removing group test_group to replace it by new one.
Removing node /test_group in content index....
item testG : /test_group removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node /test_group sucessfully removed
Creating Group group `test_group` in file tutorial_dataset.h5 at /
This time we got no error, the previously created group has been deleted, and the new one created. We also assigned the return Group object to the variable group. Let us verify:
[13]:
print(data)
print(group)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : Gtest H5_Path : /test_group
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
/test_group (Group) 'test_group'
As explained in the first section, to get this group data item from the dataset, you can use the dictionary or attribute like data item access. Both mechanisms will call the get_node SampleData method.
This method is meant to return simple data items under the form of a numpy array, or a Pytables node/group format. It takes one of the 4 possible data item identificators (name, indexname, path or alias) as argument. In this case, it should return a Group object, that should be the same as the one return by the add_group method.
Let us verify it:
[14]:
# get the Group object
group2 = data.get_node('test_group')
print(group2)
# Group objects can be compared:
print(f' Does the two Group instances represent the same group ? {group == group2}')
# get again with dictionary like access
group2 = data['test_group']
print(group2)
# get again with attribute like access
group2 = data.Gtest
print(group2)
/test_group (Group) 'test_group'
Does the two Group instances represent the same group ? True
/test_group (Group) 'test_group'
/test_group (Group) 'test_group'
As you can see, the get_node method and the attribute/dictionary like access return the same Group instance, taht was return by the add_group method.
Now you now how to create and retrieve Groups.
III - HDF5 attributes¶
As explained in the previous section, one of the use of Groups can be to contain Attributes, to organize metadata. In this section, we will see how to create attributes. It is actually very simple to add attribute to a data item with SampleData.
Let us see an example. Suppose that we want to add to our new group test_group the name of the tutorial notebook file that created it, and the tutorial section where it is created. We will start by creating a dictionary gathering this metadata:
[15]:
metadata = {'tutorial_file':'2_SampleData_basic_data_items.ipynb',
'tutorial_section':'Section II'}
Then, we simply add this metadata to test_group with the add_attributes method:
[16]:
data.add_attributes(metadata, nodename='Gtest')
Let us look at the content of your group to verify that the attributes have been added:
[17]:
data.print_node_attributes('Gtest')
-- test_group attributes :
* group_type : Group
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section II
As you can see, the metadata that we just added to the Group is printed. You can also observe that the Group already had metadata, the group_type attribute. Here this attribute is Group, which indicates that it is a standard HDF5 group, and not a Grid group (image or mesh, see data model here).
The methods to get attributes have been presented in the tutorial 1 (sec. II-8). We will reuse them here:
[18]:
tutorial_file = data.get_attribute('tutorial_file','Gtest')
tutorial_sec = data.get_attribute('tutorial_section', 'Gtest')
print(f'The group Gtest has been created with the notebook {tutorial_file}, at the section {tutorial_sec}')
The group Gtest has been created with the notebook 2_SampleData_basic_data_items.ipynb, at the section Section II
[19]:
Gtest_attrs = data.get_dic_from_attributes('Gtest')
print(f'The group Gtest has been created with the notebook {Gtest_attrs["tutorial_file"]},'
f' at the section {Gtest_attrs["tutorial_section"]}')
The group Gtest has been created with the notebook 2_SampleData_basic_data_items.ipynb, at the section Section II
To conclude this section on attribute, we will introduce the set_description and get_description methods. These methods are a shortcut to create or get the content of a specific data item attribute, description. This attribute is intended to be a string of one or a few sentences that explains the content, origin and purpose of the data item:
[20]:
data.set_description(description="Just a short example to see how to use descriptions. This is a description.",
node='Gtest')
data.print_node_attributes('Gtest')
-- test_group attributes :
* description : Just a short example to see how to use descriptions. This is a description.
* group_type : Group
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section II
[21]:
print(data.get_description('Gtest'))
Just a short example to see how to use descriptions. This is a description.
IV - Data arrays¶
Now that we can create Group and organize our datasets, we will want to add actual data in it.
The most common form of scientific data is an array of numbers. The most common and powerfull Python package used to manipulate large numeric arrays is the Numpy package. Through its implementation, and the support of the Pytables package, the SampleData class can directly load and return Numpy arrays, for the storage of numerical arrays in the datasets.
The method that you will need to use to add a numeric array is add_data_array. It accepts the following arguments:
location: the parent group that will contain the created group. Mandatory argumentname: the data array to create will have this Namearray
indexname: the data array to create will have this Indexname. If none is provided, the indexname will be duplicated from the Namereplace: if an array with the same Name exists, the SampleData class will remove it to create the new one only if this argument is set toTrue. By default, it is set toFalsearray: anumpy.ndarray, the numeric array to be stored into the dataset
In addition, it accepts two arguments linked to data compression, the chunkshape and compression_options arguments, that will not be discussed here, but rather in the tutorial dedicated to data compression. The name, indexname, location and replace work exactly as for the add_group method presented in the previous section, and the array argument is pretty explicit.
It is allowed to create an empty data array item in the dataset. The main purpose of this option will be highlighted in the tutorial on SampleData derived classes. For now, note that it allows to create the internal organization of your dataset without having to add any data to it. It allows for instance to preempt some data item names, indexnames, and to already add metadata. We will see below how to create an empty data array item, and later, add actual data to it.
Let us start by creating a random array of data with the numpy.random package, and store it into a data array in the group test_group.
[22]:
# we start by importing the numpy package
import numpy as np
# we create a random array of 20 elements
A = np.random.rand(20)
print(A)
[0.51535012 0.56508785 0.53913262 0.64040211 0.55415212 0.23042938
0.67191761 0.12123758 0.88332728 0.32097958 0.36914473 0.244528
0.74187258 0.280634 0.68578653 0.6472718 0.16596613 0.39701755
0.24730563 0.61975144]
Now we add the array A to our dataset with name test_array, indexname Tarray, to the group test_group:
[23]:
data.add_data_array(location='Gtest', name='test_array', indexname='Tarray', array=A)
Adding array `test_array` into Group `Gtest`
[23]:
/test_group/test_array (CArray(20,)) 'Tarray'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (8192,)
As for the group creation in the previous section, the verbose mode of the class informed us that the array has been added to the dataset, in the desired group. Once again, the method has return a Pytables object. Here it is a tables.Node object and not a tables.Group object.
You can see that this object contains a array, that is a Carray. A Carray is a chunkable array. It is a specific type of HDF5 array, whose data are split into multiple chunks which are all stored separately in the file. This enables a strong optimization of the reading speed when dealing with multidimensional arrays. We will not detail this feature of the HDF5 library in this tutorial, but rather in the tutorial dedicated to data compression.
Though, it is strongly advised to study the concept ouf HDF5 chuncked layout to optimally use SampleData datasets (see Pytables optimization tips or HDF5 group dedicated page).
Let us look at the content of our dataset:
[24]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : Gtest H5_Path : /test_group
Name : Tarray H5_Path : /test_group/test_array
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
We now see that our array has been added as a children of the test_group Group, as requested. Like for the Group created in the section II, we can add metadata to this dat item:
[25]:
metadata = {'tutorial_file':'2_SampleData_basic_data_items.ipynb',
'tutorial_section':'Section IV'}
data.add_attributes(metadata, 'Tarray')
data.print_node_attributes('Tarray')
-- test_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
As you can observe, array data items have an empty attribute, that indicates if the data item is associated or not to an empty data array (see a few cell above). They also have a node_type attribute, indicating their data item nature, here a Data Array.
Let us try to create a empty array now. In this case, you just have to remove the array argument from the method call:
[26]:
data.add_data_array(location='Gtest', name='empty_array', indexname='emptyA')
Adding array `empty_array` into Group `Gtest`
[26]:
/test_group/empty_array (CArray(1,)) 'emptyA'
atom := Int64Atom(shape=(), dflt=0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (8192,)
The verbose mode of the class indeed informs us that we created an empty data array item. Let us print again the content of our dataset, this time with the detailed format to also see the attributes of our data items:
[27]:
data.print_dataset_content(short=False)
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: tutorial_dataset.h5
-- Size: 71.492 Kb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
* sample_name : test_sample
-- Childrens : Index, test_group,
----------------
************************************************
GROUP test_group
=====================
-- Parent Group : /
-- Group attributes :
* description : Just a short example to see how to use descriptions. This is a description.
* group_type : Group
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section II
-- Childrens : test_array, empty_array,
----------------
****** Group /test_group CONTENT ******
NODE: /test_group/empty_array
====================
-- Parent Group : test_group
-- Node name : empty_array
-- empty_array attributes :
* empty : True
* node_type : data_array
-- content : /test_group/empty_array (CArray(1,)) 'emptyA'
-- Compression options for node `/test_group/empty_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_group/test_array
====================
-- Parent Group : test_group
-- Node name : test_array
-- test_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
-- content : /test_group/test_array (CArray(20,)) 'Tarray'
-- Compression options for node `/test_group/test_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
************************************************
We have as requested our array with the 20 element array, and the empty array. You can see that SampleData stores a 1 element array in empty arrays, as it is not possible to create a node with a 0d array. That is why the empty attribute is attached to array data items
We will try to get our newly added data items as numpy arrays now. As for the test_array Group, we can use the get_node method for this. We can now introduce its second argument, the as_numpy option. If this argument is set to True, get_node return the data item as a numpy.ndarray. If it is set to False (its default value), the method returns a tables.Node object from the Pytables class, identical to the one returned by the add_data_array method when the
data item was created.
Let us see some example:
[28]:
array_node = data.get_node('test_array')
array = data.get_node('test_array', as_numpy=True)
print('array_node returned with "as_nump=False":\n',array_node,'\n')
print('array returned with "as_nump=True":\n', array, type(array))
array_node returned with "as_nump=False":
/test_group/test_array (CArray(20,)) 'Tarray'
array returned with "as_nump=True":
[0.51535012 0.56508785 0.53913262 0.64040211 0.55415212 0.23042938
0.67191761 0.12123758 0.88332728 0.32097958 0.36914473 0.244528
0.74187258 0.280634 0.68578653 0.6472718 0.16596613 0.39701755
0.24730563 0.61975144] <class 'numpy.ndarray'>
What happens if we try to get our empty array ?
[29]:
empty_array_node = data.get_node('empty_array')
empty_array = data.get_node('empty_array', as_numpy=True)
print(f'The empty array node {empty_array_node} is not truly empty in the dataset','\n')
print(f'It actually contains ... {empty_array} ... a one element array with value 0')
The empty array node /test_group/empty_array (CArray(1,)) 'emptyA' is not truly empty in the dataset
It actually contains ... [0] ... a one element array with value 0
As mentioned earlier, the empty array is not truly empty.
We have seen that the get_node method can have two different behaviors with data arrays. So what happens if we try to get a data array from our dataset using one of the two generic getter mechanisms explained in section I ?
[30]:
array = data['test_array']
print(f'What we got with the dictionary like access is a {type(array)}')
array = data.test_array
print(f'What we got with the attribute like access is a {type(array)}')
What we got with the dictionary like access is a <class 'numpy.ndarray'>
What we got with the attribute like access is a <class 'numpy.ndarray'>
As you can see, the generic mechanisms call the get_node method with the as_numpy=True option.
The last thing we have to discuss about data arrays, is how to add actual data to an empty data array item. Actually, when calling the add_data_array method with the name/location of an empty array, the method behaves as if it had been called with replace=True. However, in this case, all metadata that was attached to the empty node is preserved and reattached to the data item created with the inputed array.
Let us add some metadata to the empty array, to test this feature:
[31]:
metadata = {'tutorial_file':'2_SampleData_basic_data_items.ipynb',
'tutorial_section':'Section IV'}
data.add_attributes(metadata, 'empty_array')
data.print_node_attributes('empty_array')
-- empty_array attributes :
* empty : True
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
Let us create a data array and try to add it to the empty array.
[32]:
A = np.arange(20)
data.add_data_array(location='Gtest', name='empty_array', indexname='emptyA', array=A)
Adding array `empty_array` into Group `Gtest`
Removing node /test_group/empty_array in content index....
item emptyA : /test_group/empty_array removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node /test_group/empty_array sucessfully removed
[32]:
/test_group/empty_array (CArray(20,)) 'emptyA'
atom := Int64Atom(shape=(), dflt=0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (8192,)
[33]:
print(data['emptyA'])
data.print_node_attributes('empty_array')
[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19]
-- empty_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
As you can see thanks to the verbose mode, the old empty data array node is removed, and replaced with a new one containing our data, but also the metadata previously attached to the empty array. A node_type as additionally been attached to it to account for the nature of the data newly stored in the data item.
Except for the control of data compression parameters, you now know all that is to know to create and retrieve data arrays with SampleData.
V - String arrays¶
We now move to another usefull type of data item. In many cases, it may be usefull to store long lists of strings. Data arrays are restricted to numerical arrays. Attributes are meant to store data of small size and are thus not suited fot it either.
To realize this task, you will need to rely on String arrays, that can be added thanks to the add_string_array method. It is basically a mapping of a Python string list to a HDF5 Pytables data item. Hence, this method has arguments that you now know well: name, location, indexname, replace. In addition, it has a data argument that must be the Python list of strings that you want to store in the dataset.
Let us see an example:
[34]:
List = ['this','is','a','not so long','list','of strings','for','the tutorial !!']
data.add_string_array(name='string_array', location='test_group', indexname='Sarray', data=List)
Adding String Array `string_array` into Group `test_group`
[34]:
/test_group/string_array (EArray(8,)) ''
atom := StringAtom(itemsize=255, shape=(), dflt=b'')
maindim := 0
flavor := 'numpy'
byteorder := 'irrelevant'
chunkshape := (257,)
Like the previous ones, this method verbose mode informs you that the string array has been created, and returns the Pytables node object associated to the created data item.
Let us look at the dataset content:
[35]:
data.print_dataset_content(short=False)
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: tutorial_dataset.h5
-- Size: 138.141 Kb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
* sample_name : test_sample
-- Childrens : Index, test_group,
----------------
************************************************
GROUP test_group
=====================
-- Parent Group : /
-- Group attributes :
* description : Just a short example to see how to use descriptions. This is a description.
* group_type : Group
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section II
-- Childrens : test_array, empty_array, string_array,
----------------
****** Group /test_group CONTENT ******
NODE: /test_group/empty_array
====================
-- Parent Group : test_group
-- Node name : empty_array
-- empty_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
-- content : /test_group/empty_array (CArray(20,)) 'emptyA'
-- Compression options for node `/test_group/empty_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_group/string_array
====================
-- Parent Group : test_group
-- Node name : string_array
-- string_array attributes :
* empty : False
* node_type : string_array
-- content : /test_group/string_array (EArray(8,)) ''
-- Compression options for node `/test_group/string_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_group/test_array
====================
-- Parent Group : test_group
-- Node name : test_array
-- test_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
-- content : /test_group/test_array (CArray(20,)) 'Tarray'
-- Compression options for node `/test_group/test_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
************************************************
You can see in the information printed about the string array that we just created, that it has a node_type attribute, indicating that it is a String Array.
To manipulate a string array use the ‘get_node’ method to get the array, and then manipulate it as a list of binary strings. Indeed, strings are automatically converted to bytes when creating this type of data item. You will hence need to use the str.decode() method to get the elements of the string_array as UTF-8 or ASCII formatted strings:
[36]:
sarray = data['Sarray']
S1 = sarray[0] # here we get a Python bytes string
S2 = sarray[0].decode('utf-8') # here we get a utf-8 string
print(S1)
print(S2,'\n')
# Let us print all strings contained in the string array:
for string in sarray:
print(string.decode('utf-8'), end=' ')
b'this'
this
this is a not so long list of strings for the tutorial !!
The particularity of String arrays, is that they are enlargeable. To add additionnal elements to them, you may use the append_string_array method, that takes as arguments the name of the string array, and a list of strings:
[37]:
# we add 3 new elements to the array
data.append_string_array(name='Sarray', data=['We can make','it','bigger !'])
# now Let us print the enlarged list of strings:
for string in data['Sarray']:
print(string.decode('utf-8'), end=' ')
this is a not so long list of strings for the tutorial !! We can make it bigger !
And that is all that it is to know about String arrays !
VI - Structured arrays¶
The last data item type that we will review in this tutorial is analogous to the data array item type (section IV), but is meant to store Numpy structured arrays (you are strongly encouraged ). Those are ndarrays whose datatype is a composition of simpler datatypes organized as a sequence of named fields, in other words, heterogeneous arrays.
The Pytables package, which handles the HDF5 dataset within the SampleData class, use a class called table to store structured arrays. This termonology is reused within the SampleData class. Hence, to add a structured array, you may use the add_table method. This method accepts the same arguments as add_data_array, plus a description argument, that may be an instance of the tables.IsDescription class (see
here), or a numpy.dtype object. It is an object whose role is to describe the structure of the array (name and type of array Fields). The data argument value must be a numpy.ndarray whose dtype is consistent with the description.
Let us see an example. Imagine that we want to create a structured array to store data describing material particles, containing for each particle, its nature, an identity number, its dimensions, and a boolean value indicating the presence of damage at the particle.
To do this, we have to create a suitable numpy.dtype and numpy structured array:
[38]:
# creation of a numpy dtype --> takes as input a list of tuples ('field_name', 'field_type')
# Numpy dtype reminder: S25: binary strings with 25 characters, ?: boolean values
#
sample_type = np.dtype([('Nature','S25'), ('Id_number',np.int16), ('Dimensions',np.double,(3,)), ('Damaged','?')])
# now we create an empty array of 2 elements of this type
sample_array = np.empty(shape=(2,), dtype=sample_type)
# now we create data to represent 2 samples
sample_array['Nature'] = ['Intermetallic', 'Carbide']
sample_array['Id_number'] = [1,2]
sample_array['Dimensions'] = [[20,20,50],[2,2,3]]
sample_array['Damaged'] = [True,False]
print(sample_array)
[(b'Intermetallic', 1, [20., 20., 50.], True)
(b'Carbide', 2, [ 2., 2., 3.], False)]
Now that we have our numpy.dtype, and our structured array, we can create the table:
[39]:
# create the structured array data item
tab = data.add_table(name='test_table', location='test_group', indexname='tableT', description=sample_type,
data=sample_array)
# adding one attribute to the table
data.add_attributes({'tutorial_section':'VI'},'tableT')
# printing information on the table
data.print_node_info('tableT')
Adding table `test_table` into Group `test_group`
-- Compression Options for dataset test_table
NODE: /test_group/test_table
====================
-- Parent Group : test_group
-- Node name : test_table
-- test_table attributes :
* node_type : structured_array
* tutorial_section : VI
-- content : /test_group/test_table (Table(2,)) ''
-- table description :
{
"Nature": StringCol(itemsize=25, shape=(), dflt=b'', pos=0),
"Id_number": Int16Col(shape=(), dflt=0, pos=1),
"Dimensions": Float64Col(shape=(3,), dflt=0.0, pos=2),
"Damaged": BoolCol(shape=(), dflt=False, pos=3)}
-- Compression options for node `tableT`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1260,)
-- Node memory size : 63.984 Kb
----------------
You see above that the method print_node_info prints the description of the structured array stored in the dataset, which allows you to know what are their fields and associated data types. You can observe however, that these fields and their types are specific object from the Pytable package: column objects (StringCol, Int16Col…). You can also see here a node_type attribute indicating that the node is a Structured Array data item.
The method returned like in for the previous data item studied, the equivalent Pytables Node object. This table object interestingly has a description attribute, and a numpy.dtype attribute:
[40]:
print(tab.description)
print(tab.dtype)
Description([('Nature', '()S25'), ('Id_number', '()i2'), ('Dimensions', '(3,)f8'), ('Damaged', '()b1')])
[('Nature', 'S25'), ('Id_number', '<i2'), ('Dimensions', '<f8', (3,)), ('Damaged', '?')]
Once the table is created, it is stil possible to expand it in two ways: 1. adding new rows 2. adding new columns
To add new rows to the table, you will need to create a numpy.ndarray that is compatible with the table, i.e. meeting the 2 criteria: 1. having a dtype compatible with the table description (same fields associated to same types) 2. having a compatible shape (all dimensions except last must have identical shapes)
Let us try to add two new sample rows to our table:
[41]:
sample_array['Nature'] = ['Intermetallic', 'Carbide']
sample_array['Id_number'] = [3,4]
sample_array['Dimensions'] = [[50,20,30],[3,2,3]]
sample_array['Damaged'] = [True,True]
[42]:
data.append_table(name='tableT', data=sample_array)
We should now look at our dataset to see if the data has been appended to our structured array. Let us see what happens if we use a generic getter mechanism on our structured array:
[43]:
print(type(data['tableT']),'\n')
print(data['tableT'].dtype, '\n')
print(data['tableT'],'\n')
# you can also directly get table columns:
print('Damaged particle ?',data['tableT']['Damaged'],'\n')
print('Particles Nature:',data['tableT']['Nature'])
<class 'numpy.ndarray'>
[('Nature', 'S25'), ('Id_number', '<i2'), ('Dimensions', '<f8', (3,)), ('Damaged', '?')]
[(b'Intermetallic', 1, [20., 20., 50.], True)
(b'Carbide', 2, [ 2., 2., 3.], False)
(b'Intermetallic', 3, [50., 20., 30.], True)
(b'Carbide', 4, [ 3., 2., 3.], True)]
Damaged particle ? [ True False True True]
Particles Nature: [b'Intermetallic' b'Carbide' b'Intermetallic' b'Carbide']
As for the data array item type, the generic mechanisms here return the data item as a numpy.ndarray, with the dtype of the structured table. We can indeed read the right number of lines, and the right values. Note that strings are necessarily stored as bytes in the dataset, so you must decode them to print or use them in standard string format:
[44]:
print(data['tableT']['Nature'][0].decode('ascii'))
Intermetallic
We will now see how to add new columns to the table, using the add_tablecols method. It allows to add a structured numpy.ndarray as additional columns to an already existing table. It takes 3 arguments: tablename (Name, Path, Indexname or Alias of the table to which you want to add columns), description and data. As for the add_table method, the data argument dtype must be consistent with the description argument.
Let us add two columns to our structured array, to store for instance the particle position and chemical composition :
[45]:
# we create a new dtype with the new fields
cols_dtype = np.dtype([('Position',np.double,(3,)), ('Composition','S25')])
# now we create an empty array of 2 elements of this type
new_cols = np.empty(shape=(4,), dtype=cols_dtype)
# now we create data to fill the new columns
new_cols['Position'] = [[100.,150.,300],[10,25,10],[520,300,450],[56,12,45]]
new_cols['Composition'] = ['Cr3Si','Fe3C','MgZn2','SiC']
[46]:
data.add_tablecols(tablename='tableT', description=cols_dtype, data=new_cols)
Updating `tableT` with fields {'Nature': (dtype('S25'), 0), 'Id_number': (dtype('int16'), 25), 'Dimensions': (dtype(('<f8', (3,))), 27), 'Damaged': (dtype('bool'), 51), 'Position': (dtype(('<f8', (3,))), 52), 'Composition': (dtype('S25'), 76)}
New table description is `{'Nature': (dtype('S25'), 0), 'Id_number': (dtype('int16'), 25), 'Dimensions': (dtype(('<f8', (3,))), 27), 'Damaged': (dtype('bool'), 51), 'Position': (dtype(('<f8', (3,))), 52), 'Composition': (dtype('S25'), 76)}`
data is: [(b'0.0', 0, [0., 0., 0.], False, [0., 0., 0.], b'0.0')
(b'0.0', 0, [0., 0., 0.], False, [0., 0., 0.], b'0.0')
(b'0.0', 0, [0., 0., 0.], False, [0., 0., 0.], b'0.0')
(b'0.0', 0, [0., 0., 0.], False, [0., 0., 0.], b'0.0')]
(get_tablecol) Getting column Nature from : tutorial_dataset.h5:/test_group/test_table
(get_tablecol) Getting column Id_number from : tutorial_dataset.h5:/test_group/test_table
(get_tablecol) Getting column Dimensions from : tutorial_dataset.h5:/test_group/test_table
(get_tablecol) Getting column Damaged from : tutorial_dataset.h5:/test_group/test_table
Removing node /test_group/test_table in content index....
item tableT : /test_group/test_table removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node test_table sucessfully removed
Adding table `test_table` into Group `/test_group`
-- Compression Options for dataset test_table
(get_tablecol) Getting column Position from : tutorial_dataset.h5:/test_group/test_table
(get_tablecol) Getting column Composition from : tutorial_dataset.h5:/test_group/test_table
[47]:
data.print_node_info('tableT')
print(data['tableT'],'\n')
NODE: /test_group/test_table
====================
-- Parent Group : test_group
-- Node name : test_table
-- test_table attributes :
* node_type : structured_array
* tutorial_section : VI
-- content : /test_group/test_table (Table(4,)) ''
-- table description :
{
"Nature": StringCol(itemsize=25, shape=(), dflt=b'', pos=0),
"Id_number": Int16Col(shape=(), dflt=0, pos=1),
"Dimensions": Float64Col(shape=(3,), dflt=0.0, pos=2),
"Damaged": BoolCol(shape=(), dflt=False, pos=3),
"Position": Float64Col(shape=(3,), dflt=0.0, pos=4),
"Composition": StringCol(itemsize=25, shape=(), dflt=b'', pos=5)}
-- Compression options for node `tableT`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1260,)
-- Node memory size : 124.277 Kb
----------------
[(b'Intermetallic', 1, [20., 20., 50.], True, [100., 150., 300.], b'Cr3Si')
(b'Carbide', 2, [ 2., 2., 3.], False, [ 10., 25., 10.], b'Fe3C')
(b'Intermetallic', 3, [50., 20., 30.], True, [520., 300., 450.], b'MgZn2')
(b'Carbide', 4, [ 3., 2., 3.], True, [ 56., 12., 45.], b'SiC')]
As you can see from the verbose mode prints, when adding new columns to a table, the SampleData class get the data from the original table, and creates a new tables including the additional columns. From the print_node_info output, you can verify that in the process, the metadata attached to the original table has been preserved. You can also observe that the table description has been correctly enriched with the Position and Composition fields.
Note that if you provide an additional columns array that do not match the shape of the stored table, you will get a mismatch error.
To conclude this section on structured arrays, we will see a method allowing to set the values for a full column of a stored structured array, the set_tablecol method. You have to pass as arguments, the name of the table and the name of the column field you want to modify, and a numpy array to set the new values of the column. Of course, the array type must be consistent with the type of the modified column.
To see an example, Let us set all the values of the 'Damaged' column of our table to True:
[48]:
data.set_tablecol(tablename='tableT', colname='Damaged', column=np.array([True,True,True,True]))
print('\n',data['tableT'],'\n')
print(data['tableT']['Damaged'],'\n')
(get_tablecol) Getting column Damaged from : tutorial_dataset.h5:/test_group/test_table
[(b'Intermetallic', 1, [20., 20., 50.], True, [100., 150., 300.], b'Cr3Si')
(b'Carbide', 2, [ 2., 2., 3.], True, [ 10., 25., 10.], b'Fe3C')
(b'Intermetallic', 3, [50., 20., 30.], True, [520., 300., 450.], b'MgZn2')
(b'Carbide', 4, [ 3., 2., 3.], True, [ 56., 12., 45.], b'SiC')]
[ True True True True]
You have now learn how to create and get values from all basic data item types that can be stored into SampleData datasets.
Before closing this tutorial, we will see how to remove data items from datasets.
VII - Removing data items from datasets¶
Removing data items is very easy, you juste have to call the remove_node method, and provide the name of the data item you want to remove. When removing non empty Groups from the dataset, the optional recursive argument should be set to True, to allow the method to remove the group and all of its childrens. If this is not the case, the method will not remove Groups that have childrens.
Let us try to remove our test_array from the dataset:
[49]:
data.remove_node('test_array')
data.print_dataset_content()
Removing node /test_group/test_array in content index....
item Tarray : /test_group/test_array removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node test_array sucessfully removed
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: tutorial_dataset.h5
-- Size: 398.229 Kb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
* sample_name : test_sample
-- Childrens : Index, test_group,
----------------
************************************************
GROUP test_group
=====================
-- Parent Group : /
-- Group attributes :
* description : Just a short example to see how to use descriptions. This is a description.
* group_type : Group
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section II
-- Childrens : empty_array, string_array, test_table,
----------------
****** Group /test_group CONTENT ******
NODE: /test_group/empty_array
====================
-- Parent Group : test_group
-- Node name : empty_array
-- empty_array attributes :
* empty : False
* node_type : data_array
* tutorial_file : 2_SampleData_basic_data_items.ipynb
* tutorial_section : Section IV
-- content : /test_group/empty_array (CArray(20,)) 'emptyA'
-- Compression options for node `/test_group/empty_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_group/string_array
====================
-- Parent Group : test_group
-- Node name : string_array
-- string_array attributes :
* empty : False
* node_type : string_array
-- content : /test_group/string_array (EArray(11,)) ''
-- Compression options for node `/test_group/string_array`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /test_group/test_table
====================
-- Parent Group : test_group
-- Node name : test_table
-- test_table attributes :
* node_type : structured_array
* tutorial_section : VI
-- content : /test_group/test_table (Table(4,)) ''
-- table description :
{
"Nature": StringCol(itemsize=25, shape=(), dflt=b'', pos=0),
"Id_number": Int16Col(shape=(), dflt=0, pos=1),
"Dimensions": Float64Col(shape=(3,), dflt=0.0, pos=2),
"Damaged": BoolCol(shape=(), dflt=False, pos=3),
"Position": Float64Col(shape=(3,), dflt=0.0, pos=4),
"Composition": StringCol(itemsize=25, shape=(), dflt=b'', pos=5)}
-- Compression options for node `/test_group/test_table`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1260,)
-- Node memory size : 124.277 Kb
----------------
************************************************
The array data item has indeed been removed from the dataset. Let us now try to remove our test group, and all of its childrens. We should end up with an empty dataset at the end:
[50]:
data.remove_node('test_group', recursive=True)
data.print_dataset_content()
Removing node /test_group in content index....
Removing node /test_group/empty_array in content index....
item emptyA : /test_group/empty_array removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node <closed tables.carray.CArray at 0x7f60c5d95d70> sucessfully removed
Removing node /test_group/string_array in content index....
item Sarray : /test_group/string_array removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node <closed tables.earray.EArray at 0x7f60c5d95cd0> sucessfully removed
Removing node /test_group/test_table in content index....
item tableT : /test_group/test_table removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node <closed tables.table.Table at 0x7f60c5d33440> sucessfully removed
item Gtest : /test_group removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node test_group sucessfully removed
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: tutorial_dataset.h5
-- Size: 271.905 Kb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
* sample_name : test_sample
-- Childrens : Index,
----------------
************************************************
You can see from the verbose mode output that the method has indeed removed the group, and all of its childrens. You may also remove node attributes using the remove_attribute and remove_attributes methods.
That’s it ! You know now how to remove data items from your datasets. This tutorial is finished, we can now close our test dataset.
[51]:
del data
Deleting DataSample object
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
SampleData Autodelete:
Removing hdf5 file tutorial_dataset.h5 and xdmf file tutorial_dataset.xdmf
4 - Creating, getting and visualizing Image Groups and Image Fields¶
This Notebook will introduce you to:
what is an Image Group
the Image Group data model
how to create an image Group
the Image Field data model
how to add a field to an Image Group
how to get image groups and image fields
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the first notebook of this User Guide of this User Guide.
I - SampleData Image Groups¶
SampleData Grids¶
In the last tutorial, the basic data item types that can be loaded into a SampleData dataset have been presented. They consist in various types of data arrays (numeric arrays, structured arrays, string arrays) and attributes, that can be stored in Groups to organize them.
However, the main focus of the class has been put on spatially organized data. This type of data, which includes field measurements, imaging techniques and numerical simulation outputs, is central to modern materials and microstructure science. Each of those outputs can be seen as a set of fields supported by a grid.
A grid is a set of nodes and elements that define a discretization of a geometrical domain. A field is a function defined on this geometrical domain. On the discrete grid, field are defined by an array containing the values that it takes at on grid nodes or grid elements. Hence, geometrical data and field data are intrinsically linked. Therefore, in addition to storing the data arrays containing the node/elements/fields values data, some geometrical organization connecting these data items must be stored as well in the datasets.
To connect data together within SampleData datasets, we can use Groups and Attributes (metadata). So, to handle spatially organized data, SampleData defines Grid groups, that have a specific data model (sub-groups, arrays and metadata). In addition to this, these group are syncrhronized with a Grid node in the XDMF dataset file, to ensure that the data stored in the Grid group can be visualized with its correct spatial organization in the Paraview software.
There are two types of Grid groups: Image groups, and Mesh groups. In this tutorial, we will learn everything that is to know on Image groups.
SampleData Images¶
As indicated by their name, Image Groups are a specific type of Grid group whose data model is adapted to store images. Images are 2D or 3D regular grids of pixels(2D)/voxels(3D), all having the same dimensions. To this pixel/voxel grid is also associated a node grid, composed by the vertexes of the pixels/voxels. Three types of Image Groups can be handled by SampleData:
emptyImage: an empty group that can be used to set the organization and metadata of the dataset before adding actual data to it (similar to empty data arrays, see tutorial 2, section IV)2DImage: a regular grid of pixels3DImage: a regular grid of voxels
An Image field is a data array that has one value associated to each pixel/voxel or each node of the image.
Hence, Image Groups contain metadata indicating the image topology, data arrays containing the image fields values, and metadata to synchronize with the XDMF Grid node associated to the Image. To explore in details this data model, we will once again open the reference test dataset of the SampleData unit tests, in the next section.
II - Image Groups Data Model¶
[1]:
from pymicro.core.samples import SampleData as SD
[2]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'test_sampledata_ref') # test dataset file path
data = SD(filename=dataset_file)
Let us print the content of the dataset to remember its composition:
[3]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
The test dataset contains one 3DImage Group, the test_image group, with indexname image. Let us print more information about this Group:
[4]:
data.print_node_info('image')
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field,
----------------
As you can observe, this 3DImage group already contains a lot of metadata. All of them are automatically created when adding a new Image to a SampleData dataset, and are part of the SampleData Image data model. We will review them in detail, except for the generic description attribute, already discussed in the last tutorial.
First, the group_type attribute informs us the the Group is a 3DImage group. In the case of a 2D image, it would have the value 2DImage.
Then, you can see several attributes linked to the topology of the image:
dimension: the number of pixels/voxels in each direction of the gridnodes_dimension: the number of nodes in each direction of the grid (alwaysdimension+ 1)origin: the coordinate of the first node of the gridspacing: the size along each direction of each pixel/voxel of the grid
For all those attributes, the order of the values in the arrays correspond in the same order to the directions (X,Y) or (X,Y,Z).
You may have noticed the nodes_dimension_xdmf attribute, that is in this case equal to the nodes_dimension attribute. This is due to the fact that, the Paraview software interprets the dimensions of the arrays loaded from a SampleData dataset with an inverse coordinate order (Z,Y,X). For that reason, some data need a transposition to yield a correct visualization through Paraview. The nodes_dimension_xdmf is one of those, and is in practice the transposition of
nodes_dimension where the X and Z dimensions have been swapped.
You can also note that the no length unit attribute is stored in the data model: origin and spacing attributes are thus dimensionless values. In fact, no *unit length* is accounted for by *SampleData*, the user as to ensure that the unit of geometrical data provided for various grid groups are consistent with each other. However, user are free to create attribute to specify the unit length corresponding to the numbers in the dataset.
The empty attribute is False, indicating that the image contains non empty fields.
Finally, note that there is a xdmf_gridname attribute. This one indicates the value of the Name tag of the Grid node in the XDMF dataset file, that is synchronized with this Image Group. Let us print the content of the XDMF tree to see if we can find it:
[5]:
data.print_xdmf()
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd">
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_mesh" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_image_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10" NumberType="Int" Precision="16">test_sampledata_ref.h5:/test_image/test_image_field</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
You can see that the XDMF tree contains 2 grids: one named test_mesh, the other one named test_image, which is the value of the xdmf_gridname attribute of our Image Group. Hence, our Group is synchronized with the second node of the XDMF tree of our dataset. You can see that this node has two sub-nodes that are Topology and Geometry. In the first one, you see the indication that the grid is a regular 3D grid (a 3DCoRectMesh), with its dimensions detailed. In the second
one, you recognize the Image origin and spacing values. Finally, you can see that it has another sub-node, named Attribute. These nodes allow to describe grid fields in the XDMF data model. Here, our image only has one field, named test_image_field.
As already explained in the tutorial 1 (sec. V), the XDMF file allows to directly visualize the data described in it with the Paraview software. Before closing our reference dataset, Let us try to visualize its image Group, with the pause_for_visualization method and the Paraview software.
You have to choose the XdmfReader to open the file, so that Paraview may properly read the data. You will see in the Property panel, the presence of two Blocks of data, one for each grid of the dataset:
To easily visualize the Image group, untick the mesh group box in the appropriate panel, like in the image below:
Now you can click on the Apply button: you should now see the 3D image. You can choose to plot test_image_field, which should render like this:
[6]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='/home/amarano/Sources/ParaView-5.9.0-RC1-MPI-Linux-Python3.8-64bit/bin/paraview')
We will print again the information on the Image Group, to focus on another aspect of the data model:
[7]:
data.print_node_info('image')
GROUP test_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [9 9 9]
* empty : False
* group_type : 3DImage
* nodes_dimension : [10 10 10]
* nodes_dimension_xdmf : [10 10 10]
* origin : [-1. -1. -1.]
* spacing : [0.2 0.2 0.2]
* xdmf_gridname : test_image
-- Childrens : Field_index, test_image_field,
----------------
You can see that the Image group has 2 childrens, test_image_field and Field_index. Let us look at their content:
[8]:
data.print_node_info('test_image_field')
NODE: /test_image/test_image_field
====================
-- Parent Group : test_image
-- Node name : test_image_field
-- test_image_field attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_image
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : test_image_field
* xdmf_gridname : test_image
-- content : /test_image/test_image_field (CArray(10, 10, 10)) 'image_test_image_field'
-- Compression options for node `test_image_field`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (327, 10, 10)
-- Node memory size : 63.867 Kb
----------------
This node is a field data item, as indicated by the node_type attribute. Those are data item designed to store the spatially organized data that are defined on grids. They will be discussed in details in section IV below.
[9]:
data.print_node_info('image_Field_index')
NODE: /test_image/Field_index
====================
-- Parent Group : test_image
-- Node name : Field_index
-- Field_index attributes :
* node_type : string array
-- content : /test_image/Field_index (EArray(1,)) ''
-- Compression options for node `image_Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
The Field Index is a String Array (see previous tutorial (section V)) that stores a list of strings that are the names of the fields stored in this Image group. You can get the list of field stored on the group by looking at this string array content:
[10]:
data['image_Field_index'][0].decode('utf-8')
[10]:
'image_test_image_field'
It can also be accessed easily using the get_grid_field_list method:
[11]:
data.get_grid_field_list('image')
[11]:
['image_test_image_field']
Now that we know what composes an Image Group, and how to visualize it, we will close the reference dataset and create one to train ourselves to create images.
[12]:
del data
III - Creating Image Groups¶
There are three ways to create an Image group in a SampleData dataset: 1. creating an image from a data array 2. creating an image from an image object indicating its topology 3. creating an empty image
We will try all three. But first, we will once again create a temporary dataset, to try the examples of this tutorial, with verbose mode on:
[13]:
data = SD(filename='tutorial_dataset', sample_name='test_sample', verbose = True, autodelete=True, overwrite_hdf5=True)
-- File "tutorial_dataset.h5" not found : file created
-- File "tutorial_dataset.xdmf" not found : file created
.... writing xdmf file : tutorial_dataset.xdmf
Minimal data model initialization....
Minimal data model initialization done
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Creation an image from a data array¶
We will start by creating two numpy arrays that will be image fields. Their dimension will define the topology of the image. To have simple and visual examples, we will create: 1. a 2D field of dimension 51x51 representing a matrix with a circular fiber at its center, with a diameter of half the image 2. a random 3D field of dimension 50x50x50x3
We will start with the bidimensional field.
[14]:
# first we need to import the Numpy package
import numpy as np
[15]:
dX = 1/101
# we create a grid of coordinates to compute the distance to the center of the image
X = np.linspace(dX, 1-dX,101)
Y = np.linspace(dX, 1-dX,101)
XX, YY = np.meshgrid(X,Y)
# compute the distance function
Dist = np.sqrt(np.square(XX - 0.5) + np.square(YY - 0.5))
# creation of the 2D array: 0 indicate the matrix, 1 the fiber
field_2D = np.int16(Dist <= 0.25)
The add_image_from_field method¶
Now that our field is created, we can introduce the method add_image_from_field. It will create an appropriate Image group and store the data array inputed as an image field.
[16]:
data.add_image_from_field(field_array=field_2D, fieldname='test_2D_field', imagename='first_2D_image',
indexname='image2D', location='/',
description="Test image group created from a bidimensional Numpy array.")
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `first_2D_image` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `test_2D_field` into Grid `/first_2D_image`
Adding array `test_2D_field` into Group `/first_2D_image`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/first_2D_image`
The verbose mode of the class instructed us that the image group has been created, along with a field/array object. You see that the method accepts similar arguments that the ones studied in the previous tutorial. Note that here the indexname argument is for the image group indexname, the field indexname being constructed from the field_name argument (see below). To add another name to the field, you may use the add_alias method. Note also
that the description argument here allows to set the description attribute of the created image group.
We will now print the content of our dataset to see the results of the image group creation:
[17]:
data.print_dataset_content()
Printing dataset content with max depth 3
****** DATA SET CONTENT ******
-- File: tutorial_dataset.h5
-- Size: 3.469 Kb
-- Data Model Class: SampleData
GROUP /
=====================
-- Parent Group : /
-- Group attributes :
* description :
* sample_name : test_sample
-- Childrens : Index, first_2D_image,
----------------
************************************************
GROUP first_2D_image
=====================
-- Parent Group : /
-- Group attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [0. 0.]
* spacing : [1. 1.]
* xdmf_gridname : first_2D_image
-- Childrens : test_2D_field, Field_index,
----------------
****** Group /first_2D_image CONTENT ******
NODE: /first_2D_image/Field_index
====================
-- Parent Group : first_2D_image
-- Node name : Field_index
-- Field_index attributes :
* empty : True
* node_type : string_array
-- content : /first_2D_image/Field_index (EArray(1,)) ''
-- Compression options for node `/first_2D_image/Field_index`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
NODE: /first_2D_image/test_2D_field
====================
-- Parent Group : first_2D_image
-- Node name : test_2D_field
-- test_2D_field attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /first_2D_image
* transpose_indices : [1, 0]
* xdmf_fieldname : test_2D_field
* xdmf_gridname : first_2D_image
-- content : /first_2D_image/test_2D_field (CArray(101, 101)) 'image2D_test_2D_field'
-- Compression options for node `/first_2D_image/test_2D_field`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (324, 101)
-- Node memory size : 63.914 Kb
----------------
************************************************
Reading its attributes, we can observe that a 2DImage group has been created, with a dimension of 51x51 pixels, in accordance with the inputed numpy array shape. We also can see that two childrens have been created for this group, a Field_index children, and a test_2D_field containing our data array (51x51 Carray).
If we look at the dataset Index, we obtain:
[18]:
data.print_index()
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : image2D H5_Path : /first_2D_image
Name : image2D_test_2D_field H5_Path : /first_2D_image/test_2D_field
Name : image2D_Field_index H5_Path : /first_2D_image/Field_index
You can see that the indexnames of the field and Field Index data items have been automatically constructed, following the rule:
indexname = grid_name + '_' + node_name.
We will look now at the other topology attributes of our Image group:
[19]:
data.print_node_attributes('image2D')
-- first_2D_image attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [0. 0.]
* spacing : [1. 1.]
* xdmf_gridname : first_2D_image
You can observe that the origin and spacing attributes have been set to their default value ([0.,0.] and [1.,1.]), as they have not been specified in add_image_from_field arguments. We will see how to modify values for these attributes in the next subsection of this tutorial.
Create an image with specific pixel/voxel size and origin¶
We will now use again the image group add method to recreate our image, this time specifying the value of the grid spacing and origin:
[20]:
data.add_image_from_field(field_array=field_2D, fieldname='test_2D_field', imagename='first_2D_image',
indexname='image2D', location='/', replace=True,
description="Test image group created from a bidimensional Numpy array.",
origin=[0.,10.], spacing=[2.,2.])
Removing group first_2D_image to replace it by new one.
Removing node /first_2D_image in content index....
Removing node /first_2D_image/test_2D_field in content index....
item image2D_test_2D_field : /first_2D_image/test_2D_field removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node <closed tables.carray.CArray at 0x7ff2a31adc30> sucessfully removed
Removing node /first_2D_image/Field_index in content index....
item image2D_Field_index : /first_2D_image/Field_index removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node <closed tables.earray.EArray at 0x7ff2a31adcd0> sucessfully removed
item image2D : /first_2D_image removed from context index dictionary
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Node /first_2D_image sucessfully removed
Creating Group group `first_2D_image` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `test_2D_field` into Grid `/first_2D_image`
Adding array `test_2D_field` into Group `/first_2D_image`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/first_2D_image`
Note that we had to set the ``replace`` argument to ``True`` to be able to overwrite our Image group in the dataset.
[21]:
data.print_node_attributes('image2D')
-- first_2D_image attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [ 0. 10.]
* spacing : [2. 2.]
* xdmf_gridname : first_2D_image
This time, the origin and spacing attributes have been set accordingly to our choice. Have they also been set in the XDMF file ?
[22]:
data.print_xdmf()
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="first_2D_image" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2">10. 0.</DataItem>
<DataItem Format="XML" Dimensions="2">2. 2.</DataItem>
</Geometry>
<Attribute Name="test_2D_field" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/first_2D_image/test_2D_field</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
The answer is yes !
You may note that in the XDMF file, the grid origin is stored as \([10, 0]\) and not \([0, 10]\), as it was inputed. This is due to the inverse interpretation of coordinate order by Paraview when visually rendering data. Hence, the values of the various data items in the XDMF file are written with an inverse coordinate order.
To modify an image spacing or origin, you can use the set_voxel_size and set_origin methods:
[23]:
data.set_voxel_size(image_group='image2D', voxel_size=np.array([4.,4.]))
data.set_origin(image_group='image2D', origin=np.array([10.,0.]))
data.print_node_attributes('image2D')
data.print_xdmf()
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
-- first_2D_image attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [10. 0.]
* spacing : [4. 4.]
* xdmf_gridname : first_2D_image
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="first_2D_image" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2"> 0. 10.</DataItem>
<DataItem Format="XML" Dimensions="2">4. 4.</DataItem>
</Geometry>
<Attribute Name="test_2D_field" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/first_2D_image/test_2D_field</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
The Image Group spacing and origin have indeed been modified in the group attributes, and the XDMF file.
You can use those methods to translate and dilate your image group in the visualization rendered by Paraview, which can be usefull to avoid grid superposition if you want to visualize them in the same RenderView, or, on the contrary, to force visual superposition of grids that had not the same position/scale when added to the dataset.
Creating images from pixel wise or nodal value defined fields¶
By using the is_elemField argument of add_image_from_field, we can change this standard behavior. By setting it to False, the method will consider the inputed array as a nodal value field: each value will be associated with a node of the image regular grid. We can test it by creating another image group from the same array:
[24]:
data.add_image_from_field(field_array=field_2D, fieldname='field_nodes_2D', imagename='image_2D_bis',
indexname='image2D_bis', location='/',
description="Test image group created from a bidimensional Numpy array.",
origin=[0.,10.], spacing=[2.,2.], is_elemField=False)
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `image_2D_bis` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `field_nodes_2D` into Grid `/image_2D_bis`
Adding array `field_nodes_2D` into Group `/image_2D_bis`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/image_2D_bis`
[25]:
data.print_xdmf()
data.print_node_info('image2D_bis')
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="first_2D_image" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2"> 0. 10.</DataItem>
<DataItem Format="XML" Dimensions="2">4. 4.</DataItem>
</Geometry>
<Attribute Name="test_2D_field" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/first_2D_image/test_2D_field</DataItem>
</Attribute>
</Grid>
<Grid Name="image_2D_bis" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="101 101"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2">10. 0.</DataItem>
<DataItem Format="XML" Dimensions="2">2. 2.</DataItem>
</Geometry>
<Attribute Name="field_nodes_2D" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/image_2D_bis/field_nodes_2D</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
GROUP image_2D_bis
=====================
-- Parent Group : /
-- Group attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [100 100]
* empty : False
* group_type : 2DImage
* nodes_dimension : [101 101]
* nodes_dimension_xdmf : [101 101]
* origin : [ 0. 10.]
* spacing : [2. 2.]
* xdmf_gridname : image_2D_bis
-- Childrens : field_nodes_2D, Field_index,
----------------
As you can see, the second image group has now dimensions reduced by one for each coordinate compared to the first image group, and the associated xdmf Grid node has an attribute that is defined on Nodes.
We can now visualize the fields to see what differencies does it make. Use the code of the following cell to open the dataset with Paraview. Then, you can open again the same file in Paraview, and only activate the rendering of one image group for each file, by ticking it in the property panel, as shown on the image below:
[26]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')
What you should see, is, for the pixel wise field, an rendering close to this:
And, for the nodal value field, you should have a rendering close to this:
The visual difference is clear, even if it is exactly the same data array that is plotted in both cases. In the first case, with Cell centered data, Paraview interprets the data as a field constant within each element. On the contrary, in the second case, with Node centered data, Paraview interprets the data as values that the field takes at grid vertexes, and interpolate linearly field values in the pixels (the grid elements) to render the visualization. That is why the field appears continuous at the interface between the fiber and the matrix.
Creating images from scalar, vector or tensor fields¶
There is another argument that changes the way add_image_from_field interprets the field data array: the is_scalar argument. Its default value is True. In this case the shape of the field is interpreted as a the dimension of the image to create. In the case just above, we inputed a (101x101) field, and the method created an image group of 101x101 pixels, containing a scalar field, i.e. a field with only one value per pixel.
However, we may want to create an image from a field whose values are vector or tensors. In that case, there is an ambiguity for arrays with 3 dimensions. They can either represent a \((x,y,z)\) array, or a \((x,y,i)\) array, where \(i\) denotes the field component index.
The ``is_scalar`` argument role is to avoid this ambiguity. If it is ``True`` (default) value, a 3D array will be interpreted as a 3D :math:`(x,y,z)` scalar field, and if it is ``False``, it will be interpreted as a :math:`(x,y,i)` 2D vector or tensor field.
Time to test this feature:
[27]:
# creation of a random 3D field :
field_3D = np.random.rand(50,50,3)
# creation of a 3D image group --> is_scalar set to True (default value)
data.add_image_from_field(field_array=field_3D, fieldname='field_3D', imagename='image_3D',
indexname='image3D', location='/',
description="""
Test 3D image group created from a tridimensional Numpy array with `is_scalar` = True.""")
# creation of a 2D image group --> is_scalar set to False
data.add_image_from_field(field_array=field_3D, fieldname='field_vect', imagename='image_2D_3',
indexname='image2D_3', location='/', is_scalar=False,
description="""
Test 2D image group created from a tridimensional Numpy array with `is_scalar` = False.""")
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `image_3D` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `field_3D` into Grid `/image_3D`
Adding array `field_3D` into Group `/image_3D`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/image_3D`
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `image_2D_3` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `field_vect` into Grid `/image_2D_3`
Adding array `field_vect` into Group `/image_2D_3`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/image_2D_3`
[28]:
# Getting information on the dataset and the created image groups
print(data)
data.print_node_info('image_3D')
data.print_node_info('image_2D_3')
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : image2D H5_Path : /first_2D_image
Name : image2D_test_2D_field H5_Path : /first_2D_image/test_2D_field
Name : image2D_Field_index H5_Path : /first_2D_image/Field_index
Name : image2D_bis H5_Path : /image_2D_bis
Name : image2D_bis_field_nodes_2D H5_Path : /image_2D_bis/field_nodes_2D
Name : image2D_bis_Field_index H5_Path : /image_2D_bis/Field_index
Name : image3D H5_Path : /image_3D
Name : image3D_field_3D H5_Path : /image_3D/field_3D
Name : image3D_Field_index H5_Path : /image_3D/Field_index
Name : image2D_3 H5_Path : /image_2D_3
Name : image2D_3_field_vect H5_Path : /image_2D_3/field_vect
Name : image2D_3_Field_index H5_Path : /image_2D_3/Field_index
Printing dataset content with max depth 3
|--GROUP first_2D_image: /first_2D_image (2DImage)
--NODE Field_index: /first_2D_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE test_2D_field: /first_2D_image/test_2D_field (field_array) ( 63.914 Kb)
|--GROUP image_2D_3: /image_2D_3 (2DImage)
--NODE Field_index: /image_2D_3/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_vect: /image_2D_3/field_vect (field_array) ( 63.281 Kb)
|--GROUP image_2D_bis: /image_2D_bis (2DImage)
--NODE Field_index: /image_2D_bis/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_nodes_2D: /image_2D_bis/field_nodes_2D (field_array) ( 63.914 Kb)
|--GROUP image_3D: /image_3D (3DImage)
--NODE Field_index: /image_3D/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_3D: /image_3D/field_3D (field_array) ( 58.594 Kb)
GROUP image_3D
=====================
-- Parent Group : /
-- Group attributes :
* description :
Test 3D image group created from a tridimensional Numpy array with `is_scalar` = True.
* dimension : [50 50 3]
* empty : False
* group_type : 3DImage
* nodes_dimension : [51 51 4]
* nodes_dimension_xdmf : [ 4 51 51]
* origin : [0. 0. 0.]
* spacing : [1. 1. 1.]
* xdmf_gridname : image_3D
-- Childrens : field_3D, Field_index,
----------------
GROUP image_2D_3
=====================
-- Parent Group : /
-- Group attributes :
* description :
Test 2D image group created from a tridimensional Numpy array with `is_scalar` = False.
* dimension : [50 50]
* empty : False
* group_type : 2DImage
* nodes_dimension : [51 51]
* nodes_dimension_xdmf : [51 51]
* origin : [0. 0.]
* spacing : [1. 1.]
* xdmf_gridname : image_2D_3
-- Childrens : field_vect, Field_index,
----------------
[29]:
# Getting information on created image fields
data.print_node_info('image3D_field_3D')
data.print_node_info('image2D_3_field_vect')
NODE: /image_3D/field_3D
====================
-- Parent Group : image_3D
-- Node name : field_3D
-- field_3D attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /image_3D
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : field_3D
* xdmf_gridname : image_3D
-- content : /image_3D/field_3D (CArray(3, 50, 50)) 'image3D_field_3D'
-- Compression options for node `image3D_field_3D`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (3, 50, 50)
-- Node memory size : 58.594 Kb
----------------
NODE: /image_2D_3/field_vect
====================
-- Parent Group : image_2D_3
-- Node name : field_vect
-- field_vect attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /image_2D_3
* transpose_indices : [1, 0, 2]
* xdmf_fieldname : field_vect
* xdmf_gridname : image_2D_3
-- content : /image_2D_3/field_vect (CArray(50, 50, 3)) 'image2D_3_field_vect'
-- Compression options for node `image2D_3_field_vect`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (54, 50, 3)
-- Node memory size : 63.281 Kb
----------------
If you look at the two created image groups, you see indeed that one has been created as a 3DImage group (with is_scalar=True), and one as a 2DImage Group (with is_scalar=False). Moreover, the field_dimensionality attribute of the two created image fields state clearly that the field of the 3D image is a scalar field, whereas the other one is a vector field.
We can now visualize them:
[30]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')
The rendering of the 3D image group with Paraview, containing a random 3D scalar field, should look the image below:
To obtain the rendering above, we only activated the rendering of the ``image_3D`` block in Paraview’s window Property Panel, and plotted it with the Surface renderer.
The rendering of the 2D image group with Paraview, containing a random 2D vector field, should look the this:
To obtain the rendering above, we only activated the rendering of the ``image_2D_3`` block in Paraview’s window Property Panel, added a Glyph filter in Arrow mode (that is how to plot vector field with Paraview) and colored the arrows with the vector field magnitude. We also bounded the maximum number of plotted arrows to 500.
We will stress again that here, those two very different visual rendering where obtained by creating two image groups with stricly the same input field data array. This illustrate the formatting possibilities offered by the SampleData class, but also highlight the need to be familiar with them, and to clearly specify in what form the data must be stored, to avoid unwanted behaviors.
Note that you can see in the attributes of the 3D image group the dimension transposition between \(X\) and \(Z\) coordinates, mentioned in section II, when comparing the values of attributes nodes_dimension and nodes_dimension_xdmf.
Note that we could have executed the same lines with an array of shape \((50,50,6)\) or \((50,50,9)\). The resulting 2D image would have in those cases supported a symetric tensor or tensor field.
This closes the presentation of the first and more straightforward way to create an Image group in a SampleData dataset.
Creating images from image objects¶
The second way to create Image groups, requires the creation of an image object. To handle grids, the SampleData class relies on the BasicTools Python package, developed by the industrial group Safran. This package offers some classes to represent meshes, and in particular rectilinear meshes, i.e. regular grids. What whe refer to as an image object, is in fact a BasicTools rectilinear mesh object. Let us see how it works with an example.
First, we need to import the rectilinear mesh class from Basictools, that should be accessible in your python environment as it is a SampleData dependence:
[31]:
from BasicTools.Containers.ConstantRectilinearMesh import ConstantRectilinearMesh
Then, we will create our image object as an instance of this class. For that, we need to set its dimenion (2D or 3D) image. For this example, we will try to recreate the 3D image with the random scalar field:
[32]:
image_object = ConstantRectilinearMesh(dim=3)
print(type(image_object))
<class 'BasicTools.Containers.ConstantRectilinearMesh.ConstantRectilinearMesh'>
We can now set the topology of the image, by using the following methods:
[33]:
image_object.SetDimensions((50,50,3))
image_object.SetOrigin([0.,0.,0.])
image_object.SetSpacing([1.,1.,1.])
Finally, image objects have two attributes that are dictionaries, the elemFields and the nodeFields, that allow to add fields to the image. Of course, these fields should have dimensions that match the image object dimensions. We will then add our random 3D array as a field of this image:
[34]:
image_object.elemFields['im_object_field'] = field_3D
We can print our image object to get information on it:
[35]:
print(image_object)
ConstantRectilinearMesh
Number of Nodes : 7500
Tags :
Number of Elements : 4802
dimensions : [50 50 3]
origin : [0. 0. 0.]
spacing : [1. 1. 1.]
ConstantRectilinearElementContainer, Type : (hex8,4802), Tags :
Node Tags : []
Cell Tags : []
elemFields : ['im_object_field']
You can see that it has an attribute that is an Element container, that has 4802 elements of type hex8, i.e. linear hexaedra, which is consistent with a grid of voxels.
Now we can add the image into the dataset with the add_image method. Its arguments are exactly the same as those of the add_image_from_field method, without the field_array, fieldname, is_scalar, elemField arguments:
[36]:
data.add_image(image_object, imagename='image_from_object', indexname='imageO', location='/',
description="""Test 3D image group created from an image object.""")
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `image_from_object` in file tutorial_dataset.h5 at /
Updating xdmf tree...
Adding field `im_object_field` into Grid `/image_from_object`
Adding array `im_object_field` into Group `/image_from_object`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/image_from_object`
[36]:
<BasicTools.Containers.ConstantRectilinearMesh.ConstantRectilinearMesh at 0x7ff2a30ef050>
[37]:
data.print_node_info('imageO')
GROUP image_from_object
=====================
-- Parent Group : /
-- Group attributes :
* description : Test 3D image group created from an image object.
* dimension : [49 49 2]
* empty : False
* group_type : 3DImage
* nodes_dimension : [50 50 3]
* nodes_dimension_xdmf : [ 3 50 50]
* origin : [0. 0. 0.]
* spacing : [1. 1. 1.]
* xdmf_gridname : image_from_object
-- Childrens : im_object_field, Field_index,
----------------
The image group has been created. Though, you can see that the dimension we indicated, actually prescribed the image nodes_dimension. Hence, remember that the ``SetDimensions`` method of image objects set the dimension of the node grid. The pixel/voxel grid has the same dimensions minus one along each direction.
As a result, we should expect here that the field loaded into the image object has been added to the image group as a nodal field. We can verify it by printing its attributes:
[38]:
data.print_node_attributes('imageO_im_object_field')
-- im_object_field attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /image_from_object
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : im_object_field
* xdmf_gridname : image_from_object
which, confirms our guess.
All the variations linked to the fields presented in above can still be achieved with this way of adding images in the datasets. For instance, if you passed to SetDimension the tuple \((N_x,N_y,N_z)\), to add a nodal vector field to your image group add a \((N_x,N_y,N_z,3)\) array to the image object elemFields list.
Creating empty images¶
SampleData also offers the possibility to create empty image objects. To do this, you proceed also with the add_image method, but without providing any image object:
[39]:
data.add_image(imagename='image_empty', indexname='imageE', location='/',
description="""Test empty image group.""")
(get_attribute) neither indexname nor node_path passed, node return aborted
Creating Group group `image_empty` in file tutorial_dataset.h5 at /
[40]:
data.print_node_info('imageE')
GROUP image_empty
=====================
-- Parent Group : /
-- Group attributes :
* empty : True
* group_type : emptyImage
-- Childrens :
----------------
Like empty data arrays presented in the tutorial 2 (section IV), empty images are used to create the internal organization of your dataset without having to add any data to it. It allows for instance to preempt some data item names, indexnames, and to already add metadata. When you have data to add, you can then create an image group with the same name/location, the empty image will be replace by an actual image group, but all metadata that was added to it before will be preserved.
To test this, we start by adding metadata to our empty image group:
[41]:
data.add_attributes({'tutorial_file':'3_SampleData_Image_groups.ipynb', 'tutorial_section':'III'}, 'imageE')
data.print_node_attributes('imageE')
-- image_empty attributes :
* empty : True
* group_type : emptyImage
* tutorial_file : 3_SampleData_Image_groups.ipynb
* tutorial_section : III
Now we will overwrite the empty image group by creating a 3D image group (same as in last subsection). In this case we do not need to set the replace argument to True.
[42]:
# first, we will change the name of the field stored in the image object ti avoid duplicates in the dataset
data.add_image(image_object, imagename='image_empty', indexname='imageE', location='/',
description="""Test empty image group overwritten with actual data.""")
data.print_node_info('imageE')
item imageE : /image_empty removed from context index dictionary
Updating xdmf tree...
Adding field `im_object_field` into Grid `/image_empty`
Adding array `im_object_field` into Group `/image_empty`
(get_node) ERROR : Node name does not fit any hdf5 path nor index name.
Adding String Array `Field_index` into Group `/image_empty`
GROUP image_empty
=====================
-- Parent Group : /
-- Group attributes :
* description : Test empty image group overwritten with actual data.
* dimension : [49 49 2]
* empty : False
* group_type : 3DImage
* nodes_dimension : [50 50 3]
* nodes_dimension_xdmf : [ 3 50 50]
* origin : [0. 0. 0.]
* spacing : [1. 1. 1.]
* tutorial_file : 3_SampleData_Image_groups.ipynb
* tutorial_section : III
* xdmf_gridname : image_empty
-- Childrens : im_object_field, Field_index,
----------------
The creation of the image group with actual data has indeed preserved the attributes attached to the former empty node.
Now, you know all about the three methods to create image groups in SampleData datasets.
IV - Image Fields data model¶
In this new section, we will now focus on the definition of image fields data items. Fields are data arrays with additional metadata allowing to interpret them as spatially organized data. For the SampleData class, these metadata main roles are to ensure that: 1. the correspondance between dimensions of the field array values and their physical meaning is preserved from the user perspective 2. that the fields visualization with Paraview through the XDMF format properly renders the geometry and dimensionality of fields values
To that, some conventions have been defined.
SampleData Image fields conventions¶
Image grids coordinates¶
Two grids are associated to Image Groups:
a grid of pixel/voxel centers, of dimensions \((N_x, N_y)\) or \((N_x, N_y, N_z)\), that can be seen as a set of points indentified by a doublet/triplet index:
\(P^c (i,j) = (x^c_i,y^c_j)\) (2D images)
$ P^c (i,j,k) = (x:sup:c_i,yc_j,z^c_k)$ (3D images)
a grid of nodes (pixel/voxel vertexes), of dimensions \((N_x+1, N_y+1)\) or \((N_x+1, N_y+1, N_z+1)\), similarly defined:
\(P^n (i,j) = (x^n_i, y^n_ j)\) (2D images)
\(P^n (i,j,k) = (x^n_i, y^n_ j, z^n_ k)\) (3D images)
with \(x^c,y^c\) and \(z^c\) representing the coordinates or pixel/voxel centers and \(x^n,y^n\) and \(z^n\) representing the coordinates or pixel/voxel centers.
Fields array possible shapes¶
You can add numpy arrays as fields to SampleData image groups. This array specifies the data of the field for each point of one of those two grids. It must have at least as many dimensions as the grid. Fields array are only allowed to have one more dimension that the grid they are defined on, and their first dimension sizes must comply with the grid dimensions. This implies that field arrays can only have 8 different types of shapes:
2D grids:
\(F(i,j) \longrightarrow\) size \((N_x,N_y)\)
\(F(i,j,c) \longrightarrow\) size \((N_x,N_y,N_c)\)
\(F(i,j) \longrightarrow\) size \((N_x+1,N_y+1)\)
\(F(i,j,c) \longrightarrow\) size \((N_x+1,N_y+1,N_c+1)\)
3D grids:
\(F(i,j,k) \longrightarrow\) size \((N_x,N_y, N_z)\)
\(F(i,j,k,c) \longrightarrow\) size \((N_x,N_y, N_c)\)
\(F(i,j,k) \longrightarrow\) size \((N_x +1,N_y +1, N_z +1)\)
\(F(i,j,k,c) \longrightarrow\) size \((N_x +1,N_y +1, N_c +1)\)
Field dimensionality¶
The size of the last dimension of the array determines the number of field components \(N_c\), which defines the field dimensionality. If the array has no supplementary dimension with respect to the grid, i.e. no \(c\) index, then \(N_c=1\).
SampleData will interpret the field dimensionality (scalar, vector, or tensor) depending on the grid topology and the array shape. All possibilities are listed hereafter: * \(N_c=1\): scalar field * \(N_c=3\): vector field * \(N_c=6\): symetric tensor field (2nd order tensor) * \(N_c=9\): tensor field (2nd order tensor)
The dimensionality of the field has the following implications: * XDMF: the dimensionality of the field is one of the metadata stored in the XDMF file, for Fields (Attribute nodes) * visualization: as it is stored in the XDMF file, Paraview can correctly interpret the fields according to their dimensionality. Il allows to plot separately each field component, and the field norm (magnitude). It also allows to use the Paraview Glyph filter to plot vector fields with arrows *
indexing: The order of the value in the last dimension for non-scalar fields correspond to a specific order of the field components, according to a specific convention. These conventions will be detailed in the next subsection. * compression: Specific lossy compression options exist for fields. See dedicated tutorial. * interface with external tools: when interfacing SampleData with external softwares, such as numerical simulation tools, it is very practical to have fields with
appropriate dimensionality accounted for (for instance to use a displacement or temperature gradient vector field stored in a SampleData dataset as input for a finite element simulation).
Field components indexing¶
We will now review the conventions for the correspondance between field data arrays indexing and field components, hence field arrays whose indexing is \(F[i,j,c]\) or \(F[i,j,k,c]\). To detail the convention, we will omit the spatial indexes \(i,j\) or \(k\) and only consider the last dimension of the field: \(F[i,j,k,c] = F[c]\). We additionally use the subscript notation \(F[c] = F_c\).
The indexing conventions are:
For vector fields (3 components), the convention is \([F_0,F_1,F_2] = [F_x,F_y,F_z]\)
For symetric tensor fields (2nd order, 6 components), the convention is \([F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx}]\)
For tensor fields (2nd order, 9 components), the convention is \([F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx},F_{yx},F_{zy},F_{xz}]\)
For the first indices, the convention is \(F[i,j,c] = F(x_i,y_i)[c]\) (2D) and \(F[i,j,k,c] = F(x_i,y_j,z_k)[c]\) (3D)
Here are a few examples to illustrate those conventions:
\(F[10,23]\) can only be interpreted as:
the field value at the grid point \((x_{10},y_{23})\)
\(F[10,23,2]\) can be interpreted as:
if \(F\) is a scalar field: the field value at the grid point \((x_{10},y_{23}, z_2)\)
if \(F\) is a vector field: the \(F_z\) component value at the grid point \((x_{10},y_{23})\)
if \(F\) is a tensor field: the \(F_{zz}\) component value at the grid point \((x_{10},y_{23})\)
\(F[10,23,7]\) can be interpreted as:
if \(F\) is a scalar field: the field value at the grid point \((x_{10},y_{23}, z_{7})\)
if \(F\) is a tensor field (non-symetric): the \(F_{zy}\) component value at the grid point \((x_{10},y_{23})\)
\(F[10,23,24,1]\) can be interpreted as:
if \(F\) is a vector field: the \(F_y\) component value at the grid point \((x_{10},y_{23},z_{24})\)
if \(F\) is a tensor field: the \(F_{yy}\) component value at the grid point \((x_{10},y_{23},z_{24})\)
\(F[10,23,24,5]\) can only be interpreted as:
the \(F_{zy}\) component value (tensor field) at the grid point \((x_{10},y_{23},z_{24})\)
Warning
Field components (example: \(x,y,xx,zy\)…) are assumed to be defined in the same frame as the grid. However, that cannot be ensured simply by providing a data array. Hence, the user must ensure to have this coincidence between the grid and the data. It is not mandatory, but not respecting this implicit convention may lead to confusions and geometrical misinterpretation of the data by other users or software. If you want to do it, a good idea would be to add attributes to the field to specify and explain the nature of the field components.
Field indices in-memory transposition¶
The indexing convention presented in the last subsection apply to data array that we add to Image groups as fields, and to data array that are retrieved from dataset Image group fields. However, you will see below that the fields are not stored in the HDF5 file with the same ordering. In practice, two transpositions are applied to the data arrays before loading them into the dataset, to ensure proper visualization with the Paraview software. There are two reasons for it, linked to Paraview indexing conventions:
Paraview convention for spatial indices is inverse to *SampleData* condition:
2D: the dimensions \(0\) and \(1\) must be inverted: \(F[i,j,...] \longrightarrow F[j,i,...]\)
3D: the dimensions \(0\) and \(2\) must be inverted: \(F[i,j,k,...] \longrightarrow F[k,j,i,...]\)
For these reason, fields are stored with indices transposition in SampleData datasets so that their visualization with Paraview yield a rendering that is consistent with the ordering convention presented in the previous subsection. These transposition are automatically handled, as well as the back transposition when a field is retrieved, as you will see in the last section of this tutorial.
An attribute ``transpose_indices`` is added to field data items. Its values represent the order in the first columns of the original array are stored in memory (see examples below).
Paraview component order convention is different from *SampleData* condition for tensors. The convention in Paraview is:
\([F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{xy},F_{xz},F_{yy},F_{yz},F_{zz}]\) for symetric tensors
\([F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{xy},F_{xz},F_{yx},F_{yy},F_{yz},F_{zx},F_{zy},F_{zz}]\) for non symetric tensors
For these reason, fields are stored with indices transposition in SampleData datasets so that their visualization with Paraview yield a rendering that is consistent with the ordering convention presented in the previous subsection. These transposition are automatically handled, as well as the back transposition when a field is retrieved, as you will see in the last section of this tutorial.
An attribute ``transpose_components`` is added to field data items. Its values represent the order in which components or the original array are stored in memory (see examples below).
Note
When visualizing a field in paraview, you may choose which component (including field magnitude) you are plotting in the box (highlighted in red in the image below) located in between the boxes for the choice of the visualization mode (Surface in the image below) and the data item choice (tensor_field2D in the image below).
In this box, you will have to choose between 9 (0 to 8) components, even for symetric tensors. In the later case, the corresponding equal components (1 and 3: \(xy\)&\(yx\), 2 and 6: \(xz\)&\(zx\), 5 and 7: \(yz\)&\(zy\)) will yield the same visualization.
Field type¶
In addition to their dimensionality, Image Fields data items also have a field type. You already encountered the two field types that can be supported by image groups in section III:
element field:
described by an array of values defined at pixel/voxel centers \(P^c\) (dimension \((N_x,N_y,N_c)\) or \((N_x,N_y,N_z,N_c)\) )
visualization: a pixel/voxel wise constant fields
nodal field:
described by an array of values defined at grid nodes (pixel/voxel vertexes) \(P^n\) (dimension \((N_x+1,N_y+1, N_c)\) or \((N_x+1,N_y+1,N_z+1,N_c)\) )
visualization: field linearly interpolated within each pixel/voxel from values at nodes
This feature is stored in the field data item attribute field_type (see examples above in section III).
Warning
Paraview seems to be unable to read non-scalar nodal fields XDMF Attributes for nodal regular grids. However, element vector or tensor fields can be visualized normally.
Field attributes¶
For each field, the value of all features presented above are stored as attributes. Let us see an example:
[43]:
data.print_index()
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : image2D H5_Path : /first_2D_image
Name : image2D_test_2D_field H5_Path : /first_2D_image/test_2D_field
Name : image2D_Field_index H5_Path : /first_2D_image/Field_index
Name : image2D_bis H5_Path : /image_2D_bis
Name : image2D_bis_field_nodes_2D H5_Path : /image_2D_bis/field_nodes_2D
Name : image2D_bis_Field_index H5_Path : /image_2D_bis/Field_index
Name : image3D H5_Path : /image_3D
Name : image3D_field_3D H5_Path : /image_3D/field_3D
Name : image3D_Field_index H5_Path : /image_3D/Field_index
Name : image2D_3 H5_Path : /image_2D_3
Name : image2D_3_field_vect H5_Path : /image_2D_3/field_vect
Name : image2D_3_Field_index H5_Path : /image_2D_3/Field_index
Name : imageO H5_Path : /image_from_object
Name : imageO_im_object_field H5_Path : /image_from_object/im_object_field
Name : imageO_Field_index H5_Path : /image_from_object/Field_index
Name : imageE H5_Path : /image_empty
Name : imageE_im_object_field H5_Path : /image_empty/im_object_field
Name : imageE_Field_index H5_Path : /image_empty/Field_index
[44]:
data.print_node_info('image2D_3_field_vect')
NODE: /image_2D_3/field_vect
====================
-- Parent Group : image_2D_3
-- Node name : field_vect
-- field_vect attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /image_2D_3
* transpose_indices : [1, 0, 2]
* xdmf_fieldname : field_vect
* xdmf_gridname : image_2D_3
-- content : /image_2D_3/field_vect (CArray(50, 50, 3)) 'image2D_3_field_vect'
-- Compression options for node `image2D_3_field_vect`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (54, 50, 3)
-- Node memory size : 63.281 Kb
----------------
The attribute printed above inform us on the field data item features. The field_dimensionality attribute indicates that the field is a vector field. Has its data array has a dimension of 3 (see the content line in the output), it is a scalar field of a 2D image. The field_type attribute indicates that the field is defined at pixel centers.
The transpose_indices show that the first and second column of the original inputed array have been inverted in memory (inversion of \(X\) and \(Y\) direction between SampleData and Paraview conventions).
The parent_grid_path and xdmf_gridname attribute provides the path of the Image Group and the XDMF Grid Node name to which this field belongs. The xdmf_fieldname provides the name of the XDMF Attribute Node associated to the field. The padding attribute is of no use for Image fields. It will be presented in the next tutorial for mesh fields.
V - Adding Fields to Image Groups¶
Now that we know all elements that compose and define a field data item in a SampleData dataset, we will see how to create fields. As for other creation tasks, the SampleData class provides a method to do it with an explicit name: the add_field method. It is very similar to the add_data_array method, with two main additions presented below.
Adding a field to a grid¶
To add a field to a grid from a numpy array, you just have to call add_field like you would have called add_data_array, with an additional argument gridname, that indicates the grid group to which you want to add the field. In the case of the present tutorial, we can hence provide the Name, Indexname, Path or Alias of an image group to add a field.
The analysis of the field dimensionality, nature and required transpositions is automatically handled by the class.
We will try to add a symetric tensor element field to the first 2D image that we have created, at the beginning of section III. Let us recall the image group features:
[45]:
data.print_node_info('image2D')
GROUP first_2D_image
=====================
-- Parent Group : /
-- Group attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [10. 0.]
* spacing : [4. 4.]
* xdmf_gridname : first_2D_image
-- Childrens : test_2D_field, Field_index,
----------------
The image has a nodes_dimension of 102x102. We can retrieve this attribute to presscribe the shape of the array that we have to create. As we want to create a symetric tensor field, we also need to add a last dimension of size 6.
We will create a very simple field, whose values are defined by \(F(i,j,c) = c\) for \(y <= 51\) and \(F(i,j,c) = 2c\) for \(y > 51\):
[46]:
# first we get the image nodal grid dimensions
dim = data.get_attribute('dimension','image2D')
# then, we create a field of 0 with the right dimensions
Nc = 6
tensor_field = np.zeros(shape=(dim[0],dim[1],Nc))
# finally we set the values of our tensor field
for c in range(Nc):
tensor_field[:,:52,c] = c
tensor_field[:,52:,c] = 2*c
[47]:
print(tensor_field)
[[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]
[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]
[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]
...
[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]
[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]
[[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
[ 0. 1. 2. 3. 4. 5.]
...
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]
[ 0. 2. 4. 6. 8. 10.]]]
[48]:
# now we can add our field to the image group
data.add_field(gridname='image2D', fieldname='tensor_field2D', location='image2D', indexname='tensorF',
array=tensor_field, replace=True)
Adding field `tensor_field2D` into Grid `image2D`
Adding array `tensor_field2D` into Group `image2D`
[48]:
/first_2D_image/tensor_field2D (CArray(101, 101, 6)) 'tensorF'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (13, 101, 6)
[49]:
data.print_node_info('tensorF')
NODE: /first_2D_image/tensor_field2D
====================
-- Parent Group : first_2D_image
-- Node name : tensor_field2D
-- tensor_field2D attributes :
* empty : False
* field_dimensionality : Tensor6
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /first_2D_image
* transpose_components : [0, 3, 5, 1, 4, 2]
* transpose_indices : [1, 0, 2]
* xdmf_fieldname : tensor_field2D
* xdmf_gridname : first_2D_image
-- content : /first_2D_image/tensor_field2D (CArray(101, 101, 6)) 'tensorF'
-- Compression options for node `tensorF`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (13, 101, 6)
-- Node memory size : 492.375 Kb
----------------
As you can observe, the field has automatically been created as a symetric tensor (Tensor6) nodal field, with the transposition of its first and second column (see transpose_indices value) and of its components (see transpose_indices value).
We can already visualize our created field:
[50]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')
You should be able to visualization the tensor field magnitude and components individually:
You can try to redo these last cells by changing the dimension of the field, to try to change its type, its dimensionality, or the image group on which you add it.
Adding time serie of fields¶
The XDMF format also allows to have a temporal organization of data, by attaching a Time node to Grid nodes. The SampleData allows to leverage this possibility and create a set of fields array that are related in time. Each of them will represent the value of one field at different instants.
To do that, you just have to set the value of the time argument when calling the add_field method. This value will be registered as the instant at which the field takes the value in the provided data array. We will illustrate this possibility with an example.
We will create a 2D vector field that will take different values at 3 different instants, and add it to same image to which we added the previous field. At each instant, the whole field will be uniform and have the value of one of the 3 base vectors. As we are providing different time values of the same field, we should provide the same Name each time we provide an time value of this field.
[51]:
# get the image nodal grid dimensions
dim = data.get_attribute('dimension','image2D')
# create of a data array of 0 of dimension (Nx,Ny,3,Ninstants)
temporal_field = np.zeros((dim[0],dim[1],3,3))
# Set the value of the field to the first unit vector at instant 0
temporal_field[:,:,0,0] = 1
# Set the value of the field to the second unit vector at instant 1
temporal_field[:,:,1,1] = 1
# Set the value of the field to the third unit vector at instant 2
temporal_field[:,:,2,2] = 1
# Set the value of time instants:
instants = [1.,10., 100.]
[52]:
# now we can add for each instant the right slice of the array as a field to the image group
# instant 0
data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',
array=temporal_field[:,:,:,0], time=instants[0])
# instant 1
data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',
array=temporal_field[:,:,:,1], time=instants[1])
# instant 2
data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',
array=temporal_field[:,:,:,1], time=instants[2])
Adding field `time_field` into Grid `image2D`
Adding array `time_field_T0` into Group `image2D`
(get_attribute) neither indexname nor node_path passed, node return aborted
Adding field `time_field` into Grid `image2D`
Adding array `time_field_T1` into Group `image2D`
(get_attribute) neither indexname nor node_path passed, node return aborted
Adding field `time_field` into Grid `image2D`
Adding array `time_field_T2` into Group `image2D`
(get_attribute) neither indexname nor node_path passed, node return aborted
[52]:
/first_2D_image/time_field_T2 (CArray(101, 101, 3)) 'Field_T2'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (27, 101, 3)
Time series Grids and time series attributes¶
You will see that creating this time serie of fields has slighlty modified the dataset. We will explore these modifications, and start by looking at our dataset organization to see if it has changed:
[53]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : image2D H5_Path : /first_2D_image
Name : image2D_test_2D_field H5_Path : /first_2D_image/test_2D_field
Name : image2D_Field_index H5_Path : /first_2D_image/Field_index
Name : image2D_bis H5_Path : /image_2D_bis
Name : image2D_bis_field_nodes_2D H5_Path : /image_2D_bis/field_nodes_2D
Name : image2D_bis_Field_index H5_Path : /image_2D_bis/Field_index
Name : image3D H5_Path : /image_3D
Name : image3D_field_3D H5_Path : /image_3D/field_3D
Name : image3D_Field_index H5_Path : /image_3D/Field_index
Name : image2D_3 H5_Path : /image_2D_3
Name : image2D_3_field_vect H5_Path : /image_2D_3/field_vect
Name : image2D_3_Field_index H5_Path : /image_2D_3/Field_index
Name : imageO H5_Path : /image_from_object
Name : imageO_im_object_field H5_Path : /image_from_object/im_object_field
Name : imageO_Field_index H5_Path : /image_from_object/Field_index
Name : imageE H5_Path : /image_empty
Name : imageE_im_object_field H5_Path : /image_empty/im_object_field
Name : imageE_Field_index H5_Path : /image_empty/Field_index
Name : tensorF H5_Path : /first_2D_image/tensor_field2D
Name : Field_T0 H5_Path : /first_2D_image/time_field_T0
Name : Field_T1 H5_Path : /first_2D_image/time_field_T1
Name : Field_T2 H5_Path : /first_2D_image/time_field_T2
Printing dataset content with max depth 3
|--GROUP first_2D_image: /first_2D_image (2DImage)
--NODE Field_index: /first_2D_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE tensor_field2D: /first_2D_image/tensor_field2D (field_array) ( 492.375 Kb)
--NODE test_2D_field: /first_2D_image/test_2D_field (field_array) ( 63.914 Kb)
--NODE time_field_T0: /first_2D_image/time_field_T0 (field_array) ( 255.656 Kb)
--NODE time_field_T1: /first_2D_image/time_field_T1 (field_array) ( 255.656 Kb)
--NODE time_field_T2: /first_2D_image/time_field_T2 (field_array) ( 255.656 Kb)
|--GROUP image_2D_3: /image_2D_3 (2DImage)
--NODE Field_index: /image_2D_3/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_vect: /image_2D_3/field_vect (field_array) ( 63.281 Kb)
|--GROUP image_2D_bis: /image_2D_bis (2DImage)
--NODE Field_index: /image_2D_bis/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_nodes_2D: /image_2D_bis/field_nodes_2D (field_array) ( 63.914 Kb)
|--GROUP image_3D: /image_3D (3DImage)
--NODE Field_index: /image_3D/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_3D: /image_3D/field_3D (field_array) ( 58.594 Kb)
|--GROUP image_empty: /image_empty (3DImage)
--NODE Field_index: /image_empty/Field_index (string_array - empty) ( 63.999 Kb)
--NODE im_object_field: /image_empty/im_object_field (field_array) ( 58.594 Kb)
|--GROUP image_from_object: /image_from_object (3DImage)
--NODE Field_index: /image_from_object/Field_index (string_array - empty) ( 63.999 Kb)
--NODE im_object_field: /image_from_object/im_object_field (field_array) ( 58.594 Kb)
The structure of the HDF5 dataset features no surprises. The three fields have been added as arrays under the Image Group, and added to the index. As you can observe, the names and indexnames of the fields have automatically been completed with a suffix indicating the instant they represent: time_field_T0, time_field_T1 etc…
We can now look at the XDMF file:
[54]:
data.print_xdmf()
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="image_2D_bis" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="101 101"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2">10. 0.</DataItem>
<DataItem Format="XML" Dimensions="2">2. 2.</DataItem>
</Geometry>
<Attribute Name="field_nodes_2D" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/image_2D_bis/field_nodes_2D</DataItem>
</Attribute>
</Grid>
<Grid Name="image_3D" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions=" 4 51 51"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">0. 0. 0.</DataItem>
<DataItem Format="XML" Dimensions="3">1. 1. 1.</DataItem>
</Geometry>
<Attribute Name="field_3D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="3 50 50" NumberType="Float" Precision="64">tutorial_dataset.h5:/image_3D/field_3D</DataItem>
</Attribute>
</Grid>
<Grid Name="image_2D_3" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="51 51"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2">0. 0.</DataItem>
<DataItem Format="XML" Dimensions="2">1. 1.</DataItem>
</Geometry>
<Attribute Name="field_vect" AttributeType="Vector" Center="Cell">
<DataItem Format="HDF" Dimensions="50 50 3" NumberType="Float" Precision="64">tutorial_dataset.h5:/image_2D_3/field_vect</DataItem>
</Attribute>
</Grid>
<Grid Name="image_from_object" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions=" 3 50 50"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">0. 0. 0.</DataItem>
<DataItem Format="XML" Dimensions="3">1. 1. 1.</DataItem>
</Geometry>
<Attribute Name="im_object_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="3 50 50" NumberType="Float" Precision="64">tutorial_dataset.h5:/image_from_object/im_object_field</DataItem>
</Attribute>
</Grid>
<Grid Name="image_empty" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions=" 3 50 50"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">0. 0. 0.</DataItem>
<DataItem Format="XML" Dimensions="3">1. 1. 1.</DataItem>
</Geometry>
<Attribute Name="im_object_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="3 50 50" NumberType="Float" Precision="64">tutorial_dataset.h5:/image_empty/im_object_field</DataItem>
</Attribute>
</Grid>
<Grid Name="first_2D_image" GridType="Collection" CollectionType="Temporal">
<Grid Name="first_2D_image_T0" GridType="Uniform">
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2"> 0. 10.</DataItem>
<DataItem Format="XML" Dimensions="2">4. 4.</DataItem>
</Geometry>
<Attribute Name="test_2D_field" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101" NumberType="Int" Precision="16">tutorial_dataset.h5:/first_2D_image/test_2D_field</DataItem>
</Attribute>
<Attribute Name="tensor_field2D" AttributeType="Tensor6" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101 6" NumberType="Float" Precision="64">tutorial_dataset.h5:/first_2D_image/tensor_field2D</DataItem>
</Attribute>
<Time Value="1.0"/>
<Attribute Name="time_field" AttributeType="Vector" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101 3" NumberType="Float" Precision="64">tutorial_dataset.h5:/first_2D_image/time_field_T0</DataItem>
</Attribute>
</Grid>
<Grid Name="first_2D_image_T1" GridType="Uniform">
<Time Value="10.0"/>
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2"> 0. 10.</DataItem>
<DataItem Format="XML" Dimensions="2">4. 4.</DataItem>
</Geometry>
<Attribute Name="time_field" AttributeType="Vector" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101 3" NumberType="Float" Precision="64">tutorial_dataset.h5:/first_2D_image/time_field_T1</DataItem>
</Attribute>
</Grid>
<Grid Name="first_2D_image_T2" GridType="Uniform">
<Time Value="100.0"/>
<Topology TopologyType="2DCoRectMesh" Dimensions="102 102"/>
<Geometry Type="ORIGIN_DXDY">
<DataItem Format="XML" Dimensions="2"> 0. 10.</DataItem>
<DataItem Format="XML" Dimensions="2">4. 4.</DataItem>
</Geometry>
<Attribute Name="time_field" AttributeType="Vector" Center="Cell">
<DataItem Format="HDF" Dimensions="101 101 3" NumberType="Float" Precision="64">tutorial_dataset.h5:/first_2D_image/time_field_T2</DataItem>
</Attribute>
</Grid>
</Grid>
</Domain>
</Xdmf>
The XDMF file of the dataset has become rather large now. As the creation of a time serie of grids requires to rewrite the 2D image XDMF grid node, you will find it at the end of the file. You see that the Grid node named first_2D_image has changed, and is no more a Uniform type of grid, like the other images.
It is now a Collection of grid, and has the collection type Temporal. Three sub grid nodes that are Uniform have been created, each with a Time element, corresponding to one of the instant values that we provided. Ultimately, note that the gris of instants 2 and 3 only have one Attribute field. The fields that were attached to the image group before adding this time serie have been automatically attached to the grid of the first instant.
We can now look at our image group, to find out how it has changed:
[55]:
data.print_node_info('image2D')
GROUP first_2D_image
=====================
-- Parent Group : /
-- Group attributes :
* description : Test image group created from a bidimensional Numpy array.
* dimension : [101 101]
* empty : False
* group_type : 2DImage
* nodes_dimension : [102 102]
* nodes_dimension_xdmf : [102 102]
* origin : [10. 0.]
* spacing : [4. 4.]
* time_list : [1.0, 10.0, 100.0]
* xdmf_gridname : first_2D_image
-- Childrens : test_2D_field, Field_index, tensor_field2D, time_field_T0, time_field_T1, time_field_T2,
----------------
The only difference that we can see, is the creation of a time_list attribute, that gather all the time instants that have been defined for the fields of this image group.
To conclude this subsection, we will look at the content of one of the field data items added in the time serie:
[56]:
data.print_node_info('time_field_T1')
NODE: /first_2D_image/time_field_T1
====================
-- Parent Group : first_2D_image
-- Node name : time_field_T1
-- time_field_T1 attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /first_2D_image
* time : 10.0
* time_serie_name : time_field
* transpose_indices : [1, 0, 2]
* xdmf_fieldname : time_field
* xdmf_gridname : first_2D_image_T1
-- content : /first_2D_image/time_field_T1 (CArray(101, 101, 3)) 'Field_T1'
-- Compression options for node `time_field_T1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (27, 101, 3)
-- Node memory size : 255.656 Kb
----------------
This field node is absolutely similar to the field data items that we created earlier without time value. The only difference is that it has two additional attributes time and time_serie_name that indicate at which instant its values correspond, and was is the name of the time serie field.
VI - Getting Images and Fields¶
To conclude this tutorial, we will now try to retrieve the data that we created, in the form of field arrays or image objects.
Getting image objects from datasets¶
Getting an image object from a dataset image Group is very straightforward with the SampleData class. You just have to use the get_image method. It takes two arguments, the name (or indexname etc…) of the image Group, and a with_fields boolean argument, that allow you to chooe if you want or not to load the field data arrays into the image object.
We will start by printing again the content of our dataset, to remember the image groups and fields stored in it:
[57]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : image2D H5_Path : /first_2D_image
Name : image2D_test_2D_field H5_Path : /first_2D_image/test_2D_field
Name : image2D_Field_index H5_Path : /first_2D_image/Field_index
Name : image2D_bis H5_Path : /image_2D_bis
Name : image2D_bis_field_nodes_2D H5_Path : /image_2D_bis/field_nodes_2D
Name : image2D_bis_Field_index H5_Path : /image_2D_bis/Field_index
Name : image3D H5_Path : /image_3D
Name : image3D_field_3D H5_Path : /image_3D/field_3D
Name : image3D_Field_index H5_Path : /image_3D/Field_index
Name : image2D_3 H5_Path : /image_2D_3
Name : image2D_3_field_vect H5_Path : /image_2D_3/field_vect
Name : image2D_3_Field_index H5_Path : /image_2D_3/Field_index
Name : imageO H5_Path : /image_from_object
Name : imageO_im_object_field H5_Path : /image_from_object/im_object_field
Name : imageO_Field_index H5_Path : /image_from_object/Field_index
Name : imageE H5_Path : /image_empty
Name : imageE_im_object_field H5_Path : /image_empty/im_object_field
Name : imageE_Field_index H5_Path : /image_empty/Field_index
Name : tensorF H5_Path : /first_2D_image/tensor_field2D
Name : Field_T0 H5_Path : /first_2D_image/time_field_T0
Name : Field_T1 H5_Path : /first_2D_image/time_field_T1
Name : Field_T2 H5_Path : /first_2D_image/time_field_T2
Printing dataset content with max depth 3
|--GROUP first_2D_image: /first_2D_image (2DImage)
--NODE Field_index: /first_2D_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE tensor_field2D: /first_2D_image/tensor_field2D (field_array) ( 492.375 Kb)
--NODE test_2D_field: /first_2D_image/test_2D_field (field_array) ( 63.914 Kb)
--NODE time_field_T0: /first_2D_image/time_field_T0 (field_array) ( 255.656 Kb)
--NODE time_field_T1: /first_2D_image/time_field_T1 (field_array) ( 255.656 Kb)
--NODE time_field_T2: /first_2D_image/time_field_T2 (field_array) ( 255.656 Kb)
|--GROUP image_2D_3: /image_2D_3 (2DImage)
--NODE Field_index: /image_2D_3/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_vect: /image_2D_3/field_vect (field_array) ( 63.281 Kb)
|--GROUP image_2D_bis: /image_2D_bis (2DImage)
--NODE Field_index: /image_2D_bis/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_nodes_2D: /image_2D_bis/field_nodes_2D (field_array) ( 63.914 Kb)
|--GROUP image_3D: /image_3D (3DImage)
--NODE Field_index: /image_3D/Field_index (string_array - empty) ( 63.999 Kb)
--NODE field_3D: /image_3D/field_3D (field_array) ( 58.594 Kb)
|--GROUP image_empty: /image_empty (3DImage)
--NODE Field_index: /image_empty/Field_index (string_array - empty) ( 63.999 Kb)
--NODE im_object_field: /image_empty/im_object_field (field_array) ( 58.594 Kb)
|--GROUP image_from_object: /image_from_object (3DImage)
--NODE Field_index: /image_from_object/Field_index (string_array - empty) ( 63.999 Kb)
--NODE im_object_field: /image_from_object/im_object_field (field_array) ( 58.594 Kb)
Let us retrieve the image group image_from_object, with its internal fields:
[58]:
im_object = data.get_image('imageO', with_fields=True)
print(im_object)
ConstantRectilinearMesh
Number of Nodes : 7500
Tags :
Number of Elements : 4802
dimensions : [50 50 3]
origin : [0. 0. 0.]
spacing : [1. 1. 1.]
ConstantRectilinearElementContainer, Type : (hex8,4802), Tags :
Node Tags : []
Cell Tags : []
nodeFields : ['imageO_im_object_field']
It is as simple as that. Setting the with_fields argument to False would have resulted in the same image object, but with en empty nodeFields attribute.
Now you have an image object that contains all the relevant information to recreate a SampleData image group with the same fields, and the same topology as the retrieved group image0. This is particularly usefull to transfer an image group from a dataset to another, as illustrated in the next cell:
[59]:
# We want to transfer our image group to a new dataset --> here we create a new dataset
# (we could have also opened a pre-existing one)
data2 = SD(filename='tmp_dataset', autodelete=True)
# We add our image object
data2.add_image(im_object, imagename='transfered_image', indexname='imageTr', location='/')
# We print the data set content to verify that our image group has been created
print(data2)
# We compare information about the two images group (the one from the original dataset and the created one)
data.print_node_info('imageO')
data2.print_node_info('imageTr')
# We close our new dataset
del data2
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : imageTr H5_Path : /transfered_image
Name : imageTr_imageO_im_object_field H5_Path : /transfered_image/imageO_im_object_field
Name : imageTr_Field_index H5_Path : /transfered_image/Field_index
Printing dataset content with max depth 3
|--GROUP transfered_image: /transfered_image (3DImage)
--NODE Field_index: /transfered_image/Field_index (string_array - empty) ( 63.999 Kb)
--NODE imageO_im_object_field: /transfered_image/imageO_im_object_field (field_array) ( 58.594 Kb)
GROUP image_from_object
=====================
-- Parent Group : /
-- Group attributes :
* description : Test 3D image group created from an image object.
* dimension : [49 49 2]
* empty : False
* group_type : 3DImage
* nodes_dimension : [50 50 3]
* nodes_dimension_xdmf : [ 3 50 50]
* origin : [0. 0. 0.]
* spacing : [1. 1. 1.]
* xdmf_gridname : image_from_object
-- Childrens : im_object_field, Field_index,
----------------
GROUP transfered_image
=====================
-- Parent Group : /
-- Group attributes :
* description :
* dimension : [49 49 2]
* empty : False
* group_type : 3DImage
* nodes_dimension : [50 50 3]
* nodes_dimension_xdmf : [ 3 50 50]
* origin : [0. 0. 0.]
* spacing : [1. 1. 1.]
* xdmf_gridname : transfered_image
-- Childrens : imageO_im_object_field, Field_index,
----------------
SampleData Autodelete:
Removing hdf5 file tmp_dataset.h5 and xdmf file tmp_dataset.xdmf
You see that the image group has been easily transfered too another dataset, with its fields and metadata linked to the Image group and Image field data models.
Note
Other attributes that belonged to the original image group have not been transfered (see for instance the description attribute). Transfering these attributes between the two datasets would require to use the get_dic_from_attribute and add_attributes methods.
The get_node method has been presented in the last tutorial (section IV), which is the generic SampleData method to retrieve data. What happens if we use it on an Image Group ?
[60]:
image = data.get_node('imageO')
print(image)
print(type(image))
/image_from_object (Group) 'image_from_object'
<class 'tables.group.Group'>
What we get is simply a Pytables HDF5 Group object, that contains no data or metadata stored into the Image object that we tried to retrieve. Trying to get the image node with the dictionary or attribute like access would yield the same behavior.
Getting field arrays¶
As for images, it is straightforward to retrieve image fields stored in a SampleData dataset. The associated method is get_field, and requires only the name of the field (or indexname etc…).
[61]:
data.print_node_info('tensorF')
field_tmp = data.get_field('tensorF')
print(type(field_tmp))
print(field_tmp.shape,'\n')
print(f' Is the field the same as the one used to create the field data item ? {np.all(field_tmp == tensor_field)}')
NODE: /first_2D_image/tensor_field2D
====================
-- Parent Group : first_2D_image
-- Node name : tensor_field2D
-- tensor_field2D attributes :
* empty : False
* field_dimensionality : Tensor6
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /first_2D_image
* transpose_components : [0, 3, 5, 1, 4, 2]
* transpose_indices : [1, 0, 2]
* xdmf_fieldname : tensor_field2D
* xdmf_gridname : first_2D_image
-- content : /first_2D_image/tensor_field2D (CArray(101, 101, 6)) 'tensorF'
-- Compression options for node `tensorF`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (13, 101, 6)
-- Node memory size : 492.375 Kb
----------------
<class 'numpy.ndarray'>
(101, 101, 6)
Is the field the same as the one used to create the field data item ? True
You see that all the transformation applied to the data before in-memory storage (transposition of columns, of the field component indices) have been transformed back automatically: the method returned exactly the same numpy array passed as argument of add_field to create the field data item in the dataset.
What would the get_node method return here ? That depends on the value of its as_numpy argument. What about the attribute or dictionary like access ? Here follow some examples:
[62]:
test_field1 = data.get_node('tensorF')
test_field2 = data.get_node('tensorF', as_numpy=True)
test_field3 = data['tensorF']
print('test 1 : ', type(test_field1))
print('test 2 : ', type(test_field2))
print('test 3 : ', type(test_field3))
test 1 : <class 'tables.carray.CArray'>
test 2 : <class 'numpy.ndarray'>
test 3 : <class 'numpy.ndarray'>
We see that the get_nodemethod returned a Pytables node object with no option specified, and a numpy array with the as_numpy=True option.
As you can see below, the dictionary like access method returned the same results as the get_node method with as_numpy=True option:
[63]:
np.all(test_field2 == test_field3)
[63]:
True
Are those outputs equivalent to the original field ?
[64]:
np.all(test_field2 == tensor_field)
[64]:
True
You see that the ``get_field(…)``, ``get_node(…, as_numpy=True)`` and dictionary like access to fields data items methods are strictly equivalent.
Now, note that Pytables node objects behave in some ways as numpy arrays. Then, are there equivalent to the original data array ?
[65]:
np.all(test_field1 == tensor_field)
[65]:
False
No, they are not ! The reason behind that is that the ``get_node(…, as_numpy=False)`` method returns a Node object that is a link to the data stored in memory in the HDF5 dataset. As explained before, this data has been transformed in memory to comply with indexing conventions, that is why the arrays are no longer equivalent !
You can remember that:
To access field array values as they have been added to the dataset, you need to retrieve them with one of the following methods:
add_fieldmethodget_node(..., as_numpy=True)methoddictionary or attribute like access:
data['fieldname']ordata.fieldname
To access field array values are they are stored in memory in the HDF5 file, you must use the ``get_node(…,as_numpy=False)`` method.
This is the end of this tutorial on SampleData Image Groups and Image Fields. We can now close our dataset.
[66]:
del data
Deleting DataSample object
.... Storing content index in tutorial_dataset.h5:/Index attributes
.... writing xdmf file : tutorial_dataset.xdmf
.... flushing data in file tutorial_dataset.h5
File tutorial_dataset.h5 synchronized with in memory data tree
Dataset and Datafiles closed
SampleData Autodelete:
Removing hdf5 file tutorial_dataset.h5 and xdmf file tutorial_dataset.xdmf
5 - Creating, getting and visualizing Mesh groups and Mesh Fields¶
This Notebook will introduce you to:
what is a Mesh Group
the Mesh Group data model
how to create a mesh Group
the Mesh Field data model
how to add a field to a Mesh Group
how to get Mesh Groups and Mesh Fields
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the first notebook of this User Guide of this User Guide.
I - SampleData Mesh Groups¶
SampleData Grid Groups have been introduced in the last tutorial, which also presented in details one of the two type of Grids, the Image Groups, representing regular grids (rectilinear meshes). This tutorial focuses on the second type on Grid Groups, the Mesh groups.
SampleData meshes¶
Mesh Groups represent unstructured grids, defined by a set of Nodes with their associate coordinates, a set of Elements, that are defined by a connectivity matrix, indicating which nodes define each element, and a set of Fields, describing data defined on this grid. They are typically used to store Finite Element meshes, and Finite Element fields. For a precise mathematical definition of what Nodes and Elements are, the reader can refer to standard concepts of finite element theory, which is not in the scpe of this tutorial.
Like for Image Groups, three types of Mesh Groups can be handled by SampleData:
emptyMesh: an empty group that can be used to set the organization and metadata of the dataset before adding actual data to it (similar to empty data arrays, see tutorial 2, section IV)IV](./2_SampleData_basic_data_items.ipynb))2DMesh: a grid of Nodes defined by two coordinates \(x\) and \(y\)3DMesh: a grid of Nodes defined by three coordinates \(x\), \(y\) and \(z\)
An Mesh field is a data array that has one value associated to each node, each element, or each integration point of the mesh.
For many applications, it may be usefull to define specific sets of Nodes or sets of Elements (for instance to define distinct components of a microstructure, define an area where specific boundary conditions are applied for numerical simulation etc…). Therefore, SampleData Mesh Groups also store data defining those Node and Element sets.
Hence, Mesh Groups contain data and metadata defining the mesh Nodes, Node Sets, Elements, Element types and Sets, Mesh Fields, and metadata to synchronize with the XDMF Grid node associated to the Mesh. To explore in details this data model, we will once again open the reference test dataset of the SampleData unit tests.
Warning 1:
The SampleData class is not designed to create meshes , in particular complex meshes of microstructures that are of interest for material science. It is designed mainly to serve as a I/O interface to load/store meshes from mesh data files, visualize them, and provide them to other numerical tools that may need them (finite element solvers for instance).
An automatic mesher that directly outputs a SampleData Mesh Group has been implemented in the Pymicro package. It relies on an automatic meshing code that is not open-source, and is the property of the laboratory Centre des Matériaux of Mines ParisTech. Please contact us if you want to know more and use this tool. If you are from le Centre des Matériaux, a dedicated tutorial for this tool exists. Please contact H. Proudhon, B. Marchand or L. Lacourt to get it.
If you need to create a meshing tool that automatically outputs results as a SampleData dataset, please contact us to see if we may help you or provide you with guidelines to develop your mesher interface.
Warning 2:
The SampleData class currently relies on the BasicTools package to handle grids. For now, BasicTools integration within SampleData covers mainly the input and output of BasicTools mesh objects in/out of SampleData datasets.
The *BasicTools* package offers many I/O functionalities with many classical mesh formats, and mesh creation tools. Their are not currently all integrated within the *SampleData* class. They will be in the future, but for now, you will need to directly use *BasicTools* code to access those possibilities. If you want to do so, feel free to contact us to get help.
Basictools is a required package to use Pymicro, so you should have it installed in your Python environment if you are using Pymicro.
II - Mesh Groups Data Model¶
This section will introduce you to the various data items constituting the Mesh Group data model, and to the Sampledata methods that you can use to retrieve them.
[1]:
# Import the Sample Data class
from pymicro.core.samples import SampleData as SD
[2]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'test_sampledata_ref') # test dataset file path
data = SD(filename=dataset_file)
Let us print the content of the dataset to remember its composition:
[3]:
data.print_index()
data.print_dataset_content(short=True)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : array H5_Path : /test_group/test_array
Name : group H5_Path : /test_group
Name : image H5_Path : /test_image
Name : image_Field_index H5_Path : /test_image/Field_index
Name : image_test_image_field H5_Path : /test_image/test_image_field
Name : mesh H5_Path : /test_mesh
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Printing dataset content with max depth 3
|--GROUP test_group: /test_group (Group)
--NODE test_array: /test_group/test_array (data_array) ( 64.000 Kb)
|--GROUP test_image: /test_image (3DImage)
--NODE Field_index: /test_image/Field_index (string array) ( 63.999 Kb)
--NODE test_image_field: /test_image/test_image_field (field_array) ( 63.867 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string array) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
The test dataset contains one 3DMesh Group, the test_mesh group, with indexname mesh. It has four fields defined on it, Test_field1, Test_field2, Test_field3 and Test_field4.
Let us print more information about this Mesh Group:
[4]:
data.print_node_info('mesh')
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4,
----------------
Like for Image Groups, a lot of metadata is attached to a Mesh Group. All the attribute that you see attached to the test_mesh group are part of the Mesh Group data model, and are automatically created upon creation of Mesh Groups. Like for Image Groups, you can see a group_type attribute, indicating that our group stores data representing a three dimensional mesh.
Mesh Group indexnames¶
Let us print more precisely the index
[5]:
data.print_index(local_root='test_mesh')
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `test_mesh`
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
As you can see, all the elements of the *Mesh Group* data model have an indexname constructed following the same pattern: ``mesh_indexname + _ + item_name``. Their ``item_name`` is also their Name in the dataset as you can see in their Pathes.
For our reference dataset, and for the Mesh Group test_mesh, the mesh_indexname is mesh.
The various elements of this data model an their item_name are all printed above in the print_index output, and will be all presented below.
The Mesh Geometry Group¶
You can see in the output of the print_dataset_content method above that the mesh contains 3 data array nodes, and one Group, which has a lot of childrens. This group, the Geometry Group, contains all the data necessary to define the grid topology. This group item_name is Geometry.
[6]:
data.print_node_info('mesh_Geometry')
data.print_group_content('mesh_Geometry', short=True)
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : ElementsTags, NodeTags, Elem_tag_type_list, Elem_tags_list, Elements, Node_tags_list, Nodes, Nodes_ID,
----------------
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
This Group contains a lot of data arrays, and 2 groups. You see from their names that all of its children are related to the grid Nodes and Elements. We will now explore in details how this essential data is stored.
Mesh Nodes¶
Mesh Nodes define the geometry of the grid. They are defined by an array of coordinates, of size :math:`[N_{nodes},N_{coordinates}]`.
You can see a few cells above that some attributes of the Mesh Group provide information on the mesh Nodes:
number_of_nodes: \(N_{nodes}\), the number of nodes in the gridnodes_path: path of a data array containing the node coordinates arraynodesID_path: path of a data array containing Identification numbers for each node of the previous array
As you can see, the node coordinates and identification number arrays are stored in the Geometry Group. These two data items have respectively the item name Nodes and Nodes_ID. We can print more information about them:
[7]:
data.print_node_info('mesh_Nodes')
data.print_node_info('mesh_Nodes_ID')
NODE: /test_mesh/Geometry/Nodes
====================
-- Parent Group : Geometry
-- Node name : Nodes
-- Nodes attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes (CArray(6, 3)) 'mesh_Nodes'
-- Compression options for node `mesh_Nodes`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
NODE: /test_mesh/Geometry/Nodes_ID
====================
-- Parent Group : Geometry
-- Node name : Nodes_ID
-- Nodes_ID attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/Nodes_ID (CArray(6,)) 'mesh_Nodes_ID'
-- Compression options for node `mesh_Nodes_ID`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
Nodes and Nodes ID arrays¶
You see that they are simple Data array items, with no specific metadata. You can see that the shape of the Nodes array is \([N_{nodes}, N_{coordinates}]\), which is \([6,3]\) in this case. This is consistent with the dimensionality of the mesh, which is 3 (tridimensional mesh). The Nodes_Id array has one value per nodes, which is also consistent. Let us print the content of those arrays:
[8]:
mesh_nodes = data['mesh_Nodes']
mesh_nodes_id = data['mesh_Nodes_ID']
for i in range(len(mesh_nodes_id)):
print(f'Mesh node number {mesh_nodes_id[i]} coordinates: {mesh_nodes[i,:]}')
Mesh node number 0 coordinates: [-1. -1. 0.]
Mesh node number 1 coordinates: [-1. 1. 0.]
Mesh node number 2 coordinates: [1. 1. 0.]
Mesh node number 3 coordinates: [ 1. -1. 0.]
Mesh node number 4 coordinates: [0. 0. 1.]
Mesh node number 5 coordinates: [ 0. 0. -1.]
If you see that the 4 first nodes form a square in the \((X,Y)\) plane, and that two other nodes are symetrically located on top and bottom of the center of this square. They are the vertexes of a regular octahedron, which we will verify later by visualizing the mesh.
To get those arrays, the most convenient solution is to use the get_mesh_nodes and get_mesh_nodesID methods:
[9]:
mesh_nodes = data.get_mesh_nodes('mesh')
mesh_nodes_id = data.get_mesh_nodesID('mesh')
for i in range(len(mesh_nodes_id)):
print(f'Mesh node number {mesh_nodes_id[i]} coordinates: {mesh_nodes[i,:]}')
Mesh node number 0 coordinates: [-1. -1. 0.]
Mesh node number 1 coordinates: [-1. 1. 0.]
Mesh node number 2 coordinates: [1. 1. 0.]
Mesh node number 3 coordinates: [ 1. -1. 0.]
Mesh node number 4 coordinates: [0. 0. 1.]
Mesh node number 5 coordinates: [ 0. 0. -1.]
Node Sets/Tags¶
You can also see that the Geometry contains a String Array labelled Nodes_tag_list, with item_name NodeTagsList. This group has the following content:
[10]:
data.print_node_info('mesh_NodeTagsList')
NODE: /test_mesh/Geometry/Node_tags_list
====================
-- Parent Group : Geometry
-- Node name : Node_tags_list
-- Node_tags_list attributes :
* node_type : string array
-- content : /test_mesh/Geometry/Node_tags_list (EArray(2,)) ''
-- Compression options for node `mesh_NodeTagsList`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (257,)
-- Node memory size : 63.999 Kb
----------------
You see that this node is a String array, containing 2 elements. They are the names of the NODE SETS, or also NODE TAGS (notation used in SampleData’s code), defined on this mesh, and stored on the dataset. You can access them easily (see String Array section in tutorial 2):
[11]:
for tag in data['mesh_NodeTagsList']:
print(tag.decode('utf-8'))
Z0_plane
out_of_plane
An easier way to get this information is to use the get_mesh_node_tags method:
[12]:
data.get_mesh_node_tags_names('test_mesh')
[12]:
['Z0_plane', 'out_of_plane']
The two node tags are names Z0_plane, and out_of_plane. From the value of the node coordinates printed a few cells above, we can expect that the Node Set Z0_plane contains the 4 first nodes, and that out_of_plane set the two last nodes.
To see their content, you can use the dedicated methods get_mesh_node_tag and get_mesh_node_tag_coordinates:
[13]:
# Get the node IDs for the Node sets defined on the mesh
Z0_plane = data.get_mesh_node_tag(meshname='test_mesh',node_tag='Z0_plane')
out_of_plane = data.get_mesh_node_tag(meshname='test_mesh',node_tag='out_of_plane')
# Get the node coordinates for the Node sets defined on the mesh
Z0_plane_xyz = data.get_mesh_node_tag_coordinates(meshname='test_mesh',node_tag='Z0_plane')
out_of_plane_xyz = data.get_mesh_node_tag_coordinates(meshname='test_mesh',node_tag='out_of_plane')
# Print informations
print(f'Node tag "Z0_plane" is composed of nodes {Z0_plane} withi coordinates \n {Z0_plane_xyz}','\n')
print(f'Node tag "out_of_plane" is composed of nodes {out_of_plane} withi coordinates \n {out_of_plane_xyz}')
Node tag "Z0_plane" is composed of nodes [0 1 2 3] withi coordinates
[[-1. -1. 0.]
[-1. 1. 0.]
[ 1. 1. 0.]
[ 1. -1. 0.]]
Node tag "out_of_plane" is composed of nodes [4 5] withi coordinates
[[ 0. 0. 1.]
[ 0. 0. -1.]]
The Node Tags are stored in the Group NodeTags, children of the Group Geometry:
[14]:
data.print_node_info('mesh_NodeTags')
data.print_group_content('mesh_NodeTags')
GROUP NodeTags
=====================
-- Parent Group : Geometry
-- Group attributes :
* group_type : Group
-- Childrens : NT_Z0_plane, NT_out_of_plane, field_Z0_plane, field_out_of_plane,
----------------
****** Group mesh_NodeTags CONTENT ******
****** Group mesh_NodeTags CONTENT ******
NODE: /test_mesh/Geometry/NodeTags/NT_Z0_plane
====================
-- Parent Group : NodeTags
-- Node name : NT_Z0_plane
-- NT_Z0_plane attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/NodeTags/NT_Z0_plane (CArray(4,)) 'NT_Z0_plane'
-- Compression options for node `/test_mesh/Geometry/NodeTags/NT_Z0_plane`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Geometry/NodeTags/NT_out_of_plane
====================
-- Parent Group : NodeTags
-- Node name : NT_out_of_plane
-- NT_out_of_plane attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/NodeTags/NT_out_of_plane (CArray(2,)) 'NT_out_of_plane'
-- Compression options for node `/test_mesh/Geometry/NodeTags/NT_out_of_plane`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Geometry/NodeTags/field_Z0_plane
====================
-- Parent Group : NodeTags
-- Node name : field_Z0_plane
-- field_Z0_plane attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : field_Z0_plane
* xdmf_gridname : test_mesh
-- content : /test_mesh/Geometry/NodeTags/field_Z0_plane (CArray(6, 1), shuffle, zlib(1)) 'mesh_field_Z0_plane'
-- Compression options for node `/test_mesh/Geometry/NodeTags/field_Z0_plane`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (65536, 1)
-- Node memory size : 310.000 bytes
----------------
NODE: /test_mesh/Geometry/NodeTags/field_out_of_plane
====================
-- Parent Group : NodeTags
-- Node name : field_out_of_plane
-- field_out_of_plane attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : field_out_of_plane
* xdmf_gridname : test_mesh
-- content : /test_mesh/Geometry/NodeTags/field_out_of_plane (CArray(6, 1), shuffle, zlib(1)) 'mesh_field_out_of_plane'
-- Compression options for node `/test_mesh/Geometry/NodeTags/field_out_of_plane`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (65536, 1)
-- Node memory size : 312.000 bytes
----------------
You can observe that the the group contains four data items, two for each node set in our mesh group, as their name indicates.
Two of those data items have a name that starts with ``NT_``, which stand for NodeTag. They contain the data items associated to the node tags, that can be retrieved with the get_mesh_node_tag method (see above).
The two other data item have a name that starts with ``_field``. They are Field Arrays data items. These fields are nodal value fields (see below the section dedicated to Mesh Fields), and have a value of 1 on the node set, and 0 out of the node set.
We have now covered all the elements of the Mesh Group data model related to mesh nodes.
Mesh Elements¶
Mesh Elements define the geometry of the geometrical domain represented by the grid. They are polylines, polygons or polyhedra whose vertices are mesh Nodes. They are defined by a connectivity array indicating for each element the IDs of the nodes defining the element..
Let us print again the Mesh Group information to find out which data items relate to mesh elements:
[15]:
data.print_node_info('mesh')
GROUP test_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [0]
* Number_of_boundary_elements : [0]
* Number_of_bulk_elements : [8]
* Number_of_elements : [8]
* Topology : Uniform
* Xdmf_elements_code : ['Triangle']
* description :
* element_type : ['tri3']
* elements_path : /test_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /test_mesh/Geometry/Nodes_ID
* nodes_path : /test_mesh/Geometry/Nodes
* number_of_nodes : 6
* xdmf_gridname : test_mesh
-- Childrens : Geometry, Field_index, Test_field1, Test_field2, Test_field3, Test_field4,
----------------
As shown by this print, many Mesh Group attributes are related to Mesh Elements.
The elements_path attribute indicates the path of the Data Array item containing the elements connectivity array.
The Topology attribute can have two values, Uniform (mesh composed of only one type of elements) or Mixed (mesh composed of several types of elements). Here the mesh is uniform. You will see an example of Mixed topology in the next section of this tutorial. All items of the Mesh Group data Model related to mesh elements will be reviewed again for the Mixed topology case.
Element types¶
You can see 6 attributes that store list values. All those have one value for each type of element that composes the mesh. Here, as the mesh has a Uniform topology, they have only one value. Two of those attributes provide information on the type of elements:
``elements_type``: a list of the types of elements that is used in the mesh. These type names consist of a geometrical prefix indicating the geometrical nature of the element, followed by a number indicating the number of nodes in the element. Elements types are stored by decreasding dimensionality, in the case of a Mixed topology: the first elements of the lists are the elements types with the higher dimensionality in the mesh (bulk element types), and then element types with a lower dimensionality (boundary element types) are stored. The different type of elements supported, and their dimensionality, are:
Element type
Description
point1A simple point
bar2/bar3A linear/quadratic line segment element (1D)
tri3andtri6A linear/quadratic triangle element (2D)
quad4andquad8A linear/quadratic quadrilateral element (2D)
tet4andtet10A linear/quadratic tetrahedron element (3D)
pyr5andpyr13A linear/quadratic pyramid (square basis) element (3D)
wed6andwed15A linear/quadratic wedge element (3D)
hex8andhex20A linear/quadratic hexaedron element (3D)
``Xdmf_elements_code``: same as
elements_typebut with the element type names of the XDMF data model, that can be found here (used for synchronization with XDMF file)
Note
You will find the geometric definition of these nodes into the BasicTools/Containers/ElementNames.py file of the BasicTools package. They are not presented here for the sake of brevity.
Elements count¶
Other attributes are used to keep track of the number of elements: Number_of_elements, Number_of_bulk_elements Number_of_boundary_elements. They contain integer values indicating respectively how many elements, bulk elements (i.e. that have the same dimensionality of the mesh, 3D elements for 3D mesh for instance) or boundary elements (elements with a lower dimensionality than the mesh, 2D elements for a 3D mesh for instance) compose the mesh.
These three attributes contain lists, that have a value for each element type of the list, and their order correspond to the element types stored in the elements_type attribute list.
The Elements_offset attribute also is a list with one element per element in elements_type. It contains the position of the first element of each type of element in the complete set of mesh elements. It is used by the class for consistency when interacting with BasicTools Elements objects. It is not relevant for SampleData users.
Using all this information, we can determine that our mesh contains 8 elements, that are linear triangle elements (tri3 elements). They are the 8 faces of our octahedron. A quicker way to retrieve this is to use the get_mesh_elem_types_and_number method, that returns a dictionary whose keys are the element type in the mesh, and values are the number of element for each type:
[16]:
data.get_mesh_elem_types_and_number('mesh')
[16]:
{'tri3': 8}
Elements connectivity array¶
Like for the Mesh Nodes, all the data describing the Mesh elements is also gathered in the Mesh Geometry Group. Let us print again the content of this group:
[17]:
data.print_group_content('mesh_Geometry', short=True)
data.print_index(local_root='mesh_Geometry')
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `mesh_Geometry`
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Like the nodes coordinate array, the element connectivity array is a Data Array node stored into the Mesh Geometry Group. Its item name is Elements. Let us print the content of this array:
[18]:
print(f'The shape of the mesh Elements connectivity array is {data["mesh_Elements"].shape}','\n')
print(f'Its content is {data["mesh_Elements"]}')
The shape of the mesh Elements connectivity array is (24,)
Its content is [0 1 4 0 1 5 1 2 4 1 2 5 2 3 4 2 3 5 3 0 4 3 0 5]
This array contains for each element the IDs of the Nodes that compose the element : 24 nodes IDS, three for each triangle. We can see that the first element of the mesh is composed of the Nodes 0, 1 and 4.
The connectivity array is used as grid Topology array in the XDMF file of the dataset. Hence, it must comply with the XDMF format. For this reason, it is always stored as a 1D array. This format is detailed on the dedicated XDMF data format webpage, `here <https://www.xdmf.org/index.php/XDMF_Model_and_Format#Topology>`__.
In the case of a Uniform topology, the connectivities of the elements are stored consecutively. As the number of nodes of each element is identical, the interpretation of the array is possible. For the Mixed topology case (see example later in this tutorial), the connectivity of each element must be preceded by a number that identifies the element type. This number is the XMDF code for the element type.
The SampleData class offers more convenient ways to get the elements connectivity array. The first is the get_mesh_xdmf_connectivity method, that return the Elements Data array:
[19]:
print(f'The mesh elements connectivity array is {data.get_mesh_xdmf_connectivity("mesh")}')
The mesh elements connectivity array is [0 1 4 0 1 5 1 2 4 1 2 5 2 3 4 2 3 5 3 0 4 3 0 5]
The second is the get_mesh_elements method. With the meshname and one if its element types as arguments, it allows to get the connectivity of a type of elements with a classical 2D array format:
[20]:
data.get_mesh_elements(meshname='mesh', get_eltype_connectivity='tri3')
[20]:
array([[0, 1, 4],
[0, 1, 5],
[1, 2, 4],
[1, 2, 5],
[2, 3, 4],
[2, 3, 5],
[3, 0, 4],
[3, 0, 5]])
Elements Sets/Tags¶
Like Node Tags, Mesh Group can store Element Tags (= element sets) in the Mesh Geometry Group), that are stored in the ElementsTags Group, and listed in the ElTagsList String array. A second string array is associated to element tags: the ElTagsTypeList that contains the type of element constituting each tag (element tags can only be composed of a single type of elements). Like for node tags, they are explicit methods that allow to retrieve information on element tags:
[21]:
data.get_mesh_elem_tags_names('mesh')
[21]:
{'2D': 'tri3', 'Top': 'tri3', 'Bottom': 'tri3'}
Our Mesh Group contains 3 element tags, all containing tri3 elements. The elements IDs and the specific connectivity of an element tag can easily be accessed using their name and the Mesh Group name:
[22]:
el_tag = data.get_mesh_elem_tag(meshname='mesh', element_tag='2D')
el_tag_connect = data.get_mesh_elem_tag_connectivity(meshname='mesh', element_tag='Bottom')
# el_tag_connect = data.get_mesh_elem_tags_connectivity(meshname='mesh', element_tag='2D')
print(f'The element tag `2D` contains the elements: {el_tag}.')
print(f'The connectivity of the `Bottom` element tag is: \n {el_tag_connect}')
The element tag `2D` contains the elements: [0 1 2 3 4 5 6 7].
The connectivity of the `Bottom` element tag is:
[[0 1 5]
[1 2 5]
[2 3 5]
[3 0 5]]
You can observe that the get_mesh_elem_tag_connectivity reshapes the connectivity array to a more usual 2D format, each row of the array corresponding to the connectivity of 1 element.
Like for Node Tags, the arrays containing the Element Tags data are stored in a specific group ElementsTags, subgroup of the Mesh Geometry Group. They contain the ID numbers of the elements constituting the Element Tag, and are named following the pattern : ET_+tag name (ET standing for element tag).
Like for Nodes, Mesh Groups can also store Mesh fields for Element Tags to allow to visualize them. These fields have one boolean value per mesh element, which is equal to 1 on elements that are part of the Element Tag. The associated data arrays are also stored in the ElementsTags Group, and are named following the pattern : field_+tag name.
Let us look at the content of this Group:
[23]:
data.print_group_content('mesh_ElemTags')
****** Group mesh_ElemTags CONTENT ******
****** Group mesh_ElemTags CONTENT ******
NODE: /test_mesh/Geometry/ElementsTags/ET_2D
====================
-- Parent Group : ElementsTags
-- Node name : ET_2D
-- ET_2D attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/ElementsTags/ET_2D (CArray(8,)) 'ET_2D'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/ET_2D`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Geometry/ElementsTags/ET_Bottom
====================
-- Parent Group : ElementsTags
-- Node name : ET_Bottom
-- ET_Bottom attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/ElementsTags/ET_Bottom (CArray(4,)) 'ET_Bottom'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/ET_Bottom`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Geometry/ElementsTags/ET_Top
====================
-- Parent Group : ElementsTags
-- Node name : ET_Top
-- ET_Top attributes :
* empty : False
* node_type : data_array
-- content : /test_mesh/Geometry/ElementsTags/ET_Top (CArray(4,)) 'ET_Top'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/ET_Top`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192,)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Geometry/ElementsTags/field_2D
====================
-- Parent Group : ElementsTags
-- Node name : field_2D
-- field_2D attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : field_2D
* xdmf_gridname : test_mesh
-- content : /test_mesh/Geometry/ElementsTags/field_2D (CArray(8, 1), shuffle, zlib(1)) 'mesh_field_2D'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/field_2D`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 325.000 bytes
----------------
NODE: /test_mesh/Geometry/ElementsTags/field_Bottom
====================
-- Parent Group : ElementsTags
-- Node name : field_Bottom
-- field_Bottom attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : field_Bottom
* xdmf_gridname : test_mesh
-- content : /test_mesh/Geometry/ElementsTags/field_Bottom (CArray(8, 1), shuffle, zlib(1)) 'mesh_field_Bottom'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/field_Bottom`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 328.000 bytes
----------------
NODE: /test_mesh/Geometry/ElementsTags/field_Top
====================
-- Parent Group : ElementsTags
-- Node name : field_Top
-- field_Top attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : field_Top
* xdmf_gridname : test_mesh
-- content : /test_mesh/Geometry/ElementsTags/field_Top (CArray(8, 1), shuffle, zlib(1)) 'mesh_field_Top'
-- Compression options for node `/test_mesh/Geometry/ElementsTags/field_Top`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 327.000 bytes
----------------
We see that our 3 element tags have a field associated in the Mesh Group.
Mesh fields¶
Like Image Groups, Mesh Groups can store fields. The list of the names of fields defined on the mesh group is stored in the Field_Index string array, like for Image Groups ( see previous tutorial). It can be accessed easily using the get_grid_field_list method:
[24]:
data.get_grid_field_list('mesh')
[24]:
['mesh_field_Z0_plane',
'mesh_field_out_of_plane',
'mesh_field_2D',
'mesh_field_Top',
'mesh_field_Bottom',
'mesh_Test_field1',
'mesh_Test_field2',
'mesh_Test_field3',
'mesh_Test_field4']
We see in this list the various Node/Element tag fields, but also additional fields called mesh_test_fieldX. A specific section is dedicated to Mesh fields in this tutorial, so no additional detail on them will be provided here.
Mesh XDMF grids¶
We have now seen all the data contained in a Mesh Group in the HDF5 dataset of a SampleData object. To conclude this section on the Mesh Group data model, we will observe how this group is stored in the XDMF dataset:
[25]:
data.print_xdmf()
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd">
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_mesh" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_sampledata_ref.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_sampledata_ref.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
</Grid>
<Grid Name="test_image" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions="10 10 10"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">-1. -1. -1.</DataItem>
<DataItem Format="XML" Dimensions="3">0.2 0.2 0.2</DataItem>
</Geometry>
<Attribute Name="test_image_field" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="10 10 10" NumberType="Int" Precision="16">test_sampledata_ref.h5:/test_image/test_image_field</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
You can see that the test_mesh is stored as a Grid Node. This node contains a Geometry node, defined by a data array, which is the Node coordinate data array stored in the HDF5 dataset. It also contains a Topology Group, defined by the Elements connectivity data array stored in the HDF5 dataset. Finally, it contains one Attribute node for each field listed in the Field Index string array of the HDF5 dataset, defined by the associated data array in the HDF5 dataset.
As already explained in the tutorial 1 (sec. V), the XDMF file allows to directly visualize the data described in it with the Paraview software. Before closing our reference dataset, Let us try to visualize the Mesh Group that we have studied, with the pause_for_visualization method and the Paraview software.
You have to choose the XdmfReader to open the file, so that Paraview may properly read the data. In the Property panel, tick only the test_mesh block to only plot the content of the Mesh Group, and then click on the Apply button.
[26]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')
You should be able to produce visualization of the mesh geometry:
or the mesh fields:
Before moving to the next section on the creation of Mesh Groups, let us close our reference dataset:
[27]:
del data
III - Creating Mesh Groups¶
Like for Image Groups (see tutorial 4 section III), Mesh Groups are created in SampleData instances through Basictools mesh objects. Creating meshes is a complex geometrical and mathematical task, that require specific tools (meshers like gmsh for instance). The SampleData code has not been designed to create meshes, but rather to manipulate them. In relevant application of the class, mesh data stored into SampleData datasets come from external tools. The main tasks are then to load and manipulate the data. This tutorial will mainly focus on those aspect, rather than mesh creation.
Hence, in this section, we will only review some quick ways to create mesh objects, from simple mesh data, mesh files or other Python packages; and how to add them to a SampleData dataset.
Creating mesh objects with BasicTools¶
The BasicTools package implements a few methods that allow to create meshes with simple topologies. We will rely on them to recreate the mesh that we studied before, and introduce you to mesh objects.
BasicTools mesh creation tools¶
The mesh creation tools of the BasicTools package can be imported with the following code line:
[28]:
import BasicTools.Containers.UnstructuredMeshCreationTools as UMCT
This module contains functions that allow, from a set of points and a connectivity matrix, to create uniform meshes of one type of element. Learning in details how to use them is beyond the scope of this tutorial. Likewise, how to add data such as tags or fields to a mesh object will not be detailed in this tutorial. The reader is refered to BasicTools code to learn more on those methods/objects.
These method include:
CreateUniformMeshOfBarsCreateMeshOfTrianglesCreateMeshOfCreateSquareCreateCubeCreateMeshFromConstantRectilinearMesh
The UnstructuredMeshCreationTools contains test methods for each of these methods that can be read to understand how to use them.
To recreate the octahedron mesh of the reference dataset studied in section II, we will use the CreateMeshOfTriangles method. For that, we will need to create two arrays, one for the node coordinates, one for the elements connectivity, and simply pass them to the BasicTools method:
[29]:
import numpy as np
# Mesh node coordinates array
mesh_nodes = np.array([[-1.,-1., 0.],
[-1., 1., 0.],
[ 1., 1., 0.],
[ 1.,-1., 0.],
[ 0., 0., 1.],
[ 0., 0.,-1.]])
# Mesh connectivity array
mesh_elements = np.array([[0, 1, 4],
[0, 1, 5],
[1, 2, 4],
[1, 2, 5],
[2, 3, 4],
[2, 3, 5],
[3, 0, 4],
[3, 0, 5]])
mesh = UMCT.CreateMeshOfTriangles(mesh_nodes, mesh_elements)
Mesh objects¶
The method return a BasicTools unstructured mesh object, a class dedicated to the manipulation of finite element meshes data. Printing this object provides information on the content of the mesh:
[30]:
print(mesh)
UnstructuredMesh
Number Of Nodes : 6
Tags :
Number Of Elements : 8
ElementsContainer, Type : (tri3,8), Tags : (2D:8)
We will now complete this mesh object with additional data, by defining fields and node/element tags:
[31]:
# Here we recreate the Node tags of the reference dataset `test_mesh`
mesh.nodesTags.CreateTag('Z0_plane', False).SetIds([0,1,2,3])
mesh.nodesTags.CreateTag('out_of_plane', False).SetIds([4,5])
# Here we recreate the Element tags of the reference dataset `test_mesh`
mesh.GetElementsOfType('tri3').GetTag('Top').SetIds([0,2,4,6])
mesh.GetElementsOfType('tri3').GetTag('Bottom').SetIds([1,3,5,7])
# Here we add mesh node fields
mesh.nodeFields['Test_field1'] = np.array([0., 0., 0., 0., 1., 0.])
mesh.nodeFields['Test_field2'] = np.array([0., 0., 0., 0., 0., 1.])
# Here we add mesh element fields
mesh.elemFields['Test_field3'] = np.array([0., 1., 2., 3., 4., 5., 6., 7.])
mesh.elemFields['Test_field4'] = np.array([1., 1., -1., -1., 1., 1., -1., -1.])
[32]:
print(mesh)
UnstructuredMesh
Number Of Nodes : 6
Tags : (Z0_plane:4) (out_of_plane:2)
Number Of Elements : 8
ElementsContainer, Type : (tri3,8), Tags : (2D:8) (Top:4) (Bottom:4)
nodeFields : ['Test_field1', 'Test_field2']
elemFields : ['Test_field3', 'Test_field4']
Creating a Mesh Group from a Meshobject¶
Our mesh object now contain all the data that is stored into the test_mesh Group of the reference dataset. We can now create a new Sampledata instance to see how we can create a MeshGroup from this mesh object:
[33]:
data = SD(filename='test_mesh_dataset', overwrite_hdf5=True, autodelete=True)
To add a Mesh Group into this dataset, use the add_mesh method. Its use is very straightforward, you just have to provide the name, indexname, location of the Mesh Group you want to create, and the mesh object from which the data must be loaded. Like all the data item creation methods, it also has a replace argument (see tutorial 3).
This methods has one specific argument, bin_fields_from_sets, that allow you to control if you want to simply store the Node/Element Tags as data arrays (in this case set it to False), or if you want to also store visualization fields for each Node/Element Tag in the mesh object.
Let us create our mesh group, with visualization fields:
[34]:
data.add_mesh(mesh_object=mesh, meshname='test_mesh', indexname='mesh', location='/', bin_fields_from_sets=True)
[34]:
<BasicTools.Containers.UnstructuredMesh.UnstructuredMesh at 0x7fc308414b90>
[35]:
print(data)
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : mesh H5_Path : /test_mesh
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Printing dataset content with max depth 3
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
That is it ! The Mesh Group has been created and all data has been stored into the SampleData dataset. Let us try to retrieve some information to check the loaded data:
[36]:
data.print_node_info('mesh_Geometry')
print('Mesh elements: ',data.get_mesh_elem_types_and_number('mesh'))
print('Mesh node tags: ', data.get_mesh_node_tags_names('mesh'))
print('Mesh element tags:',data.get_mesh_elem_tags_names('mesh'))
print('Element Tag "Bottom" connectivity: \n',data.get_mesh_elem_tag_connectivity('mesh', 'Bottom'))
GROUP Geometry
=====================
-- Parent Group : test_mesh
-- Group attributes :
* group_type : Group
-- Childrens : Nodes, Nodes_ID, Elements, NodeTags, Node_tags_list, ElementsTags, Elem_tags_list, Elem_tag_type_list,
----------------
Mesh elements: {'tri3': 8}
Mesh node tags: ['Z0_plane', 'out_of_plane']
Mesh element tags: {'2D': 'tri3', 'Top': 'tri3', 'Bottom': 'tri3'}
Element Tag "Bottom" connectivity:
[[0 1 5]
[1 2 5]
[2 3 5]
[3 0 5]]
Creating meshes from files¶
In practice, creating relevant meshes for mechanical or material science application through BasicTools mesh creation tools is very limited. The most relevant use case consist in loading meshes from mesh files created by finite element or meshing softwares. Through Basictools, the SampleData class allows to directly load Mesh Groups from various mesh file formats, and the opposite: write mesh files in various formats from Mesh Groups.
Load mesh files¶
We will use a mesh file used for Pymicro unit tests to illustrate the mesh file loading functionality. This file is a .geof file, the mesh file format used by the Zset finite element software.
To load a mesh file from into a SampleData mesh group, use the add_mesh method, but instead of providing a mesh object, use the file argument and provide the mesh file name:
[37]:
meshfile_name = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'cube_ref.geof')
print(meshfile_name)
data.add_mesh(file=meshfile_name, meshname='loaded_geof_mesh', indexname='mesh_geof', location='/', bin_fields_from_sets=True)
print(data)
/home/amarano/Codes/pymicro/examples/data/cube_ref.geof
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : mesh H5_Path : /test_mesh
Name : mesh_Geometry H5_Path : /test_mesh/Geometry
Name : mesh_Nodes H5_Path : /test_mesh/Geometry/Nodes
Name : mesh_Nodes_ID H5_Path : /test_mesh/Geometry/Nodes_ID
Name : mesh_Elements H5_Path : /test_mesh/Geometry/Elements
Name : mesh_NodeTags H5_Path : /test_mesh/Geometry/NodeTags
Name : mesh_Field_index H5_Path : /test_mesh/Field_index
Name : mesh_NodeTagsList H5_Path : /test_mesh/Geometry/Node_tags_list
Name : mesh_ElemTags H5_Path : /test_mesh/Geometry/ElementsTags
Name : mesh_ElTagsList H5_Path : /test_mesh/Geometry/Elem_tags_list
Name : mesh_ElTagsTypeList H5_Path : /test_mesh/Geometry/Elem_tag_type_list
Name : mesh_Test_field1 H5_Path : /test_mesh/Test_field1
Name : mesh_Test_field2 H5_Path : /test_mesh/Test_field2
Name : mesh_Test_field3 H5_Path : /test_mesh/Test_field3
Name : mesh_Test_field4 H5_Path : /test_mesh/Test_field4
Name : mesh_geof H5_Path : /loaded_geof_mesh
Name : mesh_geof_Geometry H5_Path : /loaded_geof_mesh/Geometry
Name : mesh_geof_Nodes H5_Path : /loaded_geof_mesh/Geometry/Nodes
Name : mesh_geof_Nodes_ID H5_Path : /loaded_geof_mesh/Geometry/Nodes_ID
Name : mesh_geof_Elements H5_Path : /loaded_geof_mesh/Geometry/Elements
Name : mesh_geof_NodeTags H5_Path : /loaded_geof_mesh/Geometry/NodeTags
Name : mesh_geof_Field_index H5_Path : /loaded_geof_mesh/Field_index
Name : mesh_geof_NodeTagsList H5_Path : /loaded_geof_mesh/Geometry/Node_tags_list
Name : mesh_geof_ElemTags H5_Path : /loaded_geof_mesh/Geometry/ElementsTags
Name : mesh_geof_ElTagsList H5_Path : /loaded_geof_mesh/Geometry/Elem_tags_list
Name : mesh_geof_ElTagsTypeList H5_Path : /loaded_geof_mesh/Geometry/Elem_tag_type_list
Printing dataset content with max depth 3
|--GROUP loaded_geof_mesh: /loaded_geof_mesh (3DMesh)
--NODE Field_index: /loaded_geof_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /loaded_geof_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /loaded_geof_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /loaded_geof_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /loaded_geof_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /loaded_geof_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /loaded_geof_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /loaded_geof_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /loaded_geof_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /loaded_geof_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
|--GROUP test_mesh: /test_mesh (3DMesh)
--NODE Field_index: /test_mesh/Field_index (string_array - empty) ( 63.999 Kb)
|--GROUP Geometry: /test_mesh/Geometry (Group)
--NODE Elem_tag_type_list: /test_mesh/Geometry/Elem_tag_type_list (string_array) ( 63.999 Kb)
--NODE Elem_tags_list: /test_mesh/Geometry/Elem_tags_list (string_array) ( 63.999 Kb)
--NODE Elements: /test_mesh/Geometry/Elements (data_array) ( 64.000 Kb)
|--GROUP ElementsTags: /test_mesh/Geometry/ElementsTags (Group)
|--GROUP NodeTags: /test_mesh/Geometry/NodeTags (Group)
--NODE Node_tags_list: /test_mesh/Geometry/Node_tags_list (string_array) ( 63.999 Kb)
--NODE Nodes: /test_mesh/Geometry/Nodes (data_array) ( 63.984 Kb)
--NODE Nodes_ID: /test_mesh/Geometry/Nodes_ID (data_array) ( 64.000 Kb)
--NODE Test_field1: /test_mesh/Test_field1 (field_array) ( 64.000 Kb)
--NODE Test_field2: /test_mesh/Test_field2 (field_array) ( 64.000 Kb)
--NODE Test_field3: /test_mesh/Test_field3 (field_array) ( 64.000 Kb)
--NODE Test_field4: /test_mesh/Test_field4 (field_array) ( 64.000 Kb)
The Mesh Group has been loaded as a Mesh Group in the dataset. Its data can be explored, and visualized, as presented above:
[38]:
data.print_node_info('mesh_geof')
print('Mesh elements: ',data.get_mesh_elem_types_and_number('mesh_geof'))
print('Mesh node tags: ', data.get_mesh_node_tags_names('mesh_geof'))
print('Mesh element tags:',data.get_mesh_elem_tags_names('mesh_geof'))
GROUP loaded_geof_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [ 0 384]
* Number_of_boundary_elements : [384]
* Number_of_bulk_elements : [384]
* Number_of_elements : [384 384]
* Topology : Mixed
* Xdmf_elements_code : [6, 4]
* description :
* element_type : ['tet4', 'tri3']
* elements_path : /loaded_geof_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /loaded_geof_mesh/Geometry/Nodes_ID
* nodes_path : /loaded_geof_mesh/Geometry/Nodes
* number_of_nodes : 125
* xdmf_gridname : loaded_geof_mesh
-- Childrens : Geometry, Field_index,
----------------
Mesh elements: {'tet4': 384, 'tri3': 384}
Mesh node tags: ['x0y0z0', 'x1y0z0', 'x0y1z0', 'x1y1z0', 'x0y0z1', 'x1y0z1', 'x0y1z1', 'x1y1z1']
Mesh element tags: {'3D': 'tet4', 'X0': 'tri3', 'Y0': 'tri3', 'Skin': 'tri3', 'X1': 'tri3', 'Y1': 'tri3', 'Z0': 'tri3', 'Z1': 'tri3'}
As you can see, this mesh has a mixed topology, and has bulk (linear tetrahedra tet4) and boundary (linear triangles tri3) elements. This is the mesh of a cube, and it contains element tags to define its faces (X0, X1, Y0 …). As a training exercise, you may try to explore and visualize all the data contained into this mesh group !
This method will in principle work with the following files extensions :
.geof .inp, .asc, .ansys, .geo, .msh, .mesh, .meshb, .sol, .solb, .gcode, .fem, .stl, .xdmf, .pxdmf, .PIPE, .odb, .ut, .utp, .vtu, .dat, .datt
Note: MeshIO Bridge
If you use the meshio package to handle meshes, you can interface it with SampleData thanks to the BasicTools package. For that, you can use the MeshToMeshIO and MeshIOToMesh methods in the BasicTools.Containers.MeshIOBridge module of the Basictools package. These methods allow you to transform a Basictools mesh object into a MeshIO object. Hence, you can load a meshio mesh into a SampleData dataset by transforming it into a Basictools mesh object, and feeding it to
the add_mesh method.
IV - Mesh Field data model¶
In this new section, we will now focus on the definition of mesh fields data items. They are very similar to image fields data items, presented in the previous tutorial, section IV. They are data arrays enriched with metadata describing how they relate to the mesh they are associated to. Like image fields, these data arrays must follow some shape, ordering and indexing conventions to be added as mesh fields.
SampleData Mesh fields conventions¶
Possible shapes and field types¶
In the following, \(N_n\), \(N_e\) and \(N_{ip}\) represent respectively the number of nodes, elements and intergration points of the mesh. Mesh fields can be added from Numpy array that can have the following shapes:
\((N_n, N_c)\) or \((N_n)\) (equivalent to \(N_c=1\)). In this case, the field is a nodal field, whose values are defined at the mesh nodes
\((N_e, N_c)\) or \((N_e)\) (equivalent to \(N_c=1\)). In this case, the field is an element field, whose values are defined on each mesh element (constant per element)
\((N_{ip},N_c)\) or \((N_{ip})\) (equivalent to \(N_c=1\)). In this case, the field is an integration point field, whose values are defined at the mesh integration points. In practice, \(N_{ip}\) can be any multiple of \(N_e\), \(N_{ip} = k*N_e\)
If an array that do not have a compatible shape is added as mesh field, the class will raise an exception.
Field padding¶
Finite element softwares may output fields only on bulk elements or boundary elements (variable flux, pressure field…) depending on the field nature. If such field must be loaded into a SampleData Mesh Group that has a mixed topology, with bulk and boundary elements, an element or integration point field array outputed by the software will not have a number of values compliant with the convention presented in the subsection above.
The SampleData class handles automatically this particular issue. Element and integration point fields may be added from field data that are defined only on bulk or boundary elements. The class will padd them with zeros values for the element over which they are not defined, to get the proper array size.
In practice, to add element fields / integration point fields, the input array can have the shape \((N_e, N_c)\) or \((N_e)\) / \((N_{ip}, N_c)\) or \((N_{ip})\), with \(N_{ip} = k*N_e\) and \(N_e\) being either the total number of elements in the mesh, the number of bulk elements in the mesh, or the number of boundary elements in the mesh.
Integration points ordering convention and visualization¶
For each component of the field, integration point field array values are ordered by increasing integration point index first, and then by increasing element index, as follows:
\(Array[:,c] = [A_{ip1/e1}, A_{ip2/e1}, ..., A_{ip1/e2}, ...,A_{ipM/eN} ]\)
where \(A_{ipi/ej}\) denotes the value of the array \(A\) for the integration point \(i\) of the element \(j\), and \(M\) and \(N\) are respectively the number of integration point per element and the number of elements in the mesh.
The XDMF format does not allow to specify integration points values for grids, and hence, these values cannot be directly visualized with Paraview and SampleData dataset yet. However, when adding a an integration point field to a dataset, the user may chose to associate to it a visualization field that is an element field (1 value per element), which is the minimum, maximum or mean of the field value on all element integration points (see next section).
Field dimensionality¶
The size of the last dimension of the array determines the number of field components \(N_c\), which defines the field dimensionality. If the array is 1D, then \(N_c=1\).
SampleData will interpret the field dimensionality (scalar, vector, or tensor) depending on the grid topology and the array shape. All possibilities are listed hereafter: * \(N_c=1\): scalar field * \(N_c=3\): vector field * \(N_c=6\): symetric tensor field (2nd order tensor) * \(N_c=9\): tensor field (2nd order tensor)
The dimensionality of the field has the following implications: * XDMF: the dimensionality of the field is one of the metadata stored in the XDMF file, for Fields (Attribute nodes) * visualization: as it is stored in the XDMF file, Paraview can correctly interpret the fields according to their dimensionality. Il allows to plot separately each field component, and the field norm (magnitude). It also allows to use the Paraview Glyph filter to plot vector fields with arrows *
indexing: The order of the value in the last dimension for non-scalar fields correspond to a specific order of the field components, according to a specific convention. These conventions will be detailed in the next subsection. * compression: Specific lossy compression options exist for fields. See dedicated tutorial. * interface with external tools: when interfacing SampleData with external softwares, such as numerical simulation tools, it is very practical to have fields with
appropriate dimensionality accounted for (for instance to use a displacement or temperature gradient vector field stored in a SampleData dataset as input for a finite element simulation).
Field components indexing¶
To detail the convention, we will omit the spatial indexes \(i,j\) or \(k\) and only consider the last dimension of the field: \(F[i,c] = F[c]\). The indexing convention to describe field components order in data arrays are:
For vector fields (3 components), the convention is \([F_0,F_1,F_2] = [F_x,F_y,F_z]\)
For symetric tensor fields (2nd order, 6 components), the convention is \([F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx}]\)
For tensor fields (2nd order, 9 components), the convention is \([F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx},F_{yx},F_{zy},F_{xz}]\)
Warning
Field components (example: \(x,y,xx,zy\)…) are assumed to be defined in the same frame as the grid. However, that cannot be ensured simply by providing a data array. Hence, the user must ensure to have this coincidence between the grid and the data. It is not mandatory, but not respecting this implicit convention may lead to confusions and geometrical misinterpretation of the data by other users or software. If you want to do it, a good idea would be to add attributes to the field to specify and explain the nature of the field components.
Components transposition¶
Paraview component order convention is different from *SampleData* condition for tensors. The convention in Paraview is:
\([F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{xy},F_{xz},F_{yy},F_{yz},F_{zz}]\) for symetric tensors
\([F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{xy},F_{xz},F_{yx},F_{yy},F_{yz},F_{zx},F_{zy},F_{zz}]\) for non symetric tensors
For these reason, fields are stored with indices transposition in SampleData datasets so that their visualization with Paraview yield a rendering that is consistent with the ordering convention presented in the previous subsection. These transposition are automatically handled, as well as the back transposition when a field is retrieved, as you will see in the last section of this tutorial.
An attribute ``transpose_components`` is added to field data items. Its values represent the order in which components or the original array are stored in memory (see examples below).
Note
When visualizing a field in paraview, you may choose which component (including field magnitude) you are plotting in the box (highlighted in red in the image below) located in between the boxes for the choice of the visualization mode (Surface in the image below) and the data item choice (tensor_field2D in the image below).
In this box, you will have to choose between 9 (0 to 8) components, even for symetric tensors. In the later case, the corresponding equal components (1 and 3: \(xy\)&\(yx\), 2 and 6: \(xz\)&\(zx\), 5 and 7: \(yz\)&\(zy\)) will yield the same visualization.
Field attributes¶
Let us look more closely on a Mesh Group Field data item, from the test mesh that we created earlier:
[39]:
data.print_node_info('mesh_Test_field1')
data.print_node_info('mesh_Test_field3')
NODE: /test_mesh/Test_field1
====================
-- Parent Group : test_mesh
-- Node name : Test_field1
-- Test_field1 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field1
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field1 (CArray(6, 1)) 'mesh_Test_field1'
-- Compression options for node `mesh_Test_field1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
NODE: /test_mesh/Test_field3
====================
-- Parent Group : test_mesh
-- Node name : Test_field3
-- Test_field3 attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : bulk
* parent_grid_path : /test_mesh
* xdmf_fieldname : Test_field3
* xdmf_gridname : test_mesh
-- content : /test_mesh/Test_field3 (CArray(8, 1)) 'mesh_Test_field3'
-- Compression options for node `mesh_Test_field3`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
As you can see, Field data item attributes provide information on the field dimensionality, type and padding. Here for instance, the two fields Test_field1 and Test_field3 are scalar fields. The Test_field3 is an element field, that has been inputed with an array containing one value per bulk elemenbt of the mesh.
The parent_grid_path and xdmf_gridname attribute provides the path of the Image Group and the XDMF Grid Node name to which this field belongs. The xdmf_fieldname provides the name of the XDMF Attribute Node associated to the field. The padding attribute is of no use for Image fields. It will be presented in the next tutorial for mesh fields.
V - Adding Fields to Image Groups¶
Like for Image Group Fields (see previous tutorial, section V), Mesh Group Fields can be created in a SampleData dataset using the add_field method.
To add a field to a grid from a numpy array, you just have to call add_field like you would have called add_data_array, with an additional argument gridname, that indicates the mesh group to which you want to add the field.
The analysis of the field dimensionality, nature, padding and required transpositions is automatically handled by the class.
If the added field is an integration point field, the value of the visualisation_type argument will control the creation of an associated visualization field. Its possible values are:
visualization_type='Elt_mean': a visualization field is created by taking the mean of integration point values in each elementvisualization_type='Elt_max': a visualization field is created by taking the mean of integration point values in each elementvisualization_type='Elt_min': a visualization field is created by taking the mean of integration point values in each elementvisualization_type='None': no visualization field is created
The add_field method has already been presented in the last tutorial. Please refer to it to see examples on how to use this method, in particular to create Field time series. The rest of this section is dedicated to a few example of Field data items creation specificities for mesh groups.
The fields will be created o,n the mesh mesh_geof. Let us print again information on this mesh topology:
[40]:
data.print_node_info('mesh_geof')
GROUP loaded_geof_mesh
=====================
-- Parent Group : /
-- Group attributes :
* Elements_offset : [ 0 384]
* Number_of_boundary_elements : [384]
* Number_of_bulk_elements : [384]
* Number_of_elements : [384 384]
* Topology : Mixed
* Xdmf_elements_code : [6, 4]
* description :
* element_type : ['tet4', 'tri3']
* elements_path : /loaded_geof_mesh/Geometry/Elements
* empty : False
* group_type : 3DMesh
* nodesID_path : /loaded_geof_mesh/Geometry/Nodes_ID
* nodes_path : /loaded_geof_mesh/Geometry/Nodes
* number_of_nodes : 125
* xdmf_gridname : loaded_geof_mesh
-- Childrens : Geometry, Field_index,
----------------
Example 1: Creating a vector node field¶
Node vector fields are a very common data in material science and material mechanics. They can be for instance a displacement or velocity field computed from a simulation solver or experimentally measured, a magnetic field…
To create a node vector field, we need to create a field data array that has one value for each node in the mesh. As we can see above, the Mesh Group has a number_of_nodes that provides us with this information (\(N_n\)). To create a vector field, each value of the field must be a 1D vector of 3 values.
We have to create a data array of shape \((N_n, 3)\). Let us do it by creating a random array:
[41]:
# random array creation
array = np.random.rand(data.get_attribute('number_of_nodes','mesh_geof'),3) - 0.5
[42]:
# creation of the mesh field
data.add_field(gridname='mesh_geof', fieldname='nodal_vectorF', array=array, indexname='vectF', replace=True)
[42]:
/loaded_geof_mesh/nodal_vectorF (CArray(125, 3)) 'vectF'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (2730, 3)
[43]:
# Printing information on our created field:
print(f'Is "vectF" in the field list of the mesh ? {"vectF" in data.get_grid_field_list("mesh_geof")}')
data.print_node_info('vectF')
Is "vectF" in the field list of the mesh ? True
NODE: /loaded_geof_mesh/nodal_vectorF
====================
-- Parent Group : loaded_geof_mesh
-- Node name : nodal_vectorF
-- nodal_vectorF attributes :
* empty : False
* field_dimensionality : Vector
* field_type : Nodal_field
* node_type : field_array
* padding : None
* parent_grid_path : /loaded_geof_mesh
* xdmf_fieldname : nodal_vectorF
* xdmf_gridname : loaded_geof_mesh
-- content : /loaded_geof_mesh/nodal_vectorF (CArray(125, 3)) 'vectF'
-- Compression options for node `vectF`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (2730, 3)
-- Node memory size : 63.984 Kb
----------------
We can verify the field attributes to check that it is indeed a vector field, a nodal field (hence with no padding), that is defined on the loaded_geof_mesh Mesh Group.
You can also visualize the field with Paraview :
[44]:
# Use the second code line if you want to specify the path of the paraview executable you want to use
# otherwise use the first line
#data.pause_for_visualization(Paraview=True)
#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')
Open the XDMF dataset with the XdmfReader of Paraview, and then, in the Property panel, tick only the box associated to the the loaded_geof_mesh block. Then you can use the Glyph filter, which allows you to plot vector fields with arrows. You should be able to get visualization looking like the image below:
Example 2: Creating a scalar element field for boundary elements¶
Scalar fields defined on boundary can correspond for instance to pressure fields applied on a body, or a surface temperature measurement. They are also a common type of data for material mechanics datasets.
To add one, we need to create a 1D array (as the field is scalar) that has one value per boundary element in the mesh. in the present case, the mesh has as many bulk as boundary elements. To solve the ambiguity, the add field method has a bulk_padding argument. If it is True (default value), the field is padded as a bulk element field. If it is set to False, the field is padded as a boundary element field.
[45]:
# Random array creation. Warning ! 'Number_of_boundary_elements' attribute is a list !
array = np.random.rand(data.get_attribute('Number_of_boundary_elements','mesh_geof')[0])
# creation of the mesh field
data.add_field(gridname='mesh_geof', fieldname='boundary_scalarF', array=array, indexname='boundaryF', replace=True,
bulk_padding=False)
[45]:
/loaded_geof_mesh/boundary_scalarF (CArray(768, 1)) 'boundaryF'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (8192, 1)
[46]:
# Printing information on our created field:
print(f'Is "boundaryF" in the field list of the mesh ? {"boundaryF" in data.get_grid_field_list("mesh_geof")}')
data.print_node_info('boundaryF')
Is "boundaryF" in the field list of the mesh ? True
NODE: /loaded_geof_mesh/boundary_scalarF
====================
-- Parent Group : loaded_geof_mesh
-- Node name : boundary_scalarF
-- boundary_scalarF attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : boundary
* parent_grid_path : /loaded_geof_mesh
* xdmf_fieldname : boundary_scalarF
* xdmf_gridname : loaded_geof_mesh
-- content : /loaded_geof_mesh/boundary_scalarF (CArray(768, 1)) 'boundaryF'
-- Compression options for node `boundaryF`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (8192, 1)
-- Node memory size : 64.000 Kb
----------------
We can verify the field attributes to check that it is indeed a scalar element field, padded as a boundary element field, that is defined on the loaded_geof_mesh Mesh Group.
Once again, you can visualize your field with the Paraview software:
Open the XDMF dataset with the XdmfReader of Paraview, and then, in the Property panel, tick only the box associated to the the loaded_geof_mesh block. Then you can use the Clip filter, which allows you to cut your grid with a customizable plane and hence plot data within the bulk of your mesh, as the right picture below:
As you can see, the visual rendering of the field displays the random values of the field on the boundary triangle elements of the external surface of the mesh, and 0 values within the bulk of the mesh: SampleData has automatically padded the field array with 0 at positions corresponding to bulk elements to ensure compatibility with the XDMF format, and hence visualization.
Example 3: Creating a time serie for a tensorial integration point field for bulk elements¶
In this last example, we will add a field representing a uniaxial stress state to the mesh, proportional to time. We need to create a symetric tensor field, with 6 components. We will suppose that this field comes from a finite element solver output, that provided the full stress tensor at each integration point in the mesh bulk elements. We will suppose that the mesh elements have 4 integration points each, and that the values have been outputed at three instants.
Hence, we have to create 3 arrays of shape \((4N_{be},6)\), where \(N_{be}\) is the number of bulk elements in the mesh.
[47]:
# Arrays creation
array_T0 = np.zeros((4*data.get_attribute('Number_of_bulk_elements','mesh_geof')[0],6))
array_T1 = np.zeros((4*data.get_attribute('Number_of_bulk_elements','mesh_geof')[0],6))
array_T2 = np.zeros((4*data.get_attribute('Number_of_bulk_elements','mesh_geof')[0],6))
# time values:
T0 = 0.
T1 = 1.
T2 = 10.
# Filling stress state values
# at second time increment --> stress = 100*t in direction Z/3 (see indexing conventions)
array_T1[:,2] = 100*T1
# at second time increment --> stress = 100*t in direction Z/3 (see indexing conventions)
array_T2[:,2] = 100*T2
# create fields
data.add_field(gridname='mesh_geof', fieldname='tensor_ipF', array=array_T0, indexname='tensorF', replace=True,
time=T0)
data.add_field(gridname='mesh_geof', fieldname='tensor_ipF', array=array_T1, indexname='tensorF', replace=True,
time=T1)
data.add_field(gridname='mesh_geof', fieldname='tensor_ipF', array=array_T2, indexname='tensorF', replace=True,
time=T2)
[47]:
/loaded_geof_mesh/tensor_ipF_T2 (CArray(1536, 6)) 'tensorF_T2'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (1365, 6)
[48]:
print(data.get_grid_field_list('mesh_geof'))
data.print_node_info('tensorF_T0')
data.print_node_info('tensorF_T2')
['mesh_geof_field_x0y0z0', 'mesh_geof_field_x1y0z0', 'mesh_geof_field_x0y1z0', 'mesh_geof_field_x1y1z0', 'mesh_geof_field_x0y0z1', 'mesh_geof_field_x1y0z1', 'mesh_geof_field_x0y1z1', 'mesh_geof_field_x1y1z1', 'mesh_geof_field_3D', 'mesh_geof_field_X0', 'mesh_geof_field_Y0', 'mesh_geof_field_Skin', 'mesh_geof_field_X1', 'mesh_geof_field_Y1', 'mesh_geof_field_Z0', 'mesh_geof_field_Z1', 'vectF', 'boundaryF', 'tensorF_T0', 'tensorF_T1', 'tensorF_T2']
NODE: /loaded_geof_mesh/tensor_ipF_T0
====================
-- Parent Group : loaded_geof_mesh
-- Node name : tensor_ipF_T0
-- tensor_ipF_T0 attributes :
* empty : False
* field_dimensionality : Tensor6
* field_type : IP_field
* node_type : field_array
* padding : bulk_IP
* parent_grid_path : /loaded_geof_mesh
* time : 0.0
* time_serie_name : tensor_ipF
* transpose_components : [0, 3, 5, 1, 4, 2]
* visualisation_field_path : /loaded_geof_mesh/tensor_ipF_T0_Elt_mean
* visualisation_type : Elt_mean
* xdmf_gridname : loaded_geof_mesh_T0
-- content : /loaded_geof_mesh/tensor_ipF_T0 (CArray(1536, 6)) 'tensorF_T0'
-- Compression options for node `tensorF_T0`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1365, 6)
-- Node memory size : 127.969 Kb
----------------
NODE: /loaded_geof_mesh/tensor_ipF_T2
====================
-- Parent Group : loaded_geof_mesh
-- Node name : tensor_ipF_T2
-- tensor_ipF_T2 attributes :
* empty : False
* field_dimensionality : Tensor6
* field_type : IP_field
* node_type : field_array
* padding : bulk_IP
* parent_grid_path : /loaded_geof_mesh
* time : 10.0
* time_serie_name : tensor_ipF
* transpose_components : [0, 3, 5, 1, 4, 2]
* visualisation_field_path : /loaded_geof_mesh/tensor_ipF_T2_Elt_mean
* visualisation_type : Elt_mean
* xdmf_gridname : loaded_geof_mesh_T2
-- content : /loaded_geof_mesh/tensor_ipF_T2 (CArray(1536, 6)) 'tensorF_T2'
-- Compression options for node `tensorF_T2`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1365, 6)
-- Node memory size : 127.969 Kb
----------------
You can see that the fields have been automatically added as integration point fields symmetric tensor (Tensor6 IP_field). You can also observe the apparition of the transpose_components attribute, that has been already introduce in the last tutorial.
You can also observe that the fields have a visualisation_field_path and a visualisation_type attributes. They indicate respectively the path of the visualization field data item associated to our added field, as well as the type of operation that has been used to create it (here, taking the mean of integration points value within each element to create an element wise constant field: Elt_mean).
You can try to modify the calls to add_field in the previous cells by adding the visualisation_type argument, and setting its value to Elt_max, or None, and observe the differences in the field data item attributes, and the field visualization.
Let us now look at the XDMf dataset:
[49]:
data.print_xdmf()
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="test_mesh" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="6 3" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Triangle" NumberOfElements="8">
<DataItem Format="HDF" Dimensions="24 " NumberType="Int" Precision="64">test_mesh_dataset.h5:/test_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="field_Z0_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/test_mesh/Geometry/NodeTags/field_Z0_plane</DataItem>
</Attribute>
<Attribute Name="field_out_of_plane" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/test_mesh/Geometry/NodeTags/field_out_of_plane</DataItem>
</Attribute>
<Attribute Name="field_2D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Geometry/ElementsTags/field_2D</DataItem>
</Attribute>
<Attribute Name="field_Top" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Geometry/ElementsTags/field_Top</DataItem>
</Attribute>
<Attribute Name="field_Bottom" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Geometry/ElementsTags/field_Bottom</DataItem>
</Attribute>
<Attribute Name="Test_field1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Test_field1</DataItem>
</Attribute>
<Attribute Name="Test_field2" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="6 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Test_field2</DataItem>
</Attribute>
<Attribute Name="Test_field3" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Test_field3</DataItem>
</Attribute>
<Attribute Name="Test_field4" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="8 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/test_mesh/Test_field4</DataItem>
</Attribute>
</Grid>
<Grid Name="loaded_geof_mesh" GridType="Collection" CollectionType="Temporal">
<Grid Name="loaded_geof_mesh_T0" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="125 3" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Mixed" NumberOfElements="768">
<DataItem Format="HDF" Dimensions="3456 " NumberType="Int" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="field_x0y0z0" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x0y0z0</DataItem>
</Attribute>
<Attribute Name="field_x1y0z0" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x1y0z0</DataItem>
</Attribute>
<Attribute Name="field_x0y1z0" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x0y1z0</DataItem>
</Attribute>
<Attribute Name="field_x1y1z0" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x1y1z0</DataItem>
</Attribute>
<Attribute Name="field_x0y0z1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x0y0z1</DataItem>
</Attribute>
<Attribute Name="field_x1y0z1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x1y0z1</DataItem>
</Attribute>
<Attribute Name="field_x0y1z1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x0y1z1</DataItem>
</Attribute>
<Attribute Name="field_x1y1z1" AttributeType="Scalar" Center="Node">
<DataItem Format="HDF" Dimensions="125 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/NodeTags/field_x1y1z1</DataItem>
</Attribute>
<Attribute Name="field_3D" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_3D</DataItem>
</Attribute>
<Attribute Name="field_X0" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_X0</DataItem>
</Attribute>
<Attribute Name="field_Y0" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_Y0</DataItem>
</Attribute>
<Attribute Name="field_Skin" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_Skin</DataItem>
</Attribute>
<Attribute Name="field_X1" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_X1</DataItem>
</Attribute>
<Attribute Name="field_Y1" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_Y1</DataItem>
</Attribute>
<Attribute Name="field_Z0" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_Z0</DataItem>
</Attribute>
<Attribute Name="field_Z1" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Int" Precision="8">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/ElementsTags/field_Z1</DataItem>
</Attribute>
<Attribute Name="nodal_vectorF" AttributeType="Vector" Center="Node">
<DataItem Format="HDF" Dimensions="125 3" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/nodal_vectorF</DataItem>
</Attribute>
<Attribute Name="boundary_scalarF" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="768 1" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/boundary_scalarF</DataItem>
</Attribute>
<Time Value="0.0"/>
<Attribute Name="tensor_ipF" AttributeType="Tensor6" Center="Cell">
<DataItem Format="HDF" Dimensions="768 6" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/tensor_ipF_T0_Elt_mean</DataItem>
</Attribute>
</Grid>
<Grid Name="loaded_geof_mesh_T1" GridType="Uniform">
<Time Value="1.0"/>
<Topology TopologyType="Mixed" NumberOfElements="768">
<DataItem Format="HDF" Dimensions="3456 " NumberType="Int" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="125 3" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="tensor_ipF" AttributeType="Tensor6" Center="Cell">
<DataItem Format="HDF" Dimensions="768 6" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/tensor_ipF_T1_Elt_mean</DataItem>
</Attribute>
</Grid>
<Grid Name="loaded_geof_mesh_T2" GridType="Uniform">
<Time Value="10.0"/>
<Topology TopologyType="Mixed" NumberOfElements="768">
<DataItem Format="HDF" Dimensions="3456 " NumberType="Int" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Elements</DataItem>
</Topology>
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="125 3" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Attribute Name="tensor_ipF" AttributeType="Tensor6" Center="Cell">
<DataItem Format="HDF" Dimensions="768 6" NumberType="Float" Precision="64">test_mesh_dataset.h5:/loaded_geof_mesh/tensor_ipF_T2_Elt_mean</DataItem>
</Attribute>
</Grid>
</Grid>
</Domain>
</Xdmf>
As expected, the Attributes of our grid time collection loaded_geof_mesh that contain the tensor field do not point towards the added data array, but towards the visualization field data array: tensor_ipF_T0_Elt_mean, tensor_ipF_T1_Elt_mean or tensor_ipF_T2_Elt_mean.
VI - Getting Mesh Groups and Mesh Fields¶
Getting mesh objects from Mesh groups¶
Like Image groups, you can easily retrieve a complete SampleData Mesh Group under the form of a BasicTools Mesh object, with the get_mesh method:
[50]:
mesh = data.get_mesh('mesh_geof', with_fields=True, with_tags=True)
print(mesh)
UnstructuredMesh
Number Of Nodes : 125
Tags : (x0y0z0:1) (x1y0z0:1) (x0y1z0:1) (x1y1z0:1) (x0y0z1:1) (x1y0z1:1) (x0y1z1:1) (x1y1z1:1)
Number Of Elements : 768
ElementsContainer, Type : (tet4,384), Tags : (3D:384)
ElementsContainer, Type : (tri3,384), Tags : (X0:32) (Y0:32) (Skin:192) (X1:32) (Y1:32) (Z0:32) (Z1:32)
nodeFields : ['vectF']
elemFields : ['boundaryF']
This method has two arguments that allows you to control how rich you want your output to be. By setting with_fields to False, the return mesh object will not contain the field arrays stored on the Mesh Group. By setting with_tags to False, the return mesh object will not contain the Node and Element tags stored in the Mesh Group. If both are False, the mesh object will only contain the mesh nodes and elements.
Getting mesh Fields¶
As for image fields, retrieveing mesh fields from Sampledata dataset is done with the get_field method. All padding, transpositions etc… are automatically reverted to return the originally inputed data array.
In the case of mesh fields, you can prevent the reversion of field padding with the unpad_field argument (set it to False). You can also choose, when getting an integration point field, to get the complete data array, or on the contrary, only the associated visualization field data array, by setting the get_visualisation_field argument to True.
Let us show these various use of the method by trying to get the tensor field created in the third exampe above.
[51]:
# getting the inputed array --> no options
O_array = data.get_field('tensorF_T1')
# getting the visualization array
V_array = data.get_field('tensorF_T1', get_visualisation_field=True)
# getting the unpadded visualization array
UPV_array = data.get_field('tensorF_T1', unpad_field=False, get_visualisation_field=True)
print(f' Original array shape is {O_array.shape}, unpadded array shape is {V_array.shape},'
f' unpadded visualization array shape is {UPV_array.shape}')
print(f'\nIs it the original inputed array ? {np.all(O_array == array_T1)} \n')
Original array shape is (1536, 6), unpadded array shape is (384, 6), unpadded visualization array shape is (768, 6)
Is it the original inputed array ? True
As you can see, using get_field allow you to retrieve exactly the original array that you inputed, reverting all transformations (\(1536 = 4*384\) here). Using the get_visualisation_field argument, you retrieve the unpadded visualization field with values only for the mesh bulk elements, as inputted (\(384\) values here); and adding the unpad_field=False argument, you get the field with the added 0, that has one value per element in the mesh (\(768\) values here).
This is the end of this tutorial on SampleData Mesh Groups ! We can now close our dataset to conclude it.
[52]:
del data
SampleData Autodelete:
Removing hdf5 file test_mesh_dataset.h5 and xdmf file test_mesh_dataset.xdmf
6 - Data Compression¶
This short Notebook will introduce you to how to efficiently compress your data within SampleData datasets.
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the first notebook of this User Guide of this User Guide.
Data compression with HDF5 and Pytables¶
HDF5 and data compression¶
HDF5 allows compression filters to be applied to datasets in a file to minimize the amount of space it consumes. These compression features allow to drastically improve the storage space required for you datasets, as well as the speed of I/O access to datasets, and can differ from one data item to another within the same HDF5 file. A detailed presentation of HDF5 compression possibilities is provided here.
The two main ingredients that control compression performances for HDF5 datasets are the compression filters used (which compression algorithm, with which parameters), and the using a chunked layout for the data. This two features are briefly developed hereafter.
Pytables and data compression¶
The application of compression filters to HDF5 files with the SampleData class is handled by the Pytable package, on which is built the SampleData HDF5 interface. Pytables implements a specific containers class, the Filters class, to gather the various settings of the compression filters to apply to the datasets in a HDF5 file.
When using the SampleData class, you will have to specify this compression filter settings to class methods dedicated to data compression. These settings are the parameters of the Pytables Filters class. These settings and their possible values are detailed in the next subsection.
Available filter settings¶
The description given below of compression options available with SampleData/Pytables is exctracted from the *Pytables* documentation of the *Filter* class.
complevel (int) – Specifies a compression level for data. The allowed range is 0-9. A value of 0 (the default) disables compression.
complib (str) – Specifies the compression library to be used. Right now,
zlib(the default),lzo,bzip2andbloscare supported. Additional compressors for Blosc likeblosc:blosclz(‘blosclz’ is the default in case the additional compressor is not specified),blosc:lz4,blosc:lz4hc,blosc:snappy,blosc:zlibandblosc:zstdare supported too. Specifying a compression library which is not available in the system issues a FiltersWarning and sets the library to the default one.shuffle (bool) – Whether or not to use the Shuffle filter in the HDF5 library. This is normally used to improve the compression ratio. A false value disables shuffling and a true one enables it. The default value depends on whether compression is enabled or not; if compression is enabled, shuffling defaults to be enabled, else shuffling is disabled. Shuffling can only be used when compression is enabled.
bitshuffle (bool) – Whether or not to use the BitShuffle filter in the Blosc library. This is normally used to improve the compression ratio. A false value disables bitshuffling and a true one enables it. The default value is disabled.
fletcher32 (bool) – Whether or not to use the Fletcher32 filter in the HDF5 library. This is used to add a checksum on each data chunk. A false value (the default) disables the checksum.
least_significant_digit (int) – If specified, data will be truncated (quantized). In conjunction with enabling compression, this produces ‘lossy’, but significantly more efficient compression. For example, if least_significant_digit=1, data will be quantized using around(scale*data)/scale, where scale = 2^bits, and bits is determined so that a precision of 0.1 is retained (in this case bits=4). Default is None, or no quantization.
Chunked storage layout¶
Compressed data is stored in a data array of an HDF5 dataset using a chunked storage mechanism. When chunked storage is used, the data array is split into equally sized chunks each of which is stored separately in the file, as illustred on the diagram below. Compression is applied to each individual chunk. When an I/O operation is performed on a subset of the data array, only chunks that include data from the subset participate in I/O and need to be uncompressed or compressed.
Chunking data allows to:
Generally improve, sometimes drastically, the I/O performance of datasets. This comes from the fact that the chunked layout removes the reading speed anisotropy for data array that depends along which dimension its elements are read (i.e the same number of disk access are required when reading data in rows or columns).
Chunked storage also enables adding more data to a dataset without rewriting the whole dataset.
By default, data arrays are stored with a chunked layout in SampleData datasets. The size of chunks is the key parameter that controls the impact on I/O performances for chunked datasets. The shape of chunks is computed automatically by the Pytable package, providing a value yielding generally good I/O performances. if you need to go further in the I/O optimization, you may consult the Pytables documentation page dedicated to compression optimization for I/O speed and storage space. In addition, it is highly recommended to read this document in order to be able to efficiently optimize I/O and storage performances for your chunked datasets. These performance issues will not be discussed in this tutorial.
Compressing your datasets with SampleData¶
Within Sampledata, data compression can be applied to:
data arrays
structured data arrays
field data arrays
There are two ways to control the compression settings of your SampleData data arrays:
Providing compression settings to data item creation methods
Using the
set_chunkshape_and_compressionandset_nodes_compression_chunkshapemethods
The compression options dictionary¶
In both cases, you will have to pass the various settings of the compression filter you want to apply to your data to the appropriate SampleData method. All of these methods accept for that purpose a compression_options argument, which must be a dictionary. Its keys and associated values can be chosen among the ones listed in the Available filter settings subsection above.
Compress already existing data arrays¶
We will start by looking at how we can change compression settings of already existing data in SampleData datasets. For that, we will use a material science dataset that is part of the Pymicro example datasets.
[1]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure') # test dataset file path
tar_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure.tar.gz') # dataset archive path
This file is zipped in the package to reduce its size. We will have to unzip it to use it and learn how to reduce its size with the SampleData methods. If you are just reading the documentation and not executing it, you may just skip this cell and the next one.
[2]:
# Save current directory
cwd = os.getcwd()
# move to example data directory
os.chdir(PYMICRO_EXAMPLES_DATA_DIR)
# unarchive the dataset
os.system(f'tar -xvf {tar_file}')
# get back to UserGuide directory
os.chdir(cwd)
Dataset presentation¶
In this tutorial, we will work on a copy of this dataset, to leave the original data unaltered. We will start by creating an autodeleting copy of the file, and print its content to discover its content.
[3]:
# import SampleData class
from pymicro.core.samples import SampleData as SD
# import Numpy
import numpy as np
[4]:
# Create a copy of the existing dataset
data = SD.copy_sample(src_sample_file=dataset_file, dst_sample_file='Test_compression', autodelete=True,
get_object=True, overwrite=True)
[5]:
print(data)
data.get_file_disk_size()
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : GrainDataTable H5_Path : /GrainData/GrainDataTable
Name : Grain_data H5_Path : /GrainData
Name : Image_data H5_Path : /CellData
Image_data aliases --> `CellData`
Name : Image_data_Amitex_stress_1 H5_Path : /CellData/Amitex_output_fields/Amitex_stress_1
Name : Image_data_grain_map H5_Path : /CellData/grain_map
Name : Image_data_grain_map_raw H5_Path : /CellData/grain_map_raw
Name : Image_data_mask H5_Path : /CellData/mask
Name : Image_data_uncertainty_map H5_Path : /CellData/uncertainty_map
Name : Mesh_data H5_Path : /MeshData
Name : Phase_data H5_Path : /PhaseData
Name : fft_fields H5_Path : /CellData/Amitex_output_fields
Name : fft_sim H5_Path : /Amitex_Results
Name : grain_map H5_Path : /CellData/grain_map
Name : grains_mesh H5_Path : /MeshData/grains_mesh
Name : grains_mesh_Geometry H5_Path : /MeshData/grains_mesh/Geometry
Name : grains_mesh_grains_mesh_elset_ids H5_Path : /MeshData/grains_mesh/grains_mesh_elset_ids
Name : mask H5_Path : /CellData/mask
Name : mean_strain H5_Path : /Amitex_Results/mean_strain
Name : mean_stress H5_Path : /Amitex_Results/mean_stress
Name : phase_01 H5_Path : /PhaseData/phase_01
Name : phase_map H5_Path : /CellData/phase_map
Name : rms_strain H5_Path : /Amitex_Results/rms_strain
Name : rms_stress H5_Path : /Amitex_Results/rms_stress
Name : simulation_iterations H5_Path : /Amitex_Results/simulation_iterations
Name : simulation_time H5_Path : /Amitex_Results/simulation_time
Printing dataset content with max depth 3
|--GROUP Amitex_Results: /Amitex_Results (Group)
--NODE mean_strain: /Amitex_Results/mean_strain (data_array) ( 63.984 Kb)
--NODE mean_stress: /Amitex_Results/mean_stress (data_array) ( 63.984 Kb)
--NODE rms_strain: /Amitex_Results/rms_strain (data_array) ( 63.984 Kb)
--NODE rms_stress: /Amitex_Results/rms_stress (data_array) ( 63.984 Kb)
--NODE simulation_iterations: /Amitex_Results/simulation_iterations (data_array) ( 64.000 Kb)
--NODE simulation_time: /Amitex_Results/simulation_time (data_array) ( 64.000 Kb)
|--GROUP CellData: /CellData (3DImage)
|--GROUP Amitex_output_fields: /CellData/Amitex_output_fields (Group)
--NODE Amitex_stress_1: /CellData/Amitex_output_fields/Amitex_stress_1 (field_array) ( 49.438 Mb)
--NODE Field_index: /CellData/Field_index (string_array) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array) ( 1.945 Mb)
--NODE grain_map_raw: /CellData/grain_map_raw (field_array) ( 1.945 Mb)
--NODE mask: /CellData/mask (field_array) ( 996.094 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
--NODE uncertainty_map: /CellData/uncertainty_map (field_array) ( 996.094 Kb)
|--GROUP GrainData: /GrainData (Group)
--NODE GrainDataTable: /GrainData/GrainDataTable (structured array) ( 63.984 Kb)
|--GROUP MeshData: /MeshData (emptyMesh)
|--GROUP grains_mesh: /MeshData/grains_mesh (3DMesh)
|--GROUP Geometry: /MeshData/grains_mesh/Geometry (Group)
--NODE grains_mesh_elset_ids: /MeshData/grains_mesh/grains_mesh_elset_ids (field_array) ( 624.343 Kb)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
File size is 83.086 Mb for file
Test_compression.h5
[5]:
(83.08567428588867, 'Mb')
As you can see, this dataset already contains a rich content. It is a digital twin of a real polycristalline microstructure of a grade 2 Titanium sample, gathering both experimental and numerical data obtained through Diffraction Contrast Tomography imaging, and FFT-based mechanical simulation.
This dataset has actually been constructed using the Microstructure class of the pymicro package, which is based on the SampleData class. The link between these classes will be discussed in the next tutorial.
This dataset contains only uncompressed data. We will try to reduce its size by using various compression methods on the large data items that it contains. You can see that most of them are stored in the 3DImage Group CellData.
Apply compression settings for a specific array¶
We will start by compressing the grain_map Field data array of the CellData image. Let us look more closely on this data item:
[6]:
data.print_node_info('grain_map')
NODE: /CellData/grain_map
====================
-- Parent Group : CellData
-- Node name : grain_map
-- grain_map attributes :
* empty : False
* field_dimensionality : Scalar
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /CellData
* transpose_indices : [2, 1, 0]
* xdmf_fieldname : grain_map
* xdmf_gridname : CellData
-- content : /CellData/grain_map (CArray(100, 100, 100)) 'Image_data_grain_map'
-- Compression options for node `grain_map`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (3, 100, 100)
-- Node memory size : 1.945 Mb
----------------
We can see above that this data item is not compressed (complevel=0), and has a disk size of almost 2 Mb.
To apply a set of compression settings to this data item, you need to:
create a dictionary specifying the compression settings:
[7]:
compression_options = {'complib':'zlib', 'complevel':1}
use the SampleData
set_chunkshape_and_compressionmethod with the dictionary and the name of the data item as arguments
[8]:
data.set_chunkshape_and_compression(nodename='grain_map', compression_options=compression_options)
data.get_node_disk_size('grain_map')
data.print_node_compression_info('grain_map')
Node grain_map size on disk is 126.297 Kb
Compression options for node `grain_map`:
complevel=1, complib='zlib', shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (3, 100, 100)
As you can see, the storage size of the data item has been greatly reduced, by more than 10 times (126 Kb vs 1.945 Mb), using this compression settings. Let us see what will change if we use different settings :
[9]:
# No `shuffle` option:
print('\nUsing the shuffle option, with the zlib compressor and a compression level of 1:')
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True}
data.set_chunkshape_and_compression(nodename='grain_map', compression_options=compression_options)
data.get_node_disk_size('grain_map')
# No `shuffle` option:
print('\nUsing no shuffle option, with the zlib compressor and a compression level of 9:')
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':False}
data.set_chunkshape_and_compression(nodename='grain_map', compression_options=compression_options)
data.get_node_disk_size('grain_map')
# No `shuffle` option:
print('\nUsing the shuffle option, with the lzo compressor and a compression level of 1:')
compression_options = {'complib':'lzo', 'complevel':1, 'shuffle':True}
data.set_chunkshape_and_compression(nodename='grain_map', compression_options=compression_options)
data.get_node_disk_size('grain_map')
# No `shuffle` option:
print('\nUsing no shuffle option, with the lzo compressor and a compression level of 1:')
compression_options = {'complib':'lzo', 'complevel':1, 'shuffle':False}
data.set_chunkshape_and_compression(nodename='grain_map', compression_options=compression_options)
data.get_node_disk_size('grain_map')
Using the shuffle option, with the zlib compressor and a compression level of 1:
Node grain_map size on disk is 179.201 Kb
Using no shuffle option, with the zlib compressor and a compression level of 9:
Node grain_map size on disk is 62.322 Kb
Using the shuffle option, with the lzo compressor and a compression level of 1:
Node grain_map size on disk is 327.034 Kb
Using no shuffle option, with the lzo compressor and a compression level of 1:
Node grain_map size on disk is 238.415 Kb
[9]:
(238.4150390625, 'Kb')
As you may observe, is significantly affected by the choice of the compression level. The higher the compression level, the higher the compression ratio, but also the lower the I/O speed. On the other hand, you can also remark that, in the present case, using the shuffle filter deteriorates the compression ratio.
Let us try to with another data item:
[10]:
data.print_node_info('Amitex_stress_1')
# No `shuffle` option:
print('\nUsing the shuffle option, with the zlib compressor and a compression level of 1:')
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
# No `shuffle` option:
print('\nUsing no shuffle option, with the zlib compressor and a compression level of 1:')
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':False}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
NODE: /CellData/Amitex_output_fields/Amitex_stress_1
====================
-- Parent Group : Amitex_output_fields
-- Node name : Amitex_stress_1
-- Amitex_stress_1 attributes :
* empty : False
* field_dimensionality : Tensor6
* field_type : Element_field
* node_type : field_array
* padding : None
* parent_grid_path : /CellData
* transpose_components : [0, 3, 5, 1, 4, 2]
* transpose_indices : [2, 1, 0, 3]
* xdmf_fieldname : Amitex_stress_1
* xdmf_gridname : CellData
-- content : /CellData/Amitex_output_fields/Amitex_stress_1 (CArray(100, 100, 100, 6)) 'Image_data_Amitex_stress_1'
-- Compression options for node `Amitex_stress_1`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1, 27, 100, 6)
-- Node memory size : 49.438 Mb
----------------
Using the shuffle option, with the zlib compressor and a compression level of 1:
Node Amitex_stress_1 size on disk is 20.975 Mb
Using no shuffle option, with the zlib compressor and a compression level of 1:
Node Amitex_stress_1 size on disk is 27.460 Mb
[10]:
(27.460082054138184, 'Mb')
On the opposite, for this second array, the shuffle filter improves significantly the compression ratio. However, in this case, you can see that the compression ratio achieved is much lower than for the grain_map array.
Warning 1
The efficiency of compression algorithms in terms of compression ratio is strongly affected by the data itself (variety, value and position of the stored values in the array). Compression filters will not have the same behavior with all data arrays, as you have observed just above. Be aware of this fact, and do not hesitate to conduct tests to find the best settings for you datasets !
Warning 2
Whenever you change the compression or chunkshape settings of your datasets, the data item is re-created into the SampleData dataset, which may be costly in computational time. Be careful if you are dealing with very large data arrays and want to try out several settings to find the best I/O speed / compression ratio compromise, with the set_chunkshape_and_compression method. You may want to try on a subset of your large array to speed up the process.
Apply same compression settings for a serie of nodes¶
If you need to apply the same compression settings to a list of data items, you may use the set_nodes_compression_chunkshape. This method works exactly like set_chunkshape_and_compression, but take a list of nodenames as arguments instead of just one. The inputted compression settings are then applied to all the nodes in the list:
[11]:
# Print current size of disks and their compression settings
data.get_node_disk_size('grain_map_raw')
data.print_node_compression_info('grain_map_raw')
data.get_node_disk_size('uncertainty_map')
data.print_node_compression_info('uncertainty_map')
data.get_node_disk_size('mask')
data.print_node_compression_info('mask')
# Compress datasets
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True}
data.set_nodes_compression_chunkshape(node_list=['grain_map_raw', 'uncertainty_map', 'mask'],
compression_options=compression_options)
# Print new size of disks and their compression settings
data.get_node_disk_size('grain_map_raw')
data.print_node_compression_info('grain_map_raw')
data.get_node_disk_size('uncertainty_map')
data.print_node_compression_info('uncertainty_map')
data.get_node_disk_size('mask')
data.print_node_compression_info('mask')
Node grain_map_raw size on disk is 1.945 Mb
Compression options for node `grain_map_raw`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (3, 100, 100)
Node uncertainty_map size on disk is 996.094 Kb
Compression options for node `uncertainty_map`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (6, 100, 100)
Node mask size on disk is 996.094 Kb
Compression options for node `mask`:
complevel=0, shuffle=False, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (6, 100, 100)
Node grain_map_raw size on disk is 134.527 Kb
Compression options for node `grain_map_raw`:
complevel=9, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (3, 100, 100)
Node uncertainty_map size on disk is 55.322 Kb
Compression options for node `uncertainty_map`:
complevel=9, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (6, 100, 100)
Node mask size on disk is 1.363 Kb
Compression options for node `mask`:
complevel=9, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (6, 100, 100)
Lossy compression and data normalization¶
The compression filters used above preserve exactly the original values of the stored data. However, it is also possible with specific filters a lossy compression, which remove non relevant part of the data. As a result, data compression ratio is usually strongly increased, at the cost that stored data is no longer exactly equal to the inputed data array.
One of the most important feature of data array that increase their compressibility, is the presence of patterns in the data. If a value or a serie of values is repeated multiple times throughout the data array, data compression can be very efficient (the pattern can be stored only once).
Numerical simulation and measurement tools usually output data in a standard simple or double precision floating point numbers, yiedling data arrays with values that have a lot of digits. Typically, these values are all different, and hence these arrays cannot be efficiently compressed.
The Amitex_stress_1 and Amitex_strain_1 data array are two tensor fields outputed by a continuum mechanics FFT-based solver, and typical fall into this category: they have almost no equal value or clear data pattern.
As you can see above, the best achieved compression ratio is 60% while for the dataset grain_map, the compression is way more efficient, with a best ratio that climbs up to 97% (62 Kb with zlib compressor and compression level of 9, versus an initial data array of 1.945 Mb). This is due to the nature of the grain_map data array, which is a tridimensional map of grains identification number in microstructure of the Titanium sample represented by the dataset. It is hence an array
containing a few integer values that are repeated many times.
Let us analyze these two data arrays values to illustrate this difference:
[12]:
import numpy as np
print(f"Data array `grain_map` has {data['grain_map'].size} elements,"
f"and {np.unique(data['grain_map']).size} different values.\n")
print(f"Data array `Amitex_stress_1` has {data['Amitex_stress_1'].size} elements,"
f"and {np.unique(data['Amitex_stress_1']).size} different values.\n")
Data array `grain_map` has 1000000 elements,and 111 different values.
Data array `Amitex_stress_1` has 6000000 elements,and 5443754 different values.
Lossy compression¶
Usually, the relevant precision of data is only of a few digits, so that many values of the array should be considered equal. The idea of lossy compression is to truncate values up to a desired precision, which increases the number of equal values in a dataset and hence increases its compressibility.
Lossy compression can be applied to floating point data arrays in SampleData datasets using the least_significant_digit compression setting. If you set the value of this option to \(N\), the data will be truncated after the \(N^{th}\) siginificant digit after the decimal point. Let us see an example.
[13]:
# We will store a value of an array to verify how it evolves after compression
original_value = data['Amitex_stress_1'][20,20,20]
# Apply lossy compression
data.get_node_disk_size('Amitex_stress_1')
# Set up compression settings with lossy compression: truncate after third digit adter decimal point
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':3}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
# Get same value after lossy compression
new_value = data['Amitex_stress_1'][20,20,20]
print(f'Original array value: {original_value} \n'
f'Array value after lossy compression: {new_value}')
Node Amitex_stress_1 size on disk is 27.460 Mb
Node Amitex_stress_1 size on disk is 15.253 Mb
Original array value: [ -27.8652935 -48.22042084 1063.31445312 11.1795702 -21.17892647
49.37730026]
Array value after lossy compression: [ -27.86523438 -48.22070312 1063.31445312 11.1796875 -21.17871094
49.37695312]
As you may observe, the compression ratio has been improved, and the retrieved values after lossy compression are effectively equal to the original array up to the third digit after the decimal point.
We will now try to increase the compression ratio by reducing the number of conserved digits to 2:
[14]:
# Set up compression settings with lossy compression: truncate after third digit adter decimal point
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':2}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
# Get same value after lossy compression
new_value = data['Amitex_stress_1'][20,20,20]
print(f'Original array value: {original_value} \n'
f'Array value after lossy compression 2 digits: {new_value}')
Node Amitex_stress_1 size on disk is 12.562 Mb
Original array value: [ -27.8652935 -48.22042084 1063.31445312 11.1795702 -21.17892647
49.37730026]
Array value after lossy compression 2 digits: [ -27.8671875 -48.21875 1063.3125 11.1796875 -21.1796875
49.375 ]
As you can see, the compression ratio has again been improved, now close to 75%. Know, you know how to do to choose the best compromise between lost precision and compression ratio.
Normalization to improve compression ratio¶
If you look more closely to the Amitex_stress_1 array values, you can observe that the value of this array have been outputed within a certain scale of values, which in particular impact the number of significant digits that come before the decimal point. Sometimes precision of the data would require less significant digits than its scale of representation.
In that case, storing the complete data array at its original scale is not necessary, and very inefficient in terms of data size. To optimize storage of such datasets, one can normalize them to a form with very few digits before the decimal point (1 or 2), and stored separately their scale to be able to revert the normalization operation when retrieiving data.
This allows to reduce the total number of significant digits of the data, and hence further improve the achievable compression ratio with lossy compression.
The SampleData class allows you to aplly automatically this operation when applying compression settings to your dataset. All you have to do is add to the compression_option dictionary the key normalization with one of its possible values.
To try it, we will close (and delete) our test dataset and recopy the original file, to apply normalization and lossy compression on the original raw data:
[15]:
# removing dataset to recreate a copy
del data
# creating a copy of the dataset to try out lossy compression methods
data = SD.copy_sample(src_sample_file=dataset_file, dst_sample_file='Test_compression', autodelete=True,
get_object=True, overwrite=True)
SampleData Autodelete:
Removing hdf5 file Test_compression.h5 and xdmf file Test_compression.xdmf
The standard normalization setting will center and reduce the data of an array \(X\) by storing a new array \(Y\) that is:
\(Y = \frac{X - \bar{X}}{\sigma(X)}\)
where \(\bar{X}\) and \(\sigma(X)\) are respectively the mean and the standard deviation of the data array \(X\).
This operation reduces the number of significant digits before the decimal point to 1 or 2 for the large majority of the data array values. After standard normalization, lossy compression will yield much higher compression ratios for data array that have a non normalized scale.
The SampleData class ensures that when data array are retrieved, or visualized, the user gets or sees the original data, with the normalization reverted.
Let us try to apply it to our stress field Amitex_stress_1.
[16]:
# Set up compression settings with lossy compression: truncate after third digit adter decimal point
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':2,
'normalization':'standard'}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
# Get same value after lossy compression
new_value = data['Amitex_stress_1'][20,20,20,:]
# Get in memory value of the node
memory_value = data.get_node('Amitex_stress_1', as_numpy=False)[20,20,20,:]
print(f'Original array value: {original_value} \n'
f'Array value after normalization and lossy compression 2 digits: {new_value}',
f'Value in memory: {memory_value}')
Node Amitex_stress_1 size on disk is 4.954 Mb
Original array value: [ -27.8652935 -48.22042084 1063.31445312 11.1795702 -21.17892647
49.37730026]
Array value after normalization and lossy compression 2 digits: [ -27.50489858 -49.33973504 1064.23692429 9.92624963 -21.26637388
50.47666019] Value in memory: [-0.515625 -0.421875 -0.3203125 -0.5703125 -0.5 2.21875 ]
As you can see, the compression ratio has been strongly improved by this normalization operation, reaching 90%. When looking at the retrieved value after compression, you can see that depending on the field component that is observed, the relative precision loss varies. The third large component value error is less than 1%, which is consistent with the truncation to 2 significant digits. However, it is not the other components, that have smaller values by two or three orders of magnitude, and that are retrieved with larger errors.
This is explained by the fact that the standard normalization option scales the array as a whole. As a result, if there are large differencies in the scale of different components of a vector or tensor field, the precision of the smaller components will be less preserved.
Another normalization option is available for SampleData field arrays, that allows to apply standard normalization individually to each component of a field in order to keep a constant relative precision for each component when applying lossy compression to the field data array.
To use this option, you will need to set the normalization value to standard_per_component:
[17]:
del data
SampleData Autodelete:
Removing hdf5 file Test_compression.h5 and xdmf file Test_compression.xdmf
[18]:
data = SD.copy_sample(src_sample_file=dataset_file, dst_sample_file='Test_compression', autodelete=True,
get_object=True, overwrite=True)
[19]:
# Set up compression settings with lossy compression: truncate after third digit adter decimal point
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':2,
'normalization':'standard_per_component'}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
# Get same value after lossy compression
new_value = data['Amitex_stress_1'][20,20,20,:]
# Get in memory value of the node
memory_value = data.get_node('Amitex_stress_1', as_numpy=False)[20,20,20,:]
print(f'Original array value: {original_value} \n'
f'Array value after normalization per component and lossy compression 2 digits: {new_value}\n',
f'Value in memory: {memory_value}')
Node Amitex_stress_1 size on disk is 8.117 Mb
Original array value: [ -27.8652935 -48.22042084 1063.31445312 11.1795702 -21.17892647
49.37730026]
Array value after normalization per component and lossy compression 2 digits: [ -27.77213549 -48.1522771 1063.34782307 11.2637209 -21.14744986
49.32227028]
Value in memory: [-0.53125 0.484375 0.984375 -0.890625 -0.9453125 -0.0859375]
As you can see, the error in the retrieved array is now less than 1% for each component of the field value. However, the cost was a reduced improvement of the compression ratio.
[20]:
data.print_xdmf()
<!DOCTYPE Xdmf SYSTEM "Xdmf.dtd">
<Xdmf xmlns:xi="http://www.w3.org/2003/XInclude" Version="2.2">
<Domain>
<Grid Name="CellData" GridType="Uniform">
<Topology TopologyType="3DCoRectMesh" Dimensions="101 101 101"/>
<Geometry Type="ORIGIN_DXDYDZ">
<DataItem Format="XML" Dimensions="3">0. 0. 0.</DataItem>
<DataItem Format="XML" Dimensions="3">0.00122 0.00122 0.00122</DataItem>
</Geometry>
<Attribute Name="grain_map" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="100 100 100" NumberType="Int" Precision="16">Test_compression.h5:/CellData/grain_map</DataItem>
</Attribute>
<Attribute Name="mask" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="100 100 100" NumberType="Int" Precision="uint8">Test_compression.h5:/CellData/mask</DataItem>
</Attribute>
<Attribute Name="grain_map_raw" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="100 100 100" NumberType="Int" Precision="16">Test_compression.h5:/CellData/grain_map_raw</DataItem>
</Attribute>
<Attribute Name="uncertainty_map" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="100 100 100" NumberType="Int" Precision="uint8">Test_compression.h5:/CellData/uncertainty_map</DataItem>
</Attribute>
<Attribute Name="Amitex_stress_1" AttributeType="Tensor6" Center="Cell">
<DataItem ItemType="Function" Function="($2*$0) + $1" Dimensions="100 100 100 6">
<DataItem Format="HDF" Dimensions="100 100 100 6" NumberType="Float" Precision="64">Test_compression.h5:/CellData/Amitex_output_fields/Amitex_stress_1</DataItem>
<DataItem Format="HDF" Dimensions="100 100 100 6" NumberType="Float" Precision="64">Test_compression.h5:/CellData/Amitex_output_fields/Amitex_stress_1_norm_mean</DataItem>
<DataItem Format="HDF" Dimensions="100 100 100 6" NumberType="Float" Precision="64">Test_compression.h5:/CellData/Amitex_output_fields/Amitex_stress_1_norm_std</DataItem>
</DataItem>
</Attribute>
</Grid>
<Grid Name="grains_mesh" GridType="Uniform">
<Geometry Type="XYZ">
<DataItem Format="HDF" Dimensions="92755 3" NumberType="Float" Precision="64">Test_compression.h5:/MeshData/grains_mesh/Geometry/Nodes</DataItem>
</Geometry>
<Topology TopologyType="Tetrahedron" NumberOfElements="490113">
<DataItem Format="HDF" Dimensions="1960452 " NumberType="Int" Precision="64">Test_compression.h5:/MeshData/grains_mesh/Geometry/Elements</DataItem>
</Topology>
<Attribute Name="grains_mesh_elset_ids" AttributeType="Scalar" Center="Cell">
<DataItem Format="HDF" Dimensions="490113 1" NumberType="Float" Precision="64">Test_compression.h5:/MeshData/grains_mesh/grains_mesh_elset_ids</DataItem>
</Attribute>
</Grid>
</Domain>
</Xdmf>
As you can see, the Amitex_stress_1 Attribute node data in the dataset XDMF file is now provided by a Function item type, involving three data array with the original field shape. This function computes:
\(X' = Y*\sigma(X) + \bar{X}\)
where \(*\) and \(+\) are element-wise product and addition operators for multidimensional arrays. This operation allows to revert the component-wise normalization of data. The Paraview software is able to interpret this syntax of the XDMF format and hence, when visualizing data, you will see the values with the original scaling.
This operation required the creation of two large arrays in the dataset, that the store the mean and standard deviation of each component of the field, repeted for each spatial dimensions of the field data array. It is mandatory to allow visualization of the data with the right scaling in Paraview. However, as these array contain a very low amount of data (\(2*N_c\): two times de number of components of the field), they can be very easily compressed and hence do not significantly affect the storage size of the data item, as you may see below:
[21]:
data.get_node_disk_size('Amitex_stress_1')
data.get_node_disk_size('Amitex_stress_1_norm_std')
data.get_node_disk_size('Amitex_stress_1_norm_mean')
Node Amitex_stress_1 size on disk is 8.117 Mb
Node Amitex_stress_1_norm_std size on disk is 75.781 Kb
Node Amitex_stress_1_norm_mean size on disk is 84.473 Kb
[21]:
(84.47265625, 'Kb')
Changing the chunksize of a node¶
Compressing all fields when adding Image or Mesh Groups¶
Changing the chunksize of a data array with SampleData is very simple. You just have to pass as a tuple the news shape of the chunks you want for your data array, and pass it as an argument to the set_chunkshape_and_compression or set_nodes_compression_chunkshape:
[22]:
data.print_node_compression_info('Amitex_stress_1')
data.get_node_disk_size('Amitex_stress_1')
Compression options for node `Amitex_stress_1`:
complevel=9, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (1, 27, 100, 6)
Node Amitex_stress_1 size on disk is 8.117 Mb
[22]:
(8.117431640625, 'Mb')
[23]:
# Change chunkshape of the array
compression_options = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':2,
'normalization':'standard_per_component'}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', chunkshape=(10,10,10,6),
compression_options=compression_options)
data.get_node_disk_size('Amitex_stress_1')
data.print_node_compression_info('Amitex_stress_1')
Node Amitex_stress_1 size on disk is 7.950 Mb
Compression options for node `Amitex_stress_1`:
complevel=9, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=None
--- Chunkshape: (10, 10, 10, 6)
As you can see, the chunkshape has been changed, which has also affected the memory size of the compressed data array. We have indeed reduced the number of chunks in the dataset, which reduces the number of data to store. This modification can also improve or deteriorate the I/O speed of access to your data array in the dataset. The reader is once again refered to dedicated documents to know more ion this matter: here and here.
Compression data and setting chunkshape upon creation of data items¶
Until here we have only modified the compression settings of already existing data items. In this process, the data items are replaced by the new compressed version of the data, which is a costly operation. For this reason, if they are known in advance, it is best to apply the compression filters and appropriate chunkshape when creating the data item.
If you have read through all the tutorials of this user guide, you should know all the method that allow to create data items in your datasets, like add_data_array, add_field, ædd_mesh… All of these methods accept the two arguments chunkshape and compression_options, that work exaclty as for the set_chunkshape_and_compression or set_nodes_compression_chunkshape methods. You hence use them to create your data items directly with the appropriate compression settings.
Let us see an example. We will get an array from our dataset, and try to recreate it with a new name and some data compression:
[24]:
# removing dataset to recreate a copy
del data
# creating a copy of the dataset to try out lossy compression methods
data = SD.copy_sample(src_sample_file=dataset_file, dst_sample_file='Test_compression', autodelete=True,
get_object=True, overwrite=True)
SampleData Autodelete:
Removing hdf5 file Test_compression.h5 and xdmf file Test_compression.xdmf
[25]:
# getting the `orientation_map` array
array = data['Amitex_stress_1']
[26]:
# create a new field for the CellData image group with the `orientation_map` array and add compression settings
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True, 'least_significant_digit':2,
'normalization':'standard'}
new_cshape = (10,10,10,3)
# Add data array as field of the CellData Image Group
data.add_field(gridname='CellData', fieldname='test_compression', indexname='testC', array=array,
chunkshape=new_cshape, compression_options=compression_options, replace=True)
[26]:
/CellData/test_compression (CArray(100, 100, 100, 6), shuffle, zlib(1)) 'testC'
atom := Float64Atom(shape=(), dflt=0.0)
maindim := 0
flavor := 'numpy'
byteorder := 'little'
chunkshape := (10, 10, 10, 3)
[27]:
# Check size and settings of new field
data.print_node_info('testC')
data.get_node_disk_size('testC')
data.print_node_compression_info('testC')
NODE: /CellData/test_compression
====================
-- Parent Group : CellData
-- Node name : test_compression
-- test_compression attributes :
* data_normalization : standard
* empty : False
* field_dimensionality : Tensor6
* field_type : Element_field
* node_type : field_array
* normalization_mean : 178.3664165861844
* normalization_std : 399.26558093714374
* padding : None
* parent_grid_path : /CellData
* transpose_components : [0, 3, 5, 1, 4, 2]
* transpose_indices : [2, 1, 0, 3]
* xdmf_fieldname : test_compression
* xdmf_gridname : CellData
-- content : /CellData/test_compression (CArray(100, 100, 100, 6), shuffle, zlib(1)) 'testC'
-- Compression options for node `testC`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=2
--- Chunkshape: (10, 10, 10, 3)
-- Node memory size : 4.750 Mb
----------------
Node testC size on disk is 4.750 Mb
Compression options for node `testC`:
complevel=1, complib='zlib', shuffle=True, bitshuffle=False, fletcher32=False, least_significant_digit=2
--- Chunkshape: (10, 10, 10, 3)
The node has been created with the desired chunkshape and compression filters.
Repacking files¶
We now recreate a new copy of the original dataset, and try to reduce the size of oll heavy data item, to reduce as much as possible the size of our dataset.
[28]:
# removing dataset to recreate a copy
del data
# creating a copy of the dataset to try out lossy compression methods
data = SD.copy_sample(src_sample_file=dataset_file, dst_sample_file='Test_compression', autodelete=True,
get_object=True, overwrite=True)
SampleData Autodelete:
Removing hdf5 file Test_compression.h5 and xdmf file Test_compression.xdmf
[29]:
compression_options1 = {'complib':'zlib', 'complevel':9, 'shuffle':True, 'least_significant_digit':2,
'normalization':'standard'}
compression_options2 = {'complib':'zlib', 'complevel':9, 'shuffle':True}
data.set_chunkshape_and_compression(nodename='Amitex_stress_1', compression_options=compression_options1)
data.set_nodes_compression_chunkshape(node_list=['grain_map', 'grain_map_raw','mask'],
compression_options=compression_options2)
Now that we have compressed a few of the items of our dataset, the disk size of its HDF5 file should have diminished. Let us check again the size of its data items, and of the file:
[30]:
data.print_dataset_content(short=True)
data.get_file_disk_size()
Printing dataset content with max depth 3
|--GROUP Amitex_Results: /Amitex_Results (Group)
--NODE mean_strain: /Amitex_Results/mean_strain (data_array) ( 63.984 Kb)
--NODE mean_stress: /Amitex_Results/mean_stress (data_array) ( 63.984 Kb)
--NODE rms_strain: /Amitex_Results/rms_strain (data_array) ( 63.984 Kb)
--NODE rms_stress: /Amitex_Results/rms_stress (data_array) ( 63.984 Kb)
--NODE simulation_iterations: /Amitex_Results/simulation_iterations (data_array) ( 64.000 Kb)
--NODE simulation_time: /Amitex_Results/simulation_time (data_array) ( 64.000 Kb)
|--GROUP CellData: /CellData (3DImage)
|--GROUP Amitex_output_fields: /CellData/Amitex_output_fields (Group)
--NODE Amitex_stress_1: /CellData/Amitex_output_fields/Amitex_stress_1 (field_array) ( 4.954 Mb)
--NODE Field_index: /CellData/Field_index (string_array) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array) ( 97.479 Kb)
--NODE grain_map_raw: /CellData/grain_map_raw (field_array) ( 134.527 Kb)
--NODE mask: /CellData/mask (field_array) ( 1.363 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
--NODE uncertainty_map: /CellData/uncertainty_map (field_array) ( 996.094 Kb)
|--GROUP GrainData: /GrainData (Group)
--NODE GrainDataTable: /GrainData/GrainDataTable (structured array) ( 63.984 Kb)
|--GROUP MeshData: /MeshData (emptyMesh)
|--GROUP grains_mesh: /MeshData/grains_mesh (3DMesh)
|--GROUP Geometry: /MeshData/grains_mesh/Geometry (Group)
--NODE grains_mesh_elset_ids: /MeshData/grains_mesh/grains_mesh_elset_ids (field_array) ( 624.343 Kb)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
File size is 83.089 Mb for file
Test_compression.h5
[30]:
(83.08916091918945, 'Mb')
The file size has not changed, surprisingly, even if the large Amitex_stress_1 array has been shrinked from almost 50 Mo to roughly 5 Mo. This is due to a specific feature of HDF5 files: they do not free up the memory space that they have used in the past. The memory space remains associated to the file, and is used in priority when new data is written into the dataset.
After changing the compression settings of one or several nodes in your dataset, if that induced a reduction of your actual data memory size, and that you want your file to be smaller on disk. To retrieve the fried up memory spacae, you may repack your file (overwrite it with a copy of itself, that has just the size require to store all actual data).
To do that, you may use the SampleData method repack_h5file:
[31]:
data.repack_h5file()
data.get_file_disk_size()
File size is 33.966 Mb for file
Test_compression.h5
[31]:
(33.96598815917969, 'Mb')
You see that repacking the file has allowed to free some memory space and reduced its size.
Note
Note that the size of the file is larger than the size of data items printed by print_dataset_content. This extra size is the memory size occupied by the data array storing Element Tags for the mesh grains_mesh. Element tags are not printed by the printing methods as they can be very numerous and pollute the lecture of the printed information.
Once again, you should repack your file at carefully chosen times, as is it a very costly operation for large datasets. The SampleData class constructor has an autorepack option. If it is set to True, the file is automatically repacked when closing the dataset.
We can now close our dataset, and remove the original unarchived file:
[32]:
# remove SampleData instance
del data
SampleData Autodelete:
Removing hdf5 file Test_compression.h5 and xdmf file Test_compression.xdmf
[33]:
os.remove(dataset_file+'.h5')
os.remove(dataset_file+'.xdmf')
7 - Sample Data Inheritance - The Microstructure Class¶
This sixth Notebook will introduce you to:
the SampleData class inheritance mechanisms
the Microstructure class of the pymicro package
the Microstructure class data model and how to browse through its content
the different ways that exist to create a Microstructure object
the basic methods to get and set the data that compose a Microstructure object
Note
Throughout this notebook, it will be assumed that the reader is familiar with the overview of the SampleData file format and data model presented in the first notebook of this User Guide of this User Guide.
SampleData Inheritance¶
The SampleData class implements a generic interface between users, numeric tools, and HDF5/XDMF multimodal datasets for material science and mechanics. It allows to create empty datasets or open existing ones, and leaves entirely to the user the definition of the dataset internal content and organization.
For specific and repeated applications, that always involve the same type of datasets, it may be convenient to standardize and predefine the internal organization of the dataset. For instance, to manage the data of a serie of material samples that are studied through SEM, EBSD imaging, and numerical simulation of the imaging digital twins, you will to define for each the same Image Group to store the imaging outputs, and a Mesh or Image group at least to store the simulation output.
For such cases, it becomes convenient to define a more specific interface, for which all the internal organization of datasets (their data mode), is already defined. For that purpose, the SampleData class offers the possibility to create inherited classes with a predefined data model through two particular and simple mechanisms, that are the subject of the present section.
Custom Data Model¶
The SampleData class defines a minimal data model for all the datasets created with the class. This data model is a collection of data item indexnames, pathes and types, in the form of two dictionaries. The keys of these two dictionaries must be identical, and define all the indexnames of the items in the data model. There values are:
minimal_content_index_dic: the path of each data item in the data modelminimal_content_type_dic: the type of each data item in the data model
The content index dictionary¶
Each item of this dictionary will define a data item of the model. Its key will be the indexname given to the data item in the dataset, and the item value must be a string giving a valid path for the data item in the dataset. For a path to be valid, the different levels of depth in it must have been declared within the dictionary.
This dictionary should hence look like this:
minimal_content_index_dic = {'item1': '/path_to_item1',
'item2': '/path_to_item1/path_to_item2',
'item3': '/path_to_item3',
'...': '...',}
An item of the form 'wrongitem': '/undeclared_item/path_to_wrong_item' would have been a non valid path.
The dictionary example just above would lead to the creation of at least 3 data items, with names item1, item2 and item3, with items 1 and 3 being directly attached to the dataset Root Group, and the item 2 being a children of item 1.
The content type dictionary¶
The second dictionary that has to be declared must have the same keys as the minimal_content_index_dic. Its values must be valid SampleData data item types. This dictionary will determine the type of data item that will be automatically created at the dataset creation, by the subclass.
Possible values and associated data types are (see previous tutorials for description of these data types):
‘Group’: creates a HDF5 group data item
‘2DImage’, ‘3DImage’, or ‘Image’: creates an empty Image group
‘2DMesh’, ‘3DMesh’, ‘Mesh’: creates an empty Mesh group
‘data_array’: creates an empty Data Array
‘field_array’: creates an empty Field Array (its path must be a children of a an Image or Mesh group)
‘string_array’: creates an empty String Array
a
numpy.dtypeor atables.IsDescriptionclass (see here and tutorial 3):
This dictionary should look like this (assuming that it corresponds to the content dictionary of the subsection above):
minimal_content_index_dic = {'item1': '3DMesh',
'item2': 'field_array',
'item3': 'data_array',
'...': '...',}
In this case, the first item would be created as a Mesh Group, the second will be created as a field data item stored in this mesh, and the last as a data array attached to the Root Group.
These two dictionaries are returned by the minimal_data_model method of the SampleData class. They are used during the dataset object initialization, to create the prescribed data model, and populate it with empty objects, with the right names and organization. This allows to prepend a set of names and pathes that form a particular data model that all objects created by the class should have.
It is labelled as a minimal data model, as it only prescribes the data items and organization that will be present in each dataset of the subclass. The user is free to enrich the datasets created with this class with any additional data item that he would want to add.
In the SampleData code, they are returned empty, so that no actual data model is created within a new SampleData dataset. This method is actually designed to create subclasses of SampleData associated to a specific data model. To achieve this, you have to:
Create a new class, inherited from SampleData
Override the
minimal_data_modelmethod and write your data model in the two dictionaries returned by the class
You will then get a class derived from SampleData (hence with all its methods and features), that creates datasets with this prescribed data model. You will see an example of it in the next section dedicated to the Microstructure class, which is designed this way.
Custom initialization¶
The other mechanisms that is important to design subclasses of SampleData, is the specification of all initialization commands that must be run each time the dataset files are closed and opened again (this happens for instance when repacking the dataset, or calling the pause_for_visualization method). These operations can include, for instance, the definition of class attributes that points toward a specific node in the dataset, the loading of data from the dataset files in some class
attributes, some sanity checks on the data etc…..
All these operations must be implemented in the _after_file_open method of the subclass. Again, the Microstructure class described in the next section will provide an example.
The Microstructure Class¶
The Microstructure class has been designed to handle multimodal datasets representing polycrystalline material samples. These materials have a specific microstructure composed of crystalline grains, that are characterized by a specific geometry and a crystalline orientation. The microstructure of polycrystalline materials strongly determines their physical and mechanical properties, and is thus extensively studied by material scientists.
The Microstructure class offers methods to easily manipulate multiomdal 4D data of granular material samples, in particular regarding geometrical and crystallographic aspects of data management and processing. As this type of data is a particular case of the of datasets for which the SampleData class has been designed, the Microstructure has been derived from the SampleData class.
A SampleData children class¶
Indeend, the Microstructure class is a subclass of the SampleData class:
class Microstructure(SampleData):
As a children of SampleData, it inherits all of its features: a Microstructure object is associated with a HDF5/XDMF file pair, and allows to create/get/remove/compress all types of data items handeled by the SampleData class, presented in the previous tutorials. As a children of SampleData, the Microstructure class benefits of the two mechanisms presented in the first section of this tutorial. We will see now how they are implemented for this class.
The minimal data model¶
This subsection will present the Microstructure class data model, and will also serve as a demonstrator of the data model mechanism described in the first section of this tutorial.
The code of the minimal_data_model method of the Microstructure class contains the following declaration of the data model dictionaries:
minimal_content_index_dic = {'Image_data': '/CellData',
'grain_map': '/CellData/grain_map',
'phase_map': '/CellData/phase_map',
'mask': '/CellData/mask',
'Mesh_data': '/MeshData',
'Grain_data': '/GrainData',
'GrainDataTable': '/GrainData/GrainDataTable',
'Phase_data': '/PhaseData'}
minimal_content_type_dic = {'Image_data': '3DImage',
'grain_map': 'field_array',
'phase_map': 'field_array',
'mask': 'field_array',
'Mesh_data': 'Mesh',
'Grain_data': 'Group',
'GrainDataTable': GrainData,
'Phase_data': 'Group'}
You can see that this data model contains a GrainData data item type. This is a tables.IsDescription object, inducing hence the creation of a Structured Array data item. The definition of this description in the Microstructure class code will be provided further in this tutorial, in the subsection dedicated to the Grain Data Table data item of the class.
As you can see, the data model contains one Image Group, with three fields declared, one Mesh Group, two Groups, one containing a Structured Array data item. It will be detailed in the next section of this tutorial.
The after file open operations¶
The _after_file_open method of the Microstructure is composed of the following lines of code:
def _after_file_open(self):
"""Initialization code to run after opening a Sample Data file."""
self.grains = self.get_node('GrainDataTable')
if self._file_exist:
self.active_grain_map = self.get_attribute('active_grain_map',
'CellData')
if self.active_grain_map is None:
self.set_active_grain_map()
self._init_phase(phase)
if not hasattr(self, 'active_phase_id'):
self.active_phase_id = 1
else:
self.set_active_grain_map()
self._init_phase(phase)
self.active_phase_id = 1
return
This method creates a class attribute grains that is associated with the Structured Array node GrainDataTable. Therefore, this attribute is an alias for the Pytable Node object associated to this array (see here how to handle these objects).
This grains attribute is used by many of the class methods, and hence must always be properly associated to the GrainDataTable. To ensure that it is the case, it is initialized in the _after_file_open method. Hence, this attribute is initialized at dataset opening, but also after in the methods that close and re-open the dataset (like pause_for_visualization or repack_h5file.
This methods also ensures that the class _phases nd active_grain_map attributes are synchronized with the dataset content, each time that the file is opened. Those two arguments are discussed later on in this tutorial.
Microstructure Data Model & Getting Microstructure data¶
What data defines a Microstructure ?¶
To define the microstructure and geometry of a polycrystalline sample, the following information are needed:
the description of the geometry of the sample
inside the sample, the description of the crystalline phases that compose the sample
within each phase, the description of the grains that compose the phase
To easily identify these elements, in a ``Microstructure`` dataset, each phase and grain in the microstructure has a identification number. These numbers are used to gather phasewise or grainwise data in data arrays, but is used as well in fields to describe the geometry of these phases/grains. These features are detailed in the following.
We will now review the various elements of the Microstructure class data model. To illustrate them, we will use a dataset from the example data base of the pymicro package.
Opening a Microstructure file¶
Opening an already existing microstructure dataset is done exactly like opening a SampleData dataset. We will look at a material science dataset that is part of the Pymicro example datasets, that is already used in the previous tutorial on data compression.
[1]:
from config import PYMICRO_EXAMPLES_DATA_DIR # import file directory path
import os
dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure') # test dataset file path
tar_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure.tar.gz') # dataset archive path
This file is zipped in the package to reduce its size. We will have to unzip it to use it and learn how to reduce its size with the SampleData methods. If you are just reading the documentation and not executing it, you may just skip this cell and the next one.
[2]:
# Save current directory
cwd = os.getcwd()
# move to example data directory
os.chdir(PYMICRO_EXAMPLES_DATA_DIR)
# unarchive the dataset
os.system(f'tar -xvf {tar_file}')
# get back to UserGuide directory
os.chdir(cwd)
Let us now open the dataset, using the Microstructure class constructor this time:
[3]:
# import SampleData class
from pymicro.crystal.microstructure import Microstructure
# import Numpy
import numpy as np
[4]:
# Open Microstructure dataset
micro = Microstructure(filename=dataset_file)
The dataset is now open, we can now look at its content:
[5]:
print(micro)
Microstructure
* name: Ti_grade2_ET7_3_crop100
* lattice: Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : GrainDataTable H5_Path : /GrainData/GrainDataTable
Name : Grain_data H5_Path : /GrainData
Name : Image_data H5_Path : /CellData
Image_data aliases --> `CellData`
Name : Image_data_Amitex_stress_1 H5_Path : /CellData/Amitex_output_fields/Amitex_stress_1
Name : Image_data_grain_map H5_Path : /CellData/grain_map
Name : Image_data_grain_map_raw H5_Path : /CellData/grain_map_raw
Name : Image_data_mask H5_Path : /CellData/mask
Name : Image_data_uncertainty_map H5_Path : /CellData/uncertainty_map
Name : Mesh_data H5_Path : /MeshData
Name : Phase_data H5_Path : /PhaseData
Name : fft_fields H5_Path : /CellData/Amitex_output_fields
Name : fft_sim H5_Path : /Amitex_Results
Name : grain_map H5_Path : /CellData/grain_map
Name : grains_mesh H5_Path : /MeshData/grains_mesh
Name : grains_mesh_Geometry H5_Path : /MeshData/grains_mesh/Geometry
Name : grains_mesh_grains_mesh_elset_ids H5_Path : /MeshData/grains_mesh/grains_mesh_elset_ids
Name : mask H5_Path : /CellData/mask
Name : mean_strain H5_Path : /Amitex_Results/mean_strain
Name : mean_stress H5_Path : /Amitex_Results/mean_stress
Name : phase_01 H5_Path : /PhaseData/phase_01
Name : phase_map H5_Path : /CellData/phase_map
Name : rms_strain H5_Path : /Amitex_Results/rms_strain
Name : rms_stress H5_Path : /Amitex_Results/rms_stress
Name : simulation_iterations H5_Path : /Amitex_Results/simulation_iterations
Name : simulation_time H5_Path : /Amitex_Results/simulation_time
Printing dataset content with max depth 3
|--GROUP Amitex_Results: /Amitex_Results (Group)
--NODE mean_strain: /Amitex_Results/mean_strain (data_array) ( 63.984 Kb)
--NODE mean_stress: /Amitex_Results/mean_stress (data_array) ( 63.984 Kb)
--NODE rms_strain: /Amitex_Results/rms_strain (data_array) ( 63.984 Kb)
--NODE rms_stress: /Amitex_Results/rms_stress (data_array) ( 63.984 Kb)
--NODE simulation_iterations: /Amitex_Results/simulation_iterations (data_array) ( 64.000 Kb)
--NODE simulation_time: /Amitex_Results/simulation_time (data_array) ( 64.000 Kb)
|--GROUP CellData: /CellData (3DImage)
|--GROUP Amitex_output_fields: /CellData/Amitex_output_fields (Group)
--NODE Amitex_stress_1: /CellData/Amitex_output_fields/Amitex_stress_1 (field_array) ( 49.438 Mb)
--NODE Field_index: /CellData/Field_index (string_array) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array) ( 1.945 Mb)
--NODE grain_map_raw: /CellData/grain_map_raw (field_array) ( 1.945 Mb)
--NODE mask: /CellData/mask (field_array) ( 996.094 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
--NODE uncertainty_map: /CellData/uncertainty_map (field_array) ( 996.094 Kb)
|--GROUP GrainData: /GrainData (Group)
--NODE GrainDataTable: /GrainData/GrainDataTable (structured array) ( 63.984 Kb)
|--GROUP MeshData: /MeshData (emptyMesh)
|--GROUP grains_mesh: /MeshData/grains_mesh (3DMesh)
|--GROUP Geometry: /MeshData/grains_mesh/Geometry (Group)
--NODE grains_mesh_elset_ids: /MeshData/grains_mesh/grains_mesh_elset_ids (field_array) ( 624.343 Kb)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
We can also print the description attribute of the dataset to read information written by its creators:
[6]:
print(micro.get_description())
Dataset representing a subvolume of a grade 2 Titanium sample.
* Obtained by cropping a full DCT microstructure of the sample to get a 100x100x100 voxels sub image
* Contains a mesh of the polycrystalline microstructure, conformal with the grains geometry
* Contains FFT simulation results
* Contains DCT imaging outputs
As you can see, this dataset is a digital twin of a real polycristalline microstructure of a grade 2 Titanium sample, gathering both experimental and numerical data obtained through Diffraction Contrast Tomography imaging, and FFT-based mechanical simulation.
This dataset will serve as an illustration of the Microstructure class data model. In the dataset content printed above, we indeed recognize the 4 groups defined by the minimal_data_model. They all contain essential data to represent the microstructure, and are presented in details in the next subsections.
Phases¶
Obviously, the first information that must be provided when describing a polycrystalline material sample, is its composition. Polycrystalline samples may be composed of multiple phases, having all distinct crystallographic and physical properties. The PhaseData group in the Microstructure data model is designed to store this essential information on the various material phases composing the sample, and their properties.
We can see in the dataset content above that this Group only contains another group phase_01. Their content is:
[7]:
micro.print_node_info('PhaseData')
micro.print_node_info('phase_01')
GROUP PhaseData
=====================
-- Parent Group : /
-- Group attributes :
* group_type : Group
-- Childrens : phase_01,
----------------
GROUP phase_01
=====================
-- Parent Group : PhaseData
-- Group attributes :
* description : Polycrystalline alpha grade 2 Titanium phase
* elastic_constants : [162000.0, 92000.0, 69000.0, 180000.0, 46700.0]
* elastic_constants_unit : MPa
* formula : Ti
* group_type : Group
* lattice_parameters : [1.0, 1.0]
* lattice_parameters_unit : nm
* name : Ti grade 2
* phase_id : 1
* symmetry : hexagonal
-- Childrens :
----------------
As you may observe, these groups only contain data in the form of attributes, i.e. metadata, on the crystalline phase that compose the dataset. The ``PhaseData`` group has as many children groups as there are phases in the dataset. Each one of these groups has a predefined name, following the pattern phase_+ phase_number. They provide the name, number, and crystallographic (symmetry, lattice parameters), chemical (formula) and physical information (elasticity constants) of each phase.
Warning
Working with multiple phases in Microstructure datasets is still a feature under development, not yet stable. It is possible to have mutliple phase groups within a dataset, but some class methods working with Phases may not yet be compatible with multiple phase datasets.
Getting Phase objects from a dataset¶
The medata content of a phase_XX Group can be retrieved as a Pymicro Phase object, which is a container object, with the get_phase method:
[8]:
phase_01 = micro.get_phase(phase_id=1)
print(phase_01)
Phase 1 (Ti grade 2)
-- Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0
-- elastic constants: [162000.0, 92000.0, 69000.0, 180000.0, 46700.0]
The class has a _phase private attribute, that is a list of Pymicro CrystallinePhase objects, synchronized with the content of the PhaseData group. Calling the sync_phases method ensures this synchronization. It also ensured by the after_file_open method at each time that the file is opened.
The CellData Group¶
Group aim and content¶
We will move now to the description of the CellData group. As you can see, it is an Image Group (3D image in the case of this example file). The ``CellData`` group is aimed at storing descriptions of the polycristalline microstructure of the sample, in the form of images. They may describe the geometry and orientation of the grains, the presence of damage or the micromechanical state of the sample.
These images can be 2D or 3D, and be:
outputs of real imaging experiments (DCT, CT, EBSD …)
digitally generated microstructure images (Voronoi tesselations, outputs from softwares like Neper, DREAM3D …)
outputs of numerical simulations techniques that provide images (such as FFT-based solvers)
This group is a classical Image Group, with the usual set of attributes (see dedicated tutorial):
[9]:
micro.print_node_info('CellData')
GROUP CellData
=====================
-- Parent Group : /
-- Group attributes :
* active_grain_map : grain_map
* description :
* dimension : [100 100 100]
* empty : False
* group_type : 3DImage
* nodes_dimension : [101 101 101]
* nodes_dimension_xdmf : [101 101 101]
* origin : [0. 0. 0.]
* spacing : [0.00122 0.00122 0.00122]
* xdmf_gridname : CellData
-- Childrens : Amitex_output_fields, Field_index, grain_map, grain_map_raw, mask, phase_map, uncertainty_map,
----------------
Group minimal content¶
The minimal_data_model of the class contains three fields data items that are attached to the CellData group. These scalar fields of integers allow to completely describe the geometry and microstructure of the polycrystalline material sample associated to the ``Microstructure`` dataset. They are:
mask: a field describing the geometry of the sample. It has a 1 value inside the sample, and a 0 value outside the sample (typically, segmented X-ray Contrast Tomography outputs can be used as Microstructure mask).phase_map: a field indicating for each pixel/voxel the ID of the phase in which it is located. In areas of the Image that are not part of the material sample, the grain_map field takes a zero or negative value.grain_map: a field indicating for each pixel/voxel the ID of the grain in which it is located. In areas of the Image that are not part of the material sample, the grain_map field takes a zero or negative value.
You can retrieve the value of these fields as shown in the previous tutorials (get_field, attribute or dict like access). You can also use the Microstructure class dedicated methods, that ensure to return arrays of dimension 3 (with a 1 sized 3 dimension when getting 2D images fields), used just below:
[10]:
mask = micro.get_mask()
phase_map = micro.get_phase_map()
grain_map = micro.get_grain_map()
print(f'List of values in the mask:\n {np.unique(mask)}\n')
print(f'List of values in the phase map:\n {np.unique(phase_map)}\n')
print(f'List of values in the grain map:\n {np.unique(grain_map)}\n')
print(f'Number of grains in the grain map: {len(np.unique(grain_map))}')
List of values in the mask:
[1]
List of values in the phase map:
[None]
List of values in the grain map:
[ 2 3 34 43 46 73 98 107 137 164 196 223 245 252
310 327 345 384 433 435 437 452 484 485 488 503 540 649
672 684 692 744 783 802 824 849 851 859 870 903 953 998
1019 1025 1054 1059 1061 1062 1096 1124 1163 1210 1251 1270 1306 1335
1346 1384 1392 1407 1438 1494 1521 1529 1534 1560 1564 1617 1634 1642
1650 1686 1772 1855 1893 1897 1900 1905 2031 2059 2225 2307 2335 2508
2558 2726 2770 2852 2895 2929 2934 3025 3048 3053 3074 3101 3154 3183
3319 3448 3575 3587 3604 3613 3620 3643 3724 3812 3827 3918 3960]
Number of grains in the grain map: 111
As you can see, the phase map is not defined here, which is not a problem when the sample only has one crystalline phase (which is the case here). The mask also contains only 1 values, which means that the dataset represents a full cube of the sample microstructure. The grain map contains 111 grain IDs values.
The active grain map class attribute¶
It may be relevant, for some datasets, to store several fields for the same information. For instance, when reconstructing a microstructure from an imaging experiment, the microstructure in the reconstruction process raw output may be incomplete, and require some image processing. In that case, storing both versions of the array (raw reconstruction output and image after processing) is relevant.
In the specific case of the grain_map field, if you have several arrays for this information in your dataset, you may use the active_grain_map class attribute to specify which one is to be considered as the sample main grain map by setting the Microstructure.active_grain_map attribute to a name of this field data item. This implies that the get_grain_map method will return specifically this field, and that all Microstructure methods working with the sample grain map will use
this field as well.
Our example dataset has two version of the grain map information: the grain_map and grain_map_raw fields. They represent the processed and raw version of the DCT reconstruction algorithm output data. We will use them to illustrate the role of the active_grain_map attribute.
You can set its value with the set_active_grain_map method:
[11]:
# print active grain map
print(f'The active grain map is {micro.active_grain_map}')
# change active grain map
micro.set_active_grain_map('grain_map_raw')
print(f'The active grain map is {micro.active_grain_map}')
The active grain map is grain_map
The active grain map is grain_map_raw
We can now chekc the return of get_grain_map:
[12]:
# get both grain maps arrays
grain_map = micro['grain_map']
grain_map_raw = micro['grain_map_raw']
# check class method return grain map
print(f'Is the active grain map equal to "grain_map" array ? {np.all(grain_map == micro.get_grain_map())}')
print(f'Is the active grain map equal to "grain_map_raw" array ? {np.all(grain_map_raw == micro.get_grain_map())}\n')
# let us change again the grain map and redo the test
micro.set_active_grain_map('grain_map')
print(f'Is the active grain map equal to "grain_map" array ? {np.all(grain_map == micro.get_grain_map())}')
print(f'Is the active grain map equal to "grain_map_raw" array ? {np.all(grain_map_raw == micro.get_grain_map())}')
Is the active grain map equal to "grain_map" array ? False
Is the active grain map equal to "grain_map_raw" array ? True
Is the active grain map equal to "grain_map" array ? True
Is the active grain map equal to "grain_map_raw" array ? False
Note
Note that the same mechanism will be implemented soon for the phase map field.
Visualize sample microstructure image: View Slice method¶
These three arrays, as Field data items of the CellData group can be visualized by opening the HDF5 dataset file with the Paraview software, as the Microstructure class inherits this feature from SampleData. In addition, the Microstructure class provides a visualization method to plot a slice of the microstructure (hence the whole image if it is a 2D microstructure) using the grain_map and mask arrays.
This method is the view_slice_method. It has many input arguments, but can be called without any one. In that, you should obtain an output like this:
If you are executing interactively this Notebook, you may try to reproduce this figure by setting the display option to True in the following line:
[13]:
micro.view_slice(display=False)
using slice value 50
[13]:
(<Figure size 640x480 with 1 Axes>, <AxesSubplot:xlabel='X', ylabel='Y'>)
When no argument is provided, the method plots the middle (X,Y)-wise slice of the sample grain map, with a random color map for the grain ids, and shows the mask on the foreground in transparency mode. Here the mask cannot be actually seen as it is uniform on the slice.
Let us plot a slice of an other example dataset that allows to see the mask:
[14]:
micro2 = Microstructure(filename=os.path.join(PYMICRO_EXAMPLES_DATA_DIR,'t5_dct_slice_data.h5'))
micro2.view_slice(display=False) # change display to True to try to reproduce the figure below
del micro2
using slice value 0
You should get a figure like this one:
The mask shows two regions, a black region representing the outside of the sample, and a red region, representing the sample geometry. The grain map appear below this transparent mask color layer.
The view_slice method has many optional arguments. The most important are:
slice: this argument allows you to choose the index of the slice that is plotted. The default value is the middle slice.color: allows you to chose the color map used to plot the grains.randomis the default value, alternatives aregrain_ids,ipf(inverse pole figure coloring) andschmid(plot intensity of grain maximal schmid factor for the load direction specified by theaxisargument, and for the slip system object provided in theslip_systemargument)show_mask: set to False if you do not want to plot the maskshow_grain_ids: set to True to annotate the grains with their ID numbershow_slip_traces: set to True to annotate the grains with the trace of a slip plane (provided as a slip system object to thehkl_planesargument)display: is set to False, the image is not plotted. The matplotlib figure and axis created are returned anyway by the method.
You will find below two examples. If you are reading through the Notebook version of the documentation, you may try to change the value of these arguments to experiment the various possibilities offered by the view_slice method.
[15]:
import matplotlib.pyplot as plt
[16]:
# set display to True to try reproducing the figure below !
micro.view_slice(slice=15, color='grain_ids', show_grain_ids=True, highlight_ids=[98,953,1019,1335,1534,1560,1905],
display=False)
[16]:
(<Figure size 640x480 with 1 Axes>, <AxesSubplot:xlabel='X', ylabel='Y'>)
You should get:
[17]:
# get one basal slip system to compute its schmid factors and plot them on the slice
from pymicro.crystal.lattice import SlipSystem
lattice = micro.get_phase().get_lattice()
slip_system = lattice.get_slip_systems('basal')[1]
# plot slice with schmid factor colormap
# set display to True to try reproducing the figure below !
micro.view_slice(slice=15, color='schmid', slip_system=slip_system, display=False)
[17]:
(<Figure size 640x480 with 2 Axes>, <AxesSubplot:xlabel='X', ylabel='Y'>)
You should get:
[18]:
# Get two slip planes (basal and one prismatic) to plot slip plane traces on the selected grains,
# coloured with inverse pole figure color map
plane_list = []
slip_system = lattice.get_slip_systems('basal')[0]
plane_list.append(slip_system.get_slip_plane())
slip_system = lattice.get_slip_systems('prism')[1]
plane_list.append(slip_system.get_slip_plane())
# set display to True to try reproducing the figure below !
micro.view_slice(slice=15, color='ipf', slip_system=slip_system, show_slip_traces=True, display=False,
hkl_planes=plane_list, highlight_ids=[98,953,1019,1335,1534,1560,1905])
print(plane_list)
[HKL Plane
Miller indices:
h : 0
k : 0
l : 1
plane normal : [0. 0. 1.]
crystal lattice : Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0, HKL Plane
Miller indices:
h : -1
k : 0
l : 0
plane normal : [-8.66025404e-01 -5.00000000e-01 1.06057524e-16]
crystal lattice : Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0]
You should get:
The GrainData Group¶
Group aim and content¶
We will move now to the description of the GrainData group. As you can see, it is a classical HDF5 Group. The ``GrainData`` group is aimed at storing statistical data describing the sample grains. In the data model as well as in the example dataset, this Group has contains only one data item, the GrainDataTable.
The GrainDataTable is a structured array that contains the statistical data describing the grains. Its description in the Pymicro code is:
class GrainData(tables.IsDescription):
"""
Description class specifying structured storage of grain data in
Microstructure Class, in HDF5 node /GrainData/GrainDataTable
"""
# grain identity number
idnumber = tables.Int32Col() # Signed 32-bit integer
# grain volume
volume = tables.Float32Col() # float
# grain center of mass coordinates
center = tables.Float32Col(shape=(3,)) # float (double-precision)
# Rodrigues vector defining grain orientation
orientation = tables.Float32Col(shape=(3,)) # float (double-precision)
# Grain Bounding box
bounding_box = tables.Int32Col(shape=(3, 2)) # Signed 64-bit integer
As you can see, each row contains:
the identity number of the grain
two columns describing the grain geometry: grain volume, position of grain center of mass
the orientation of the grain provided as a Rodrigues vector
the indices of the grain bounding box in the
CellDataimage field arrays
Getting information on grains from Grain Data Table¶
Like for the CellData group items, you can retrieve the GrainDataTable columns value as shown in the previous tutorials (get_node, attribute or dict like access). You can also use the Microstructure class dedicated methods, used just below:
[19]:
# retrieve table as numpy structured array with dictionary like access
GrainDataTable = micro['GrainDataTable']
# get table columns from class methods and compare to numpy array
grain_ids = micro.get_grain_ids()
print(f'grain ids equal ? {np.all(grain_ids == GrainDataTable["idnumber"])}')
grain_centers = micro.get_grain_centers()
print(f'grain centers equal ? {np.all(grain_centers == GrainDataTable["center"])}')
grain_volumes = micro.get_grain_volumes()
print(f'grain volumes equal ? {np.all(grain_volumes == GrainDataTable["volume"])}')
grain_bboxes = micro.get_grain_bounding_boxes()
print(f'grain bounding boxes equal ? {np.all(grain_bboxes == GrainDataTable["bounding_box"])}')
grain_rodrigues = micro.get_grain_rodrigues()
print(f'grain orientations equal ? {np.all(grain_rodrigues == GrainDataTable["orientation"])}')
grain ids equal ? True
grain centers equal ? True
grain volumes equal ? True
grain bounding boxes equal ? True
grain orientations equal ? True
Other methods to get specific grain data are also available in the class interface:
[20]:
centers = micro.get_grain_positions()
print(f'The position of the 10 first grain centers of mass are:\n {centers[:10]}\n')
volume_fractions = micro.get_grain_volume_fractions()
print(f'The 10 first grain volume fractions are:\n {volume_fractions[:10]}\n')
volume_fr = micro.get_grain_volume_fraction(1335)
print(f'Volume fraction of grain 1335 is {volume_fr*100}%')
The position of the 10 first grain centers of mass are:
[[ 0.05650836 0.05867881 -0.04994912]
[ 0.02869466 0.00739364 -0.05561062]
[-0.03373204 0.02536774 0.04139666]
[-0.00488 0.060146 0.060268 ]
[-0.00521397 -0.01333876 0.05817775]
[ 0.00187614 -0.05681766 0.05338208]
[-0.03986695 -0.04861747 -0.03584476]
[ 0.00257532 -0.0298424 0.03544907]
[ 0.05424954 -0.04179282 0.03134331]
[-0.0300348 0.01446837 -0.00940701]]
The 10 first grain volume fractions are:
[6.1100005e-04 1.2865001e-02 6.3486010e-02 2.0000001e-05 3.6640004e-03
3.7020005e-03 2.9416002e-02 8.2998008e-02 9.9510010e-03 7.1037009e-02]
Volume fraction of grain 1335 is 1.938900351524353%
Getting grain objects¶
Pymicro also has Grain objects that are specific containers equivalent to a row of the dataset GrainDataTable. You can get them with the following methods:
[21]:
# get the grain object of a specific grain
grain = micro.get_grain(1335)
print(f'Grain 1335 grain object:\n {grain}')
Schmid = grain.schmid_factor(lattice.get_slip_systems('basal')[0])
print(f'Schmid factor of grain {grain.id} for first basal slip system is {Schmid}')
Grain 1335 grain object:
Grain
* id = 1335
* Crystal Orientation
-------------------
orientation matrix =
[[ 0.86979023 -0.23651806 -0.43304061]
[-0.41716777 0.11619364 -0.90137121]
[ 0.26350713 0.96465445 0.00239638]]
Euler angles (degrees) = ( 164.722, 89.863, 205.661)
Rodrigues vector = [-0.9384652 0.35030912 0.0908527 ]
Quaternion = [ 0.70504969 0.66166461 -0.24698534 0.06405567]
* center [-0.0273755 0.0372526 -0.04865015]
* has vtk mesh ? False
Schmid factor of grain 1335 for first basal slip system is 0.001037729482748718
[22]:
# get a list of all grain objects in the microstructure
grains_list = micro.get_all_grains()
print(f'First 2 grain objects of the microstructure:\n {grains_list[:2]}')
First 2 grain objects of the microstructure:
[Grain
* id = 2
* Crystal Orientation
-------------------
orientation matrix =
[[ 0.3111274 0.59834852 0.73836226]
[ 0.11859268 0.74640592 -0.65483891]
[-0.94293985 0.2913027 0.16126758]]
Euler angles (degrees) = ( 252.832, 80.720, 131.569)
Rodrigues vector = [-0.42642024 -0.75775258 0.21622302]
Quaternion = [ 0.744782 0.31759011 0.56436047 -0.16103901]
* center [ 0.05650836 0.05867881 -0.04994912]
* has vtk mesh ? False
, Grain
* id = 3
* Crystal Orientation
-------------------
orientation matrix =
[[ 0.55125435 -0.54673326 -0.63023916]
[-0.38553982 0.50297086 -0.77354986]
[ 0.73991736 0.66940502 0.06647743]]
Euler angles (degrees) = ( 132.136, 86.188, 219.171)
Rodrigues vector = [-0.68041358 0.64608611 -0.07600945]
Quaternion = [ 0.72813162 0.49543064 -0.47043572 0.05534488]
* center [ 0.02869466 0.00739364 -0.05561062]
* has vtk mesh ? False
]
The grains class attribute¶
The Microstructure.grains attribute is, as mentioned earlier, an alias for the Pytables node associated to the GrainDataTable data item. As such, it allows to manipulate and interact with the GrainDataTable content directly in the dataset.
You can use this attribute to access grain data just like you would manipulate a Numpy structured array:
[23]:
print(micro.grains[0]['center'],'\n')
print(micro.grains[4:10]['orientation'])
[ 0.05650836 0.05867881 -0.04994912]
[[ 0.5890785 0.6887171 0.0980203 ]
[-0.76733357 0.32743722 0.07196987]
[-0.7142779 -0.61377376 0.19212927]
[ 0.51318526 0.6922173 0.09951874]
[ 0.77879316 -0.64933205 -0.16091348]
[ 0.21065354 -0.9373515 0.23172894]]
This attribute can also be iterated:
[24]:
# iterate through grains with ID number below 100
for g in micro.grains:
if g["idnumber"] > 100:
break
print(f'Grain {g["idnumber"]} center of mass is located at {g["center"]}')
Grain 2 center of mass is located at [ 0.05650836 0.05867881 -0.04994912]
Grain 3 center of mass is located at [ 0.02869466 0.00739364 -0.05561062]
Grain 34 center of mass is located at [-0.03373204 0.02536774 0.04139666]
Grain 43 center of mass is located at [-0.00488 0.060146 0.060268]
Grain 46 center of mass is located at [-0.00521397 -0.01333876 0.05817775]
Grain 73 center of mass is located at [ 0.00187614 -0.05681766 0.05338208]
Grain 98 center of mass is located at [-0.03986695 -0.04861747 -0.03584476]
The MeshData Group¶
The ``MeshData`` group is aimed at storing descriptions of the polycristalline microstructure of the sample, in the form of a mesh. In the present dataset, this group is used as a container group for a Mesh Group, grains_mesh. Mesh support has not yet been developped for Pymicro Microstructure class, and hence the data model of this group is for now empty.
Additional data items¶
As you can see above, the example dataset also contains data items that are not defined in the data model of the Microstructure class (the Amitex_Results Group, several fields of the CellData image…). Obviously, as a SampleData children class, with Microstructure class, you may add any additional data item to your datasets, as you would with SampleData datasets.
This concludes the presentation of the Microstructure class data model and data access.
Creating and setting Microstructures¶
Now that the class data model has been presented, we will now introduce how to create and fill Microstructure objects.
There are three ways to create a Microstructure dataset:
Creating an empty microstructure dataset
Copying an existing microstructure dataset, or a croped version of this microstructure
Create a Microstructure object from a compatible data file contaning microstructure data. These files can be outputs of imaging techniques (reconstruction of DCT or EBSD scans for instance) or microstructure generation tools (such as Neper)
The third point will be the subject of detailed and specific Notebook. The first will be presented further in the tutorial to build a full microstructure dataset. Hence, we will start by presenting how a Microstructure object/dataset can be created from an already existing one.
From an existing Microstructure¶
Copy an existing Microstructure¶
As for SampleData datasets, Microstructure datasets can be copied from already existing one, using the copy_sample method:
[25]:
original_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR,'t5_dct_slice_data.h5')
micro_copy = Microstructure.copy_sample(src_micro_file=original_file, dst_micro_file='micro_copy',
get_object=True, autodelete=True, overwrite=True)
print(micro_copy)
del micro_copy
Microstructure
* name: micro
* lattice: Lattice (Symmetry.cubic) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=90.0
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : Crystal_data H5_Path : /CrystalStructure
Name : GrainDataTable H5_Path : /GrainData/GrainDataTable
Name : Grain_data H5_Path : /GrainData
Name : Image_data H5_Path : /CellData
Name : Mesh_data H5_Path : /MeshData
Name : Phase_data H5_Path : /PhaseData
Name : grain_map H5_Path : /CellData/grain_map
grain_map aliases --> `grain_ids`
Name : lattice_params H5_Path : /CrystalStructure/LatticeParameters
Name : mask H5_Path : /CellData/mask
Name : phase_01 H5_Path : /PhaseData/phase_01
Name : phase_map H5_Path : /CellData/phase_map
Printing dataset content with max depth 3
|--GROUP CellData: /CellData (3DImage)
--NODE grain_map: /CellData/grain_map (None) ( 15.072 Kb)
--NODE mask: /CellData/mask (None) ( 4.714 Kb)
--NODE phase_map: /CellData/phase_map (None - empty) ( 64.000 Kb)
|--GROUP CrystalStructure: /CrystalStructure (Data)
--NODE LatticeParameters: /CrystalStructure/LatticeParameters (None) ( 64.000 Kb)
|--GROUP GrainData: /GrainData (Data)
--NODE GrainDataTable: /GrainData/GrainDataTable (None) ( 1.436 Kb)
|--GROUP MeshData: /MeshData (Mesh)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
Microstructure Autodelete:
Removing hdf5 file micro_copy.h5 and xdmf file micro_copy.xdmf
Crop an existing Microstructure¶
Sometimes, it may be desirable not to copy a complete microstructure, but only a specific region of it to build a dataset dedicated to this region. For that, you may use the crop method of the Microstructure class, that return a new microstructure object.
The method:
creates a new Microstructure dataset, with the same name plus the suffix
_crop, or the name specified by the optional argumentcrop_namecrops all fields of the
CellDatagroup of the original Microstructure, by extracting the subregion indicated by thex_start, x_end, y_start, y_end, z_start, z_endarguments (bounds indices of the cropped region). Then, it adds them to theCellDatagroup of the new Microstructure.fills the GrainDataTable of the new microstructure with only the grains contained in the cropped region, and recomputes the grains geometric data for the new grain map, unless argument
recompute_geometryis set toFalse.
Like the copy_sample method, the crop method also has an autodelete optional argument that sets the autodelete mode of the cropped microstructure instance.
Let us try to crop a small region of our example microstructure dataset:
[26]:
micro_crop = micro.crop(x_start=30, x_end=70, y_start=30, y_end=70, z_start=30, z_end=70, crop_name='test_crop',
autodelete=True)
micro_crop.print_dataset_content(short=True)
micro_crop.view_slice(display=False) # set to True to try reproducing the figure below !
new phase added: unknown
cropping microstructure to test_crop_data.h5
cropping field grain_map
cropping field mask
cropping field orientation_map
cropping field grain_map_raw
cropping field uncertainty_map
cropping field Amitex_stress_1
cropping field Amitex_strain_1
9 grains in cropped microstructure
updating grain geometry
Printing dataset content with max depth 3
|--GROUP CellData: /CellData (3DImage)
--NODE Amitex_stress_1: /CellData/Amitex_stress_1 (field_array) ( 4.980 Mb)
--NODE Field_index: /CellData/Field_index (string_array - empty) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array) ( 125.000 Kb)
--NODE grain_map_raw: /CellData/grain_map_raw (field_array) ( 125.000 Kb)
--NODE mask: /CellData/mask (field_array) ( 62.500 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
--NODE uncertainty_map: /CellData/uncertainty_map (field_array) ( 62.500 Kb)
|--GROUP GrainData: /GrainData (Group)
--NODE GrainDataTable: /GrainData/GrainDataTable (structured_array) ( 63.984 Kb)
|--GROUP MeshData: /MeshData (emptyMesh)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
using slice value 20
[26]:
(<Figure size 640x480 with 1 Axes>, <AxesSubplot:xlabel='X', ylabel='Y'>)
You should get:
You can observe that the data that was not in the CellData group nor in the class data model in the original file has not been added to the cropped Microstructure (the AmitexResults Group for instance).
Warning
Cropping a microstructure can be long if the original microstructure is heavy and has a lot a fields for the CellData group. If you only want to crop some of these fields, you may want to create a new microstructure, add to its CellData group only the fields you want to crop, and then create your crop from this new instance.
Creating and filling an Empty Microstructure¶
We will now see how to create from scratch a complete Microstructure dataset. As an exercise for this tutorial, we will attempt to recreate the cropped microstructure that has been created in the cell just above.
For that, we first need to create an empty Microstructure object. The Microstructure class can be used as the SampleData class constructor (see here):
[27]:
micro2 = Microstructure(filename='Crop_remake.h5', overwrite_hdf5=True, autodelete=True)
print(micro2)
new phase added: unknown
Microstructure
* name: micro
* lattice: Lattice (Symmetry.cubic) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=90.0
Dataset Content Index :
------------------------:
index printed with max depth `3` and under local root `/`
Name : Image_data H5_Path : /CellData
Name : Mesh_data H5_Path : /MeshData
Name : Grain_data H5_Path : /GrainData
Name : Phase_data H5_Path : /PhaseData
Name : grain_map H5_Path : /CellData/grain_map
Name : Image_data_Field_index H5_Path : /CellData/Field_index
Name : phase_map H5_Path : /CellData/phase_map
Name : mask H5_Path : /CellData/mask
Name : GrainDataTable H5_Path : /GrainData/GrainDataTable
Name : phase_01 H5_Path : /PhaseData/phase_01
Printing dataset content with max depth 3
|--GROUP CellData: /CellData (emptyImage)
--NODE Field_index: /CellData/Field_index (string_array - empty) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array - empty) ( 64.000 Kb)
--NODE mask: /CellData/mask (field_array - empty) ( 64.000 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
|--GROUP GrainData: /GrainData (Group)
--NODE GrainDataTable: /GrainData/GrainDataTable (structured_array - empty) ( 0.000 bytes)
|--GROUP MeshData: /MeshData (emptyMesh)
|--GROUP PhaseData: /PhaseData (Group)
|--GROUP phase_01: /PhaseData/phase_01 (Group)
[28]:
print(micro2.GrainDataTable)
[]
The new microstructure has been created, with the complete class data model filled with empty data items. We will now fill it with data corresponding to the cropped region in the previous subsection.
Setting CellData items¶
As an Image Group, you can add fields to the CellData with the SampleData method add_field. In the specific case of the Microstructure class, you can use specific methods to set the value of the fields that are part of the class data model: set_mask , set_grain_map and set_phase_map.
We are trying to recreate a cropped microstructure with a uniform mask and no phase map. We need to create the appropriate grain map and mask arrays to set their value in the dataset:
[29]:
# Crop manually the original grain map
grain_map_original = micro.get_grain_map()
grain_map_crop = grain_map_original[30:70,30:70,30:70]
# Create a mask array full of ones with appropriate shape
mask_crop = np.ones_like(grain_map_crop)
Now that we have created our arrays, we can set the CellData fields. Note that when you add the first CellData field, you have to specify a pixel/voxel size to set the scale of the image.
[30]:
# retrieve voxel size of original microstructure
voxel_size = micro.get_voxel_size()
# set the grain map of our new microstructure
micro2.set_grain_map(grain_map_crop, voxel_size)
# set the mask
micro2.set_mask(mask_crop)
# visualize slice of added arrays
micro2.view_slice(display=False) # set to True to try reproducing the figure below !
using slice value 20
[30]:
(<Figure size 640x480 with 1 Axes>, <AxesSubplot:xlabel='X', ylabel='Y'>)
You should get:
To add the other fields that were part of the original microstructure Image Group, we have to go back to SampleData methods:
[31]:
uncertainty_map_crop = micro['uncertainty_map'][30:70,30:70,30:70]
micro2.add_field(gridname='CellData', fieldname='uncertainty_map', array=uncertainty_map_crop)
uncertainty_map_crop = micro['grain_map_raw'][30:70,30:70,30:70]
micro2.add_field(gridname='CellData', fieldname='grain_map_raw', array=uncertainty_map_crop)
micro2.print_group_content('CellData', short=True)
--NODE Field_index: /CellData/Field_index (string_array - empty) ( 63.999 Kb)
--NODE grain_map: /CellData/grain_map (field_array) ( 125.000 Kb)
--NODE grain_map_raw: /CellData/grain_map_raw (field_array) ( 125.000 Kb)
--NODE mask: /CellData/mask (field_array) ( 125.000 Kb)
--NODE phase_map: /CellData/phase_map (field_array - empty) ( 64.000 Kb)
--NODE uncertainty_map: /CellData/uncertainty_map (field_array) ( 62.500 Kb)
Setting the phase¶
Likewise, we can take advantage of the get_phase and set_phase methods to transfer the phase data from the original dataset to our new one. The set_phase method takes as argument a pymicro CrystallinePhase object. These object contain an identification number for the phase. If a phase with same number already exists in the dataset, it is overwritten by the inputted one.
Let us add phase data to our dataset:
[32]:
phase = micro.get_phase(1)
print(phase)
micro2.set_phase(phase)
print(micro2.get_phase_ids_list())
print(micro2.get_phase(1))
Phase 1 (Ti grade 2)
-- Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0
-- elastic constants: [162000.0, 92000.0, 69000.0, 180000.0, 46700.0]
setting phase 1 with Ti grade 2
[1]
Phase 1 (Ti grade 2)
-- Lattice (Symmetry.hexagonal) a=1.000, b=1.000, c=1.000 alpha=90.0, beta=90.0, gamma=120.0
-- elastic constants: [162000.0, 92000.0, 69000.0, 180000.0, 46700.0]
Note that phases can also be added with the set_phases method, that takes as input a list of pymicro CrystallinePhase objects. You can also use the add_phase method, that adds a CrystallinePhase object to the dataset with the next available phase identification number of the microstructure.
Setting the GrainDataTable¶
The Microstructure class offers several ways to fill the GrainDataTable, that are successively reviewed in this subsection.
From the grain map¶
As detailed earlier, the GrainDataTable contains data describing the grains position and morphology. These values can be computed from the grain map, that provides the full geometry of each grain. Specific methods of the Microstucture class allow to compute those values and automatically fill the GrainDataTable with them. They are:
recompute_grain_centers: computes and fills thecentercolumn of the GrainDataTable from the grains geometry in grain maprecompute_grain_volumes: computes and fills thevolumecolumn of the GrainDataTable from the grains geometry in grain maprecompute_grain_bounding_boxes: computes and fills thebounding_boxcolumn of the GrainDataTable from the grains geometry in grain map
If you need to call them all, you can do it at once with the build_grain_table_from_grain_map, that will first synchronize the grain ids that are in the grain map and the GrainDataTable, and then call the 3 previous methods to fill the geometric grain data in the table. We will use for our tutorial exercise:
[33]:
micro2.build_grain_table_from_grain_map()
print(micro2.GrainDataTable)
adding 9 grains to the microstructure
[([[ 0, 13], [20, 40], [33, 40]], [-0.01886625, 0.01578343, 0.0216327 ], 34, [ 1.8039021e+01, -4.6116508e+01, 2.4090748e+01], 1.3164898e-06)
([[ 0, 40], [ 0, 24], [22, 40]], [-0.00033868, -0.0152649 , 0.01731798], 107, [ 2.7277954e+00, 3.8666222e+00, 3.0442362e+00], 1.1594189e-05)
([[ 0, 24], [10, 40], [ 0, 39]], [-0.01444502, 0.0103255 , -0.00216276], 164, [ 6.3165874e+01, -8.7734337e+01, 4.9011006e+00], 2.4938856e-05)
([[ 0, 23], [ 0, 17], [ 0, 40]], [-0.01349942, -0.01565468, -0.00471803], 196, [-6.0964459e-01, -1.4468047e-01, -2.3179213e-02], 1.5941330e-05)
([[ 9, 40], [26, 40], [ 0, 27]], [ 0.00772108, 0.01941636, -0.01246925], 223, [-1.1969326e+01, -8.6379910e+00, -7.3886573e-01], 8.1331827e-06)
([[11, 40], [ 3, 36], [ 0, 34]], [ 0.01088636, -0.00186796, -0.00645274], 485, [ 6.6325707e+00, -1.2901696e+00, 1.0559826e+01], 3.3618609e-05)
([[ 8, 40], [10, 40], [23, 40]], [ 0.00810409, 0.00917885, 0.01690876], 692, [ 1.4133686e+00, 8.5083926e-01, 1.8574281e-01], 1.5970383e-05)
([[ 3, 26], [ 4, 22], [ 0, 4]], [-0.00799421, -0.00828505, -0.02307991], 1019, [ 1.9616644e+00, -3.0095938e-01, 1.8194200e+00], 5.8651892e-07)
([[20, 40], [ 0, 7], [ 0, 30]], [ 0.01231845, -0.0215309 , -0.00930183], 1494, [-3.7780006e+01, 1.1798064e+00, -6.1420937e+00], 4.1147114e-06)]
The table has been updated with the geometric data of 9 grains, the grain identity number. You can also see that the method has added a random orientation to each grain. If you want to avoid this, you may use instead the compute_grains_geometry method, that do not generates random orientations for the grains but fills orientation with zeros.
From data arrays¶
So, at this point, our GrainDataTable has its geometric values in accordance with the grain map, but wrong grain orientations, that have been randomly generated. We can however get the correct grain orientations from the original example microstructure dataset, with the get_grain_rodrigues method:
[34]:
# get the list of grain ids of the new microstructure
new_ids = micro2.get_grain_ids().tolist()
# get the orientations of this list of grains
orientations = micro.get_grain_rodrigues(new_ids)
[35]:
print(orientations)
[[ 0.6985433 -0.30674216 0.23611188]
[ 0.51318526 0.6922173 0.09951874]
[ 0.21065354 -0.9373515 0.23172894]
[-0.28858265 0.75905836 0.1430808 ]
[ 0.82040954 0.45255002 -0.02743571]
[-0.04288794 -0.9335458 0.0118214 ]
[-0.5966228 0.620786 -0.18180412]
[ 0.57177645 -0.53374106 -0.03578908]
[-0.778519 -0.5381718 0.19439915]]
Now, we can use the set_orientations method to add this information to our data table:
[36]:
micro2.set_orientations(orientations)
print(micro2.GrainDataTable['orientation'])
[[ 0.6985433 -0.30674216 0.23611188]
[ 0.51318526 0.6922173 0.09951874]
[ 0.21065354 -0.9373515 0.23172894]
[-0.28858265 0.75905836 0.1430808 ]
[ 0.82040954 0.45255002 -0.02743571]
[-0.04288794 -0.9335458 0.0118214 ]
[-0.5966228 0.620786 -0.18180412]
[ 0.57177645 -0.53374106 -0.03578908]
[-0.778519 -0.5381718 0.19439915]]
Similar methods exist for the rest of the data in the table:
set_centersset_bounding_boxesset_volumes
Getting/Setting data for/from a specific grain¶
To get information from a specific grain, you use the GrainDataTable as a standard Numpy structured array:
[37]:
grain_index = np.where(micro2.GrainDataTable['idnumber'] == 485)
print(f'Grain 485 center is {micro2.GrainDataTable[grain_index]["center"]}')
print(f'Grain 485 volume is {micro2.GrainDataTable[grain_index]["volume"]}')
Grain 485 center is [[ 0.01088636 -0.00186796 -0.00645274]]
Grain 485 volume is [3.361861e-05]
But you can also iterate the grains attribute of the class. By doing this, you will get at each iteration a Pytables Row object representing a row of data in the table, i.e. a grain. You can access its values exactly as if it was a Numpy structured array:
[38]:
for g in micro2.grains:
if g['idnumber'] == 485:
print(f'Grain 485 center is {g["center"]}')
print(f'Grain 485 volume is {g["volume"]}')
Grain 485 center is [ 0.01088636 -0.00186796 -0.00645274]
Grain 485 volume is 3.361860945005901e-05
You can also use this process to set specifically some values for one grain. You can iterate the table to find your grain object, set one of its values as if it was a Numpy structured array. Then you have to use the specific update method of the Pytables Row class to set the value in the dataset, as follows:
[39]:
# get old orientation value
grain_orientation = micro2.GrainDataTable[grain_index]['orientation']
print(f'The orientation of the grain is {micro2.GrainDataTable[grain_index]["orientation"]}')
# iterate to find the grain and set its orientation to a random value
for g in micro2.grains:
if g['idnumber'] == 485:
g['orientation'] = np.random.rand(3)
g.update()
print(f'The new orientation of the grain is {micro2.GrainDataTable[grain_index]["orientation"]}')
# Set back the original value of the orientation
for g in micro2.grains:
if g['idnumber'] == 485:
g['orientation'] = grain_orientation
g.update()
print(f'The orientation of the grain is back at {micro2.GrainDataTable[grain_index]["orientation"]}')
The orientation of the grain is [[-0.04288794 -0.9335458 0.0118214 ]]
The new orientation of the grain is [[0.9739699 0.01902657 0.34096706]]
The orientation of the grain is back at [[-0.04288794 -0.9335458 0.0118214 ]]
Obviously, you can use the same method to get/set other columns of the table (centers, bounding boxes…)
We have now completed this introduction to the Microstructure class. More advanced features of the class are already implemented in the code, and many more will be in the next years. Specific tutorial Notebooks about this features will be released in the future, as well as examples presented in the documentation Cookbook.
We can now close our datasets, and remove the original unarchived file:
[40]:
# remove SampleData instance
del micro2
del micro
Microstructure Autodelete:
Removing hdf5 file Crop_remake.h5 and xdmf file Crop_remake.xdmf
[41]:
os.remove(dataset_file+'.h5')
os.remove(dataset_file+'.xdmf')
SampleData Quick Reference Sheet¶
Notation¶
In the code lines constituting this reference sheet, the following notation conventions are used in the SampleData method arguments:
node_name,group_name,data_item_nameare the Name, Indexname, Path or Alias of respectively a Data Array, Group, or either of the two previous data items.parent_nameis the the Name, Indexname, Path or Alias of a group that is the parent of thedatais a SampleData class instance synchronized with the dataset filesmy_dataset.hdf5andmy_dataset.xdmf.
SampleData Naming system¶
Interacting with data item in a SampleData dataset require to provide their name to the various class method. 4 possible types of names can be provided for each data item:
the Path of the data item in the HDF5 file.
the Name of the data item.
the Indexname of the data item
the Alias or aliases of the data item
Creating, Opening datasets, and Exploring their content¶
Dataset creation/opening¶
Import SampleData class:
[ ]:
from pymicro.core.samples import SampleData as SD
Create/Open a SampleData dataset, and activate verbose mode:
[ ]:
# CREATE dataset: the file `filename` must not exist. Verbose mode OFF
data = SD(filename='my_first_dataset', verbose=False)
# OPEN dataset: the file `filename` must exist. Verbose mode ON
data = SD(filename='my_first_dataset', verbose=True)
Copy dataset and get class instance synchronized with new dataset:
[ ]:
data = SD.copy_sample(src_sample_file='source_dataset', dst_sample_file='destination_dataset', get_object=True)
Create dataset, overwrite existing file, and automatic removal of dataset files at class instance destruction (autodelete option):
[ ]:
# Create new dataset and overwrite already existing dataset files
data = SD(filename='my_first_dataset', verbose=True, overwrite_hdf5=True)
# Create new dataset with autodelete option ON
data = SD(filename='my_first_dataset', verbose=True, autodelete=True)
# Set autodelete option on
data.autodelete = True
Getting information on datasets¶
Print informations on global content of the dataset:
[ ]:
# Print dataset index
data.print_index() # --> no option = local root '/' and max depth 3
data.print_index(max_depth=2, local_root='local_root_name') # --> with specified local root and depth
# Print dataset content (list all groups, nodes with detailed information)
data.print_dataset_content() # detailed output, printed in standard output
data.print_dataset_content(short=True, to_file='dataset_information.txt') # short output, written in text file
# Print both index and dataset content in short version --> class string representation
print(data)
# Print only grid groups informations
data.print_grids_info()
# Print content of XDMF file
data.print_xdmf()
# Get the memory disk size of the HDF5 dataset
size, unit = data.get_file_disk_size(convert=True, print_flag=False)
# value not printed, returned in method output and converted to most readable memory unit
Command line tools to get information on dataset
[ ]:
# recursive (-r) and detailed (-d) output of h5ls --> also print the content of the data arrays
!h5ls -rd ../data/test_sampledata_ref.h5
# h5dump
!ptdump -d ../data/test_sampledata_ref.h5
# detailed (-d) and verbose (-v) output of ptdump
!ptdump -dv ../data/test_sampledata_ref.h5
[ ]:
# Print information on data items content
data.print_node_info(nodename='node_name') # detailed information on a data item (group or array node)
data.print_group_content('group_name') # print information on group childrens
data.print_group_content('group_name', recursive=True) # print information on group childrens recursively
# get data item disk size
size, unit = data.get_node_disk_size(nodename='node_name', print_flag=False, convert=False)
Dataset interactive visualization¶
[ ]:
# Visualize dataset organization and content with Vitables
data.pause_for_visualization(Vitables=True, Vitables_path='Path_to_Vitables_executable')
# Visualize spatially organized data with Paraview
data.pause_for_visualization(Paraview=True, Paraview_path='Path_to_Paraview_executable')
Basic data items: creating and getting them¶
Generic methods to get data items¶
[ ]:
# Dictionary like access
data['data_item_name']
# Attribute like access
data.data_item_name
# generic getter method
data.get_node('data_item_name') # --> returns a Pytables Group or Node object
data.get_node('data_item_name', as_numpy=True) # --> for array data items, returns a numpy array
Group data items¶
group_name and group_indexnam are the Name and Indexname of a Group data item. parent_group is the Name, Path, Indexname or Alias of a Group where group_name will be stored.
[ ]:
# Create a group in a dataset with name `group_name`, stored in the group `parent_name`
data.add_group(groupname='group_name', location='parent_name', indexname='group_indexname')
# Create a group and overwrite pre-existing group with the same name + get the created Pytables Group object
group = data.add_group(groupname='group_name', location='parent_name', indexname='group_indexname', replace=True)
Data item attributes (metadata)¶
[ ]:
# Add attributes from a python dictionary (metadata_dictionary)
data.add_attributes(metadata_dictionary, nodename='node_name')
# get data item attributes (metadata)
data.print_node_attributes(nodename='node_name') # print all attributes of note
attribute_value = data.get_attribute(attrname='attribute_name', nodename='node_name') # get value of one attribute
mesh_attrs = data.get_dic_from_attributes(nodename='node_name') # get all attributes as a dictionary
# set and get specific `description` attribute for node `node_name`
data.set_description(description="Write your description text here.", node='node_name')
data.get_description('node_name')
Data arrays¶
[ ]:
# add a numpy array `array` in data item `node_name`
data.add_data_array(location='parent_name', name='node_name', indexname='array_indexname', array=array)
# replace = True allows to overwrite preexisting field with same name
data.add_data_array(location='parent_name', name='node_name', indexname='array_indexname', array=array, replace= True)
# get data array from data item `node_name`
array_node = data.get_node('node_name') # --> returns a Pytables Node object
array = data.get_node('node_name', as_numpy=True) # --> returns a Numpy array
array = data.['node_name'] # --> returns a Numpy array
array = data.node_name # --> returns a Numpy array
String arrays¶
[ ]:
# Add list of strings `List` as a string array `node_name`
data.add_string_array(name='node_name', location='parent_name', indexname='Sarray_indexname', data=List)
# get and decode binary strings stored in a String array
sarray = data['Sarray_indexname']
for string in sarray:
print(string.decode('utf-8'), end=' ') # prints each string of the String array
Structured arrays¶
[ ]:
# Add structured array from Numpy structured array `structured_array` with Numpy.dtype `table_type`
data.add_table(name='node_name', location='parent_name', indexname='table_indexname', description=table_type,
data=structured_array)
# Add lines to a structured array node from Numpy array `structured_array` (same dtype as the table)
data.append_table(name='table_indexname', data=structured_array)
# Add columns to a structured array node from a Numpy array `structured_array` with Numpy.dtype ``
data.add_tablecols(tablename='table_indexname', description=cols_dtype, data=structured_array)
# Get structured array just like Data arrays
Remove data items¶
[ ]:
data.remove_node('node_name') # removes a group without childrens or a data array item
data.remove_node('group_name', recursive=True) # remove a Group data item and all its childrens recursively
# remove one or a list of attributes (metadata) from a node
data.remove_attribute(attrname='attribute_name', nodename='node_name')
data.remove_attributes(attr_list=['list','of','attributes','to','remove','from','node'], nodename='node_name')
Image Groups and Image fields: creating and getting them¶
Creating Image groups from fields¶
[ ]:
# Create an Image Group from a Numpy array `field_array` interpreted as a pixel/voxel wise constant scalar field
data.add_image_from_field(field_array=field_array, fieldname='node_name', imagename='group_name',
indexname='image_indexname', location='parent_name',
description="Write image group description here.", origin=[0.,10.], spacing=[2.,2.])
# Create an Image Group from a Numpy array `field_array` interpreted as a node value scalar field
data.add_image_from_field(field_array=field_array, fieldname='node_name', imagename='group_name',
indexname='image_indexname', location='parent_name', is_elemField=False,
description="Write image group description here.", origin=[0.,10.], spacing=[2.,2.])
# Create an Image Group from a Numpy array `field_array` interpreted as a non scalar field
data.add_image_from_field(field_array=field_array, fieldname='node_name', imagename='group_name',
indexname='image_indexname', location='parent_name', is_scalar=False,
description="Write image group description here.", origin=[0.,10.], spacing=[2.,2.])
# Set image position and dimensions
data.set_voxel_size(image_group='image_indexname', voxel_size=np.array([4.,4.]))
data.set_origin(image_group='image_indexname', origin=np.array([10.,0.]))
Creating image groups from image objects¶
[ ]:
# import BasicTools image object `ConstantRectilinearMesh`
from BasicTools.Containers.ConstantRectilinearMesh import ConstantRectilinearMesh
# Initialize image object, image dimension, origin and pixel/voxel size
image_object = ConstantRectilinearMesh(dim=3)
image_object.SetDimensions((50,50,3))
image_object.SetOrigin([0.,0.,0.])
image_object.SetSpacing([1.,1.,1.]) # pixel/voxel size in each dimension
# Create Image Group in dataset
data.add_image(image_object, imagename='group_name', indexname='image_indexname', location='parent_name',
description="""Write image group description here.""")
Creating empty images¶
[ ]:
data.add_image(imagename='group_name', indexname='image_indexname', location='parent_name',
description="""Write image group description here.""")
Get image object from Image Group¶
[ ]:
# Get BasicTools image object from SampleData image group `group_name` including image group fields data arrays
im_object = data.get_image('group_name', with_fields=True)
Creating and getting image Fields¶
[ ]:
# Creating a field for image group `group_name` from Numpy array `tensor_field`
data.add_field(gridname='group_name', fieldname='node_name', location='parent_name', indexname='field_indexname',
array=tensor_field, replace=True) # replace = True allows to overwrite preexisting field with same name
# Getting image fields
# --> field returned as Numpy array
field = data.get_field('node_name')
field = data.get_node('node_name', as_numpy=True)
field = data['node_name']
field = data.node_name
# --> field returned as a Pytables Node object
field = data.get_node('node_name')
Creating a field time serie¶
[ ]:
instants = [1.,10., 100.]
# Add three temporal values for the field `node_name` in image group `group_name` for 3 different time values given in
# `instants` array. Field values are stored in *Numpy* arrays temporal_field_0, temporal_field_1, temporal_field_2
data.add_field(gridname='group_name', fieldname='node_name', location='parent_name', indexname='Field',
array=temporal_field_0, time=instants[0])
# instant 1
data.add_field(gridname='group_name', fieldname='node_name', location='parent_name', indexname='Field',
array=temporal_field_1, time=instants[1])
# instant 2
data.add_field(gridname='group_name', fieldname='node_name', location='parent_name', indexname='Field',
array=temporal_field_2, time=instants[2])
Mesh Groups and Mesh Fields: creating and getting them¶
Creating Mesh objects with BasicTools¶
[ ]:
# Import Basictools mesh creation tools
import BasicTools.Containers.UnstructuredMeshCreationTools as UMCT
# Create a Node and Connectivity (elements) array, then create a mesh:
mesh = UMCT.CreateMeshOfTriangles(mesh_nodes, mesh_elements) # mesh of triangles
mesh = UMCT.CreateMeshOf(mesh_nodes, mesh_elements, elemName='tet4') # mesh of tetrahedra
# Create a mesh of a cube with tetrahedron elements
mesh = UMCT.CreateCube(dimensions=[5,5,5],spacing=[2.,2.,2.],ofTetras=True)
# adding node and element tags to the mesh
mesh.nodesTags.CreateTag('nodetag_name', False).SetIds(nodetag_Id_list) # Node tag
mesh.GetElementsOfType('tri3').GetTag('elemtag_name').SetIds(elemtag_Id_list) # Element tag ( of type `tri3`)
# adding fields
mesh.nodeFields['nodal_fieldname'] = nodal_field_array
mesh.elemFields['element_fieldname'] = elem_field_array
Creating a Mesh Group in a dataset¶
[ ]:
# Creating Mesh Group from Mesh object
# mesh is a Basictools mesh object. bin_fields_from_sets options allows to load node and element tags in mesh Group
data.add_mesh(mesh_object=mesh, meshname='meshname', indexname='mesh_indexname', location='mesh_parent',
bin_fields_from_sets=True)
# Creating Mesh group from file
data.add_mesh(file=meshfile_name, meshname='meshname', indexname='mesh_indexname', location='mesh_parent',
bin_fields_from_sets=True)
Creating and getting Mesh Fields¶
[ ]:
# creation of the mesh field
data.add_field(gridname='meshname', fieldname='fieldname', array=field_data_array, indexname='field_indexname')
# Creation of a field part of a time serie
data.add_field(gridname='meshname', fieldname='fieldname', array=field_data_array, indexname='field_indexname',
time)
# Force element field to be defined on boundary elements if the mesh has same number of bulk and boundary elements
data.add_field(gridname='meshname', fieldname='fieldname', array=field_data_array, indexname='field_indexname',
time)
# getting the inputed array --> no options
field_data_array = data.get_field('fieldname')
# getting the visualization array of an integration point field
field_data_array = data.get_field('fieldname', get_visualisation_field=True)
# getting the unpadded visualization array of an integration point field
field_data_array = data.get_field('fieldname', unpad_field=False, get_visualisation_field=True)
Getting Mesh objects¶
[ ]:
# Get a Basictools mesh object with all content of Mesh group 'meshname' (fields, tags, nodes, elements)
mesh = data.get_mesh('meshname')
# Get a Basictools mesh object without fields (tags, nodes, elements) from Mesh group 'meshname'
mesh = data.get_mesh('meshname', with_fields=False)
# Get a Basictools mesh object without fields and tags (just nodes, elements) from Mesh group 'meshname'
mesh = data.get_mesh('meshname', with_fields=False, with_tags=False)
Data Compression¶
[ ]:
# Set chunckshape and compression settings for one data item
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True}
chunkshape = c_shap # tuple
data.set_chunkshape_and_compression(nodename='nodename_to_compress', compression_options=compression_options,
chunkshape=chunkshape)
# Set chunckshape and compression settings forseveral nodes
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True}
data.set_nodes_compression_chunkshape(node_list=['nodename_to_compress1', 'nodename_to_compress2',...],
compression_options=compression_options,
chunkshape=chunkshape)
# Apply lossy compression
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True, 'least_significant_digit':2}
data.set_chunkshape_and_compression(nodename='nodename_to_compress', compression_options=compression_options,
chunkshape=chunkshape)
# Apply lossy compression with normalization
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True, 'least_significant_digit':2,
normalization='standard'}
data.set_chunkshape_and_compression(nodename='nodename_to_compress', compression_options=compression_options,
chunkshape=chunkshape)
# Apply lossy compression with per-component normalization
compression_options = {'complib':'zlib', 'complevel':1, 'shuffle':True, 'least_significant_digit':2,
normalization='standard-per-component'}
data.set_chunkshape_and_compression(nodename='nodename_to_compress', compression_options=compression_options,
chunkshape=chunkshape)
# Create an array with predefined chunkshape and compression settings
data.add_data_array(name='arrayname', indexname='array_indexname', location='parent_name', array=array,
chunkshape=chunkshape, compression_options=compression_options)
Examples¶
3d_visualisation¶
2-6-2_30k00_c1_3d.py¶
AuCu_crystal.py¶
cracked_single_crystal_with_slip_systems.py¶
crystal_lattice_3d.py¶
cubic_crystal_3d.py¶
grain_3d_planes.py¶
grain_hkl_3d.py¶
hcp_crystal_3d.py¶
hexagonal_crystal_3d.py¶
mousse_3d.py¶
pa6_teg.py¶
pole_figure_3d.py¶
pure_Ti_all_grains.py¶
sand_volren.py¶
steel_damage_3d.py¶
animation¶
euler_angles_anim.py¶
grain_hkl_anim_3d.py¶
ebsd¶
load_osc.py¶
finite_elements¶
CT10_PROPAG100_3d.py¶
poly_40_grains.py¶
pack_volumes¶
plotting¶
Au_6grains_pole_figure.py¶
Cu_111_pole_figure.py¶
Cu_200_pole_figure.py¶
Ti_ebsd_slip_traces.py¶
atomic_structure_factor.py¶
contour_pole_figure.py¶
cos_fitting.py¶
cubic_elasticity.py¶
field_ipf.py¶
field_pole_figure.py¶
hexagonal_elasticity.py¶
laue_ellipse.py¶
plot_grain_rotation.py¶
pyplot_show_pixel_value.py¶
radon.py¶
random_texture_misorientation.py¶
random_texture_pole_figure_ipf.py¶
slip_traces.py¶
xray_mar165_detector.py¶
xray_trans_Al.py¶
xray_xpadS140_azimutal_regroup.py¶
xray_xpadS140_detector.py¶
Cookbook¶
pymicro tries to hide some of the complexity of vtk, manipulating 3d arrays and crystallographic structures but the code may remain complex sometimes. Each experiment is unique and may require more or less tweaking to get the desired result or 3d rendering. The cookbook gather some recipe for users to better understand how it works and how to adapt existing code. Recipes are much more detailed than examples shown in the galleries and represent complete studies from the initial data to the final result.
Using pymicro interactively¶
pymicro can be used interactively within the ipython framework without any change to your system. Just type some code in !
Example:
>>> from pymicro.crystal.lattice import Lattice
>>> al = Lattice.from_symbol('Al')
>>> from pymicro.crystal.lattice import HklPlane
>>> p111 = HklPlane(1, 1, 1, al)
>>> p111.bragg_angle(20)*180/pi
7.618
As seen it can quick become boring to import all the required modules. To use pymicro seamlessly within ipython, it is recommended to tweak the ipython config in the following way:
you can add the import_modules.py file to the ipython startup directory. This way all the pymicro modules will be imported into the namespace at startup. You can copy it there but the best way is to use a hardlink. This way if pymicro is updated and this file changes, you will have nothing special to do:
$ cd .ipython/profile_default/startup
$ ln -s /path/to/pymicro/import_modules.py
That’s it, now you may just type:
>>> al = Lattice.from_symbol('Al')
>>> p111 = HklPlane(1, 1, 1, al)
>>> p111.bragg_angle(20)*180/pi
7.618
pymicro features several static methods to directly compute or plot with one liners, for instance to plot a pole figure for a single orientation without creating a Grain instance and so on you may just type:
>>> PoleFigure.plot(Orientation.from_rodrigues([0.0885, 0.3889, 0.3268]))
To compute all Schmid factor for this orientation and the {111}[110] slip systems, you can use:
>>> o = Orientation.from_rodrigues([0.0885, 0.3889, 0.3268])
>>> o.compute_all_schmid_factors(SlipSystem.get_slip_systems(plane_type='111'))
How to read 3d files properly and ensure consistency of the coordinate system?¶
3d images may be stored in various binary formats. Here we assume a raw binary format possibly with a header (like .edf images). It is important to recognize that you must know how the file is stored since this information is usually not available from the file metadata (except for tiff files).
In pymicro we assume that the data is written to the disk as a succession of (x,y) slices, x being the fastest moving index.
Let (nx, ny, nz) be the volume dimensions in each direction. Assuming the first slice of (nx x ny) pixels is the bottom slice: x is the fastest varying index, y is the second fastest and z is the slowest. The file must thus be read with the convention (+x, +y, +z) and displayed in a right handed coordinate system. This means if you plot the first slice with x horizontal towards the right and y vertical towards the bottom (such as in ImageJ or with pyplot with the option origin=’upper’), tehn the second slice is below to ensure a right handed coordinate system.
In this recipe we will work with the polycristalline pure titanium sample, load it into memory, print some of the grain values and display it in 2d and 3d to chack the consistency.
3d analysis and rendering of damage in a steel tension sample¶
This recipe demonstrate how to analyze the cavities in a steel sample deformed under tension loading.
Get the complete Python source code: steel_damage_analysis.py
The 3d data set is a 8 bits binary file with size 341x341x246. First you need to define the path to the 3d volume:
data_dir = '../../examples/data'
scan_name = 'steel_431x431x246_uint8'
scan_path = os.path.join(data_dir, scan_name + '.raw')
Then load the volume in memory thanks to the :pymicro:file:file_utils:HST_read function:
data = HST_read(scan_path, header_size=0)
The reconstructed tomographic volume is not align with the cartesian axes, so rotate the volume by 15 degrees:
data = ndimage.rotate(data, 15.5, axes=(1, 0), reshape=False)
The result of these operations can be quickly observed using regular pyplot functions such as imshow. Here we look at slice number 87 where a ring artifact is present. We will take care of those later.
The entire volume is binarized using a simple threshold with the value 100:
data_bin = np.where(np.greater(data, 100), 0, 255).astype(np.uint8)
And cavities are labeled using standard scipy tools:
label_im, nb_labels = ndimage.label(data_bin)
3465 labels have been found, the result can be observed on slice 87 using pyplot:
We use a simple algorithm to remove ring articfacts (assuming the cavities are convex): the center of mass of each label is compute and if it does not lie inside the label, the corresponding voxels are set to 0. Here 11 rings were removed.
# simple ring removal artifact
coms = ndimage.measurements.center_of_mass(data_bin / 255, labels=label_im, \
index=range(1, nb_labels + 1))
rings = 0
for i in range(nb_labels):
com_x = round(coms[i][0])
com_y = round(coms[i][1])
com_z = round(coms[i][2])
if not label_im[com_x, com_y, com_z] == i + 1:
print 'likely found a ring artifact at (%d, %d, %d) for label = %d, value is %d' \
% (com_x, com_y, com_z, (i + 1), label_im[com_x, com_y, com_z])
data_bin[label_im == (i + 1)] = 0
rings += 1
print 'removed %d rings artifacts' % rings
The outside of the sample needs a little love, it is by far the largest label here so after inverting the labeled image and running a binary closing it can be set to a fixed value (155 here) in the data_bin array.
print('labeling and using a fixed color around the specimen')
# the outside is by far the largest label here
mask_outside = (sizes >= ndimage.maximum(sizes))
print('inverting the image so that outside is now 1')
data_inv = 1 - label_im
outside = mask_outside[data_inv]
outside = ndimage.binary_closing(outside, iterations=3)
# fix the image border
outside[0:3, :, :] = 1;
outside[-3:, :, :] = 1;
outside[:, 0:3, :] = 1;
At this point the segmented volume can be written to the disk for later use using the HST_write function:
plt.subplots_adjust(left=0.1, right=0.95, top=0.95, bottom=0.1)
Convention to read and write 2d images with pyplot¶
- When dealing with 3d images, pymicro uses the following convention:
the array in memory represents (x,y,z) data.
the data is written to the disk as a succession of (x,y) slices, x being the fastest moving index.
This matches ImageJ convention when looking at the data. For instance looking at the 100th slice of the foam data set:
When writting 2d images such as tif, jpg or png, out of a numpy array one may use existing function from matplotlib.pyplot, scipy.misc, etc… but careful checking needs to be done to remain consistent since pyplot assume a (y,x) array when plotting, writting, reading images.. This recipe demonstrate how to use the pyplot module to write / read 2d images. For other format than PNG, pyplot uses PIL under the hood so this should work with PIL as well.
Get the complete Python source code: convention_read_write_2d_images.py
In this example we use the foam 3d dataset with size 250x250x250. First you need to define the path to the 3d volume:
data_dir = '../../examples/data'
scan_name = 'mousse_250x250x250_uint8'
scan_path = os.path.join(data_dir, scan_name + '.raw')
Then load the volume in memory thanks to the :pymicro:file:file_utils:HST_read function. The zrange option is used here to load only the 100th slice.
vol = HST_read(scan_path, autoparse_filename=True, zrange=range(100, 101))
The slice is saved as a raster image (eg. png or jpg) using pyplot imsave function:
plt.imsave('%s.%s' % (scan_name, ext), np.transpose(vol[:, :, 0]), cmap=cm.gray)
The 2d image can now be loaded into memory as a numpy array using pyplot imread function:
im = np.transpose(plt.imread('%s.%s' % (scan_name, ext))[:, :, 0])
Note that imsave write png image as 4 RGBA (red, green, blue, alpha) using floatting point numbers between 0 and 1. Here we have selected only the first channel (R) since in gray level images R=G=B. To compare with our initial raw image we may just re-transforming it to 8 bits (between 0 and 255):
im_uint8 = (255 * im).astype(np.uint8)
We can print some of the pixel value to verify the data is well ordered in memory:
print 'in raw data: pixel value at [124,108] is %d, at [157,214] is %d' % (vol[124, 108, 0], vol[157, 214, 0])
print 'in 2d image: pixel value at [124,108] is %d, at [157,214] is %d' % (im_uint8[124, 108], im_uint8[157, 214])
Finally the data is compared using imshow both on the raw data and the numpy array read from the 2d image and plotting a profile along x:
plt.imshow(np.transpose(vol[:, :, 0]), cmap=cm.gray)
plt.imshow(im.T, cmap=cm.gray)
plt.plot(vol[:, 108, 0], label='raw data')
plt.plot(im_uint8[:, 108], label='%s image' % ext)
Everythings lloks in order, the changes in terms of values along x are normal since pyplot renormalize the data between 0 and 1 to write the png image (to minimize the loss of information).
In summary, 3d volumes are handled via (x,y,z) 3d numpy arrays in pymicro, pyplot assume a (y,x) array when plotting, writting, reading images. So if you care about having everything consistent you must transpose the arrays passed to pyplot.
Understanding Euler angles and the orientation matrix¶
In crystallography, the orientation of a lattice can be described with respect to the laboratory frame by a rotation. In material science, this description follows the passive convention (as used by pymicro) which means the rotation is defines such as it brings the laboratory frame in coincidence with the crystal frame.
Rotations in Euclidian space have 3 independent components as shown by Euler. They can be described in a number of ways such as:
Euler angles
Orientation matrix
Rodrigues vector
Quaternion
Euler angles are a common way of defining a rotation by combining 3 successive rotations around different axes. Here we use the convention of Bunge which is to rotate first around Z then around the new X and finally around the new Z. This example will show how the 3 successive rotations are carried out and that they indeed bring the laboratory frame (XYZ) in coincidence with the crystal frame.
Get the complete Python source code: euler_angles_anim.py
As an example, take the following triplet of Euler angles (in degrees): \((\phi_1, \Phi, \phi_2) = (142.8, 32.0, 214.4)\)
euler_angles = np.array([142.8, 32.0, 214.4])
(phi1, Phi, phi2) = euler_angles
orientation = Orientation.from_euler(euler_angles)
This instance of Orientation can be used to display a crystal lattice in a 3D scene
s3d = Scene3D(display=True, ren_size=(600, 600))
a = 0.4045 # nm, value for Al
l = Lattice.cubic(a)
cubic_lattice = lattice_3d(l, crystal_orientation=orientation, tubeRadius=0.1 * a, sphereRadius=0.2 * a)
s3d.add(cubic_lattice)
A 3D view of a cubic lattice with a given orientation.¶
Now by applying successively the 3 rotation, we show that the laboratory frame is made coincident with the crystal frame. The first rotation of angle \(\phi_1\) is around Z, the second rotation of angle \(\Phi\) is around the new X and the third rotation angle \(\phi_2\) is around the new Z.
The expression of the orientation matrix is obtained by composing the three rotations \(\mathbf{R}_{\phi_1}\), \(\mathbf{R}_{\Phi}\) and \(\mathbf{R}_{\phi_2}\):
Using the given Euler angle yield the following orientation matrix:
With the orientation matrix it is the possible to express any vector \(V_c\) from the cartesian crystal frame to the sample frame by:
From this it follows that the lines of \(\mathbf{g}\) are actually composed by the 3 cartesian lattice vectors expressed in the lab frame. Similarly, the columns of \(\mathbf{g}\) are composed by the 3 cartesian laboratory vectors expressed in the crystal frame.
For instance, using the same Euler angles and considering the vector \(V_c = (1, 0, 0)\) gives \(V_s=(0.946, -0.117, -0.299)\) which is the first line of \(\mathbf{g}\). Chosing \(V_c = (1, 1, 1)\) gives \(V_s=(1.240, 1.203, 0.111)\). This can be verified using pymicro:
>>> g = orientation.orientation_matrix()
>>> Vc = np.array([1, 0, 0])
>>> print(np.dot(g.T, Vc))
[ 0.94690263, -0.11723012, -0.2993869]
>>> Vc = np.array([1, 1, 1])
>>> print(np.dot(g.T, Vc))
[ 1.24033795 1.20380558 0.11141766]
Finally, looking at the 3d representation for direction \([1, 1, 1]\) shows the values seem correct.
3D view showing the [111] lattice vector in the sample frame XYZ.¶
Image alignment by point set registration¶
It is often the case that images are acquired with different modalities, pehaps different spatial resolutions on the same specimen and we want to register them together to have a consistent description across all available images. In this recipe we will see how to achieve that using point set registration between different images.
Get the complete Python source code: pointset_registration.py
To compute the affine transform between an image and the reference image, at least 3 pair of points must be identified. The matching pairs are usually obtained by selecting unique features in both images and measuring their coordinates. Here we will illustrate the method compute_affine_transform and then apply it on a real example.
Let’s consider 5 points given by their coordinates \((x, y)\) in the reference 2D space.
random.seed(13)
n = 5
ref_points = np.empty((n, 2))
for i in range(n):
ref_points[i, 0] = random.randint(0, 10)
ref_points[i, 1] = random.randint(0, 10)
For the sake of the example, let’s transform the 5 points by an isotropic scaling of \(s=1.4\), a rotation of \(\theta=22\) degrees and a translation of \((t_x, t_y)=(5, 3)\). The transformation is can be achieved by multiplying the augmented coordinates vector \((x, y, 1)^T\) by the affine transform matrix \(\mathbf{A}=\mathbf{T}.\mathbf{S}.\mathbf{R}\) obtained by the composition of the rotation, scaling and translation (the order of the translation is important, here it is applied after the rotation and scaling).
For this example we have chosen the following transformation:
# compute the affine transform by composing R, S and T
s = 1.4
angle = 22 * pi / 180.
tx, ty = 5, 3
S = np.array([[s, 0.0, 0.0],
[0.0, s, 0.0],
[0.0, 0.0, 1.0]])
R = np.array([[cos(angle), -sin(angle), 0.0],
[sin(angle), cos(angle), 0.0],
[0.0, 0.0, 1.0]])
T = np.array([[1.0, 0.0, tx],
[0.0, 1.0, ty],
[0.0, 0.0, 1.0]])
A = np.dot(T, np.dot(S, R))
print('full affine transform:\n{:s}'.format(A))
Which is also equivalent to:
In this case, one can verify that the full transformation is:
With either solution, we can transform the points to obtain the new coordinates:
# transform the points
tsr_points = np.empty_like(ref_points)
for i in range(n):
p = np.dot(A, [ref_points[i, 0], ref_points[i, 1], 1])
tsr_points[i, 0] = p[0]
tsr_points[i, 1] = p[1]
With the two lists (or arrays) of points, one can compute the affine transform:
# compute the affine transform from the point set
translation, transformation = compute_affine_transform(ref_points, tsr_points)
invt = np.linalg.inv(transformation)
offset = -np.dot(invt, translation)
And register the points using the inverse affine transform and the offset computed from the centroids.
ref_centroid = np.mean(ref_points, axis=0)
tsr_centroid = np.mean(tsr_points, axis=0)
new_points = np.empty_like(ref_points)
for i in range(n):
new_points[i] = ref_centroid + np.dot(transformation, tsr_points[i] - tsr_centroid)
The 5 transformed points registered by applying the inverse affine transform.¶
Atomic Scattering Factors for X-rays¶
pymicro Package¶
pymicro Package¶
A package to work with material microstructures and 3d data sets
Subpackages¶
apps Package¶
- A package regrouping some graphical utilities to help working with
synchrotron data.
View Module¶
- class pymicro.apps.View.View(arg)¶
Bases:
objectQuick visualisation of various instances at use in pymicro.
This class provides a way to quickly visualise an instance of a particular class. A 3D scene is then created with default settings and the actor added to the scene. The following types are supported:
str: the load_STL_actor method is called to create the actor;
Grain: the grain_3d method is called to create the actor;
Orientation: A cube is created and rotated according to the passive orientation matrix;
Lattice: the lattice_3d method is called to create the actor;
numpy array: the show_array method is called to create the actor;
vtk actor: the actor is directly added to the scene.
The key_pressed_callback is activated so it is possible to save an image using the ‘s’ key.
ImageViewer Module¶
core Package¶
A package implementing a generic data platform for multimodal datasets.
samples Module¶
DataPlatform module for the management of multimodal mechanical datasets.
The samples module provides a base class SampleData implementing a generic data model and data paltform API, allowing to define by inheritance specific data platform classes for specific mechanical applications. It is designed to gather and manipulate easily multimodal datasets gathering data from 3D/4D imaging experiments and simulations of mechanical material samples. Such data consists in volumic data defined on 3D meshes/images, classical data arrays, and associated metadata.
- class pymicro.core.samples.SampleData(filename='sample_data', sample_name='', sample_description=' ', verbose=False, overwrite_hdf5=False, autodelete=False, autorepack=False, after_file_open_args={})¶
Bases:
objectBase class to store multi-modal datasets for material science.
SampleData is a high level API designed to create and interact with complex datasets collecting all the data generated for a material sample by material scientists (from experiments, numerical simulation or data processing).
Each dataset consist of a pair of files: one HDF5 file containing all data and metadata, and one XDMF data gathering metadata for all spatially origanized data (grids and fields). The class ensures creation and synchronization of both XDMF and HDF5 files.
The HDF5 and XDMF data tree structure and content in both files are accessible through the h5_dataset and xdmf_tree class attributes, that are respectively instances of classes imported from the Pytables and lxml packages.
SampleData Naming system
An index of the dataset content is stored into a dictionary content_index as an attribute of the class. Additional names can be defined by users for data items, and are stored in an alias dictionary. The content_index dic is synchronized with the hdf5 Group ‘/Index’. The aliases dic is synchronized with the ‘/Index/Aliases’ Group. When an existing dataset is opened to create a SampleData instance, these attributes are initialized from these Groups in the dataset HDF5 file. Each data item can be accessed in the API methods via:
- path
it’s path in the HDF5 data tree
- indexname
it’s name in the content_index dic (the key associated to it’s path)
- alias
an alias of it’s indexname
- name
its name as a HDF5 Node if it is unique
Data Compression
HDF5 compression algorithm are available through the Pytables package. SampleData offers an interface to it with the
set_chunkshape_and_compression()method.Arguments of the Class constructor
- Filename
str basename of HDF5/XDMF files to create/read. A file pair is created if the filename do not match any existing file.
- Sample_name
str, optional (‘’) name of the sample associated to data (metadata, dataset “title”). If the class is called to open a pre-existing filename, its sample name is not overwritten.
- Sample_description
str, optional (‘’) short description of the mechanical sample (material, type of tests…. – metadata). If the class is called to open a pre-existing filename, its sample name is not overwritten.
- Verbose
bool, optional (False) set verbosity flag
- Overwrite_hdf5
bool, optional (False) set to True to overwrite existing HDF5/XDMF couple of files with the same filename.
- Autodelete
bool, optional (False) set to True to remove HDF5/XDMF files when deleting SampleData instance.
- Autorepack
bool, optional (False) if True, the HDF5 file is automatically repacked when deleting the SampleData instance, to recover the memory space freed up by data compression operations. See
repack_h5file()for more details.
Class attributes
- H5_file
basename of HDF5 file containing dataset (str)
- H5_path
full path of the HDF5 dataset file
- H5_dataset
tables.Fileinstance associated to the h5_file- Xdmf_file
name of XDMF file associated with h5_file (str)
- Xdmf_path
full path of XDMF file (str)
- Xdmf_tree
lxml.etreeXML tree associated with xdmf_file- Autodelete
autodelete flag (bool)
- Autorepack
autorepack flag (bool)
- After_file_open_args
command arguments for after_file_open (dict)
- Content_index
Dictionnary of data items (nodes/groups) names and pathes in HDF5 dataset (dic)
- Aliases
Dictionnary of list of aliases for each item in content_index (dic)
- __init__(filename='sample_data', sample_name='', sample_description=' ', verbose=False, overwrite_hdf5=False, autodelete=False, autorepack=False, after_file_open_args={})¶
Sample Data constructor, see class documentation.
- __del__()¶
Sample Data destructor.
- Deletes SampleData instance and:
closes h5_file –> writes data structure into the .h5 file
writes the .xdmf file
- __repr__()¶
Return a string representation of the dataset content.
- __contains__(name)¶
Check if name refers to an existing HDF5 node in the dataset.
- Parameters
name (str) – a string for the name / indexname / path
- Return bool
True if the dataset has a node associated with this name, False if not.
- __getitem__(key)¶
Implement dictionnary like access to hdf5 dataset items.
- __getattribute__(name)¶
Implement attribute like access to hdf5 dataset items.
- minimal_data_model()¶
Specify minimal data model to store in class instance.
This method is designed to construct derived classes from SampleData to serve as data platforms for a specific data model, that is specified by the two dictionaries returned.
The class constructor searches through these dictionaries to determine the name, pathes and types of data items constituting the data model of the class, and creates them at each instance creation. If the constructor is used to create an instance from an existing file, the compatibility with the data model is verified, and the missing data items are created if needed. The constructor ensures compatibility of previously created datasets with the class if the current data model of the class has been enriched. Note that the associated datasets can contain additional data items, those defined in this method are the minimal required content for this class of datasets.
The return dictionaries keys are the Index names of the data items of the data model. Their values are defined hereafter:
- Return dic index_dic
- Dictionary specifying the indexnames and pathes of data items inthe data model. Each entry must be of the form{‘indexname’:’/data_item/path’}. The data item path is its pathin the HDF5 tree structure.
- Return dic type_dic
- Dictionary specifying the indexnames and types of data items inthe data model. Each entry must be of the form{‘indexname’:’grouptype’}. ‘grouptype’ can be:
- ‘Group’
creates a classical HDF5 group (str)
- ‘3DImage’
creates a HDF5 group containing datasets (fields) defined on the same 3D image (str)
- ‘Mesh’
creates a HDF5 group containing datasets (fields) defined on the same mesh (str)
- ‘Array’
creates a HDF5 node containing a data array (str)
- Table_description
creates a structured storage array (
tables.Filtersclass) with the given Description. Table_description must be a subclass oftables.IsDescriptionclass.
- print_xdmf()¶
Print a readable version of xdmf_tree content.
- write_xdmf()¶
Write xdmf_tree in .xdmf file with suitable XML declaration.
- print_dataset_content(as_string=False, max_depth=3, to_file=None, short=False)¶
Print information on all nodes in the HDF5 file.
- Parameters
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
max_depth (int) – Only prints data item whose depth is equal or less than this value. The depth is the number of parents a data item has. The root Group has thus a depth of 0, its children a depth of 1, the children of its children a depth of 2…
to_file (str) – (optional) If not None, writes the dataset information to the provided text file name to_file. In that case, nothing is printed to the standard output.
short (bool) – If True, return a short description of the dataset content, reduced to hdf5 tree structure and node memory sizes.
- Return str s
string representation of HDF5 nodes information
- print_group_content(groupname, recursive=False, as_string=False, max_depth=1000, to_file=None, short=False)¶
Print information on all nodes in a HDF5 group.
- Parameters
groupname (str) – Name, Path, Index name or Alias of the HDF5 group
recursive (bool) – If True, print content of children groups
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
max_depth (int) – Only prints data item whose depth is equal or less than this value. The depth is the number of parents a data item has. The root Group has thus a depth of 0, its children a depth of 1, the children of its children a depth of 2… Note that this depth is an absolute depth. Thus, if you want for instance to print the content of a group with depth 3, with 2 levels of depth (its childrens and their childrens), you will need to specify a depth of 5 for this method.
to_file (str) – (optional) If not None, writes the group contentto the provided text file name to_file. In that case, nothing is printed to the standard output.
short (bool) – If True, return a short description of the dataset content, reduced to hdf5 tree structure and node memory sizes.
- Return str s
string representation of HDF5 nodes information
- print_node_info(nodename, as_string=False, short=False)¶
Print information on a node in the HDF5 tree.
Prints node name, content, attributes, compression settings, path, childrens list if it is a group.
- Parameters
name (str) – Name, Path, Index name or Alias of the HDF5 Node
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
short (bool) – If True, return a short description of the node content, reduced to name, hdf5 type and node memory sizs.
- Return str s
string representation of HDF5 Node information
- print_node_attributes(nodename, as_string=False)¶
Print the hdf5 attributes (metadata) of an array node.
:param str name:Name, Path, Index name or Alias of the HDF5 Node :param bool as_string: If True solely returns string representation. If False, prints the string representation. :return str s: string representation of HDF5 Node compression settings
- print_node_compression_info(name, as_string=False)¶
Print the compression settings of an array node.
:param str name:Name, Path, Index name or Alias of the HDF5 Node :param bool as_string: If True solely returns string representation. If False, prints the string representation. :return str s: string representation of HDF5 Node compression settings
- print_data_arrays_info(as_string=False, to_file=None, short=False)¶
Print information on all data array nodes in hdf5 file.
Mesh node and element sets are excluded from the output due to their possibly very high number.
- Parameters
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
to_file (str) – (optional) If not None, writes the dataset information to the provided text file name to_file. In that case, nothing is printed to the standard output.
short (bool) – If True, return a short description of the dataset content, reduced to hdf5 tree structure and node memory sizes.
- Return str s
string representation of HDF5 nodes information
- print_grids_info(as_string=False, to_file=None, short=False)¶
Print information on all grid groups in hdf5 file.
- Parameters
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
to_file (str) – (optional) If not None, writes the dataset information to the provided text file name to_file. In that case, nothing is printed to the standard output.
short (bool) – If True, return a short description of the dataset content, reduced to hdf5 tree structure and node memory sizes.
- Return str s
string representation of HDF5 nodes information
- print_index(as_string=False, max_depth=3, local_root='/')¶
Print a list of the data items in HDF5 file and their Index names.
- Parameters
as_string (bool) – If True solely returns string representation. If False, prints the string representation.
max_depth (int) – Only prints data item whose depth is equal or less than this value. The depth is the number of parents a data item has. The root Group has thus a depth of 0, its children a depth of 1, the children of its children a depth of 2…
local_root (str) – prints only the Index for data items that are children of the provided local_root. The Name, Path, Indexname, or Alias of the local_root can be passed for this argument.
- Return str s
string representation of HDF5 nodes information if as_string is True.
- sync()¶
Synchronize and flush .h5 and .xdmf files with dataset content.
After using the sync method, the XDMF file can be opened in Paraview and 3DImage and/or Mesh data visualized, even if the files are still open in the class instance.
Important
Paraview >=5 cannot read data from synchronized files, you must close them first. In this case, use method
pause_for_visualization().
- pause_for_visualization(Vitables=False, Paraview=False, **keywords)¶
Flushes data, close files and pause interpreter for visualization.
This method pauses the interpreter until you press the <Enter> key. During the pause, the HDF5 file object is closed, so that it can be read by visualization softwares like Paraview or ViTables. Two optional arguments allow to directly open the dataset with Paraview and/or Vitables, as a subprocess of Python. In these cases, the Python interpreter is paused until you close the visualization software.
Paraview allows to visualize the volumic data that is stored in the SampleData dataset, i.e. Mesh and Images groups (geometry and stored fields). Vitables allows to visualize the content of the HDF5 dataset in term of data tree, arrays content and nodes attributes. If both are requested, Vitables is executed before Paraview.
- Parameters
Vitables (bool) – set to True to launch Vitables on the HDF5 file of the instance HDF5 dataset.
Paraview (bool) – set to True to launch Paraview on the XDMF file of the instance.
- switch_verbosity()¶
Change the verbosity flag to its opposite.
- add_mesh(mesh_object=None, meshname='', indexname='', location='/', description=' ', replace=False, bin_fields_from_sets=True, file=None, compression_options={})¶
Create a Mesh group in the dataset from a MeshObject.
A Mesh group is a HDF5 Group that contains arrays describing mesh geometry, and fields defined on this mesh. The mesh geometry HDF5 nodes are: This methods adds a Mesh group to the dataset from a BasicTools
UnstructuredMeshclass instance.- Nodes
array of shape (Nnodes,Ndim) with path
'/Mesh_Path/Geometry/Nodes'and Index name'Meshname_Nodes'- Elements
array of shape (Nelements,Nelement_nodes) with path
'/Mesh_Path/Geometry/Elements'and Index name'Meshname_Elements'
Mesh group may also contain data array to describe fields, whose pathes, index names and content can be set using the class method
add_data_array(). Fields defined on nodes must have a shape equal to (Nnodes,Field_dimension). Fields defined on integration points must have a shape equal to (Nintegration_points,Field_dimension).- Parameters
mesh_object – mesh to add to dataset. It is an instance from the
pymicro.core.meshes.MeshObjectclassmeshname (str) – name used to create the Mesh group in dataset
indexname – Index name used to reference the Mesh group
description (str) – Description metadata for this mesh
replace (bool) – remove Mesh group in the dataset with the same name/location if True and such group exists
bin_fields_from_sets (bool) – If True, stores all Node and Element Sets in mesh_object as binary fields (1 on Set, 0 else)
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- Location str
Path, Name, Index Name or Alias of the parent group where the Mesh group is to be created
- add_mesh_from_image(imagename, with_fields=True, ofTetras=False, meshname='', indexname='', location='/', description=' ', replace=False, bin_fields_from_sets=True, compression_options={})¶
Create a Mesh group in the dataset from an Image dataset.
The mesh group created can represent a mesh of tetrahedra or a mesh of hexaedra, of the image domain (square/triangles in 2D). The fields in the mesh groups are restored, with an adequate shape, and a suffix ‘_msh’ in their indexname.
- Parameters
imagename (str) – Name, Path or Indexname of the mesh group to get
with_fields (bool) – If True, load the nodes and elements fields from the image group into the mesh object.
ofTetras (bool) – if True, returns a mesh with tetrahedron elements. If False, return a rectilinera mesh of hexaedron elements.
meshname (str) – name used to create the Mesh group in dataset
indexname – Index name used to reference the Mesh group
description (str) – Description metadata for this mesh
replace (bool) – remove Mesh group in the dataset with the same name/location if True and such group exists
bin_fields_from_sets (bool) – If True, stores all Node and Element Sets in mesh_object as binary fields
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- Location str
Path, Name, Index Name or Alias of the parent group where the Mesh group is to be created
- add_image(image_object=None, imagename='', indexname='', location='/', description='', replace=False, field_indexprefix='', compression_options={})¶
Create a 2D/3D Image group in the dataset from an ImageObject.
An Image group is a HDF5 Group that contains arrays describing fields defined on an image (uniform grid of voxels/pixels). This methods adds an Image group to the dataset from a BasicTools
ConstantRectilinearMeshclass instance.This class represents regular meshes of square/cubes, i.e. pixels/voxels.The image geometry and topology is defined by HDF5 attributes of the Image Group, that are:
- nodes_dimension
np.array, number of grid points along each dimension of the Image. This number is array is equal to the dimension attribute array +1 for each value.
- dimension
np.array, number of voxels along each dimension of the Image (Nx,Ny,Nz) or (Nx,Ny)
- spacing
np.array, voxel size along each dimension (dx,dy,dz) or (dx, dy)
- origin
np.array, coordinates of the image grid origin, corresponding to the first vertex of the voxel [0,0,0] or pixel [0,0]
The Image group may also contain arrays of field values on the image. These fields can be elementFields (defined at pixel/voxel centers) or nodefields (defined at pixel/voxel vertexes). Their pathes, index names and content can be set using the class method
add_field(). Fields defined on nodes must have a shape equal to the nodes_dimension attribute. Fields defined on elements must have a shape equal to the dimension attribute. Both can have an additional last dimension if they have a higher dimensionality than scalar fields (for instance [Nx,Ny,Nz,3] for a vector field).- Parameters
image_object – image to add to dataset. It is an instance from the
ConstantRectilinearMeshclass of the BasicTools Python package.imagename (str) – name used to create the Image group in dataset
indexname (str) – Index name used to reference the Image. If none is provided, imagename is used.
description (str) – Description metadata for this 3D image
replace (bool) – remove Image group in the dataset with the same name/location if True and such group exists
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- Location str
Path, Name, Index Name or Alias of the parent group where the Image group is to be created
- add_image_from_field(field_array, fieldname, imagename='', indexname='', location='/', description=' ', replace=False, origin=None, spacing=None, is_scalar=True, is_elemField=True, compression_options={})¶
Create a 2D/3M Image group in the dataset from a field data array.
Construct an image object from the inputed field array. This array is interpreted by default as an element field of a pixelized/voxelized grid. Hence, if the field is of shape (Nx,Ny), the image group will store a (Nx,Ny) image (i.e. a regular grid of Nx+1,Ny+1 nodes). If specified, the field can be interpreted as a nodal field (values at pixels/voxels vertexes). In this case the method will create a (Nx-1,Ny-1) image of (Nx,Ny) nodes. The same applies in 3D.
If the field is not a scalar field, the last dimension of the field array is interpreted as the dimension containing the field components
- Parameters
field_array (numpy.array) – data array of the field values on the image regular grid.
fieldname (str) – add the field to HDF5 dataset and image Group with this name.
imagename (str) – name used to create the Image group in dataset
indexname (str) – Index name used to reference the Image. If none is provided, imagename is used.
description (str) – Description metadata for this 3D image
replace (bool) – remove Image group in the dataset with the same name/location if True and such group exists
origin (np.array(3,)) – Coordinates of the first node of the regular grid of squares/cubes constituting the image geometry
spacing (np.array(3,)) – Size along each dimension of the pixels/voxels composing the image.
is_scalar (bool) – If True (default value), the field is considered as a scalar field to compute the image dimensions from the field array shape.
is_elemField (bool) – If True (default value), the array is considered as a pixel/voxel wise field value array. If False, the field is considered as a nodal value array.
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- Location str
Path, Name, Index Name or Alias of the parent group where the Image group is to be created
- add_grid_time(gridname, time_list)¶
Add a list of time values to a grid data group.
If the grid has no time value, an xdmf grid temporal collection is created.
- Parameters
gridname (str) – Path, name or indexname of the grid Group where to add time values.
time_list (list(float)) – List of times to add to the grid. Can also be passed as a numpy array.
- add_group(groupname, location='/', indexname='', replace=False)¶
Create a standard HDF5 group at location with no grid properties.
If the group parents in location do not exist, they are created.
- Parameters
groupname (str) – Name of the group to create
location (str) – Path where the group will be added in the HDF5 dataset
indexname (str) – Index name used to reference the Group. If none is provided, groupname is used.
replace (bool) – remove 3DImage group in the dataset with the same name/location if True and such group exists
createparents (bool) – if True, create parent nodes in path if they are not present in the dataset
- add_field(gridname, fieldname, array=None, location=None, indexname=None, chunkshape=None, replace=False, visualisation_type='Elt_mean', compression_options={}, time=None, bulk_padding=True)¶
Add a field to a grid (Mesh or 2D/3DImage) group from a numpy array.
This methods checks the compatibility of the input field array with the grid dimensionality and geometry, adds it to the HDF5 dataset, and the XDMF file. Metadata describing the field type, dimensionality are stored as field HDF node attributes. The path of the field is added to the grid Group as a HDF5 attribute.
- Parameters
gridname (str) – Path, name or indexname of the grid Group on which the field will be added
fieldname (str) – Name of the HDF5 node to create that will contain the field value array
array (np.array) – Array containing the field values to add in the dataset
- ;param str location: Path, name or indexname of the Group in which the
field array will be stored. This Group must be a children of the gridname Group. If not provided, the field is stored in the gridname Group.
- Parameters
indexname (str) – Index name used to reference the field node
chunkshape (tuple) – The shape of the data chunk to be read or written in a single HDF5 I/O operation
replace (bool) – remove 3DImage group in the dataset with the same name/location if True and such group exists
filters (Filters) – instance of
tables.Filtersclass specifying compression settings.visualisation_type (str) – Type of visualisation used to represent integration point fields with an element wise constant field. Possibilities are ‘Elt_max’ (maximum value per element), ‘Elt_mean’ (mean value per element), ‘None’ (no visualisation field). Default value is ‘Elt_mean’
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
time (float) – Associate a time value for this field. IF a time value is provided, the suffix ‘_T{time_index}’ is appended to the fieldname and indexname
bulk_padding (bool) – If adding a field on a mesh that has as many bulk as boundary elements, forces field padding to bulk if True, or to boundary if false
- add_data_array(location, name, array=None, indexname=None, chunkshape=None, replace=False, compression_options={})¶
Add a data array node at the given location in the HDF5 dataset.
The method uses the
CArrayandtables.Filtersclasses of the Pytables package to add data arrays in the dataset and control their chunkshape and compression settings.- Parameters
location (str) – Path where the array will be added in the dataset
name (str) – Name of the array to create
array (np.array) – Array to store in the HDF5 node
indexname (str) – Index name used to reference the node
chunkshape (tuple) – The shape of the data chunk to be read or written in a single HDF5 I/O operation
replace (bool) – remove 3DImage group in the dataset with the same name/location if True and such group exists
filters (Filters) – instance of
tables.Filtersclass specifying compression settings.empty (bool) – if True create the path, Index Name in dataset and store an empty array. Set the node attribute empty to True.
compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- add_table(location, name, description, indexname=None, chunkshape=None, replace=False, data=None, compression_options={})¶
Add a structured storage table in HDF5 dataset.
- Parameters
location (str) – Path where the array will be added in the dataset
name (str) – Name of the array to create
description (IsDescription) – Definition of the table rows, can be a numpy dtype.
indexname (str) –
Index name used to reference the node composition as a sequence of named fields (analogous to Numpy structured arrays). It must be an instance of the
tables.IsDescriptionclass from the Pytables packagechunkshape (tuple) – The shape of the data chunk to be read or written in a single HDF5 I/O operation
replace (bool) – remove 3DImage group in the dataset with the same name/location if True and such group exists
data (np.array(np.void)) – Array to store in the HDF5 node. dtype must be consistent with the table description.
filters (Filters) – instance of
tables.Filtersclass specifying compression settings.compression_options (dict) – Dictionary containing compression options items, see set_chunkshape_and_compression method for more details.
- add_tablecols(tablename, description, data=None)¶
Add new columns to a table node.
- Parameters
tablename (str) – Name, Path or Indexname of the table where the columns must be added.
description (np.dtype or tables.IsDescription) – Description of the fields constituting the new columns to add the table.
data (np.array, optional) – values to add into the new columns, defaults to None. The dtype of this array must be constitent with the description argument.
- Raises
ValueError – If data.dtype and description do no match
- add_string_array(name, location, indexname=None, replace=False, data=[])¶
Add an enlargeable array to store strings of max 255 characters.
String arrays are typically used to store large list of strings that are too large to be stored as HDF5 attributes into the dataset.
Warning
The string are stored as byte strings. You will need to use the str.decode() method to get the elements of the string_array as UTF-8 or ASCII formatted strings.
To manipulate a string array use the ‘get_node’ method to get the array, and then manipulate as a list of binary strings.
- Parameters
name (str) – Name of the array to create
location (str) – Path where the array will be added in the dataset
indexname (str) –
Index name used to reference the node composition as a sequence of named fields (analogous to Numpy structured arrays). It must be an instance of the
tables.IsDescriptionclass from the Pytables packagereplace (bool) – remove array in the dataset with the same name/location if True
data (list[str]) – List of strings to add to the string array upon creation.
- append_string_array(name, data=[])¶
Append a list of strings to a string array node in the dataset.
- Parameters
name (str) – Path, Indexname, Name or Alias of the string array to which append the list of strings.
data (list(str)) – List of strings to append to the string array.
- append_table(name, data)¶
Append a numpy structured array to a table in the dataset.
- Parameters
name (str) – Path, Indexname, Name or Alias of the string array to which append the list of strings.
data (numpy.ndarray) – array to append to the table? Its dtype must match de table description.
- add_attributes(dic, nodename)¶
Add a dictionary entries as HDF5 Attributes to a Node or Group.
- Parameters
dic (dic) – Python dictionary of items to store in HDF5 file as HDF5 Attributes
nodename (str) – Path, Index name or Alias of the HDF5 node or group receiving the Attributes
- add_alias(aliasname, path=None, indexname=None)¶
Add alias name to reference Node with inputed path or index name.
- Parameters
aliasname (str) – name to add as alias to reference the node
path (str) – Path of the node to reference with aliasname
indexname (str) – indexname of the node to reference with aliasname
- add_to_index(indexname, path, colname=None)¶
Add path to index if indexname is not already in content_index.
- Parameters
indexname (str) – name to add as indexname to reference the node
path (str) – Path of the node to reference with aliasname
colname (str) – if the node is a table node, set colname to reference a column (named field) of the table with this indexname
- compute_mesh_elements_normals(meshname, element_tag, Normal_fieldname=None, align_vector=<Mock name='mock.random.rand()' id='139894657890512'>, as_nodal_field=False)¶
Compute the normals of a set of boundary elements of a mesh group.
The normals are stored as en element wise constant field in the mesh group.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
element_tag (str) – name of the element tag or element type whose normals must be computed
Normal_fieldname (TYPE, optional) – Name of the normals field to store on the mesh group. Defaults to ‘Normals_element_tag_name’
align_vector (np.array(3), optional) – All normals are oriented to have positive dot product with align_vector, defaults to [0,0,1]
- Raises
ValueError – Can only process elements of bidimensional topology (surface elements, like triangles, quadrilaterals…)
- get_indexname_from_path(node_path)¶
Return the Index name of the node at given path.
- Parameters
node_path (str) – Path of the node in the HDF5 data tree
- Returns
Returns the Index name of the node
- get_mesh(meshname, with_tags=True, with_fields=True, as_numpy=True)¶
Return data of a mesh group as BasicTools UnstructuredMesh object.
This methods gathers the data of a 2DMesh or 3DMesh group, including nodes coordinates, elements types and connectivity and fields, into a BasicTools
ConstantRectilinearMeshobject.- Parameters
meshname (str) – Name, Path or Indexname of the mesh group to get
with_tags (bool) – If True, store the nodes and element tags (sets) into the mesh object
with_fields (bool, optional) – If True, store mesh group fields into the mesh_object, defaults to True
- Returns
Mesh_object containing all data (nodes, elements, nodes and elements sets, fields), contained in the mesh data group.
- Return type
BasicTools
UnstructuredMeshobject.
- get_mesh_from_image(imagename, with_fields=True, ofTetras=False)¶
Return an UnstructuredMesh instance from an Image data group.
- Parameters
imagename (str) – Name, Path or Indexname of the mesh group to get
with_fields (bool) – If True, load the nodes and elements fields from the image group into the mesh object.
ofTetras (bool) – if True, returns a mesh with tetrahedron elements. If False, return a rectilinera mesh of hexaedron elements.
- Returns
Mesh_object containing all data (nodes, elements, nodes and elements sets, fields), corresponding to the Image data group content.
- Return type
BasicTools
UnstructuredMeshobject.
- get_mesh_nodes(meshname, as_numpy=False)¶
Return the mesh node coordinates as a HDF5 node or Numpy array.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
as_numpy (bool) – if True, returns the Node as a numpy.array. If False, returns the node as a Node or Group object.
- Returns
Return the mesh Nodes coordinates array as a
tables.Nodeobject or a numpy.array
- get_mesh_nodesID(meshname, as_numpy=False)¶
Return the mesh node ID as a HDF5 node or Numpy array.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
as_numpy (bool) – if True, returns the Node as a numpy.array. If False, returns the node as a Node or Group object.
- Returns
Return the mesh Nodes ID array as a
tables.Nodeobject or a numpy.array
- get_mesh_node_tag(meshname, node_tag, as_numpy=True)¶
Returns the node IDs of a node tag of the mesh group.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
node_tag (str) – name of the node tag whose IDs must be returned.
as_numpy (bool) – if True, returns arrays in elements container as numpy array
- get_mesh_node_tag_coordinates(meshname, node_tag)¶
Returns the node coordinates of a node tag of the mesh group.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
node_tag (str) – name of the node tag whose IDs must be returned.
as_numpy (bool) – if True, returns arrays in elements container as numpy array
- get_mesh_xdmf_connectivity(meshname, as_numpy=True)¶
Return the mesh elements connectivity as HDF5 node or Numpy array.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
as_numpy (bool) – if True, returns the Node as a numpy.array. If False, returns the node as a Node or Group object.
- Returns
Return the mesh elements connectivity referenced in the XDMF file as a
tables.Nodeobject or a numpy.array
- get_mesh_elements(meshname, with_tags=True, as_numpy=True, get_eltype_connectivity=None)¶
Return the mesh elements connectivity as HDF5 node or Numpy array.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
with_tags (bool) – if True, loads the element tags in the returned elements container
as_numpy (bool) – if True, returns arrays in elements container as numpy array
get_eltype_connectivity (str) – if this argument is set to the name of an element type contained in the mesh, the method retuns the connectivity array of these elements, and not the BasicTools elements container.
- Returns
Return the mesh elements container as a BasicTools
AllElementsobject
- get_mesh_elem_types_and_number(meshname)¶
Returns the list and types of elements tags defined on a mesh.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
- Return dict
keys are element types in the mesh and values are the number of elements for each element type.
- get_mesh_elem_tag(meshname, element_tag, as_numpy=True, local_IDs=False)¶
Returns the elements IDs of an element tag of the mesh group.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
element_tag (str) – name of the element tag whose connectivity must be returned. Can also be one of the element types contained in the mesh. In this case, the complete global element IDs for these elements is return
as_numpy (bool) – if True, returns arrays in elements container as numpy array
local_IDs (bool) – if True, returns the local elements IDs for the element type, i.e. the indexes of elements in the local connectivity array that can be obtain with get_mesh_elements.
- get_mesh_elem_tags_names(meshname)¶
Returns the list and types of elements tags defined on a mesh.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
- Return dict
keys are element tag names in the mesh and values are the element type for each element tag.
- get_mesh_elem_tag_connectivity(meshname, element_tag)¶
Returns the list and types of elements tags defined on a mesh.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
element_tag (str) – name of the element tag whose connectivity must be returned.
- get_mesh_node_tags_names(meshname)¶
Returns the list of node tags defined on a mesh.
- Parameters
meshname (str) – Name, Path, Index name or Alias of the Mesh group in dataset
- Return list node_tags
list of node tag names defined on this mesh
- get_image(imagename, with_fields=True)¶
Return data of an image group as a BasicTools mesh object.
This methods gathers the data of a 2DImage or 3DImage group, including grid geometry and fields, into a BasicTools
ConstantRectilinearMeshobject.- Parameters
imagename (str) – Name, Path or Indexname of the image group to get
with_fields (bool) – If True, load the nodes and elements fields from the image group into the mesh object.
- Returns
Returns a BasicTools rectilinear mesh object with image group data.
- Return type
ConstantRectilinearMesh
- get_tablecol(tablename, colname)¶
Return a column of a table as a numpy array.
- Parameters
tablename (str) – Name, Path, Index name or Alias of the table in dataset
colname (str) – Name of the column to get in the table (analogous to name of the field to get in a Numpy structured array)
- Return numpy.array data
returns the queried column data as a numpy.array
- get_table_description(tablename, as_dtype=False)¶
Get the description of the table as a description or Numpy dtype.
- get_grid_field_list(gridname)¶
Return the list of fields stored on the grid.
- Parameters
gridname (str) – Path, name or indexname of the grid Group on which the field will be added
- Return str Field_list
List of the name of the fields dataset stored in the hdf5 file and defined on the grid.
- get_field(fieldname, unpad_field=True, get_visualisation_field=False)¶
Return a padded or unpadded field from a grid data group as array.
Use this method to get a mesh element wise field in its original form, i.e. bulk element fields (defined on elements of the same dimensonality than the mesh) or a boundary field (defined on elements of a lower dimensionality than the mesh).
- Parameters
fieldname (str) – Name, Path, Index, Alias or Node of the field in dataset
unpad_field (bool) – if True (default), remove the zeros added to to the field to comply with the mesh topology and return it with its original size (bulk or boundary field).
- get_node(name, as_numpy=False)¶
Return a HDF5 node in the dataset.
The node is returned as a
tables.Node,tables.Groupor anumpy.ndarraydepending on its nature, and the value of the as_numpy argument.- Parameters
name (str) – Name, Path, Index name or Alias of the Node in dataset
as_numpy (bool) – if True, returns the Node as a numpy.array. If False, returns the node as a Node or Group object.
- Returns
Return the node as a a
tables.Nodeortables.Groupobject depending on the nature of the node, or, returns it as a numpy.array if required and if the node is an array node.
- get_dic_from_attributes(nodename)¶
Get all attributes from a HDF5 Node in the dataset as a dictionary.
- Parameters
nodename (str) – Name, Path, Index name or Alias of the HDF5 group
- Returns
Dictionary of the form
{'Attribute_name': Attribute_value}
- get_attribute(attrname, nodename)¶
Get a specific attribute value from a HDF5 Node in the dataset.
- Parameters
attrname (str) – name of the attribute to get
nodename (str) – Name, Path, Index name or Alias of the HDF5 group
- Returns
Value of the attribute
- get_file_disk_size(print_flag=True, convert=True)¶
Get the disk size of the dataset.
- Parameters
print_flag (bool) – print the disk size if True
convert (bool) – convert disk size to a suitable memory unit if True. If False, return result in bytes.
- Return float fsize
Disk size of the dataset in unit
- Return str unit
Unit of fsize. bytes is the default
- get_node_disk_size(nodename, print_flag=True, convert=True)¶
Get the disk size of a HDF5 node.
- Parameters
nodename (str) – Name, Path, Index name or Alias of the HDF5 node
print_flag (bool) – print the disk size if True
convert (bool) – convert disk size to a suitable memory unit if True. If False, return result in bytes.
- Return float fsize
Disk size of the dataset in unit
- Return str unit
Unit of fsize. bytes is the default
- get_sample_name()¶
Return the sample name.
- set_sample_name(sample_name)¶
Set the sample name.
- Parameters
sample_name (str) – a string for the sample name.
- get_description(node='/')¶
Get the string describing this node.
By defaut the sample description is returned, from the root HDF5 Group.
- Parameters
nodename (str) – the path or name of the node of interest.
- set_description(description, node='/')¶
Set the description of a node.
By defaut this method sets the description of the complete sample, in the root HDF5 Group.
- Parameters
description (str) – a string for the description of the node or sample.
node (str) – the path or name of the node of interest (‘/’ by default).
- set_new_indexname(nodename, new_indexname)¶
Change the indexname of a node in the dataset.
Usefull to solve indexname duplicates issues that can arises when automatically adding elements to the dataset, that have the same name.
- Parameters
nodename (str) – Name, Path, Indexname or Alias of the node whose indexname is to be changed
new_indexname (str) – New indexname for the node
- set_voxel_size(image_group, voxel_size)¶
Set voxel size for an HDF5/XDMF image data group.
The values are registered in the spacing Attribute of the 3DImage group.
- Parameters
image_data_group (str) – Name, Path, Index name or Alias of the 3DImage group
voxel_size (np.array) – (dx, dy, dz) array of the voxel size in each dimension of the 3Dimage
- set_origin(image_group, origin)¶
Set origin coordinates for an HDF5/XDMF image data group.
The origin corresponds to the first vertex of the first voxel, that is referenced by the [0,0,0] elements of arrays in the 3DImage group. The values are registered in the origin Attribute of the 3DImage group.
- Parameters
image_data_group (str) – Name, Path, Index name or Alias of the 3DImage group
voxel_size (np.array) – (Ox, Oy, Oz) array of the coordinates in each dimension of the origin of the 3Dimage
- set_tablecol(tablename, colname, column)¶
Store an array into a structured table column.
If the column is not in the table description, a new field corresponding to the inputed column is added to the table description.
- Parameters
tablename (str) – Name, Path, Index name or Alias of the table
colname (str) – Name of the column to set (analogous to name of a field in a Numpy structured array)
column (np.array) – array of values to set as column of the table. It’s shape must match the column shape in table description.
- set_nodes_compression_chunkshape(node_list=None, chunkshape=None, compression_options={})¶
Set compression options for a list of nodes in the dataset.
This methods sets the same set of compression options for a list of nodes in the dataset.
- Parameters
node_list (list) – list of Name, Path, Index name or Alias of the HDF5 array nodes where to set the compression settings.
chunkshape (tuple) – The shape of the data chunk to be read or written in a single HDF5 I/O operation
compression_options (dict) – Dictionary containing compression options items (keys are options names, values are )
Compression Options
Compression settings can be passed through the compression_options dictionary as follows:
compression_options[option_name] = option_value
These options are the Pytables package Filters class constructor parameters (see PyTables documentation for details) The list of available compression options is provided here:
complevel: Compression level for data. Allowed range is 0-9. A value of 0 (the default) disables compression.
complib: Compression library to use. Possibilities are: zlib’ (the default), ‘lzo’, ‘bzip2’ and ‘blosc’.
shuffle: Whether or not to use the Shuffle filter in the HDF5 library (may improve compression ratio).
bitshuffle: Whether or not to use the BitShuffle filter in the Blosc library (may improve compression ratio).
fletcher32: Whether or not to use the Fletcher32 filter in the HDF5 library. This is used to add a checksum on each data chunk.
least_significant_digit: If specified, data will be truncated using
around(scale*data)/scale, wherescale = 2**least_significant_digit. In conjunction with enabling compression, this produces ‘lossy’, but significantly more efficient compression.
Important
If the new compression settings reduce the size of the node in the dataset, the file size will not be changed. This is a standard behavior for HDF5 files, that preserves freed space in disk to add additional data in the future. If needed, use the
repack_file()method to reduce file disk size after changing compression settings. This method is also called by the class instance destructor.Note
If compression settings are passed as additional keyword arguments, they are prioritised over the settings in the inputed Filter object.
- set_chunkshape_and_compression(nodename, chunkshape=None, compression_options={})¶
Set the chunkshape and compression settings for a HDF5 array node.
- Parameters
node (str) – Name, Path, Index name or Alias of the node
chunkshape (tuple) – The shape of the data chunk to be read or written in a single HDF5 I/O operation
compression_options (dict) – Dictionary containing compression options items (keys are options names, values are )
Compression options
Compression settings can be passed through the compression_options dictionary as follows:
compression_options[option_name] = option_value
See
set_nodes_compression_chunkshape(), for the list of available compression options.Important
If the new compression settings reduce the size of the node in the dataset, the file size will not be changed. This is a standard behavior for HDF5 files, that preserves freed space in disk to add additional data in the future. If needed, use the
repack_file()method to reduce file disk size after changing compression settings. This method is also called by the class instance destructor.Note
If compression settings are passed as additional keyword arguments, they are prioritised over the settings in the inputed Filter object.
- set_verbosity(verbosity=True)¶
Set the verbosity of the instance methods to inputed boolean.
- remove_attribute(attrname, nodename)¶
Remove an attribute from a node in the dataset.
- Parameters
attrname (str) – name of the attribute to remove
nodename (str) – Name, Path or Index name of the node to modify
- remove_attributes(attr_list, nodename)¶
Remove an attribute from a node in the dataset.
- Parameters
attr_list (list) – list of the names of the attribute to remove
nodename (str) – Name, Path or Index name of the node to modify
- rename_node(nodename, newname, replace=False, new_indexname=None)¶
Rename a node in the HDF5 tree, XDMF file and content index.
This method do not change the indexname of the node, if one exists.
- Parameters
nodename (str) – Name, Path or Index name of the node to modify
newname (str) – New name to give to the HDF5 node
replace (bool, optional) – If True, overwrite a possibily existing node with name newname defaults to False
- remove_node(name, recursive=False)¶
Remove a node from the dataset.
- Parameters
name (str) – Name, Path, Index name or Alias of the node to remove
recursive (bool) – if True and the node is a Group, removes all childrens of the node as well.
Important
After node removal, the file size will not be changed. This is a standard behavior for HDF5 files, that preserves freed space in disk to add additional data in the future. If needed, use the
repack_file()method to reduce file disk size after changing compression settings. This method is also called by the class instance destructor.
- repack_h5file()¶
Overwrite hdf5 file with a copy of itself to recover disk space.
Manipulation to recover space leaved empty when removing data from the HDF5 tree or reducing a node space by changing its compression settings. This method is called also by the class destructor if the autorepack flag is True.
- static copy_sample(src_sample_file, dst_sample_file, overwrite=False, get_object=False, new_sample_name=None, autodelete=False)¶
Initiate a new SampleData object and files from existing dataset.
- Parameters
src_sample_file (src) – name of the dataset file to copy.
dst_sample_file (src) – name of the new dataset files.
overwrite (bool) – set to True to overwrite an existing dataset file with name dst_sample_file when copying.
get_object (bool) – if True returns the SampleData instance
new_sample_name (str) – name of the sample in the new dataset
autodelete (bool) – remove copied dataset files when copied instance is destroyed.
- create_elset_ids_field(meshname=None, store=True, fieldname=None, get_sets_IDs=True, tags_prefix='elset', remove_elset_fields=False)¶
Create an element tag Id field on the inputed mesh.
Creates a element wise field from the provided mesh, adding to each element the value of the Elset it belongs to.
Warning
CAUTION : the methods is designed to work with non intersecting element tags/sets. In this case, the produce field will indicate the value of the last elset containing it for each element.
- Parameters
mesh (str) – Name, Path or index name of the mesh on which an orientation map element field must be constructed
store (bool) – If True, store the field on the mesh
get_sets_IDs (bool) – If True, get the sets ID numbers from their names by substracting the input prefix. If False, use the set position in the mesh elset list as ID number.
tags_prefix (str) – Remove from element sets/tags names prefix to determine the set/tag ID. This supposes that sets names have the form prefix+ID
remove_elset_fields (bool) – If True, removes the elset indicator fields after construction of the elset id field. (default is False)
crystal Package¶
A package to specifically handle crystal structures and microstructures.
Subpackages¶
lattice Module¶
The lattice module define the class to handle 3D crystal lattices (the 14 Bravais lattices).
- class pymicro.crystal.lattice.Crystal(lattice, basis=None, basis_labels=None, basis_sizes=None, basis_colors=None)¶
Bases:
objectThe Crystal class to create any particular crystal structure.
A crystal instance is composed by:
one of the 14 Bravais lattice
a point basis (or motif)
- __init__(lattice, basis=None, basis_labels=None, basis_sizes=None, basis_colors=None)¶
Create a Crystal instance with the given lattice and basis.
This create a new instance of a Crystal object. The given lattice is assigned to the crystal. If the basis is not specified, it will be one atom at (0., 0., 0.).
- Parameters
lattice – the
Latticeinstance of the crystal.basis (list) – A list of tuples containing the position of the atoms in the motif.
basis_labels (list) – A list of strings containing the description of the atoms in the motif.
basis_labels – A list of float between 0. and 1. (default 0.1) to sale the atoms in the motif.
basis_colors (list) – A list of vtk colors of the atoms in the motif.
- class pymicro.crystal.lattice.CrystallinePhase(phase_id=1, name='unknown', lattice=None)¶
Bases:
object- __init__(phase_id=1, name='unknown', lattice=None)¶
Create a new crystalline phase.
The phase_id attribute is used to identify the phase in data sets where it can be referred to in phase_map for instance.
- get_lattice()¶
Returns the crystal lattice.
- get_symmetry()¶
Returns the type of Symmetry of the Lattice.
- to_dict()¶
- static from_dict(d)¶
- class pymicro.crystal.lattice.Symmetry(value)¶
Bases:
EnumClass to describe crystal symmetry defined by its Laue class symbol.
- cubic = 'm3m'¶
- hexagonal = '6/mmm'¶
- orthorhombic = 'mmm'¶
- tetragonal = '4/mmm'¶
- trigonal = 'bar3m'¶
- monoclinic = '2/m'¶
- triclinic = 'bar1'¶
- static from_string(s)¶
- to_string()¶
- static from_space_group(space_group_number)¶
Create an instance of the Symmetry class from a TSL symmetry number.
- Raises
ValueError – if the space_group_number is not between 1 and 230.
- Parameters
space_group_number (int) – the number asociated with the
space group (between 1 and 230). :return: an instance of the Symmetry class
- static from_tsl(tsl_number)¶
Create an instance of the Symmetry class from a TSL symmetry number.
- Returns
an instance of the Symmetry class
- symmetry_operators(use_miller_bravais=False)¶
Define the equivalent crystal symmetries.
Those come from Randle & Engler, 2000. For instance in the cubic crystal struture, for instance there are 24 equivalent cube orientations.
- Returns array
A numpy array of shape (n, 3, 3) where n is the number of symmetries of the given crystal structure.
- move_vector_to_FZ(v)¶
Move the vector to the Fundamental Zone of a given Symmetry instance.
- Parameters
v – a 3 components vector.
- Returns
a new 3 components vector in the fundamental zone.
- move_rotation_to_FZ(g, verbose=False)¶
Compute the rotation matrix in the Fundamental Zone of a given Symmetry instance.
- Parameters
g – a 3x3 matrix representing the rotation.
verbose – flag for verbose mode.
- Returns
a new 3x3 matrix for the rotation in the fundamental zone.
- stiffness_matrix(elastic_constants)¶
Build the stiffness matrix for this symmetry using Voigt convention.
- Parameters
elastic_constants (list) – the elastic constants (the number must correspond to the type of symmetry, eg 3 for cubic).
- Return ndarray
a numpy array of shape (6, 6) representing the stiffness matrix.
- static orthotropic_constants_from_stiffness(C)¶
Return orthotropic elastic constants from stiffness matrix.
- Parameters
C (ndarray) – a numpy array of shape (6, 6) representing the stiffness matrix.
- Return dict OrthoElas
Dict of orthotropic elastic constants corresponding to the input stiffness matrix. Keys are ‘E1’,’E2’,’E3’,’nu12’,’nu13’,’nu23’,’G12’,’G13’,’G23’
- class pymicro.crystal.lattice.Lattice(matrix, centering='P', symmetry=None)¶
Bases:
objectThe Lattice class to create one of the 14 Bravais lattices.
This particular class has been partly inspired from the pymatgen project at https://github.com/materialsproject/pymatgen
Any of the 7 lattice systems (each corresponding to one point group) can be easily created and manipulated.
The lattice centering can be specified to form any of the 14 Bravais lattices:
Primitive (P): lattice points on the cell corners only (default);
Body (I): one additional lattice point at the center of the cell;
Face (F): one additional lattice point at the center of each of the faces of the cell;
Base (A, B or C): one additional lattice point at the center of each of one pair of the cell faces.
a = 0.352 # FCC Nickel l = Lattice.face_centered_cubic(a) print(l.volume())
Additionnally the point-basis can be controlled to address non Bravais lattice cells. It is set to a single atoms at (0, 0, 0) by default so that each cell is a Bravais lattice but may be changed to something more complex to achieve HCP structure or Diamond structure for instance.
- __init__(matrix, centering='P', symmetry=None)¶
Create a crystal lattice (unit cell).
Create a lattice from a 3x3 matrix. Each row in the matrix represents one lattice vector. The unit is nm.
- Parameters
matrix (ndarray) – the 3x3 matrix representing the crystal lattice.
centering (str) –
- reciprocal_lattice()¶
Compute the reciprocal lattice.
The reciprocal lattice defines a crystal in terms of vectors that are normal to a plane and whose lengths are the inverse of the interplanar spacing. This method computes the three reciprocal lattice vectors defined by:
\[* a.a^* = 1 * b.b^* = 1 * c.c^* = 1\]
- property matrix¶
Returns a copy of matrix representing the Lattice.
- get_symmetry()¶
Returns the type of Symmetry of the Lattice.
- static symmetry(crystal_structure=Symmetry.cubic, use_miller_bravais=False)¶
Define the equivalent crystal symmetries.
Those come from Randle & Engler, 2000. For instance in the cubic crystal struture, for instance there are 24 equivalent cube orientations.
- Parameters
crystal_structure – an instance of the Symmetry class describing the crystal symmetry.
- Raises
ValueError – if the given symmetry is not supported.
- Returns array
A numpy array of shape (n, 3, 3) where n is the number of symmetries of the given crystal structure.
- get_lattice_parameters()¶
This function create a list of the independent lattice parameters depending on the symmetry.
- Returns
a list of the lattice parameters.
- guess_symmetry()¶
Guess the lattice symmetry from the geometry.
- static guess_symmetry_from_parameters(a, b, c, alpha, beta, gamma)¶
Guess the lattice symmetry from the geometrical parameters.
- static from_cif(file_path)¶
Create a crystal Lattice using information contained in a given CIF file (Crystallographic Information Framework, a standard for information interchange in crystallography).
Reference: S. R. Hall, F. H. Allen and I. D. Brown, The crystallographic information file (CIF): a new standard archive file for crystallography, Acta Crystallographica Section A, 47(6):655-685 (1991) doi = 10.1107/S010876739101067X
Note
Lattice constants are given in Angstrom in CIF files and so converted to nanometer.
- Parameters
file_path (str) – The path to the CIF file representing the crystal structure.
- Returns
A Lattice instance corresponding to the given CIF file.
- static from_symbol(symbol)¶
Create a crystal Lattice using information contained in a unit cell.
Parameters
symbol: The chemical symbol of the crystal (eg ‘Al’)
Returns
A Lattice instance corresponding to the given element.
- static cubic(a)¶
Create a cubic Lattice unit cell.
Parameters
a: first lattice length parameter (a = b = c here)
Returns
A Lattice instance corresponding to a primitice cubic lattice.
- static body_centered_cubic(a)¶
Create a body centered cubic Lattice unit cell.
Parameters
a: first lattice length parameter (a = b = c here)
Returns
A Lattice instance corresponding to a body centered cubic lattice.
- static face_centered_cubic(a)¶
Create a face centered cubic Lattice unit cell.
Parameters
a: first lattice length parameter (a = b = c here)
Returns
A Lattice instance corresponding to a face centered cubic lattice.
- static tetragonal(a, c)¶
Create a tetragonal Lattice unit cell.
Parameters
a: first lattice length parameter
c: third lattice length parameter (b = a here)
Returns
A Lattice instance corresponding to a primitive tetragonal lattice.
- static body_centered_tetragonal(a, c)¶
Create a body centered tetragonal Lattice unit cell.
Parameters
a: first lattice length parameter
c: third lattice length parameter (b = a here)
Returns
A Lattice instance corresponding to a body centered tetragonal lattice.
- static orthorhombic(a, b, c)¶
Create a tetragonal Lattice unit cell with 3 different length parameters a, b and c.
- static base_centered_orthorhombic(a, b, c)¶
Create a based centered orthorombic Lattice unit cell.
Parameters
a: first lattice length parameter
b: second lattice length parameter
c: third lattice length parameter
Returns
A Lattice instance corresponding to a based centered orthorombic lattice.
- static body_centered_orthorhombic(a, b, c)¶
Create a body centered orthorombic Lattice unit cell.
Parameters
a: first lattice length parameter
b: second lattice length parameter
c: third lattice length parameter
Returns
A Lattice instance corresponding to a body centered orthorombic lattice.
- static face_centered_orthorhombic(a, b, c)¶
Create a face centered orthorombic Lattice unit cell.
Parameters
a: first lattice length parameter
b: second lattice length parameter
c: third lattice length parameter
Returns
A Lattice instance corresponding to a face centered orthorombic lattice.
- static hexagonal(a, c)¶
Create a hexagonal Lattice unit cell with length parameters a and c.
- static rhombohedral(a, alpha)¶
Create a rhombohedral Lattice unit cell with one length parameter a and the angle alpha.
- static monoclinic(a, b, c, alpha)¶
Create a monoclinic Lattice unit cell with 3 different length parameters a, b and c. The cell angle is given by alpha. The lattice centering id primitive ie. ‘P’
- static base_centered_monoclinic(a, b, c, alpha)¶
Create a based centered monoclinic Lattice unit cell.
Parameters
a: first lattice length parameter
b: second lattice length parameter
c: third lattice length parameter
alpha: first lattice angle parameter
Returns
A Lattice instance corresponding to a based centered monoclinic lattice.
- static triclinic(a, b, c, alpha, beta, gamma)¶
Create a triclinic Lattice unit cell with 3 different length parameters a, b, c and three different cell angles alpha, beta and gamma.
..note:
This method is here for the sake of completeness since one can create the triclinic cell directly using the `from_parameters` method.
- static from_symmetry(symmetry, parameters)¶
Create a new lattice based on a type of symmetry and a list of lattice parameters.
The type of symmetry should be an instance of Symmetry and the list of parameters should contain the appropriate number: 1 for cubic, 2 for hexagonal, tetragonal or trigonal, 3 for orthorhombic, 4 for monoclinic or 6 for triclinic.
- Parameters
symmetry – an instance of Symmetry.
parameters (list) – a list of the lattice parameters.
- Returns
the newly created Lattice instance.
- static from_parameters(a, b, c, alpha, beta, gamma, x_aligned_with_a=True, centering='P', symmetry=Symmetry.triclinic)¶
Create a Lattice using unit cell lengths and angles (in degrees). The lattice centering can also be specified (among ‘P’, ‘I’, ‘F’, ‘A’, ‘B’ or ‘C’).
- Parameters
a (float) – first lattice length parameter.
b (float) – second lattice length parameter.
c (float) – third lattice length parameter.
alpha (float) – first lattice angle parameter.
beta (float) – second lattice angle parameter.
gamma (float) – third lattice angle parameter.
x_aligned_with_a (bool) – flag to control the convention used to define the Cartesian frame.
centering (str) – lattice centering (‘P’ by default) passed to the Lattice class.
symmetry – a Symmetry instance to be passed to the lattice.
- Returns
A Lattice instance with the specified lattice parameters and centering.
- volume()¶
Compute the volume of the unit cell.
- get_hkl_family(hkl)¶
Get a list of the hkl planes composing the given family for this crystal lattice.
Parameters
hkl: miller indices of the requested family
Returns
A list of the hkl planes in the given family.
- get_slip_systems(slip_type='oct')¶
Create a list of the slip systems of a given type for this lattice.
- Parameters
slip_type (str) – a string describing the slip system type, should be in (oct, 111, cube, 001, 112, basal, prism)
- class pymicro.crystal.lattice.SlipSystem(plane, direction)¶
Bases:
objectA class to represent a crystallographic slip system.
A slip system is composed of a slip plane (most widely spaced planes in the crystal) and a slip direction (highest linear density of atoms in the crystal).
- __init__(plane, direction)¶
Create a new slip system object with the given slip plane and slip direction.
- get_slip_plane()¶
- get_slip_direction()¶
- static from_indices(plane_indices, direction_indices, lattice=None)¶
create a slip system from the indices of the plane and the direction.
This method create a SlipSystem instance by associating a slip plane and a slip direction both given by their Miller indices. In the case of a hexagonal crystal lattice, the Miller-Bravais (4 indices) notation can be used. If this notation is used without specifying any lattice, a default hexagonal lattice is created.
- Parameters
plane_indices (tuple) – the miller indices for the slip plane.
direction_indices (tuple) – the miller indices for the slip direction.
lattice (Lattice) – the crystal lattice.
- Returns
the new SlipSystem instance.
- Raise
ValueError if the 4 indices notation is used with a non hexagonal crystal lattice.
- static get_slip_systems(slip_type='oct', lattice=None)¶
Get all slip systems for a given hkl plane family.
- A string is used to describe the slip system type:
cube or 001, for [110] slip in (001) planes
oct or 111 for [110] slip in (111) planes
112 for [111] slip in (112) planes
basal for [11-20] slip in (0001) planes (hexagonal)
prism for [11-20] slip in (1-100) planes (hexagonal)
- Parameters
slip_type (str) – a string describing the slip system type.
- Return list
a list of
SlipSystem.
- class pymicro.crystal.lattice.HklObject(h, k, l, lattice=None)¶
Bases:
objectAn abstract class to represent an object related to a crystal lattice and which can be described by Miller indices.
- __init__(h, k, l, lattice=None)¶
Create a new hkl object with the given Miller indices and crystal lattice.
- property lattice¶
- set_lattice(lattice)¶
Assign a new Lattice to this instance.
- Parameters
lattice – the new crystal lattice.
- property h¶
- property k¶
- property l¶
- miller_indices()¶
Returns an immutable tuple of the plane Miller indices.
- static skip_higher_order(hkl_list, keep_friedel_pair=False, verbose=False)¶
Create a copy of a list of some hkl object retaining only the first order.
- Parameters
hkl_list (list) – The list of HklObject.
keep_friedel_pair (bool) – flag to keep order -1 in the list.
verbose (bool) – activate verbose mode.
- Returns list
A new list of
HklObjectwithout any multiple reflection.
- class pymicro.crystal.lattice.HklDirection(h, k, l, lattice=None)¶
Bases:
HklObject- direction()¶
Returns a normalized vector, expressed in the cartesian coordinate system, corresponding to this crystallographic direction.
- angle_with_direction(hkl)¶
Computes the angle between this crystallographic direction and the given direction (in radian).
- static angle_between_directions(hkl1, hkl2, lattice=None)¶
Computes the angle between two crystallographic directions (in radian).
- Parameters
hkl1 (tuple) – The triplet of the miller indices of the first direction.
hkl2 (tuple) – The triplet of the miller indices of the second direction.
lattice (Lattice) – The crystal lattice, will default to cubic if not specified.
- Returns float
The angle in radian.
- static three_to_four_indices(u, v, w)¶
Convert from Miller indices to Miller-Bravais indices. this is used for hexagonal crystal lattice.
- static four_to_three_indices(U, V, T, W)¶
Convert from Miller-Bravais indices to Miller indices. this is used for hexagonal crystal lattice.
- static angle_between_4indices_directions(hkil1, hkil2, ac)¶
Computes the angle between two crystallographic directions in a hexagonal lattice.
The solution was derived by F. Frank in: On Miller - Bravais indices and four dimensional vectors. Acta Cryst. 18, 862-866 (1965)
- Parameters
hkil1 (tuple) – The quartet of the indices of the first direction.
hkil2 (tuple) – The quartet of the indices of the second direction.
ac (tuple) – the lattice parameters of the hexagonal structure in the form (a, c).
- Returns float
The angle in radian.
- find_planes_in_zone(max_miller=5)¶
This method finds the hkl planes in zone with the crystallographic direction. If (u,v,w) denotes the zone axis, this means finding all hkl planes which verify \(h.u + k.v + l.w = 0\).
- Parameters
max_miller – The maximum miller index to limt the search`
- Returns list
A list of
HklPlaneobjects describing all the planes in zone with the direction.
- class pymicro.crystal.lattice.HklPlane(h, k, l, lattice=None)¶
Bases:
HklObjectThis class define crystallographic planes using Miller indices.
A plane can be create by speficying its Miller indices and the crystal lattice (default is cubic with lattice parameter of 1.0).
a = 0.405 # FCC Aluminium l = Lattice.cubic(a) p = HklPlane(1, 1, 1, lattice=l) print(p) print(p.scattering_vector()) print(p.interplanar_spacing())
Note
Miller indices are defined in terms of the inverse of the intercept of the plane on the three crystal axes a, b, and c.
- normal()¶
Returns the unit vector normal to the plane.
We use of the repiprocal lattice to compute the normal to the plane and return a normalised vector.
- scattering_vector()¶
Calculate the scattering vector of this HklPlane.
The scattering vector (or reciprocal lattice vector) is normal to this HklPlane and its length is equal to the inverse of the interplanar spacing. In the cartesian coordinate system of the crystal, it is given by:
..math
G_c = h.a^* + k.b^* + l.c^*
- Returns
a numpy vector expressed in the cartesian coordinate system of the crystal.
- friedel_pair()¶
Create the Friedel pair of the HklPlane.
- interplanar_spacing()¶
Compute the interplanar spacing. For cubic lattice, it is:
\[d = a / \sqrt{h^2 + k^2 + l^2}\]The general formula comes from ‘Introduction to Crystallography’ p. 68 by Donald E. Sands.
- bragg_angle(lambda_keV, verbose=False)¶
Compute the Bragg angle for this HklPlane at the given energy.
Note
For this calculation to work properly, the lattice spacing needs to be in nm units.
- static four_to_three_indices(U, V, T, W)¶
Convert four to three index representation of a slip plane (used for hexagonal crystal lattice).
- static three_to_four_indices(u, v, w)¶
Convert three to four index representation of a slip plane (used for hexagonal crystal lattice).
- is_in_list(hkl_planes, friedel_pair=False)¶
Check if the hkl plane is in the given list.
By default this relies on the built in in test from the list type which in turn calls in the __eq__ method. This means it will return True if a plane with the exact same miller indices (and same lattice) is in the list. Turning on the friedel_pair flag will allow to test also the Friedel pair (-h, -k, -l) and return True if it is in the list. For instance (0,0,1) and (0,0,-1) are in general considered as the same lattice plane.
- static is_same_family(hkl1, hkl2, crystal_structure=Symmetry.cubic)¶
Static mtd to test if both lattice planes belongs to same family.
A family {hkl} is composed by all planes that are equivalent to (hkl) using the symmetry of the lattice. The lattice assoiated with hkl2 is not taken into account here.
- static get_family(hkl, lattice=None, include_friedel_pairs=False, crystal_structure=Symmetry.cubic)¶
Static method to obtain a list of the different crystallographic planes in a particular family.
- Parameters
hkl (str) – a sequence of 3 (4 for hexagonal) numbers corresponding to the miller indices.
lattice (Lattice) – The reference crystal lattice (default None).
include_friedel_pairs (bool) – Flag to include the Friedel pairs in the list (False by default).
crystal_structure (str) – A string descibing the crystal structure (cubic by default).
- Raises
ValueError – if the given string does not correspond to a supported family.
- Returns list
a list of the
HklPlanein the given hkl family.
Note
The method account for the lattice symmetry to create a list of equivalent lattice plane from the point of view of the point group symmetry. A flag can be used to include or not the Friedel pairs. If not, the family is contstructed using the miller indices limited the number of minus signs. For instance (1,0,0) will be in the list and not (-1,0,0).
- multiplicity(symmetry=Symmetry.cubic)¶
compute the general multiplicity for this HklPlane and the given Symmetry.
- Parameters
symmetry (Symmetry) – The crystal symmetry to take into account.
- Returns
the number of equivalent planes in the family.
- slip_trace(orientation, n_int=<Mock name='mock.array()' id='139894746709968'>, view_up=<Mock name='mock.array()' id='139894746709968'>, trace_size=100, verbose=False)¶
Compute the intersection of the lattice plane with a particular plane defined by its normal.
- Parameters
orientation – The crystal orientation.
n_int – normal to the plane of intersection (laboratory local frame).
view_up – vector to place upwards on the plot.
trace_size (int) – size of the trace.
verbose – activate verbose mode.
- Returns
a numpy array with the coordinates of the two points defining the trace.
- static plot_slip_traces(orientation, hkl='111', n_int=<Mock name='mock.array()' id='139894746709968'>, view_up=<Mock name='mock.array()' id='139894746709968'>, verbose=False, title=True, legend=True, trans=False, str_plane=None)¶
A method to plot the slip planes intersection with a particular plane (known as slip traces if the plane correspond to the surface). A few parameters can be used to control the plot looking. Thank to Jia Li for starting this code.
- Parameters
orientation – The crystal orientation.
hkl – a string representing the slip plane family (eg. 111 or 110)
or the list of HklPlane instances. :param n_int: normal to the plane of intersection. :param view_up: vector to place upwards on the plot. :param verbose: activate verbose mode. :param title: display a title above the plot. :param legend: display the legend. :param trans: use a transparent background for the figure (useful to overlay the figure on top of another image). :param str_plane: particular string to use to represent the plane in the image name.
- static plot_XY_slip_traces(orientation, hkl='111', title=True, legend=True, trans=False, verbose=False)¶
Helper method to plot the slip traces on the XY plane.
- static plot_YZ_slip_traces(orientation, hkl='111', title=True, legend=True, trans=False, verbose=False)¶
Helper method to plot the slip traces on the YZ plane.
- static plot_XZ_slip_traces(orientation, hkl='111', title=True, legend=True, trans=False, verbose=False)¶
Helper method to plot the slip traces on the XZ plane.
- static indices_from_two_directions(uvw1, uvw2)¶
Two crystallographic directions \(uvw_1\) and \(uvw_2\) define a unique set of hkl planes. This does not depends on the crystal symmetry.
\[\begin{split}h = v_1 . w_2 - w_1 . v_2 \\ k = w_1 . u_2 - u_1 . w_2 \\ l = u_1 . v_2 - v_1 . u_2\end{split}\]- Parameters
uvw1 – The first instance of the HklDirection class.
uvw2 – The second instance of the HklDirection class.
- Return h, k, l
the miller indices of the HklPlane defined by the two directions.
microstructure Module¶
The microstructure module provide elementary classes to describe a crystallographic granular microstructure such as mostly present in metallic materials.
It contains several classes which are used to describe a microstructure composed of several grains, each one having its own crystallographic orientation:
- class pymicro.crystal.microstructure.Orientation(matrix)¶
Bases:
objectCrystallographic orientation class.
This follows the passive rotation definition which means that it brings the sample coordinate system into coincidence with the crystal coordinate system. Then one may express a vector \(V_c\) in the crystal coordinate system from the vector in the sample coordinate system \(V_s\) by:
\[V_c = g.V_s\]and inversely (because \(g^{-1}=g^T\)):
\[V_s = g^T.V_c\]Most of the code to handle rotations has been written to comply with the conventions laid in [2].
- __init__(matrix)¶
Initialization from the 9 components of the orientation matrix.
- orientation_matrix()¶
Returns the orientation matrix in the form of a 3x3 numpy array.
- to_crystal(v)¶
Transform a vector or a matrix from the sample frame to the crystal frame.
- Parameters
v (ndarray) – a 3 component vector or a 3x3 array expressed in the sample frame.
- Returns
the vector or matrix expressed in the crystal frame.
- to_sample(v)¶
Transform a vector or a matrix from the crystal frame to the sample frame.
- Parameters
v (ndarray) – a 3 component vector or a 3x3 array expressed in the crystal frame.
- Returns
the vector or matrix expressed in the sample frame.
- static cube()¶
Create the particular crystal orientation called Cube and which corresponds to euler angle (0, 0, 0).
- static brass()¶
Create the particular crystal orientation called Brass and which corresponds to euler angle (35.264, 45, 0).
- static copper()¶
Create the particular crystal orientation called Copper and which corresponds to euler angle (90, 35.264, 45).
- static s3()¶
Create the particular crystal orientation called S3 and which corresponds to euler angle (59, 37, 63).
- static goss()¶
Create the particular crystal orientation called Goss and which corresponds to euler angle (0, 45, 0).
- static shear()¶
Create the particular crystal orientation called shear and which corresponds to euler angle (45, 0, 0).
- static random()¶
Create a random crystal orientation.
- ipf_color(axis=<Mock name='mock.array()' id='139894746709968'>, symmetry=Symmetry.cubic, saturate=True)¶
Compute the IPF (inverse pole figure) colour for this orientation.
This method has bee adapted from the DCT code.
Note
This method coexist with the get_ipf_colour for the moment.
- Parameters
axis (ndarray) – the direction to use to compute the IPF colour.
symmetry (Symmetry) – the symmetry operator to use.
- Return bool saturate
a flag to saturate the RGB values.
- get_ipf_colour(axis=<Mock name='mock.array()' id='139894746709968'>, symmetry=Symmetry.cubic)¶
Compute the IPF (inverse pole figure) colour for this orientation.
Given a particular axis expressed in the laboratory coordinate system, one can compute the so called IPF colour based on that direction expressed in the crystal coordinate system as \([x_c,y_c,z_c]\). There is only one tuple (u,v,w) such that:
\[[x_c,y_c,z_c]=u.[0,0,1]+v.[0,1,1]+w.[1,1,1]\]and it is used to assign the RGB colour.
- Parameters
axis (ndarray) – the direction to use to compute the IPF colour.
symmetry (Symmetry) – the symmetry operator to use.
- Return tuple
a tuple contining the RGB values.
- static compute_mean_orientation(rods, symmetry=Symmetry.cubic)¶
Compute the mean orientation.
This function computes a mean orientation from several data points representing orientations. Each orientation is first moved to the fundamental zone, then the corresponding Rodrigues vectors can be averaged to compute the mean orientation.
- Parameters
rods (ndarray) – a (n, 3) shaped array containing the Rodrigues
vectors of the orientations. :param Symmetry symmetry: the symmetry used to move orientations to their fundamental zone (cubic by default) :returns: the mean orientation as an Orientation instance.
- static fzDihedral(rod, n)¶
check if the given Rodrigues vector is in the fundamental zone.
After book from Morawiec :cite`Morawiec_2004`:
- inFZ(symmetry=Symmetry.cubic)¶
Check if the given Orientation lies within the fundamental zone.
For a given crystal symmetry, several rotations can describe the same physcial crystllographic arangement. The Rodrigues fundamental zone (also called the asymmetric domain) restricts the orientation space accordingly.
- Parameters
symmetry – the Symmetry to use.
- Return bool
True if this orientation is in the fundamental zone,
False otherwise.
- move_to_FZ(symmetry=Symmetry.cubic, verbose=False)¶
Compute the equivalent crystal orientation in the Fundamental Zone of a given symmetry.
- Parameters
symmetry (Symmetry) – an instance of the Symmetry class.
verbose – flag for verbose mode.
- Returns
a new Orientation instance which lies in the fundamental zone.
- static misorientation_MacKenzie(psi)¶
Return the fraction of the misorientations corresponding to the given \(\psi\) angle in the reference solution derived By MacKenzie in his 1958 paper [1].
- Parameters
psi – the misorientation angle in radians.
- Returns
the value in the cummulative distribution corresponding to psi.
- static misorientation_axis_from_delta(delta)¶
Compute the misorientation axis from the misorientation matrix.
- Parameters
delta – The 3x3 misorientation matrix.
- Returns
the misorientation axis (normalised vector).
- misorientation_axis(orientation)¶
Compute the misorientation axis with another crystal orientation. This vector is by definition common to both crystalline orientations.
- Parameters
orientation – an instance of
Orientationclass.- Returns
the misorientation axis (normalised vector).
- static misorientation_angle_from_delta(delta)¶
Compute the misorientation angle from the misorientation matrix.
Compute the angle associated with this misorientation matrix \(\Delta g\). It is defined as \(\omega = \arccos(\text{trace}(\Delta g)/2-1)\). To avoid float rounding error, the argument is rounded to 1.0 if it is within 1 and 1 plus 32 bits floating point precison.
Note
This does not account for the crystal symmetries. If you want to find the disorientation between two orientations, use the
disorientation()method.- Parameters
delta – The 3x3 misorientation matrix.
- Returns float
the misorientation angle in radians.
- disorientation(orientation, crystal_structure=Symmetry.triclinic)¶
Compute the disorientation another crystal orientation.
Considering all the possible crystal symmetries, the disorientation is defined as the combination of the minimum misorientation angle and the misorientation axis lying in the fundamental zone, which can be used to bring the two lattices into coincidence.
Note
Both orientations are supposed to have the same symmetry. This is not necessarily the case in multi-phase materials.
- Parameters
orientation – an instance of
Orientationclass describing the other crystal orientation from which to compute the angle.crystal_structure – an instance of the Symmetry class describing the crystal symmetry, triclinic (no symmetry) by default.
- Returns tuple
the misorientation angle in radians, the axis as a numpy vector (crystal coordinates), the axis as a numpy vector (sample coordinates).
- phi1()¶
Convenience methode to expose the first Euler angle.
- Phi()¶
Convenience methode to expose the second Euler angle.
- phi2()¶
Convenience methode to expose the third Euler angle.
- compute_XG_angle(hkl, omega, verbose=False)¶
Compute the angle between the scattering vector \(\mathbf{G_{l}}\) and \(\mathbf{-X}\) the X-ray unit vector at a given angular position \(\omega\).
A given hkl plane defines the scattering vector \(\mathbf{G_{hkl}}\) by the miller indices in the reciprocal space. It is expressed in the cartesian coordinate system by \(\mathbf{B}.\mathbf{G_{hkl}}\) and in the laboratory coordinate system accounting for the crystal orientation by \(\mathbf{g}^{-1}.\mathbf{B}.\mathbf{G_{hkl}}\).
The crystal is assumed to be placed on a rotation stage around the laboratory vertical axis. The scattering vector can finally be written as \(\mathbf{G_l}=\mathbf{\Omega}.\mathbf{g}^{-1}.\mathbf{B}.\mathbf{G_{hkl}}\). The X-rays unit vector is \(\mathbf{X}=[1, 0, 0]\). So the computed angle is \(\alpha=acos(-\mathbf{X}.\mathbf{G_l}/||\mathbf{G_l}||\)
The Bragg condition is fulfilled when \(\alpha=\pi/2-\theta_{Bragg}\)
- Parameters
hkl – the hkl plane, an instance of
HklPlaneomega – the angle of rotation of the crystal around the laboratory vertical axis.
verbose (bool) – activate verbose mode (False by default).
- Return float
the angle between \(-\mathbf{X}\) and \(\mathbf{G_{l}}\) in degrees.
- static solve_trig_equation(A, B, C, verbose=False)¶
Solve the trigonometric equation in the form of:
\[A\cos\theta + B\sin\theta = C\]- Parameters
A (float) – the A constant in the equation.
B (float) – the B constant in the equation.
C (float) – the C constant in the equation.
- Return tuple
the two solutions angular values in degrees.
- dct_omega_angles(hkl, lambda_keV, verbose=False)¶
Compute the two omega angles which satisfy the Bragg condition.
For a given crystal orientation sitting on a vertical rotation axis, there is exactly two \(\omega\) positions in \([0, 2\pi]\) for which a particular \((hkl)\) reflexion will fulfil Bragg’s law.
According to the Bragg’s law, a crystallographic plane of a given grain will be in diffracting condition if:
\[\sin\theta=-[\mathbf{\Omega}.\mathbf{g}^{-1}\mathbf{G_c}]_1\]with \(\mathbf{\Omega}\) the matrix associated with the rotation axis:
\[\begin{split}\mathbf{\Omega}=\begin{pmatrix} \cos\omega & -\sin\omega & 0 \\ \sin\omega & \cos\omega & 0 \\ 0 & 0 & 1 \\ \end{pmatrix}\end{split}\]This method solves the associated second order equation to return the two corresponding omega angles.
- Parameters
hkl – The given cristallographic plane
HklPlanelambda_keV (float) – The X-rays energy expressed in keV
verbose (bool) – Verbose mode (False by default)
- Returns tuple
\((\omega_1, \omega_2)\) the two values of the rotation angle around the vertical axis (in degrees).
- rotating_crystal(hkl, lambda_keV, omega_step=0.5, display=True, verbose=False)¶
- static compute_instrument_transformation_matrix(rx_offset, ry_offset, rz_offset)¶
Compute instrument transformation matrix for given rotation offset.
This function compute a 3x3 rotation matrix (passive convention) that transforms the sample coordinate system by rotating around the 3 cartesian axes in this order: rotation around X is applied first, then around Y and finally around Z.
A sample vector \(V_s\) is consequently transformed into \(V'_s\) as:
\[V'_s = T^T.V_s\]- Parameters
rx_offset (double) – value to apply for the rotation around X.
ry_offset (double) – value to apply for the rotation around Y.
rz_offset (double) – value to apply for the rotation around Z.
- Returns
a 3x3 rotation matrix describing the transformation applied by the diffractometer.
- topotomo_tilts(hkl, T=None, verbose=False)¶
Compute the tilts for topotomography alignment.
- Parameters
hkl – the hkl plane, an instance of
HklPlaneT (ndarray) – transformation matrix representing the diffractometer direction at omega=0.
verbose (bool) – activate verbose mode (False by default).
- Returns tuple
(ut, lt) the two values of tilts to apply (in radians).
- static from_euler(euler, convention='Bunge')¶
Rotation matrix from Euler angles.
This is the classical method to obtain an orientation matrix by 3 successive rotations. The result depends on the convention used (how the successive rotation axes are chosen). In the Bunge convention, the first rotation is around Z, the second around the new X and the third one around the new Z. In the Roe convention, the second one is around Y.
- static from_rodrigues(rod)¶
- static from_Quaternion(q)¶
- static Zrot2OrientationMatrix(x1=None, x2=None, x3=None)¶
Compute the orientation matrix from the rotated coordinates given in the .inp file for Zebulon’s computations.
The function needs two of the three base vectors, the third one is computed using a cross product.
Note
Still need some tests to validate this function.
- Parameters
x1 – the first basis vector.
x2 – the second basis vector.
x3 – the third basis vector.
- Returns
the corresponding 3x3 orientation matrix.
- static OrientationMatrix2EulerSF(g)¶
Compute the Euler angles (in degrees) from the orientation matrix in a similar way as done in Mandel_crystal.c
- static OrientationMatrix2Euler(g)¶
Compute the Euler angles from the orientation matrix.
This conversion follows the paper of Rowenhorst et al. [2]. In particular when \(g_{33} = 1\) within the machine precision, there is no way to determine the values of \(\phi_1\) and \(\phi_2\) (only their sum is defined). The convention is to attribute the entire angle to \(\phi_1\) and set \(\phi_2\) to zero.
- Parameters
g – The 3x3 orientation matrix
- Returns
The 3 euler angles in degrees.
- static OrientationMatrix2Rodrigues(g)¶
Compute the rodrigues vector from the orientation matrix.
- Parameters
g – The 3x3 orientation matrix representing the rotation.
- Returns
The Rodrigues vector as a 3 components array.
- static OrientationMatrix2Quaternion(g, P=1)¶
- static Rodrigues2OrientationMatrix(rod)¶
Compute the orientation matrix from the Rodrigues vector.
- Parameters
rod – The Rodrigues vector as a 3 components array.
- Returns
The 3x3 orientation matrix representing the rotation.
- static Rodrigues2Axis(rod)¶
Compute the axis/angle representation from the Rodrigues vector.
- Parameters
rod – The Rodrigues vector as a 3 components array.
- Returns
A tuple in the (axis, angle) form.
- static Axis2OrientationMatrix(axis, angle)¶
Compute the (passive) orientation matrix associated the rotation defined by the given (axis, angle) pair.
- Parameters
axis – the rotation axis.
angle – the rotation angle (degrees).
- Returns
the 3x3 orientation matrix.
- static Euler2Axis(euler)¶
Compute the (axis, angle) representation associated to this (passive) rotation expressed by the Euler angles.
- Parameters
euler – 3 euler angles (in degrees).
- Returns
a tuple containing the axis (a vector) and the angle (in radians).
- static Euler2Quaternion(euler, P=1)¶
Compute the quaternion from the 3 euler angles (in degrees).
- Parameters
euler (tuple) – the 3 euler angles in degrees.
P (int) – +1 to compute an active quaternion (default), -1 for a passive quaternion.
- Returns
a Quaternion instance representing the rotation.
- static Euler2Rodrigues(euler)¶
Compute the rodrigues vector from the 3 euler angles (in degrees).
- Parameters
euler – the 3 Euler angles (in degrees).
- Returns
the rodrigues vector as a 3 components numpy array.
- static eu2ro(euler)¶
Transform a series of euler angles into rodrigues vectors.
- Parameters
euler (ndarray) – the (n, 3) shaped array of Euler angles (radians).
- Returns
a (n, 3) array with the rodrigues vectors.
- static Euler2OrientationMatrix(euler)¶
Compute the orientation matrix \(\mathbf{g}\) associated with the 3 Euler angles \((\phi_1, \Phi, \phi_2)\).
The matrix is calculated via (see the euler_angles recipe in the cookbook for a detailed example):
\[\begin{split}\mathbf{g}=\begin{pmatrix} \cos\phi_1\cos\phi_2 - \sin\phi_1\sin\phi_2\cos\Phi & \sin\phi_1\cos\phi_2 + \cos\phi_1\sin\phi_2\cos\Phi & \sin\phi_2\sin\Phi \\ -\cos\phi_1\sin\phi_2 - \sin\phi_1\cos\phi_2\cos\Phi & -\sin\phi_1\sin\phi_2 + \cos\phi_1\cos\phi_2\cos\Phi & \cos\phi_2\sin\Phi \\ \sin\phi_1\sin\Phi & -\cos\phi_1\sin\Phi & \cos\Phi \\ \end{pmatrix}\end{split}\]- Parameters
euler – The triplet of the Euler angles (in degrees).
- Return g
The 3x3 orientation matrix.
- static Quaternion2Euler(q)¶
Compute Euler angles from a Quaternion :param q: Quaternion :return: Euler angles (in degrees, Bunge convention)
- static Quaternion2OrientationMatrix(q)¶
- static read_euler_txt(txt_path)¶
Read a set of euler angles from an ascii file.
This method is deprecated, please use read_orientations.
- Parameters
txt_path (str) – path to the text file containing the euler angles.
- Returns dict
a dictionary with the line number and the corresponding orientation.
- static read_orientations(txt_path, data_type='euler', **kwargs)¶
Read a set of grain orientations from a text file.
The text file must be organised in 3 columns (the other are ignored), corresponding to either the three euler angles or the three rodrigues vector components, depending on the data_type). Internally the ascii file is read by the genfromtxt function of numpy, to which additional keyworks (such as the delimiter) can be passed to via the kwargs dictionnary.
- Parameters
txt_path (str) – path to the text file containing the orientations.
data_type (str) – ‘euler’ (default) or ‘rodrigues’.
kwargs (dict) – additional parameters passed to genfromtxt.
- Returns dict
a dictionary with the line number and the corresponding orientation.
- static read_euler_from_zset_inp(inp_path)¶
Read a set of grain orientations from a z-set input file.
In z-set input files, the orientation data may be specified either using the rotation of two vector, euler angles or rodrigues components directly. For instance the following lines are extracted from a polycrystalline calculation file using the rotation keyword:
**elset elset1 *file au.mat *integration theta_method_a 1.0 1.e-9 150 *rotation x1 0.438886 -1.028805 0.197933 x3 1.038339 0.893172 1.003888 **elset elset2 *file au.mat *integration theta_method_a 1.0 1.e-9 150 *rotation x1 0.178825 -0.716937 1.043300 x3 0.954345 0.879145 1.153101 **elset elset3 *file au.mat *integration theta_method_a 1.0 1.e-9 150 *rotation x1 -0.540479 -0.827319 1.534062 x3 1.261700 1.284318 1.004174 **elset elset4 *file au.mat *integration theta_method_a 1.0 1.e-9 150 *rotation x1 -0.941278 0.700996 0.034552 x3 1.000816 1.006824 0.885212 **elset elset5 *file au.mat *integration theta_method_a 1.0 1.e-9 150 *rotation x1 -2.383786 0.479058 -0.488336 x3 0.899545 0.806075 0.984268
- Parameters
inp_path (str) – the path to the ascii file to read.
- Returns dict
a dictionary of the orientations associated with the elset names.
- slip_system_orientation_tensor(s)¶
Compute the orientation strain tensor m^s for this
Orientationand the given slip system.- Parameters
s – an instance of
SlipSystem
\[M^s_{ij} = \left(l^s_i.n^s_j)\]
- slip_system_orientation_strain_tensor(s)¶
Compute the orientation strain tensor m^s for this
Orientationand the given slip system.- Parameters
s – an instance of
SlipSystem
\[m^s_{ij} = \frac{1}{2}\left(l^s_i.n^s_j + l^s_j.n^s_i)\]
- slip_system_orientation_rotation_tensor(s)¶
Compute the orientation rotation tensor q^s for this
Orientationand the given slip system.- Parameters
s – an instance of
SlipSystem
\[q^s_{ij} = \frac{1}{2}\left(l^s_i.n^s_j - l^s_j.n^s_i)\]
- schmid_factor(slip_system, load_direction=[0.0, 0.0, 1])¶
Compute the Schmid factor for this crystal orientation and the given slip system.
- Parameters
slip_system – a SlipSystem instance.
load_direction – a unit vector describing the loading direction (default: vertical axis [0, 0, 1]).
- Return float
a number between 0 ad 0.5.
- compute_all_schmid_factors(slip_systems, load_direction=[0.0, 0.0, 1], verbose=False)¶
Compute all Schmid factors for this crystal orientation and the given list of slip systems.
- Parameters
slip_systems – a list of the slip systems from which to compute the Schmid factor values.
load_direction – a unit vector describing the loading direction (default: vertical axis [0, 0, 1]).
verbose (bool) – activate verbose mode.
- Return list
a list of the schmid factors.
- static compute_m_factor(o1, ss1, o2, ss2)¶
Compute the m factor with another slip system.
- Parameters
o1 (Orientation) – the orientation the first grain.
ss1 (SlipSystem) – the slip system in the first grain.
o2 (Orientation) – the orientation the second grain.
ss2 (SlipSystem) – the slip system in the second grain.
- Returns
the m factor as a float number < 1
- class pymicro.crystal.microstructure.Grain(grain_id, grain_orientation)¶
Bases:
objectClass defining a crystallographic grain.
A grain has a constant crystallographic Orientation and a grain id. The center attribute is the center of mass of the grain in world coordinates. The volume of the grain is expressed in pixel/voxel unit.
- __init__(grain_id, grain_orientation)¶
- get_volume()¶
- get_volume_fraction(total_volume=None)¶
Compute the grain volume fraction.
- Parameters
total_volume (float) – the total volume value to use.
- Return float
the grain volume fraction as a number in the range [0, 1].
- schmid_factor(slip_system, load_direction=[0.0, 0.0, 1])¶
Compute the Schmid factor of this grain for the given slip system and loading direction.
- Parameters
slip_system – a SlipSystem instance.
load_direction – a unit vector describing the loading direction (default: vertical axis [0, 0, 1]).
- Return float
a number between 0 ad 0.5.
- SetVtkMesh(mesh)¶
Set the VTK mesh of this grain.
- Parameters
mesh – the grain mesh in VTK format.
- add_vtk_mesh(array, contour=True, verbose=False)¶
Add a mesh to this grain.
This method process a labeled array to extract the geometry of the grain. The grain shape is defined by the pixels with a value of the grain id. A vtkUniformGrid object is created and thresholded or contoured depending on the value of the flag contour. The resulting mesh is returned, centered on the center of mass of the grain.
- Parameters
array (ndarray) – a numpy array from which to extract the grain shape.
contour (bool) – a flag to use contour mode for the shape.
verbose (bool) – activate verbose mode.
- vtk_file_name()¶
- save_vtk_repr(file_name=None)¶
- load_vtk_repr(file_name, verbose=False)¶
- orientation_matrix()¶
A method to access the grain orientation matrix.
- Returns
the grain 3x3 orientation matrix.
- dct_omega_angles(hkl, lambda_keV, verbose=False)¶
Compute the two omega angles which satisfy the Bragg condition.
For a grain with a given crystal orientation sitting on a vertical rotation axis, there is exactly two omega positions in [0, 2pi] for which a particular hkl reflexion will fulfil Bragg’s law. See
dct_omega_angles()of theOrientationclass.- Parameters
hkl – The given cristallographic
HklPlanelambda_keV (float) – The X-rays energy expressed in keV
verbose (bool) – Verbose mode (False by default)
- Return tuple
(w1, w2) the two values of the omega angle.
- static from_dct(label=1, data_dir='.')¶
Create a Grain instance from a DCT grain file.
- Parameters
label (int) – the grain id.
data_dir (str) – the data root from where to fetch data files.
- Returns
a new grain instance.
- class pymicro.crystal.microstructure.Microstructure(filename=None, name='micro', description='empty', verbose=False, overwrite_hdf5=False, phase=None, autodelete=False)¶
Bases:
SampleDataClass used to manipulate a full microstructure derived from the SampleData class.
As SampleData, this class is a data container for a mechanical sample and its microstructure, synchronized with a HDF5 file and a XML file Microstructure implements a hdf5 data model specific to polycrystalline sample data.
The dataset maintains a GrainData instance which inherits from tables.IsDescription and acts as a structured array containing the grain attributes such as id, orientations (in form of rodrigues vectors), volume and bounding box.
A crystal Lattice is also associated to the microstructure and used in all crystallography calculations.
- __init__(filename=None, name='micro', description='empty', verbose=False, overwrite_hdf5=False, phase=None, autodelete=False)¶
Sample Data constructor, see class documentation.
- minimal_data_model()¶
Data model for a polycrystalline microstructure.
Specify the minimal contents of the hdf5 (Group names, paths and group types) in the form of a dictionary {content: location}. This extends ~pymicro.core.SampleData.minimal_data_model method.
- Returns
a tuple containing the two dictionnaries.
- sync_phases()¶
This method sync the _phases attribute with the content of the hdf5 file.
- set_phase(phase)¶
Set a phase for the given phase_id.
If the phase id does not correspond to one of the existing phase, nothing is done.
- Parameters
phase (CrystallinePhase) – the phase to use.
phase_id (int) –
- set_phases(phase_list)¶
Set a list of phases for this microstructure.
The different phases in the list are added in that order.
- Parameters
phase_list (list) – the list of phases to use.
- get_number_of_phases()¶
Return the number of phases in this microstructure.
Each crystal phase is stored in a list attribute: _phases. Note that it may be different (although it should not) from the different phase ids in the phase_map array.
- Return int
the number of phases in the microstructure.
- get_number_of_grains(from_grain_map=False)¶
Return the number of grains in this microstructure.
- Returns
the number of grains in the microstructure.
- add_phase(phase)¶
Add a new phase to this microstructure.
Before adding this phase, the phase id is set to the corresponding id.
- Parameters
phase (CrystallinePhase) – the phase to add.
- get_phase_ids_list()¶
Return the list of the phase ids.
- get_phase(phase_id=None)¶
Get a crystalline phase.
If no phase_id is given, the active phase is returned.
- Parameters
phase_id (int) – the id of the phase to return.
- Returns
the CrystallinePhase corresponding to the id.
- get_lattice(phase_id=None)¶
Get the crystallographic lattice associated with this microstructure.
If no phase_id is given, the Lattice of the active phase is returned.
- Returns
an instance of the Lattice class.
- get_grain_map(as_numpy=True)¶
- get_phase_map(as_numpy=True)¶
- get_mask(as_numpy=False)¶
- get_ids_from_grain_map()¶
Return the list of grain ids found in the grain map.
By convention, only positive values are taken into account, 0 is reserved for the background and -1 for overlap regions.
- Returns
a 1D numpy array containing the grain ids.
- get_grain_ids()¶
Return the grain ids found in the GrainDataTable.
- Returns
a 1D numpy array containing the grain ids.
- static id_list_to_condition(id_list)¶
Convert a list of id to a condition to filter the grain table.
The condition will be interpreted using Numexpr typically using a read_where call on the grain data table.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
the condition as a string .
- get_grain_volumes(id_list=None)¶
Get the grain volumes.
The grain data table is queried and the volumes of the grains are returned in a single array. An optional list of grain ids can be used to restrict the grains, by default all the grain volumes are returned.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
a numpy array containing the grain volumes.
- get_grain_centers(id_list=None)¶
Get the grain centers.
The grain data table is queried and the centers of the grains are returned in a single array. An optional list of grain ids can be used to restrict the grains, by default all the grain centers are returned.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
a numpy array containing the grain centers.
- get_grain_rodrigues(id_list=None)¶
Get the grain rodrigues vectors.
The grain data table is queried and the rodrigues vectors of the grains are returned in a single array. An optional list of grain ids can be used to restrict the grains, by default all the grain rodrigues vectors are returned.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
a numpy array containing the grain rodrigues vectors.
- get_grain_orientations(id_list=None)¶
Get a list of the grain orientations.
The grain data table is queried to retreiv the rodrigues vectors. An optional list of grain ids can be used to restrict the grains. A list of Orientation instances is then created and returned.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
a list of the grain orientations.
- get_grain_bounding_boxes(id_list=None)¶
Get the grain bounding boxes.
The grain data table is queried and the bounding boxes of the grains are returned in a single array. An optional list of grain ids can be used to restrict the grains, by default all the grain bounding boxes are returned.
- Parameters
id_list (list) – a non empty list of the grain ids.
- Returns
a numpy array containing the grain bounding boxes.
- get_voxel_size()¶
Get the voxel size for image data of the microstructure.
If this instance of Microstructure has no image data, None is returned.
- get_grain(gid)¶
Get a particular grain given its id.
This method browses the microstructure and return the grain corresponding to the given id. If the grain is not found, the method raises a ValueError.
- Parameters
gid (int) – the grain id.
- Returns
The method return a new Grain instance with the corresponding id.
- get_all_grains()¶
Build a list of Grain instances for all grains in this Microstructure.
- Returns
a list of the grains.
- get_grain_positions()¶
Return all the grain positions as a numpy array of shape (n, 3) where n is the number of grains.
- Returns
a numpy array of shape (n, 3) of the grain positions.
- get_grain_volume_fractions()¶
Compute all grains volume fractions.
- Returns
a 1D numpy array with all grain volume fractions.
- get_grain_volume_fraction(gid, use_total_volume_value=None)¶
Compute the volume fraction of this grain.
- Parameters
gid (int) – the grain id.
use_total_volume_value (float) – the total volume value to use.
- Return float
the grain volume fraction as a number in the range [0, 1].
- set_orientations(orientations)¶
Store grain orientations array in GrainDataTable
orientation : (Ngrains, 3) array of rodrigues orientation vectors
- set_centers(centers)¶
Store grain centers array in GrainDataTable
centers : (Ngrains, 3) array of grain centers of mass
- set_bounding_boxes(bounding_boxes)¶
Store grain bounding boxes array in GrainDataTable
- set_volumes(volumes)¶
Store grain volumes array in GrainDataTable
- set_lattice(lattice, phase_id=None)¶
Set the crystallographic lattice associated with this microstructure.
If no phase_id is specified, the lattice will be set for the active phase.
- Parameters
lattice (Lattice) – an instance of the Lattice class.
phase_id (int) – the id of the phase to set the lattice.
- set_active_grain_map(map_name='grain_map')¶
Set the active grain map name to inputed string.
The active_grain_map string is used as Name to get the grain_map field in the dataset through the SampleData “get_field” method.
- set_grain_map(grain_map, voxel_size=None, map_name='grain_map')¶
Set the grain map for this microstructure.
- Parameters
grain_map (ndarray) – a 2D or 3D numpy array.
voxel_size (float) – the size of the voxels in mm unit. Used only if the CellData image Node must be created.
- set_phase_map(phase_map, voxel_size=None)¶
Set the phase map for this microstructure.
- Parameters
phase_map (ndarray) – a 2D or 3D numpy array.
voxel_size (float) – the size of the voxels in mm unit. Used only if the CellData image Node must be created.
- set_mask(mask, voxel_size=None)¶
Set the mask for this microstructure.
- Parameters
mask (ndarray) – a 2D or 3D numpy array.
voxel_size (float) – the size of the voxels in mm unit. Used only if the CellData image Node must be created.
- set_random_orientations()¶
Set random orientations for all grains in GrainDataTable
- remove_grains_not_in_map()¶
Remove from GrainDataTable grains that are not in the grain map.
- remove_small_grains(min_volume=1.0, sync_table=False, new_grain_map_name=None)¶
Remove from grain_map and grain data table small volume grains.
Removed grains in grain map will be replaced by background ID (0). To be sure that the method acts consistently with the current grain map, activate sync_table options.
- Parameters
min_volume (float) – Grains whose volume is under or equal to this value willl be suppressed from grain_map and grain data table.
sync_table (bool) – If True, synchronize gran data table with grain map before removing grains.
new_grain_map_name (str) – If provided, store the new grain map with removed grain with this new name. If not, overright the current active grain map
- remove_grains_from_table(ids)¶
Remove from GrainDataTable the grains with given ids.
- Parameters
ids (list) – Array of grain ids to remove from GrainDataTable
- add_grains(euler_list, grain_ids=None)¶
A a list of grains to this microstructure.
This function adds a list of grains represented by a list of Euler angles triplets, to the microstructure. If provided, the grain_ids list will be used for the grain ids.
- Parameters
euler_list (list) – the list of euler angles (Bunge passive convention).
grain_ids (list) – an optional list for the ids of the new grains.
- add_grains_in_map()¶
Add to GrainDataTable the grains in grain map missing in table.
The grains are added with a random orientation by convention.
- static random_texture(n=100)¶
Generate a random texture microstructure.
parameters:
n The number of grain orientations in the microstructure.
- set_mesh(mesh_object=None, file=None, meshname='micro_mesh')¶
Add a mesh of the microstructure to the dataset.
Mesh can be inputed as a BasicTools mesh object or as a mesh file. Handled file format for now only include .geof (Zset software fmt).
- create_grain_ids_field(meshname=None, store=True)¶
Create a grain Id field of grain orientations on the input mesh.
Creates a element wise field from the microsctructure mesh provided, adding to each element the value of the Rodrigues vector of the local grain element set, as it is and if it is referenced in the GrainDataTable node.
- Parameters
mesh (str) – Name, Path or index name of the mesh on which an orientation map element field must be constructed
store (bool) – If True, store the orientation map in CellData image group, with name orientation_map
- create_orientation_field(meshname=None, store=True)¶
Create a vector field of grain orientations on the inputed mesh.
Creates a element wise field from the microsctructure mesh provided, adding to each element the value of the Rodrigues vector of the local grain element set, as it is and if it is referenced in the GrainDataTable node.
- Parameters
mesh (str) – Name, Path or index name of the mesh on which an orientation map element field must be constructed
store (bool) – If True, store the orientation map in CellData image group, with name orientation_map
- create_orientation_map(store=True)¶
Create a vector field in CellData of grain orientations.
Creates a (Nx, Ny, Nz, 3) or (Nx, Ny, 3) field from the microsctructure grain_map, adding to each voxel the value of the Rodrigues vector of the local grain Id, as it is and if it is referenced in the GrainDataTable node.
- Parameters
store (bool) – If True, store the orientation map in CellData image group, with name orientation_map
- add_IPF_maps()¶
Add IPF maps to the data set.
IPF colors are computed for the 3 cartesian directions and stored into the h5 file in the CellData image group, with names ipf_map_100, ipf_map_010, ipf_map_001.
- create_IPF_map(axis=<Mock name='mock.array()' id='139894746709968'>)¶
Create a vector field in CellData to store the IPF colors.
Creates a (Nx, Ny, Nz, 3) field with the IPF color for each voxel. Note that this function assumes a single orientation per grain.
- Parameters
axis – the unit vector for the load direction to compute IPF colors.
- view_slice(slice=None, color='random', show_mask=True, show_grain_ids=False, highlight_ids=None, slip_system=None, axis=[0.0, 0.0, 1], show_slip_traces=False, hkl_planes=None, display=True)¶
A simple utility method to show one microstructure slice.
The microstructure can be colored using different fields such as a random color (default), the grain ids, the Schmid factor or the grain orientation using IPF coloring. The plot can be customized in several ways. Annotations can be added in the grains (ids, lattice plane traces) and the list of grains where annotations are shown can be controled using the highlight_ids argument. By default, if present, the mask will be shown.
- Parameters
slice (int) – the slice number
color (str) – a string to chose the colormap from (‘random’, ‘grain_ids’, ‘schmid’, ‘ipf’)
show_mask (bool) – a flag to show the mask by transparency.
show_grain_ids (bool) – a flag to annotate the plot with the grain ids.
highlight_ids (list) – a list of grain ids to restrict the annotations (by default all grains are annotated).
slip_system – an instance (or a list of instances) of the class SlipSystem to compute the Schmid factor.
axis – the unit vector for the load direction to compute the Schmid factor or to display IPF coloring.
show_slip_traces (bool) – activate slip traces plot in each grain.
hkl_planes (list) – the list of planes to plot the slip traces.
display (bool) – if True, the show method is called, otherwise, the figure is simply returned.
- static rand_cmap(n=4096, first_is_black=False, seed=13)¶
Creates a random color map to color the grains.
The first color can be enforced to black and usually figure out the background. The random seed is fixed to consistently produce the same colormap.
- Parameters
n (int) – the number of colors in the list.
first_is_black (bool) – set black as the first color of the list.
seed (int) – the random seed.
- Returns
a matplotlib colormap.
- ipf_cmap()¶
Return a colormap with ipf colors.
Warning
This function works only for a microstructure with the cubic symmetry due to current limitation in the Orientation get_ipf_colour method.
- Returns
a color map that can be directly used in pyplot.
- static from_grain_file(grain_file_path, col_id=0, col_phi1=1, col_phi=2, col_phi2=3, col_x=4, col_y=5, col_z=None, col_volume=None, autodelete=True)¶
Create a Microstructure reading grain infos from a file.
This file is typically created using EBSD. the usual pattern is: grain_id, phi1, phi, phi2, x, y, volume. The column number are tunable using the function arguments.
- print_grains_info(grain_list=None, as_string=False)¶
Print informations on the grains in the microstructure
- match_grains(micro2, mis_tol=1, use_grain_ids=None, verbose=False)¶
Match grains from a second microstructure to this microstructure.
This function try to find pair of grains based on their orientations.
Warning
This function works only for microstructures with the same symmetry.
- Parameters
micro2 – the second instance of Microstructure from which to match the grains.
mis_tol (float) – the tolerance is misorientation to use to detect matches (in degrees).
use_grain_ids (list) – a list of ids to restrict the grains in which to search for matches.
verbose (bool) – activate verbose mode.
- Raises
ValueError – if the microstructures do not have the same symmetry.
- Return tuple
a tuple of three lists holding respectively the matches, the candidates for each match and the grains that were unmatched.
- match_orientation(orientation, use_grain_ids=None)¶
Find the best match between an orientation and the grains from this microstructure.
- Parameters
orientation – an instance of Orientation to match the grains.
use_grain_ids (list) – a list of ids to restrict the grains in which to search for matches.
- Return tuple
the grain id of the best match and the misorientation.
- find_neighbors(grain_id, distance=1)¶
Find the neighbor ids of a given grain.
This function find the ids of the neighboring grains. A mask is constructed by dilating the grain to encompass the immediate neighborhood of the grain. The ids can then be determined using numpy unique function.
- Parameters
grain_id (int) – the grain id from which the neighbors need to be determined.
distance (int) – the distance to use for the dilation (default is 1 voxel).
- Returns
a list (possibly empty) of the neighboring grain ids.
- dilate_grain(grain_id, dilation_steps=1, use_mask=False)¶
Dilate a single grain overwriting the neighbors.
- Parameters
grain_id (int) – the grain id to dilate.
dilation_steps (int) – the number of dilation steps to apply.
use_mask (bool) – if True and that this microstructure has a mask, the dilation will be limited by it.
- static dilate_labels(array, dilation_steps=1, mask=None, dilation_ids=None, struct=None)¶
Dilate labels isotropically to fill the gap between them.
This code is based on the gtDilateGrains function from the DCT code. It has been extended to handle both 2D and 3D cases.
- Parameters
array (ndarray) – the numpy array to dilate.
dilation_steps (int) – the number of dilation steps to apply.
mask (ndarray) – a msk to constrain the dilation (None by default).
dilation_ids (list) – a list to restrict the dilation to the given ids.
struct (ndarray) – the structuring element to use (strong connectivity by default).
- Returns
the dilated array.
- dilate_grains(dilation_steps=1, dilation_ids=None, new_map_name='dilated_grain_map')¶
Dilate grains to fill the gap between them.
This function calls dilate_labels with the grain map of the microstructure.
- Parameters
dilation_steps (int) – the number of dilation steps to apply.
dilation_ids (list) – a list to restrict the dilation to the given ids.
- clean_grain_map(new_map_name='grain_map_clean')¶
Apply a morphological cleaning treatment to the active grain map.
A Matlab morphological cleaner is called to smooth the morphology of the different IDs in the grain map.
This cleaning treatment is typically used to improve the quality of a mesh produced from the grain_map, or improved image based mechanical modelisation techniques results, such as FFT-based computational homogenization of the polycrystalline microstructure.
..Warning:
This method relies on the code of the 'core.utils' and on Matlab code developed by F. Nguyen at the 'Centre des Matériaux, Mines Paris'. These tools and codes must be installed and referenced in the PATH of your workstation for this method to work. For more details, see the 'utils' package.
- mesh_grain_map(mesher_opts={}, print_output=False)¶
Create a 2D or 3D conformal mesh from the grain map.
A Matlab multiphase_image mesher is called to create a conformal mesh of the grain map that is stored as a SampleData Mesh group in the MeshData Group of the Microstructure dataset. The mesh data will contain an element set per grain in the grain map.
..Warning:
This method relies on the code of the 'core.utils', on Matlab code developed by F. Nguyen at the 'Centre des Matériaux, Mines Paris', on the Zset software and the Mesh GMS software. These tools and codes must be installed and referenced in the PATH of your workstation for this method to work. For more details, see the 'utils' package.
- crop(x_start=None, x_end=None, y_start=None, y_end=None, z_start=None, z_end=None, crop_name=None, autodelete=False, recompute_geometry=True, verbose=False)¶
Crop the microstructure to create a new one.
This method crops the CellData image group to a new microstructure, and adapts the GrainDataTable to the crop.
- Parameters
x_start (int) – start value for slicing the first axis.
x_end (int) – end value for slicing the first axis.
y_start (int) – start value for slicing the second axis.
y_end (int) – end value for slicing the second axis.
z_start (int) – start value for slicing the third axis.
z_end (int) – end value for slicing the third axis.
name (str crop) – the name for the cropped microstructure (the default is to append ‘_crop’ to the initial name).
autodelete (bool) – a flag to delete the microstructure files on the disk when it is not needed anymore.
recompute_geometry (bool) – If True (defaults), recompute the grain centers, volumes, and bounding boxes in the croped micro. Use False when using a crop that do not cut grains, for instance when cropping a microstructure within the mask, to avoid the heavy computational cost of the grain geometry data update.
- Returns
a new Microstructure instance with the cropped grain map.
- sync_grain_table_with_grain_map(sync_geometry=False)¶
Update GrainDataTable with only grain IDs from active grain map.
- Parameters
sync_geometry (bool) – If True, recomputes the geometrical parameters of the grains in the GrainDataTable from active grain map.
- renumber_grains(sort_by_size=False, new_map_name=None, only_grain_map=False)¶
Renumber the grains in the microstructure.
Renumber the grains from 1 to n, with n the total number of grains that are found in the active grain map array, so that the numbering is consecutive. Only positive grain ids are taken into account (the id 0 is reserved for the background).
- Parameters
sort_by_size (bool) – use the grain volume to sort the grain ids (the larger grain will become grain 1, etc).
overwrite_active_map (bool) – if ‘True’, overwrites the active grain map with the renumbered map. If ‘False’, the active grain map is kept and a ‘renumbered_grain_map’ is added to CellData.
new_map_name (str) – Used as name for the renumbered grain map field if is not None and overwrite_active_map is False.
only_grain_map (bool) – If True, do not modify the grain map and GrainDataTable in dataset, but return the renumbered grain_map as a numpy array.
- compute_grain_volume(gid)¶
Compute the volume of the grain given its id.
The total number of voxels with the given id is computed. The value is converted to mm unit using the voxel_size. The unit will be squared mm for a 2D grain map or cubed mm for a 3D grain map.
Warning
This function assume the grain bounding box is correct, call recompute_grain_bounding_boxes() if this is not the case.
- Parameters
gid (int) – the grain id to consider.
- Returns
the volume of the grain.
- compute_grain_center(gid)¶
Compute the center of masses of a grain given its id.
Warning
This function assume the grain bounding box is correct, call recompute_grain_bounding_boxes() if this is not the case.
- Parameters
gid (int) – the grain id to consider.
- Returns
a tuple with the center of mass in mm units (or voxel if the voxel_size is not specified).
- compute_grain_bounding_box(gid, as_slice=False)¶
Compute the grain bounding box indices in the grain map.
- Parameters
gid (int) – the id of the grain.
as_slice (bool) – a flag to return the grain bounding box as a slice.
- Returns
the bounding box coordinates.
- compute_grain_equivalent_diameters(id_list=None)¶
Compute the equivalent diameter for a list of grains.
- Parameters
id_list (list) – the list of the grain ids to include (compute for all grains by default).
- Returns
a 1D numpy array of the grain diameters.
- compute_grain_sphericities(id_list=None)¶
Compute the equivalent diameter for a list of grains.
The sphericity measures how close to a sphere is a given grain. It can be computed by the ratio between the surface area of a sphere with the same volume and the actual surface area of that grain.
\[\psi = \dfrac{\pi^{1/3}(6V)^{2/3}}{A}\]- Parameters
id_list (list) – the list of the grain ids to include (compute for all grains by default).
- Returns
a 1D numpy array of the grain diameters.
- compute_grain_aspect_ratios(id_list=None)¶
Compute the aspect ratio for a list of grains.
The aspect ratio is defined by the ratio between the major and minor axes of the equivalent ellipsoid of each grain.
- Parameters
id_list (list) – the list of the grain ids to include (compute for all grains by default).
- Returns
a 1D numpy array of the grain aspect ratios.
- recompute_grain_volumes(verbose=False)¶
Compute the volume of all grains in the microstructure.
Each grain volume is computed using the grain map. The value is assigned to the volume column of the GrainDataTable node. If the voxel size is specified, the grain centers will be in mm unit, if not in voxel unit.
Note
A grain map need to be associated with this microstructure instance for the method to run.
- Parameters
verbose (bool) – flag for verbose mode.
- Returns
a 1D array with all grain volumes.
- recompute_grain_centers(verbose=False)¶
Compute and assign the center of all grains in the microstructure.
Each grain center is computed using its center of mass. The value is assigned to the grain.center attribute. If the voxel size is specified, the grain centers will be in mm unit, if not in voxel unit.
Note
A grain map need to be associated with this microstructure instance for the method to run.
- Parameters
verbose (bool) – flag for verbose mode.
- Returns
a 1D array with all grain centers.
- recompute_grain_bounding_boxes(verbose=False)¶
Compute and assign the center of all grains in the microstructure.
Each grain center is computed using its center of mass. The value is assigned to the grain.center attribute. If the voxel size is specified, the grain centers will be in mm unit, if not in voxel unit.
Note
A grain map need to be associated with this microstructure instance for the method to run.
- Parameters
verbose (bool) – flag for verbose mode.
- compute_grains_geometry(overwrite_table=False)¶
Compute grain centers, volume and bounding box from grain_map
- compute_grains_map_table_intersection(verbose=False)¶
Return grains that are both in grain map and grain table.
The method also returns the grains that are in the grain map but not in the grain table, and the grains that are in the grain table, but not in the grain map.
- Return array intersection
array of grain ids that are in both grain map and grain data table.
- Return array not_in_map
array of grain ids that are in grain data table but not in grain map.
- Return array not_in_table
array of grain ids that are in grain map but not in grain data table.
- build_grain_table_from_grain_map()¶
Synchronizes and recomputes GrainDataTable from active grain map.
- static voronoi(self, shape=(256, 256), n_grain=50)¶
Simple voronoi tesselation to create a grain map.
The method works both in 2 and 3 dimensions. The grains are labeled from 1 to n_grains (included).
- Parameters
shape (tuple) – grain map shape in 2 or 3 dimensions.
n_grains (int) – number of grains to generate.
- Returns
a numpy array grain_map
- to_amitex_fftp(binary=True, mat_file=True, elasaniso_path='', add_grips=False, grip_size=10, grip_constants=(104100.0, 49440.0), add_exterior=False, exterior_size=10, use_mask=False)¶
Write orientation data to ascii files to prepare for FFT computation.
AMITEX_FFTP can be used to compute the elastoplastic response of polycrystalline microstructures. The calculation needs orientation data for each grain written in the form of the coordinates of the first two basis vectors expressed in the crystal local frame which is given by the first two rows of the orientation matrix. The values are written in 6 files N1X.txt, N1Y.txt, N1Z.txt, N2X.txt, N2Y.txt, N2Z.txt, each containing n values with n the number of grains. The data is written either in BINARY or in ASCII form.
Additional options exist to pad the grain map with two constant regions. One region called grips can be added on the top and bottom (third axis). The second region is around the sample (first and second axes).
- Parameters
binary (bool) – flag to write the files in binary or ascii format.
mat_file (bool) – flag to write the material file for Amitex.
elasaniso_path (str) – path for the libUmatAmitex.so in the Amitex_FFTP installation.
add_grips (bool) – add a constant region at the beginning and the end of the third axis.
grip_size (int) – thickness of the region.
grip_constants (tuple) – elasticity values for the grip (lambda, mu).
add_exterior (bool) – add a constant region around the sample at the beginning and the end of the first two axes.
exterior_size (int) – thickness of the exterior region.
use_mask (bool) – use mask to define exterior material, and use mask to extrude grips with same shapes as microstructure top and bottom surfaces.
- from_amitex_fftp(results_basename, grip_size=0, ext_size=0, grip_dim=2, sim_prefix='Amitex', int_var_names={}, finite_strain=False, load_fields=True, compression_options={})¶
Read output of a Amitex_fftp simulation and stores in dataset.
Read a Amitex_fftp result directory containing a mechanical simulation of the microstructure. See method ‘to_amitex_fftp’ to generate input files for such simulation of Microstructure instances.
The results are stored as fields of the CellData group by default. If generated by the simulation, the strain and stress tensor fields are stored, as well as the internal variables fields.
Mechanical fields and macroscopical curves are stored. The latter is stored in the data group ‘/Mechanical_simulation’ as a structured array.
- Parameters
results_basename (str) – Basename of Amitex .std, .vtk output files to load in dataset.
grip_size (int, optional) – Thickness of the grips added to simulation unit cell by the method ‘to_amitex_fftp’ of this class, defaults to 0. This value corresponds to a number of voxels on both ends of the cell.
grip_dim (int, optional) – Dimension along which the tension test has been simulated (0:x, 1:y, 2:z)
ext_size (int, optional) – Thickness of the exterior region added to simulation unit cell by the method ‘to_amitex_fftp’ of this class, defaults to 0. This value corresponds to a number of voxels on both ends of the cell.
sim_prefix (str, optional) – Prefix of the name of the fields that will be stored on the CellData group from simulation results.
int_var_names (dict, optional) – Dictionnary whose keys are the names of internal variables stored in Amitex output files (varInt1, varInt2…) and values are corresponding names for these variables in the dataset.
- print_zset_material_block(mat_file, grain_prefix='_ELSET')¶
Outputs the material block corresponding to this microstructure for a finite element calculation with z-set.
- Parameters
mat_file (str) – The name of the file where the material behaviour is located
grain_prefix (str) – The grain prefix used to name the elsets corresponding to the different grains
- to_dream3d()¶
Write the microstructure as a hdf5 file compatible with DREAM3D.
- static from_dream3d(file_path, main_key='DataContainers', data_container='DataContainer', grain_data='FeatureData', grain_orientations='AvgEulerAngles', orientation_type='euler', grain_centroid='Centroids')¶
Read a microstructure from a hdf5 file.
- Parameters
file_path (str) – the path to the hdf5 file to read.
main_key (str) – the string describing the root key.
data_container (str) – the string describing the data container group in the hdf5 file.
grain_data (str) – the string describing the grain data group in the hdf5 file.
grain_orientations (str) – the string describing the average grain orientations in the hdf5 file.
orientation_type (str) – the string describing the descriptor used for orientation data.
grain_centroid (str) – the string describing the grain centroid in the hdf5 file.
- Returns
a Microstructure instance created from the hdf5 file.
- static copy_sample(src_micro_file, dst_micro_file, overwrite=False, get_object=False, dst_name=None, autodelete=False)¶
Initiate a new SampleData object and files from existing one
- static from_neper(neper_file_path)¶
Create a microstructure from a neper tesselation.
Neper is an open source program to generate polycristalline microstructure using voronoi tesselations. It is available at https://neper.info
- Parameters
neper_file_path (str) – the path to the tesselation file generated by Neper.
- Returns
a pymicro Microstructure instance.
- static from_labdct(labdct_file, data_dir='.', include_IPF_map=False, include_rodrigues_map=False)¶
Create a microstructure from a DCT reconstruction.
- Parameters
labdct_file (str) – the name of the file containing the labDCT data.
data_dir (str) – the path to the folder containing the HDF5 reconstruction file.
include_IPF_map (bool) – if True, the IPF maps will be included in the microstructure fields.
include_rodrigues_map (bool) – if True, the rodrigues map will be included in the microstructure fields.
- Returns
a Microstructure instance created from the labDCT reconstruction file.
- static from_dct(data_dir='.', grain_file='index.mat', vol_file='phase_01_vol.mat', mask_file='volume_mask.mat', use_dct_path=True, verbose=True)¶
Create a microstructure from a DCT reconstruction.
DCT reconstructions are stored in several files. The indexed grain informations are stored in a matlab file in the ‘4_grains/phase_01’ folder. Then, the reconstructed volume file (labeled image) is stored in the ‘5_reconstruction’ folder as an hdf5 file, possibly stored alongside a mask file coming from the absorption reconstruction.
- Parameters
data_dir (str) – the path to the folder containing the reconstruction data.
grain_file (str) – the name of the file containing grains info.
vol_file (str) – the name of the volume file.
mask_file (str) – the name of the mask file.
use_dct_path (bool) – if True, the grain_file should be located in 4_grains/phase_01 folder and the vol_file and mask_file in the 5_reconstruction folder.
verbose (bool) – activate verbose mode.
- Returns
a Microstructure instance created from the DCT reconstruction.
- static from_legacy_h5(file_path, filename=None)¶
read a microstructure object from a HDF5 file created by pymicro until version 0.4.5.
- Parameters
file_path (str) – the path to the file to read.
- Returns
the new Microstructure instance created from the file.
- static from_ebsd(file_path, roi=None, tol=5.0, min_ci=0.2)¶
“Create a microstructure from an EBSD scan.
- Parameters
file_path (str) – the path to the file to read.
roi (list) – a list of 4 integers in the form [x1, x2, y1, y2] to crop the EBSD scan.
tol (float) – the misorientation angle tolerance to segment the grains (default is 5 degrees).
min_ci (float) – minimum confidence index for a pixel to be a valid EBSD measurement.
- Returns
a new instance of Microstructure.
- static merge_microstructures(micros, overlap, translation_offset=[0, 0, 0], plot=False)¶
Merge two Microstructure instances together.
The function works for two microstructures with grain maps and an overlap between them. Temporarily Microstructures restricted to the overlap regions are created and grains are matched between the two based on a disorientation tolerance.
Note
The two microstructure must have the same crystal lattice and the same voxel_size for this method to run.
- Parameters
micros (list) – a list containing the two microstructures to merge.
overlap (int) – the overlap to use.
translation_offset (list) – a manual translation (in voxels) offset to add to the result.
plot (bool) – a flag to plot some results.
- Returns
a new Microstructure instance containing the merged microstructure.
texture Module¶
The texture module provide some utilities to generate, analyse and plot crystallographic textures.
- class pymicro.crystal.texture.PoleFigure(microstructure=None, lattice=None, axis='Z', hkl='111', proj='stereo', verbose=False)¶
Bases:
objectA class to create pole figures.
A pole figure is a useful tool to plot multiple crystal orientations, either in the sample coordinate system (direct pole figure) or alternatively plotting a particular direction in the crystal coordinate system (inverse pole figure).
- __init__(microstructure=None, lattice=None, axis='Z', hkl='111', proj='stereo', verbose=False)¶
Create an empty PoleFigure object associated with a Microstructure.
Warning
Any crystal structure is now supported (you have to set the proper crystal lattice) but it has only really be tested for cubic.
- Parameters
microstructure – the
Microstructurecontaining the collection of orientations to plot (None by default).lattice – the crystal
Lattice.axis (str) – the pole figure axis (‘Z’ by default), vertical axis in the direct pole figure and direction plotted on the inverse pole figure.
hkl (str) – slip plane family (‘111’ by default)
proj (str) – projection type, can be either ‘stereo’ (default) or ‘flat’
verbose (bool) – verbose mode (False by default)
- get_orientations()¶
Get the list of orientations in the PoleFigure.
- Returns
a list of Orientation instances.
- set_hkl_poles(hkl='111')¶
Set the pole (aka hkl planes) list to to use in the PoleFigure.
The list of poles can be given by the family type or directly by a list of HklPlanes objects.
- Params str/list hkl
slip plane family (‘111’ by default)
- set_map_field(field_name, field=None, field_min_level=None, field_max_level=None, lut='hot')¶
Set the PoleFigure to color poles with the given field.
This method activates a mode where each symbol in the pole figure is color coded with respect to a field, which can be either the grain id, or a given field given in form of a list. If the grain volume or strain. For the grain id, the color is set according the each grain id in the
Microstructureand therand_cmap()function. For a given field, the color is set from the lookup table and according to the value in the given list. The list must contain a record for each grain. Minimum and maximum value to map the field values and the colors can be specify, if not they are directly taken as the min() and max() of the field.- Parameters
field_name (str) – The field name, could be ‘grain_id’, ‘ipf’, ‘grain_size’ or any other name describing the field.
field (list) – A list containing a record for each grain.
field_min_level (float) – The minimum value to use for this field.
field_max_level (float) – The maximum value to use for this field.
lut (str) – A string describing the colormap to use (among matplotlib ones available).
- Raises
ValueError – If the given field does not contain enough values.
- plot_pole_figures(plot_sst=True, display=True, save_as='pdf')¶
Plot and save a picture with both direct and inverse pole figures.
- Parameters
plot_sst (bool) – controls wether to plot the full inverse pole figure or only the standard stereographic triangle (True by default).
display (bool) – display the plot if True, else save a picture of the pole figures (True by default)
save_as (str) – File format used to save the image such as pdf or png (‘pdf’ by default)
micro = Microstructure(name = 'AlLi_sam8') micro.grains.append(Grain(11, Orientation.from_euler(np.array([262.364, 16.836, 104.691])))) Al_fcc = Lattice.face_centered_cubic(0.405) # not really necessary since default lattice is cubic pf = PoleFigure(microstructure=micro, proj='stereo', lattice=Al_fcc, hkl='111') pf.mksize = 12 pf.set_map_field('grain_id') pf.pflegend = True # this works well for a few grains pf.plot_pole_figures()
A 111 pole figure plotted for a single crystal orientation.¶
- plot_crystal_dir(c_dir, **kwargs)¶
Function to plot a crystal direction on a pole figure.
- Parameters
c_dir – A vector describing the crystal direction.
kwargs (dict) – a dictionnary of keyword/values to control the plot, it should at least contain a reference to a pyplot axes to draw the pole using keyword ‘ax’.
- Raises
ValueError – if the projection type is not supported
- plot_line_between_crystal_dir(c1, c2, ax=None, steps=11, col='k')¶
Plot a curve between two crystal directions.
The curve is actually composed of several straight lines segments to draw from direction 1 to direction 2.
- Parameters
c1 – vector describing crystal direction 1
c2 – vector describing crystal direction 2
ax – a reference to a pyplot ax to draw the line
steps (int) – number of straight lines composing the curve (11 by default)
col – line color (black by default)
- plot_pf_background(ax, labels=True)¶
Function to plot the background of the pole figure.
- Parameters
ax – a reference to a pyplot ax to draw the backgroud.
labels (bool) – add lables to axes (True by default).
- plot_pf_dir(c_dir, **kwargs)¶
Plot a crystal direction in a direct pole figure.
- Parameters
c_dir – a vector describing the crystal direction.
kwargs (dict) – a dictionnary of keyword/values to control the plot, it should at least contain a reference to a pyplot axes to draw the pole using keyword ‘ax’.
- plot_pf(ax=None, mk='o', ann=False)¶
Create the direct pole figure.
- Parameters
ax – a reference to a pyplot ax to draw the poles.
mk – marker used to plot the poles (disc by default).
ann (bool) – Annotate the pole with the coordinates of the vector if True (False by default).
- create_pf_contour(ax=None, ang_step=10)¶
Compute the distribution of orientation and plot it using contouring.
This plot the distribution of orientation in the microstructure associated with this PoleFigure instance, as a continuous distribution using angular bining with the specified step. the distribution is constructed at runtime by discretizing the angular space and counting the number of poles in each bin. Then the plot_pf_contour method is called to actually plot the data.
Warning
This function has not been tested properly, use at your own risk.
- Parameters
ax – a reference to a pyplot ax to draw the contours.
ang_step (int) – angular step in degrees to use for constructing the orientation distribution data (10 degrees by default)
- plot_pf_contour(ax, x, y, values)¶
Plot the direct pole figure using contours.
Warning
This function has not been tested properly, use at your own risk.
- sst_symmetry(v)¶
Transform a given vector according to the lattice symmetry associated with the pole figure.
This function transform a vector so that it lies in the smallest symmetry equivalent zone.
- Parameters
v – the vector to transform.
- Returns
the transformed vector.
- static sst_symmetry_cubic(z_rot)¶
Transform a given vector according to the cubic symmetry.
This function transform a vector so that it lies in the unit SST triangle.
- Parameters
z_rot – vector to transform.
- Returns
the transformed vector.
- get_color_from_field(grain)¶
Get the color of the given grain according to the chosen field.
This function will return the color associated with the given grain. Depending on how the pole figure has been configured (see the set_map_field function), it will be obtained from:
the grain id, according to the Microstructure.rand_cmap function
- ipf the colour will reflect the orientation according to the IPF
coloring scheme
- the field value mapped on a pyplot color map if the lut field of
the PoleFigure instance is a string.
- a color directly read from the lut field; in this case the field
value must reflect the category of the given grain.
- Parameters
grain – the Grain instance.
- Returns
the color as a 3 element numpy array representing the rgb values.
- plot_sst(**kwargs)¶
Create the inverse pole figure in the unit standard triangle.
- Parameters
ax – a reference to a pyplot ax to draw the poles.
mk – marker used to plot the poles (square by default).
ann (bool) – Annotate the pole with the coordinates of the vector if True (False by default).
- plot_ipf(**kwargs)¶
Create the inverse pole figure for direction Z.
- Parameters
ax – a reference to a pyplot ax to draw the poles.
mk – marker used to plot the poles (square by default).
ann (bool) – Annotate the pole with the coordinates of the vector if True (False by default).
- static plot(orientations, **kwargs)¶
Plot a pole figure (both direct and inverse) for a list of crystal orientations.
- Parameters
orientations – the list of crystalline
Orientationto plot.
- static plot_euler(phi1, Phi, phi2, **kwargs)¶
Directly plot a pole figure for a single orientation given its three Euler angles.
PoleFigure.plot_euler(10, 20, 30)
- Parameters
phi1 (float) – first Euler angle (in degree).
Phi (float) – second Euler angle (in degree).
phi2 (float) – third Euler angle (in degree).
file Package¶
file_utils Module¶
- pymicro.file.file_utils.read_image_sequence(data_dir, prefix, num_images, start_index=0, image_format='png', zero_padding=0, crop=None, verbose=False)¶
Read a series of images into a list of numpy arrays.
- Parameters
data_dir (str) – directory where the image files are located.
prefix (str) – a string to construct the image file names.
num_images (int) – the number of images to read.
start_index (int) – the index to start loading the images (0 by default).
image_format (str) – can be tif or png (png by default).
zero_padding (int) – number of zero to use in zero padding (0 by default).
crop (list) – bounds to crop the images (None by default)
verbose (bool) – activate verbose mode (False by default).
- Returns
the list of the images read from the disk.
- pymicro.file.file_utils.unpack_header(header)¶
Unpack an ascii header.
Form a string with the read binary data and then split it into string tokens which are put in a dictionnary.
- Parameters
header (str) – the ascii header to unpack.
- Returns
a dictionnary with the (key, value) fields contained in the header.
- pymicro.file.file_utils.edf_info(file_name, header_size=None, verbose=False)¶
Read and return informations contained in the header of a .edf file.
Edf files always start with a header (of variable length) containing information about the file such as acquisition conditions, image dimensions… This function reads a certain amount of bytes of a given file as ascii data and unpack it. If not specified, the header size is determined automatically by substracting the data size (read as ascii at the begining of the file) to the total file size.
- Parameters
file_name (str) – the name of the edf file to read.
header_size (int) – number of bytes to read as a multiple of 512 (None by default).
verbose (bool) – flag to activate verbose mode.
- Returns
a dictionary containing the file information.
- pymicro.file.file_utils.edf_read(file_name, verbose=False)¶
Read an edf file.
edf stands for ESRF data file. It has a variable header size which is a multiple of 512 bytes and contains the image meta in ASCII format (eg. image size, data type, motor positions).
The ascii header is parsed automatically by edf_info to retreive the image size and data type. Depending on the information enclosed in the header, this function may return a 1d, 2d or 3d array.
>>> im = edf_read('radio.edf') >>> im.shape (2048, 2048)
- Parameters
file_name (str) – the name of the edf file to read.
verbose (bool) – flag to activate verbose mode.
- Returns
a numpy array containing the data
- pymicro.file.file_utils.esrf_to_numpy_datatype(data_type)¶
- pymicro.file.file_utils.numpy_to_esrf_datatype(data_type)¶
- pymicro.file.file_utils.edf_write(data, file_name, header_size=1024)¶
Write a binary edf file with the appropriate header.
This function write a (x,y,z) 3D dataset to the disk. The file is written as a Z-stack. It means that the first nx*ny bytes represent the first slice and so on…
- Parameters
data (ndarray) – the data array to write to the file.
file_name (str) – the file name to use.
header_size (int) – the size of te header (a multiple of 512).
- pymicro.file.file_utils.HST_info(info_file)¶
Read the given info file and returns a dictionary containing the data size and type.
Note
The first line of the file must begin by ! PyHST or directly by NUM_X. Also note that if the data type is not specified, it will not be present in the dictionary.
- Parameters
info_file (str) – path to the ascii file to read.
- Returns
a dictionary with the values for x_dim, y_dim, z_dim and data_type if needed.
- pymicro.file.file_utils.HST_read(scan_name, zrange=None, data_type=<Mock name='mock.uint8' id='139894658854800'>, verbose=False, header_size=0, autoparse_filename=False, dims=None, mmap=False, pack_binary=False)¶
Read a volume file stored as a concatenated stack of binary images.
The volume size must be specified by dims=(nx, ny, nz) unless an associated .info file is present in the same location to determine the volume size. The data type is unsigned short (8 bits) by default but can be set to any numpy type (32 bits float for example).
The autoparse_filename can be activated to retreive image type and size:
HST_read(myvol_100x200x50_uint16.raw, autoparse_filename=True)
will read the 3d image as unsigned 16 bits with size 100 x 200 x 50.
Note
If you use this function to read a .edf file written by matlab in +y+x+z convention (column major order), you may want to use: np.swapaxes(HST_read(‘file.edf’, …), 0, 1)
- Parameters
scan_name (str) – path to the binary file to read.
zrange – range of slices to use.
data_type – numpy data type to use.
verbose (bool) – flag to activate verbose mode.
header_size (int) – number of bytes to skeep before reading the payload.
autoparse_filename (bool) – flag to parse the file name to retreive the dims and data_type automatically.
dims (tuple) – a tuple containing the array dimensions.
mmap (bool) – activate the memory mapping mode.
pack_binary (bool) – this flag should be true when reading a file written with the binary packing mode.
- pymicro.file.file_utils.rawmar_read(image_name, size, verbose=False)¶
Read a square 2D image plate MAR image.
These binary images are typically obtained from the marcvt utility.
Note
This method assume Big endian byte order.
- pymicro.file.file_utils.HST_write(data, file_name, mode='w', verbose=True, pack_binary=False)¶
Write data as a raw binary file.
This function write a (x,y,z) 3D dataset to the disk. The actual data type is used, you can convert your data array on the fly using data.astype if you want to change the type. The file is written as a Z-stack. It means that the first nx*ny bytes written represent the first slice and so on… For binary data files (stored in memory as integer or bool data type), binary packing mode can be activated which stores 8 values on each byte (saving 7/8 of the disk space).
A .info file containing the volume size and data type is also written.
- Parameters
data – the 3d array to write to the disk in [x, y, z] form.
file_name (str) – the name of the file to write, including file extension.
mode (char) – file write mode, change to ‘a’ to append to a file.
verbose (bool) – flag to activate verbose mode.
pack_binary (bool) – flag to activate binary packing.
- pymicro.file.file_utils.recad_vol(vol_filename, min, max, verbose=False)¶
Recad a 32 bit vol file into 8 bit raw file.
This function reads a 3D volume file into a numpy float32 array and applies the recad function with the [min, max] range. The result is saved into a .raw file with the same name as the input file.
In verbose mode, a piture to compare mid slices in the two volumes and another one to compare the histograms are saved.
Note
To read the vol file, the presence of a .info file is assumed, see HST_read.
Parameters
vol_filename: the path to the binary vol file.
min: value to use as the minimum (will be 0 in the casted array).
max: value to use as the maximum (will be 255 in the casted array).
verbose: activate verbose mode (False by default).
- pymicro.file.file_utils.Vtk_write(data, fname)¶
Write a data array into old style (V3.0) VTK format.
An ascii header is written to which the binary data is appended.
Note
The header assumes uint8 data type.
view Package¶
vol_utils Module¶
- pymicro.view.vol_utils.hist(data, nb_bins=256, data_range=(0, 255), show=True, save=False, prefix='data', density=False)¶
Histogram of a data array.
Compute and plot the gray level histogram of the provided data array.
Parameters
data: the data array to analyse.
nb_bins: the number of bins in the histogram.
data_range: the data range to use in the histogram, (0,255) by default.
show: boolean to display the figure using pyplot (defalut True).
save: boolean to save the figure using pyplot (defalut False).
prefix: a string to use in the file name when saving the histogram as an image (defaut ‘data’).
density: a boolean to control wether the histogram is plotted using the probability density function, see numpy histogram function for more details (default False).
Gray level histogram computed on a 512x512x512 8 bits image.¶
- pymicro.view.vol_utils.flat(img, ref, dark)¶
Apply flat field correction to an image.
- Parameters
img (np.array) – A 2D array representing the image to correct.
ref (np.array) – The reference image (without the sample), same shape as the image to correct.
dark (np.array) – A 2D numpy array representing the dark image (thermal noise of the camera).
- Returns np.array float
the flat field corrected image (between 0 and 1) as a float32 numpy array.
- pymicro.view.vol_utils.min_max_cumsum(data, cut=0.001, verbose=False)¶
Compute the indices corresponding to the edge of a distribution.
The min and max indices are calculated according to a cutoff value (0.001 by default) meaning 99.99% of the distribution lies within this range.
- pymicro.view.vol_utils.auto_min_max(data, cut=0.0002, nb_bins=256, verbose=False)¶
Compute the min and max values in a numpy data array.
The min and max values are calculated based on the histogram of the array which is limited at both ends by the cut parameter (0.0002 by default). This means 99.98% of the values are within the [min, max] range.
- Parameters
data – the data array to analyse.
cut (float) – the cut off value to use on the histogram (0.0002 by default).
nb_bins (int) – number of bins to use in the histogram (256 by default).
verbose (bool) – activate verbose mode (False by default).
- Returns tuple (val_mini, val_maxi)
a tuple containing the min and max values.
- pymicro.view.vol_utils.region_growing(data, seed, min_thr, max_thr, structure=None)¶
Simple region growing segmentation algorithm.
This function segment a part of an image by selecting the connected region within a given gray value range. Underneath, the method uses the label function of the ndimage package.
- Parameters
data – the numpy data array to segment.
seed (tuple) – the seed point.
min_thr (float) – the lower bound of the gray value range.
max_thr (float) – the upper bound of the gray value range.
structure – structuring element, to use (None by default).
- Returns region
a bool data array with True values for the segmented region.
- pymicro.view.vol_utils.recad(data, mini, maxi)¶
Cast a numpy array into 8 bit data type.
This function change the data type of a numpy array into 8 bit. The data values are interpolated from [min, max] to [0, 255]. Both min and max values may be chosen manually or computed by the function auto_min_max.
- Parameters
data – the data array to cast to uint8.
mini (float) – value to use as the minimum (will be 0 in the casted array).
maxi (float) – value to use as the maximum (will be 255 in the casted array).
- Returns data_uint8
the data array casted to uint8.
- pymicro.view.vol_utils.alpha_cmap(color='red', opacity=1.0)¶
Creating a particular colormap with transparency.
Only values equal to 255 will have a non zero alpha channel. This is typically used to overlay a binary result on initial data.
- Parameters
color – the color to use for non transparent values (ie. 255).
opacity (float) – opacity value to use for visible pixels.
- Returns mycmap
a fully transparent colormap except for 255 values.
- pymicro.view.vol_utils.stitch(image_stack, nh=2, nv=1, pattern='E', hmove=None, vmove=None, adjust_bc=False, adjust_bc_nbins=256, verbose=False, show=True, save=False, save_name='stitch', save_ds=1, check=None)¶
Stich a series of images together.
- Parameters
image_stack (list) – a list of the images to stitch.
nh (int) – number of images to stitch horizontally.
nv (int) – number of images to stitch vertically.
pattern (str) – stitching pattern (‘E’ or ‘S’).
hmove (tuple) – horizontal move between 2 images (first image width by default).
vmove (tuple) – vertical move between 2 images (first image height by default).
adjust_bc (bool) – adjust gray levels by comparing the histograms in the overlapping regions (False by default).
adjust_bc_nbins (int) – numbers of bins to use tin the histograms when adjusting the gray levels (256 by default).
verbose (bool) – activate verbose mode (False by default).
show (bool) – show the stitched image (True by default).
save (bool) – flag to save the image as png to the disk (False by default).
save_name (str) – name to use to save the stitched image (stitch by default).
save_ds (int) – downsampling factor using when saving the stitched image (1 by default).
check (int) – trigger plotting for the given image if set.
- Returns full_im
the stitched image in [x, y] form.
- pymicro.view.vol_utils.compute_affine_transform(fixed, moving)¶
Compute the affine transform by point set registration.
The affine transform is the composition of a translation and a linear map. The two ordered lists of points must be of the same length larger or equal to 3. The order of the points in the two list must match.
The 2D affine transform \(\mathbf{A}\) has 6 parameters (2 for the translation and 4 for the linear transform). The best estimate of \(\mathbf{A}\) can be computed using at least 3 pairs of matching points. Adding more pair of points will improve the quality of the estimate. The matching pairs are usually obtained by selecting unique features in both images and measuring their coordinates.
- Parameters
fixed (list) – a list of the reference points.
moving (list) – a list of the moving points to register on the fixed point.
- Returns translation, linear_map
the computed translation and linear map affine transform.
Thanks to Will Lenthe for helping with this code.
vtk_anim Module¶
The vtk_anim module define a set of classes to generate 3d animations with vtk in the form of a series of png images.
- class pymicro.view.vtk_anim.vtkAnimationScene(ren, ren_size=(600, 600))¶
Bases:
object- add_animation(anim)¶
- write_image()¶
- execute(iren, event)¶
- render()¶
- render_at(time=0.0)¶
- class pymicro.view.vtk_anim.vtkAnimation(t, duration=10)¶
Bases:
object- pre_execute()¶
- post_execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkAnimCameraAroundZ(t, cam, turn=360)¶
Bases:
vtkAnimationAnimate the camera around the vertical axis.
This class can be used to generate a series of images (default 36) while the camera rotate around the vertical axis (defined by the camera SetViewUp method).
- execute(iren, event)¶
Execute method called to rotate the camera.
- class pymicro.view.vtk_anim.vtkRotateActorAroundAxis(t=0, duration=10, axis=(0.0, 0.0, 1.0), angle=360)¶
Bases:
vtkAnimation- set_actor(actor)¶
Set the actor for this animation.
This also keep a record of the actor initial transformation matrix.
- Parameters
actor (vtkActor) – the actor on which to apply the animation.
- execute(iren, event)¶
instruction block executed when a TimerEvent is captured by the vtkRotateActorAroundAxis.
If the time is not in [start, end] nothing is done. Otherwise the transform matrix corresponding to the 3D rotation is applied to the actor.
The transform matrix for this increment is the result of the multiplication of the rotation matrix for the current angle with the initial 4x4 matrix before any rotation (we keep a record of this in the user_transform_matrix attribute).
- Parameters
iren (vtkRenderWindowInteractor) – the vtk render window interactor.
event – the captures event.
- class pymicro.view.vtk_anim.vtkRotateActorAroundZAxis(t=0)¶
Bases:
vtkRotateActorAroundAxis
- class pymicro.view.vtk_anim.vtkAnimCameraToZ(t, cam)¶
Bases:
vtkAnimation- execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkZoom(t, cam, zoom)¶
Bases:
vtkAnimation- execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkSetVisibility(t, actor, visible=1, max_opacity=1, gradually=False)¶
Bases:
vtkAnimation- execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkMoveActor(t, actor, motion)¶
Bases:
vtkAnimation- execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkAnimLine(points, t1, t2)¶
Bases:
vtkAnimation- execute(iren, event)¶
- class pymicro.view.vtk_anim.vtkUpdateText(text_actor, str_method, t=0, duration=10)¶
Bases:
vtkAnimation- execute(iren, event)¶
vtk_utils Module¶
- pymicro.view.vtk_utils.to_vtk_type(type)¶
Function to get the VTK data type given a numpy data type.
- Parameters
type (str) – The numpy data type like ‘uint8’, ‘uint16’…
- Returns
A VTK data type.
- pymicro.view.vtk_utils.rand_cmap(N=256, first_is_black=False, table_range=(0, 255))¶
Create a VTK lookup table with random colors.
The first color can be enforced to black and usually figure out the image background. The random seed is fixed to 13 in order to consistently produce the same colormap.
- Parameters
N (int) – The number of colors in the colormap.
first_is_black (bool) – Force the first color to be black.
table_range (typle) – The range of the VTK lookup table
- Returns
A vtkLookupTable lookup table with N random colors.
- pymicro.view.vtk_utils.pv_rand_cmap(N=256, first_is_black=False)¶
Write out the random color map in paraview xml format.
This method print out the XML declaration of the random colormap. This may be saved to a text file and used in paraview.
- Parameters
N (int) – The number of colors in the colormap.
first_is_black (bool) – Force the first color to be black.
- pymicro.view.vtk_utils.pyplot_cmap(name='viridis', table_range=(0, 255))¶
Create a VTK colormap from pyplot.
- Parameters
name (str) – the color map name to import from pyplot.
- Returns
- pymicro.view.vtk_utils.gray_cmap(table_range=(0, 255))¶
create a black and white colormap.
Parameters
table_range: 2 values tuple (default: (0,255)) start and end values for the table range.
Returns
A vtkLookupTable from black to white.
- pymicro.view.vtk_utils.invert_cmap(ref_lut)¶
invert a VTK lookup table.
Parameters
ref_lut: The lookup table to invert.
Returns
A reverse vtkLookupTable.
- pymicro.view.vtk_utils.hsv_cmap(N=64, table_range=(0, 255))¶
Create a VTK look up table similar to matlab’s hsv.
Parameters
N: int, number of colors in the table.
table_range: 2 values tuple (default: (0,255)) start and end values for the table range.
Returns
A vtkLookupTable.
- pymicro.view.vtk_utils.jet_cmap(N=64, table_range=(0, 255))¶
Create a VTK look up table similar to matlab’s jet.
Parameters
N: int, number of colors in the table.
table_range: 2 values tuple (default: (0,255)) start and end values for the table range.
Returns
A vtkLookupTable from blue to red.
- pymicro.view.vtk_utils.hot_cmap(table_range=(0, 255))¶
Create a VTK look up table similar to matlab’s hot.
Parameters
table_range: 2 values tuple (default: (0,255)) start and end values for the table range.
Returns
A vtkLookupTable from white to red.
- pymicro.view.vtk_utils.add_hklplane_to_grain(hkl_plane, grid, orientation, origin=(0, 0, 0), opacity=1.0, show_normal=False, normal_length=1.0, show_intersection=False, color_intersection=<Mock name='mock.red' id='139894658534928'>)¶
Add a plane describing a crystal lattice plane to a VTK grid.
- Parameters
hkl_plane – an instance of HklPlane describing the lattice plane.
grid – a vtkunstructuredgrid instance representing the geometry of the grain.
orientation – the grain orientation.
origin – the origin of the plane in the grain.
opacity (float) –
show_normal – A flag to show the plane normal.
normal_length – The length of the plane normal.
show_intersection – A flag to show the intersection of the plane with the grain.
color_intersection (tuple) – The color to display the intersection of the plane with the grain.
- Returns
A VTK assembly with the grain, the plane, its normal and edge intersection if requested.
- pymicro.view.vtk_utils.add_slip_system_to_grain(slip_system, grid, orientation, origin=(0, 0, 0), opacity=1.0, show_normal=False, normal_length=1.0, show_intersection=False, color_intersection=<Mock name='mock.red' id='139894658534928'>)¶
Add a slip system to a VTK grid.
- Parameters
slip_system – an instance of SlipSystem describing the slip system.
grid – a vtkunstructuredgrid instance representing the geometry of the grain.
orientation – the grain orientation.
origin – the origin of the plane in the grain.
opacity (float) – opacity value of the plane.
show_normal – A flag to show the plane normal.
normal_length – The length of the plane normal.
show_intersection – A flag to show the intersection of the plane with the grain.
color_intersection (tuple) – The color to display the intersection of the plane with the grain.
- Returns
A VTK assembly with the grain, the plane, its normal and edge intersection if requested and the slip
direction.
- pymicro.view.vtk_utils.add_plane_to_grid(plane, grid, origin=None, color=None, opacity=0.3, show_normal=False, normal_length=1.0, show_intersection=False, color_intersection=<Mock name='mock.red' id='139894658534928'>)¶
Add a 3d plane inside another object.
This function adds a plane inside another object described by a mesh (vtkunstructuredgrid). The method is to use a vtkCutter with the mesh as input and the plane as the cut function. The plane normal can be displayed and its length controlled. This may be used directly to add hkl planes inside a lattice cell or a grain.
- Parameters
plane – A VTK implicit function describing the plane to add.
grid – A VTK unstructured grid in which the plane is to be added.
origin (tuple) – (x, y, z) tuple to define the origin of the plane.
color (tuple) – A color defined by its rgb components.
opacity (float) – Opacity value of the plane actor.
show_normal (bool) – A flag to display the plane normal.
normal_length – The length of the plane normal vector (1.0 by default).
show_intersection (bool) – Also add the intersection of the plane with the grid in the assembly.
color_intersection (tuple) – The color to display the intersection of the plane with the grid.
- Returns
A VTK assembly with the mesh, the plane, its normal and edge intersection if requested.
- pymicro.view.vtk_utils.axes_actor(length=1.0, axisLabels=('x', 'y', 'z'), fontSize=20, color=None)¶
Build an actor for the cartesian axes.
- Parameters
length (float or triple of float to specify the length of each axis individually.) – The arrow length of the axes (1.0 by default).
axisLabels (list) – Specify the axes labels (xyz by default), use axisLabels = None to hide the axis labels
fontSize (int) – Font size for the axes labels (20 by default).
color (tuple) – A single color defined by its rgb components (not set by default which keep the red, green, blue colors).
- Returns
A VTK assembly representing the cartesian axes.
- pymicro.view.vtk_utils.grain_3d(grain, hklplanes=None, show_normal=False, plane_origins=None, plane_opacity=1.0, show_orientation=False, N=2048, verbose=False)¶
Creates a 3d representation of a crystallographic grain.
This method creates a vtkActor object of the surface mesh representing a Grain object. An optional list of crystallographic planes can be given
- Parameters
grain – the Grain object to be shown in 3d.
hklplanes (list) – the list of HklPlanes object to add to the assembly.
show_normal (bool) – show also the normal to the hkl planes if True.
plane_origins (list) – the list of each plane origin (3d scene coordinates).
plane_opacity (float) – set the opacity of the grain actor.
show_orientation (bool) – show also the grain orientation with a vtkAxesActor placed at the grain center if True.
N (int) – the number of colors to use in the colormap.
verbose (bool) – activate verbose mode.
- Returns
a vtkAssembly of the grain mesh and the optional hkl planes.
- pymicro.view.vtk_utils.add_grain_to_3d_scene(grain, hklplanes, show_orientation=False)¶
- pymicro.view.vtk_utils.add_local_orientation_axes(orientation, axes_length=30)¶
- pymicro.view.vtk_utils.add_HklPlanes_with_orientation_in_grain(grain, hklplanes=[])¶
Add some plane actors corresponding to a list of (hkl) planes to a grain actor.
- pymicro.view.vtk_utils.unit_arrow_3d(start, vector, color=<Mock name='mock.orange' id='139894658552208'>, radius=0.03, make_unit=True, label=False, text=None, text_scale=0.1, vector_normal=None)¶
- pymicro.view.vtk_utils.lattice_points(lattice, origin=(0.0, 0.0, 0.0), m=1, n=1, p=1)¶
Create a vtk representation of a the lattice points.
A vtkPoints instance is used to store the lattice points, including the points not on the lattice corners according to the system centering (may be P, I, F for instance).
- Parameters
lattice (Lattice) – The Lattice instance from which to construct the points.
origin (tuple) – cartesian coordinates of the origin.
m (int) – the number of cells in the [100] direction (1 by default).
n (int) – the number of cells in the [010] direction (1 by default).
p (int) – the number of cells in the [001] direction (1 by default).
- Returns
A vtkPoints with all the lattice points ordered such that the first 8*(m*n*p) points describe the lattice cells.
- pymicro.view.vtk_utils.lattice_grid(lattice, origin=(0.0, 0.0, 0.0), m=1, n=1, p=1)¶
Create a mesh representation of a crystal lattice.
A vtkUnstructuredGrid instance is used with a hexaedron element corresponding to the lattice system. Any number of cells can be displayed (just one by default).
- Parameters
lattice (Lattice) – The Lattice instance from which to construct the grid.
origin (tuple) – cartesian coordinates of the origin.
m (int) – the number of cells in the [100] direction (1 by default).
n (int) – the number of cells in the [010] direction (1 by default).
p (int) – the number of cells in the [001] direction (1 by default).
- Returns
A vtkUnstructuredGrid with (m x n x p) hexaedron cell representing the crystal lattice.
- pymicro.view.vtk_utils.hexagonal_lattice_grid(lattice, origin=[0.0, 0.0, 0.0])¶
Create a mesh representation of a hexagonal crystal lattice.
A vtkUnstructuredGrid instance is used with a hexagonal prism element corresponding to 3 unit cells of the lattice system.
- Parameters
lattice (Lattice) – The Lattice instance from which to construct the grid.
origin (tuple) – cartesian coordinates of the origin.
- Returns
A vtkUnstructuredGrid one hexagnal prism cell representing the crystal lattice.
- pymicro.view.vtk_utils.lattice_edges(grid, tubeRadius=0.02, tubeColor=<Mock name='mock.grey' id='139894658552016'>)¶
Create the 3D representation of crystal lattice edges.
Parameters
grid: vtkUnstructuredGrid The vtkUnstructuredGrid instance representing the crystal lattice.
tubeRadius: float Radius of the tubes representing the atomic bonds (default: 0.02).
tubeColor: vtk color Color of the tubes representing the atomis bonds (default: grey).
Returns
The method return a vtk actor for lattice edges.
- pymicro.view.vtk_utils.lattice_vertices(grid, sphereRadius=0.1, sphereColor=<Mock name='mock.blue' id='139894658552144'>)¶
Create the 3D representation of crystal lattice atoms.
Parameters
grid: vtkUnstructuredGrid The vtkUnstructuredGrid instance representing the crystal lattice.
sphereRadius: float Size of the spheres representing the atoms (default: 0.1).
sphereColor: vtk color Color of the spheres representing the atoms (default: blue).
Returns
The method return a vtk actor for lattice vertices.
- pymicro.view.vtk_utils.crystal_vertices(crystal, origin=(0.0, 0.0, 0.0), m=1, n=1, p=1, hide_outside=True)¶
Create the 3D representation of the atoms in a given crystal, taking into account the crystal lattice and the basis which can be composed of any motif.
- Parameters
origin (tuple) – cartesian coordinates of the origin.
m (int) – the number of cells in the [100] direction (1 by default).
n (int) – the number of cells in the [010] direction (1 by default).
p (int) – the number of cells in the [001] direction (1 by default).
hide_outside (bool) – do not displays atoms outside the displayed unit cells if True.
- Returns
The method return a vtk actor with all the crystal atoms.
- pymicro.view.vtk_utils.crystal_3d(crystal, origin=(0.0, 0.0, 0.0), m=1, n=1, p=1, sphereRadius=0.1, tubeRadius=0.02, sphereColor=<Mock name='mock.blue' id='139894658552144'>, tubeColor=<Mock name='mock.grey' id='139894658552016'>, hide_outside=True)¶
- pymicro.view.vtk_utils.lattice_3d(lattice, origin=(0.0, 0.0, 0.0), m=1, n=1, p=1, sphereRadius=0.05, tubeRadius=0.02, sphereColor=<Mock name='mock.black' id='139894658535312'>, tubeColor=<Mock name='mock.grey' id='139894658552016'>, crystal_orientation=None, show_atoms=True, show_edges=True, cell_clip=False)¶
Create the 3D representation of a crystal lattice.
The lattice edges are shown using a vtkTubeFilter and the atoms are displayed using spheres. Both tube and sphere radius can be controlled. Crystal orientation can also be provided which rotates the whole assembly appropriately.
The origin of the actor can be t=either specified directly using a tuple or set using a string as follow:
mid the middle of the lattice cell(s)
l = Lattice.cubic(1.0) cubic = lattice_3d(l) ren = vtk.vtkRenderer() ren.AddActor(cubic) render(ren, display=True)
A 3D view of a cubic lattice.¶
- Parameters
lattice (Lattice) – The Lattice instance representing the crystal lattice.
origin (tuple or string) – cartesian coordinates of the origin.
m (int) – the number of cells in the [100] direction (1 by default).
n (int) – the number of cells in the [010] direction (1 by default).
p (int) – the number of cells in the [001] direction (1 by default).
sphereRadius (float) – Size of the spheres representing the atoms (default: 0.05).
tubeRadius (float) – Radius of the tubes representing the atomic bonds (default: 0.02).
sphereColor (tuple) – Color of the spheres representing the atoms (default: black).
tubeColor (tuple) – Color of the tubes representing the atomis bonds (default: grey).
crystal_orientation – The crystal
Orientationwith respect to the sample coordinate system (default: None).show_atoms (bool) – Control if the atoms are shown (default: True).
show_edges (bool) – Control if the eges of the lattice are shown (default: True).
cell_clip (bool) – Clip the lattice points glyphs by the cell (default: False).
- Returns
The method return a vtk assembly combining lattice edges and vertices.
- pymicro.view.vtk_utils.lattice_3d_with_planes(lattice, hklplanes, plane_origins=None, plane_colors=None, show_normal=True, plane_opacity=1.0, **kwargs)¶
Create the 3D representation of a crystal lattice.
HklPlanes can be displayed within the lattice cell with their normals. A single vtk actor in form of an assembly is returned. Additional parameters are passed to the lattice_3d method to control how the lattice is pictured.
l = Lattice.cubic(1.0) o = Orientation.from_euler([344.0, 125.0, 217.0]) hklplanes = Hklplane.get_family('111') cubic = lattice_3d_with_planes(l, hklplanes, show_normal=True, \ plane_opacity=0.5, crystal_orientation=o) s3d = Scene3D() s3d.add(cubic) s3d.render()
A 3D view of a cubic lattice with all four 111 planes displayed.¶
- Parameters
lattice – An instance of
Latticecorresponding to the crystal lattice to be displayed.hklplanes – A list of
HklPlaneinstances to add to the lattice.plane_origins – A list of tuples describing the plane origins (must be the same length as hklplanes), if None, the planes are created to pass through the middle of the lattice (default).
plane_colors – A list of tuples describing the plane colors (must be the same length as hklplanes), if None, the planes are left gray (default).
show_normal (bool) – Control if the slip plane normals are shown (default: True).
plane_opacity (float) – A float number in the [0.,1.0] range controlling the slip plane opacity.
**kwargs –
additional parameters are passed to the lattice_3d method.
- Returns
The method return a vtkAssembly that can be directly added to a renderer.
- pymicro.view.vtk_utils.lattice_3d_with_plane_series(lattice, hkl, nps=1, **kwargs)¶
Create the 3D representation of a crystal lattice with a series of hkl planes.
HklPlanes can be displayed within the lattice cell with their normals. A single vtk actor in form of an assembly is returned. Additional parameters are passed to the lattice_3d method to control how the lattice is pictured.
l = Lattice.cubic(1.0) orientation = Orientation.from_euler([0, 54.74, 135]) # correspond to 111 fiber texture copper = (1.000000, 0.780392, 0.494117) # nice copper color copper_lattice = lattice_3d_with_plane_series(l, (1, -1, 1), nps=4, crystal_orientation=orientation, \ origin='mid', show_atoms=True, sphereColor=copper, sphereRadius=0.1) s3d = Scene3D() s3d.add(cubic) s3d.render()
A 3D view of a copper lattice with a series of successive (111) planes displayed.¶
- Parameters
lattice – An instance of
Latticecorresponding to the crystal lattice to be displayed.hkl – A tuple of the 3 miller indices.
nps (int) – The number of planes to display in the series (1 by default).
**kwargs –
additional parameters are passed to the lattice_3d_with_planes method.
- Returns
The method return a vtkAssembly that can be directly added to a renderer.
- pymicro.view.vtk_utils.pole_figure_3d(pf, radius=1.0, show_lattice=False)¶
Method to display a pole figure in 3d.
This method displays a sphere of radius 1 with a crystal lattice placed at the center (only shown if the show_lattice parameter is True). The poles associated with the pole figure are shown on the outbounding sphere as well as the projection n the equatorial plane.
- Parameters
pf – the PoleFigure instance .
radius (float) – the outbounding sphere radius (1 by default).
show_lattice – a flag to show the crystal lattice in the center of the sphere.
- Returns
a vtk assembly that can be added to a Scene3D instance.
- pymicro.view.vtk_utils.apply_translation_to_actor(actor, trans)¶
Transform the actor (or whole assembly) using the specified translation.
- Parameters
actor (vtkActor) – the vtk actor.
trans – a 3 component numpy vector or sequence describing the translation to apply in scene units.
- pymicro.view.vtk_utils.apply_rotation_to_actor(actor, R)¶
Transform the actor with a given rotation matrix.
- Parameters
actor (vtkActor) – the vtk actor.
R – a (3x3) array representing the rotation matrix.
- pymicro.view.vtk_utils.apply_orientation_to_actor(actor, orientation)¶
Transform the actor (or whole assembly) using the specified orientation. Here we could use the three euler angles associated with the orientation with the RotateZ and RotateX methods of the actor but the components of the orientation matrix are used directly since they are known without any ambiguity.
- Parameters
actor (vtkActor) – the vtk actor.
orientation – an instance of the
pymicro.crystal.microstructure.Orientationclass
- pymicro.view.vtk_utils.load_STL_actor(name, ext='STL', verbose=False, color=<Mock name='mock.grey' id='139894658552016'>, feature_edges=False)¶
Read a STL file and return the corresponding vtk actor.
- Parameters
name (str) – the base name of the file to read.
ext (str) – extension of the file to read.
verbose (bool) – verbose mode.
color (tuple) – the color to use for the actor.
feature_edges (bool) – show boundary edges (default False).
- Returns
the 3d solid in the form of a vtk actor.
- pymicro.view.vtk_utils.is_in_array(cad_path, step, origin=[0.0, 0.0, 0.0])¶
Function to compute the coordinates of the points within a CAD volume, with a given step.
- pymicro.view.vtk_utils.read_image_data(file_name, size, header_size=0, data_type='uint8', verbose=False)¶
vtk helper function to read a 3d data file. The size is needed in the form (x, y, z) as well a string describing the data type in numpy format (uint8 is assumed by default). Lower file left and little endian are assumed.
Parameters
file_name: the name of the file to read.
size: a sequence of three numbers describing the size of the 3d data set
header_size: size of the header to skip in bytes (0 by default)
data_type: a string describing the data type in numpy format (‘uint8’ by default)
verbose: verbose mode (False by default)
Returns
A VTK data array
- pymicro.view.vtk_utils.data_outline(data, corner=False, color=<Mock name='mock.black' id='139894658535312'>)¶
vtk helper function to draw a bounding box around a volume.
- pymicro.view.vtk_utils.box_3d(origin=(0, 0, 0), size=(100, 100, 100), line_color=<Mock name='mock.black' id='139894658535312'>)¶
Create a box of a given size.
- Parameters
origin (tuple) – the origin of the box in the laboratory frame.
size (tuple) – the size of the box.
line_color – the color to use to draw the box.
- Returns
- pymicro.view.vtk_utils.detector_3d(detector, image_name=None, show_axes=False, see_reference=True)¶
Create a 3D detector on a 3D scene, using all tilts. See_reference allow to plot an empty detector with dashed edge without any tilts.
- Parameters
detector – RegArrayDetector2d
image_name – str of the image to plot in the 3D detector ‘image.png’
- Returns
vtk actor
- pymicro.view.vtk_utils.build_line_mesh(points)¶
Function to construct a vtkUnstructuredGrid representing a line mesh.
- Parameters
points (list) – the list of points.
- Returns line_mesh
the vtkUnstructuredGrid.
- pymicro.view.vtk_utils.line_actor(line_grid)¶
- pymicro.view.vtk_utils.line_3d(start_point, end_point)¶
Function to draw a line in a 3d scene.
- Parameters
start_point (tuple) – the line starting point.
end_point (tuple) – the line ending point.
- Returns vtkActor
The method return a vtkActor of the line that can be directly added to a 3d scene.
- pymicro.view.vtk_utils.circle_line_3d(center=(0, 0, 0), radius=1, normal=(0, 0, 1), resolution=1)¶
Function to draw a circle in a 3d scene.
- Parameters
center (tuple) – the center of the circle.
radius (float) – the radius of the circle.
normal (tuple) – the normal to the plane of the circle.
resolution (float) – the resolution in degree.
- Returns vtkActor
The method return a vtkActor that can be directly added to a 3d scene.
- pymicro.view.vtk_utils.point_cloud_3d(data_points, point_color=(0, 0, 0), point_size=1)¶
Function to display a point cloud in a 3d scene.
- Parameters
data_points (list) – the list of points in he cloud.
point_color (tuple) – the color to use to display the points.
point_size (float) – the size of the points.
- Returns
The method return a vtkActor of the point cloud that can be directly added to a 3d scene.
- pymicro.view.vtk_utils.contourFilter(data, value, color=<Mock name='mock.grey' id='139894658552016'>, diffuseColor=<Mock name='mock.grey' id='139894658552016'>, opacity=1.0, discrete=False)¶
This method create an actor running a contour filter through the given data set.
The data set can be equally given in numpy or VTK format (it will be converted to VTK if needed). The method may require a fair amount of memory so downsample your data if you can.
- Params data
the dataset to map, in numpy or VTK format.
- Params float value
numeric value to use for contouring.
- Params color
the solid color to use for the contour actor.
- Params diffuseColor
the diffusive color to use for the contour actor.
- Params float opacity
the opacity value to use for the actor (1.0 by default).
- Params bool discrete
use vtkDiscreteMarchingCubes if True (False by default).
- Returns
The method return a vtkActor that can be directly added to a renderer.
- pymicro.view.vtk_utils.volren(data, alpha_channel=None, color_function=None)¶
Volume rendering for a 3d array using standard ray casting.
- Parameters
data – the dataset to render, in numpy or VTK format.
alpha_channel – a vtkPiecewiseFunction instance, default to linear between 0 and 255 if not given.
- Returns
The method return a vtkVolume that can be added to a renderer.
- pymicro.view.vtk_utils.elevationFilter(data, value, low_high_range, low_point=None, high_point=None)¶
Create an isosurface and map it with an elevation filter.
- Parameters
data – the dataset to map, in VTK format.
value (float) – the value to use to create the isosurface.
low_high_range (tuple) – range to use in the elevation filter in the form (low, high).
low_point (tuple) – lower point defining the axis from which to compute the elevation.
If not specified, (0, 0, low) is assumed. :param tuple high_point: lower point defining the axis from which to compute the elevation. If not specified, (0, 0, high) is assumed. :returns vtkActor: The method return an actor that can be directly added to a renderer.
- pymicro.view.vtk_utils.numpy_array_to_vtk_grid(data, cell_data=True, array_name=None)¶
Transform a 3d numpy data array into a vtk uniform grid with scalar data.
- Parameters
data – the 3d numpy data array, possibly with 3 components using a 4th dimension.
cell_data (bool) – flag to assign cell data, as opposed to point data, to the grid (True by default).
array_name (str) – an optional name for the vtk array.
- Return vtkUniformGrid
The method return a vtkUniformGrid with scalar data initialized from
the provided numpy array.
- pymicro.view.vtk_utils.map_data_with_clip(data, lut=<Mock name='mock.vtkLookupTable()' id='139894662220688'>, cell_data=True)¶
This method construct an actor to map a 3d dataset.
1/8 of the data is clipped out to have a better view of the interior. It requires a fair amount of memory so downsample your data if you can (it may not be visible at all on the resulting image).
data = read_image_data(im_file, size) ren = vtk.vtkRenderer() actor = map_data_with_clip(data) ren.AddActor(actor) render(ren, display=True)
A 3D view of a polyamid sample with reinforcing glass fibers.¶
Parameters
data: the dataset to map, in numpy or VTK format.
lut: VTK look up table (default: gray_cmap).
cell_data: boolean to map cell data or point data if False (True by default)
Returns
The method return a vtkActor that can be directly added to a renderer.
- pymicro.view.vtk_utils.map_data(data, function, lut=<Mock name='mock.vtkLookupTable()' id='139894662220688'>, cell_data=True)¶
This method construct an actor to map a 3d dataset.
It requires a fair amount of memory so downsample your data if you can (it may not be visible at all on the resulting image).
Parameters
data: the dataset to map, in numpy or VTK format.
function: VTK implicit function where to map the data.
lut: VTK look up table (default: gray_cmap).
cell_data: boolean to map cell data or point data if False (True by default)
Returns
The method return a vtkActor that can be directly added to a renderer.
- pymicro.view.vtk_utils.set_opacity(assembly, opacity)¶
- pymicro.view.vtk_utils.color_bar(title, lut=None, fmt='%.1e', width=0.5, height=0.075, num_labels=7, font_size=26)¶
- pymicro.view.vtk_utils.text(text, font_size=20, color=(0, 0, 0), hor_align='center', coords=(0.5, 0.5))¶
Create a 2D text actor to add to a 3d scene.
- Params int font_size
the font size (20 by default).
- Params tuple color
the face color (black by default).
- Params str hor_align
horizontal alignment, should be ‘left’, ‘center’ or ‘right’ (center by default).
- Params tuple coords
a sequence of two values between 0 and 1.
:returns an actor for the text to add to a renderer.
- pymicro.view.vtk_utils.setup_camera(size=(100, 100, 100))¶
Setup the camera with usual viewing parameters.
The camera is looking at the center of the data with the Z-axis vertical.
Parameters
size: the size of the 3d data set (100x100x100 by default).
- pymicro.view.vtk_utils.render(ren, ren_size=(600, 600), display=True, save=False, name='render_3d.png', key_pressed_callback=None)¶
Render the VTK scene in 3D.
Given a vtkRenderer, this function does the actual 3D rendering. It can be used to display the scene interactlively and/or save a still image in png format.
Parameters
ren: the VTK renderer with containing all the actors.
ren_size: a tuple with two value to set the size of the image in pixels (defalut 600x600).
display: a boolean to control if the scene has to be displayed interactively to the user (default True).
save: a boolean to to control if the scene has to be saved as a png image (default False).
name: a string to used when saving the scene as an image (default is ‘render_3d.png’).
key_pressed_callback a function (functions are first class variables) called in interactive mode when a key is pressed.
- pymicro.view.vtk_utils.extract_poly_data(grid, inside=True, boundary=True)¶
Convert from vtkUnstructuredGrid to vtkPolyData.
- Parameters
grid – the vtkUnstructuredGrid instance.
inside (bool) – flag to extract inside cells or not.
boundary (bool) – flag to include boundary cells.
- Returns
the vtkExtractGeometry filter.
- pymicro.view.vtk_utils.select(grid, id_list, verbose=False)¶
Select cells in vtkUnstructuredGrid based on a list of indices.
- Parameters
grid (vtkUnstructuredGrid) – the grid on which to perform the selection.
id_list (list) – the indices of the cells to select.
verbose (bool) – a flag to activate verbose mode.
- Returns
a new vtkUnstructuredGrid containing the selected cells.
- pymicro.view.vtk_utils.show_array(data, map_scalars=False, lut=None, hide_zero_values=True)¶
Create a 3d actor representing a numpy array.
Given a 3d array, this function compute the skin of the volume. The scalars can be mapped to the created surface and the colormap adjusted. If the data is in numpy format it is converted to VTK first.
- Parameters
data – the dataset, in numpy or VTK format.
map_scalars (bool) – map the scalar in the data array to the created
surface (False by default). :param lut: a vtk lookup table (colormap) used to map the scalars. :param bool hide_zero_values: blank cells with a value of zero (True by default) :return: a vtk actor that can be added to a rendered to show the 3d array.
- pymicro.view.vtk_utils.show_mesh(grid, map_scalars=False, lut=None, show_edges=False, edge_color=(0.0, 0.0, 0.0), edge_line_width=1.0)¶
Create a 3d actor representing a mesh.
- Parameters
grid – the vtkUnstructuredGrid object.
map_scalars (bool) – map the scalar in the data array to the created surface (False by default).
lut – a vtk lookup table (colormap) used to map the scalars.
show_edges (bool) – display the mesh edges (False by default).
edge_color (tuple) – color to use for the mesh edges (black by default).
edge_line_width (float) – width of the edge lines (1.0 by default).
- Returns
a vtk actor that can be added to a rendered to show the 3d array.
- pymicro.view.vtk_utils.show_grains(data, num_colors=2048)¶
Create a 3d actor of all the grains in a labeled numpy array.
Given a 3d numpy array, this function compute the skin of all the grains (labels > 0). the background is assumed to be zero and is removed. The actor produced is colored by the grain ids using the random color map, see rand_cmap.
Parameters
data: a labeled numpy array.
num_colors: number of colors in the lookup table (2048 by default)
Returns a vtk actor that can be added to a rendered to show all the grains colored by their id.
- pymicro.view.vtk_utils.show_boundaries(grid, array_id=0, array_name=None, write=False)¶
Create an actor representing the boundaries separating different values of a given array. The values have to be one of the integer type.
- Parameters
grid (vtkUnstructuredGrid) – the unstructured grid referencing the data array.
array_id (int) – the index of the array to process (default 0).
array_name (str) – the name of the array to process.
write (bool) – flag to write the boundary polydata to the disk.
- Returns
a VTK actor containing the boundaries.
- pymicro.view.vtk_utils.edges_actor(polydata, linewidth=1.0, linecolor=<Mock name='mock.black' id='139894658535312'>)¶
- pymicro.view.vtk_utils.xray_arrow()¶
- pymicro.view.vtk_utils.slits(size, x_slits=0)¶
Create a 3d schematic represenation of X-ray slits.
The 3d represenation is made of 4 corners of the given size along the Y and Z axes.
Parameters:
size: a (X,Y,Z) tuple giving the size of the illuminated volume. The first value of the tuple is not used.
x_slits: position of the slits along the X axis (0 be default).
Returns:
A vtk assembly of the 4 corners representing the slits.
- pymicro.view.vtk_utils.pin_hole(inner_radius=100, outer_radius=200)¶
- pymicro.view.vtk_utils.zone_plate(thk=50, sep=25, n_rings=5)¶
Create a 3d schematic represenation of a Fresnel zone plate.
The 3d represenation is made of a number or concentric rings separated by a specific distance which control the X-ray focalisation.
Parameters:
thk: ring thickness (50 by default).
sep: ring spacing (25 by default).
Returns:
A vtk assembly of the rings composing the Fresnel zone plate.
- pymicro.view.vtk_utils.grid_vol_view(scan)¶
- pymicro.view.vtk_utils.vol_view(scan)¶
- pymicro.view.vtk_utils.ask_for_map_file(dir, scan_name)¶
xray Package¶
A package to play with X-rays
The xray package currently has five modules:
dct (experimental)
detectors
fitting
laue
xray_utils
Warning
The documentation of this package is still very poor, use with care… and add some love to the docs if you have time!
dct Module¶
The dct module provide helpers functions to work with experimental diffraction contrast tomography data.
- class pymicro.xray.dct.DctForwardSimulation(verbose=False)¶
Bases:
ForwardSimulationClass to represent a Forward Simulation.
- __init__(verbose=False)¶
- set_hkl_planes(hkl_planes)¶
- set_diffracting_famillies(hkl_list)¶
Set the list of diffracting hk planes using a set of families.
- setup(omega_step, grain_ids=None)¶
Setup the forward simulation.
- Parameters
omega_step (float) – the angular integration step (in degrees) use to compute the diffraction comditions.
grain_ids (list) – a list of grain ids to restrict the forward simulation (use all grains by default).
- load_grain(gid=1)¶
- grain_projection_image(g_uv, g_proj)¶
Produce a 2D image placing all diffraction spots of a given grain at their respective position on the detector.
Spots outside the detector are are skipped while those only partially on the detector are cropped accordingly.
- Parameters
g_proj – image stack of the diffraction spots. The first axis is so that g_proj[0] is the first spot the second axis is the horizontal coordinate of the detector (u) and the third axis the vertical coordinate of the detector (v).
g_uv – list or array of the diffraction spot position.
- Returns
a 2D composite image of all the diffraction spots.
- grain_projection_exp(gid=1)¶
Produce a composite image with all the experimental diffraction spots of this grain on the detector.
- Parameters
gid (int) – the number of the selected grain.
- Returns
a 2D composite image of all the diffraction spots.
- grain_projections(omegas, gid=1, data=None, hor_flip=False, ver_flip=False)¶
Compute the projections of a grain at different rotation angles.
The method compute each projection and concatenate them into a single 3D array in the form [n, u, v] with n the number of angles.
- Parameters
omegas (list) – the list of omega angles to use (in degrees).
gid (int) – the id of the grain to project (1 default).
data (ndarray) – the data array representing the grain.
hor_flip (bool) – a flag to apply a horizontal flip.
ver_flip (bool) – a flag to apply a vertical flip.
- Returns
a 3D array containing the n projections.
- grain_projection_simulation(gid=1)¶
Function to compute all the grain projection in DCT geometry and create a composite image.
- Parameters
gid (int) – the id of the grain to project (1 default).
- dct_projection(omega, include_direct_beam=True, att=5)¶
Function to compute a full DCT projection at a given omega angle.
- Parameters
omega (float) – rotation angle in degrees.
include_direct_beam (bool) – flag to compute the transmission through the sample.
att (float) – an attenuation factor used to limit the gray levels in the direct beam.
- Returns
the dct projection as a 2D numpy array
- pymicro.xray.dct.add_to_image(image, inset, uv, verbose=False)¶
Add an image to another image at a specified position.
The inset image may be of any size and may only overlap partly on the overall image depending on the location specified. In such a case, the inset image is cropped accordingly.
- Parameters
image (np.array) – the master image taht will be modified.
inset (np.array) – the inset to add to the image.
uv (tuple) – the location (center) where to add the inset in the form (u, v).
verbose (bool) – activate verbose mode (False by default).
- pymicro.xray.dct.merge_dct_scans(scan_list, samtz_list, use_mask=False, overlap=-1, root_dir='.', write_to_h5=True)¶
Merge two DCT scans.
This function build a Microstructure instance for each DCT scan and calls merge_microstructures. The overlap can be deduced from the samtz values or specified directly.
- Parameters
scan_list (list) – a list with the two DCT scan names.
samtz_list (list) – a list with the two samtz value (the order should match the scan names).
use_mask (bool) – a flag to also merge the absorption masks.
overlap (int) – the value to use for the overlap if not computed automatically.
root_dir (str) – the root data folder.
write_to_h5 (bool) – flag to write the result of the merging operation to an HDF5 file.
- Returns
A new Microstructure instance of the 2 merged scans.
- pymicro.xray.dct.all_dif_spots(g_proj, g_uv, verbose=False)¶
Produce a 2D image placing all diffraction spots at their respective position on the detector.
Spots outside the detector are are skipped while those only partially on the detector are cropped accordingly.
- Parameters
g_proj – image stack of the diffraction spots. The first axis is so that g_proj[0] is the first spot the second axis is the horizontal coordinate of the detector (u) and the third axis the vertical coordinate of the detector (v).
g_uv – list or array of the diffraction spot position.
verbose (bool) – activate verbose mode (False by default).
- Returns
a 2D composite image of all the diffraction spots.
- pymicro.xray.dct.plot_all_dif_spots(gid, detector, hkl_miller=None, uv=None, lattice=None, lambda_keV=None, spots=True, dif_spots_image=None, max_value=None, positions=True, debye_rings=True, suffix='')¶
- pymicro.xray.dct.output_tikzpicture(proj_dif, omegas, gid=1, d_uv=[0, 0], suffix='')¶
- pymicro.xray.dct.tt_rock(scan_name, data_dir='.', n_topo=-1, mask=None, dark_factor=1.0)¶
- pymicro.xray.dct.tt_stack(scan_name, data_dir='.', save_edf=False, n_topo=-1, dark_factor=1.0)¶
Build a topotomography stack from raw detector images.
The number of image to sum for a topograph can be determined automatically from the total number of images present in data_dir or directly specified using the variable TOPO_N.
- Parameters
scan_name (str) – the name of the scan to process.
data_dir (str) – the path to the data folder.
save_edf (bool) – flag to save the tt stack as an EDF file.
n_topo (int) – the number of images to sum for a topograph.
dark_factor (float) – a multiplicative factor for the dark image.
detectors Module¶
The detectors module define classes to manipulate X-ray detectors.
- class pymicro.xray.detectors.Detector2d(size=(2048, 2048), data_type=<Mock name='mock.uint16' id='139894657552272'>)¶
Bases:
objectClass to handle 2D detectors.
2D detectors produce array like images and may have different geometries. In particular the pixel arrangement may not necessarily be flat or regular. This abstract class regroup the generic method for those kind of detectors.
- __init__(size=(2048, 2048), data_type=<Mock name='mock.uint16' id='139894657552272'>)¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- clear_data()¶
Simply set all pixels to zeros.
- azimuthal_regroup(two_theta_mini=None, two_theta_maxi=None, two_theta_step=None, psi_mask=None, psi_min=None, psi_max=None, write_txt=False, output_image=False, debug=False)¶
- sagital_regroup(two_theta_mini=None, two_theta_maxi=None, psi_min=None, psi_max=None, psi_step=None, write_txt=False, output_image=False)¶
- class pymicro.xray.detectors.RegArrayDetector2d(size=(2048, 2048), data_type=<Mock name='mock.uint16' id='139894657552272'>, P=None, tilts=(0.0, 0.0, 0.0))¶
Bases:
Detector2dGeneric class to handle a flat detector with a regular grid of pixels.
An orthonormal local frame Rc is attached to the detector with its origin located at the center of the detector array (the ref_pos attribute). Tilts can be applied to change the orientation of the detector via a series of three intrinsic rotations in that order: kappa rotate around X, delta rotate around Y, omega rotate around Z.
The two dimensions of the pixel grid are referred by u and v. Usually u is the horizontal direction and v the vertical one, but both can be controlled by the u_dir and v_dir attributes. An attribute P is maintained to change from the detector coordinate system Rc to the pixel coordinate system which has its origin in the top left corner of the detector.
This type of detector supports binning.
- __init__(size=(2048, 2048), data_type=<Mock name='mock.uint16' id='139894657552272'>, P=None, tilts=(0.0, 0.0, 0.0))¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- static compute_tilt_matrix(tilts)¶
Compute the rotation matrix of the detector with respect to the laboratory frame.
The three tilt angles define a series of three intrinsic rotations in that order: kappa rotate around X, delta rotate around Y, omega rotate around Z.
- apply_tilts(tilts)¶
- clear_data()¶
Clear all data arrays.
- set_binning(binning)¶
Set the binning factor. This operation clears all the data arrays.
- Parameters
binning (int) – binning factor
- set_u_dir(tilts)¶
Set the coordinates of the vector describing the first (horizontal) direction of the pixels.
- set_v_dir(tilts)¶
Set the coordinates of the vector describing the second (vertical) direction of the pixels.
- get_pixel_size()¶
Return the effective pixel size, accounting for current binning settings.
- get_size_px()¶
Return the size of the detector in pixel, accounting for current binning settings.
- get_size_mm()¶
Return the size of the detector in millimeters.
- get_origin()¶
Return the detector origin in laboratory coordinates.
- get_edges(num_points=21, verbose=False)¶
Return an array describing the detector edges in pixel units.
- Parameters
num_points (int) – number of points to describe an edge (minimum 2)
verbose (bool) – activate verbose mode.
- project_along_direction(direction, origin=[0.0, 0.0, 0.0])¶
Return the intersection point of a line and the detector plane, in laboratory coordinates.
If \(\l\) is a vector in the direction of the line, \(l_0\) a point of the line, \(p_0\) a point of the plane (here the reference position), and \(n\) the normal to the plane, the intersection point can be written as \(p = d.l + l_0\) where the distance \(d\) can be computed as:
\[d=\dfrac{(p_0 - l_0).n}{l.n}\]- Parameters
direction – the direction of the projection (in the laboratory frame).
origin – the origin of the projection ([0., 0., 0.] by default).
- Returns
the point of projection in the detector plane (can be outside the detector bounds).
- lab_to_pixel(points)¶
Compute the pixel numbers corresponding to a series physical point in space on the detector. The points can be given as an array of size (n, 3) for n points or just as a 3 elements tuple for a single point.
- Parameters
points (ndarray) – the coordinates of the points in the laboratory frame.
- Return ndarray uv
the detector coordinates of the given points as an array of size (n, 2).
- pixel_to_lab(u, v)¶
Compute the laboratory coordinates of a given pixel. , if the pixels coordinates are given using 1D arrays of length n, a numpy array of size (n, 3) with the laboratory coordinates is returned.
- Parameters
u (int) – the given pixel number along the first direction (can be 1D array).
v (int) – the given pixel number along the second direction (can be 1D array).
- Return tuple (x, y, z)
the laboratory coordinates.
- load_image(image_path)¶
- compute_corrected_image()¶
- compute_geometry()¶
Calculate an array of the image size with the (2theta, psi) for each pixel.
- compute_TwoTh_Psi_arrays()¶
Calculate two arrays (2theta, psi) TwoTheta and Psi angles arrays corresponding to repectively the vertical and the horizontal pixels.
- angles_to_pixels(two_theta, psi)¶
given two values 2theta and psi in degrres (that could be arrays), compute the corresponding pixel on the detector.
- static from_poni(size, poni_path)¶
Create a new detector using settings from a poni file (PyFAI convention).
- Parameters
size (tuple) – the size of the 2D detector to create.
poni_path (str) – the path to the ascii poni file to read.
- Returns
a new instance of RegArrayDetector2d.
- class pymicro.xray.detectors.Varian2520¶
Bases:
RegArrayDetector2dClass to handle a Varian Paxscan 2520 detector.
The flat panel detector produces 16 bit unsigned (1840, 1456) images when setup in horizontal mode.
- __init__()¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- class pymicro.xray.detectors.Mar165¶
Bases:
RegArrayDetector2dClass to handle a rayonix marccd165.
The image plate marccd 165 detector produces 16 bits unsigned (2048, 2048) square images.
- __init__()¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- class pymicro.xray.detectors.PerkinElmer1620¶
Bases:
RegArrayDetector2dClass to handle a PErkin Elmer 1620 detector.
The flat panel detector produces 16 bits unsigned (2000, 2000) square images.
- __init__()¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- class pymicro.xray.detectors.Xpad¶
Bases:
Detector2dClass to handle Xpad like detectors.
Xpad are pixel detectors made with stacked array of silicon chips. Between each chip are 2 double pixels with a 2.5 times bigger size and which need to be taken into account.
Note
This code is heavily inspired by the early version of C. Mocuta, scientist on the DiffAbs beamline at Soleil synchrotron.
Warning
only tested with Xpad S140 for now…
- __init__()¶
Initialization of a Detector2d instance with a given image size (2048 pixels square array by default). The instance can be positioned in the laboratory frame using the ref_pos attribute which is the position in the laboratory frame of the middle of the detector.
- load_image(image_path, nxs_prefix=None, nxs_dataset=None, nxs_index=None, nxs_update_geometry=False, stack='first')¶
load an image from a file.
The image can be stored as a uint16 binary file (.raw) or in a nexus file (.nxs). With the nexus format, several arguments must be specified such as the prefix, the index and the dataset number (as str). :param str image_path: relative or absolute path to the file containing the image. :param str nxs_prefix: the nexus prefix hardcoded into the xml tree. :param str nxs_dataset: the dataset number. :param int nxs_index: the nexus index. :param bool nxs_update_geometry: if True the compute_TwoTh_Psi_arrays method is called after loading the image. :params str stack: indicates what to do if many images are present, ‘first’ (default) to keep only the first one, ‘median’ to compute the median over the third dimension.
- compute_geometry()¶
Calculate the array with the corrected geometry (double pixels).
- compute_corrected_image()¶
Compute a corrected image.
First the intensity is corrected either via background substraction or flat field correction. Then tiling and double pixels are accounted for to obtain a proper geometry where each pixel of the image represent the same physical zone.
- compute_TwoTh_Psi_arrays(diffracto_delta, diffracto_gamma)¶
Computes TwoTheta and Psi angles arrays corresponding to repectively the vertical and the horizontal pixels.
Parameters
diffracto_delta: diffractometer value of the delta axis
diffracto_gamma: diffractometer value of the gamma axis
Note
This assume the detector is perfectly aligned with the delta and gamma axes (which should be the case).
experiment Module¶
- class pymicro.xray.experiment.ForwardSimulation(sim_type, verbose=False)¶
Bases:
objectClass to represent a Forward Simulation.
- __init__(sim_type, verbose=False)¶
- set_experiment(experiment)¶
Attach an X-ray experiment to this simulation.
- class pymicro.xray.experiment.XraySource(position=None)¶
Bases:
objectClass to represent a X-ray source.
- __init__(position=None)¶
- set_position(position)¶
- property min_energy¶
- property max_energy¶
- set_min_energy(min_energy)¶
- set_max_energy(max_energy)¶
- set_energy(energy)¶
Set the energy (monochromatic case).
- set_energy_range(min_energy, max_energy)¶
- discretize(diameter, n=5)¶
Discretize the focus zone of the source into a regular sphere.
- Parameters
diameter (float) – the diameter of the sphere to use.
n (int) – the number of point to use alongside the source diameter.
- Returns
a numpy array of size (n_inside, 3) with the xyz coordinates of the points inside teh sphere.
- class pymicro.xray.experiment.SlitsGeometry(position=None)¶
Bases:
objectClass to represent the geometry of a 4 blades slits.
- __init__(position=None)¶
- set_position(position)¶
- class pymicro.xray.experiment.ObjectGeometry(geo_type='point', origin=None)¶
Bases:
objectClass to represent any object geometry.
The geometry may have multiple form, including just a point, a regular 3D array or it may be described by a CAD file using the STL format. The array represents the material microstructure using the grain ids and should be set in accordance with an associated Microstructure instance. In this case, zero represent a non crystalline region.
- __init__(geo_type='point', origin=None)¶
- set_type(geo_type)¶
- set_array(grain_map, voxel_size)¶
- set_origin(origin)¶
- get_bounding_box()¶
- get_positions()¶
Return an array of the positions within this sample in world coordinates.
- discretize_geometry(grain_id=None)¶
Compute the positions of material points inside the sample.
A array of size (n_vox, 3) is returned where n_vox is the number of positions. The 3 values of each position contain the (x, y, z) coordinates in mm unit. If a grain id is specified, only the positions within this grain are returned.
This is useful in forward simulation where we need to access all the locations within the sample. Three cases are available:
point Laue diffraction: uses sample origin, grains center and orientation
cad Laue diffraction: uses origin, cad file geometry and grains[0] orientation (assumes only one grain)
grain map Laue diffraction: uses origin, array, size, and grains orientations.
to set the cad geometry must be the path to the STL file.
- class pymicro.xray.experiment.Sample(name=None, position=None, geo=None, material=None, microstructure=None)¶
Bases:
objectClass to describe a material sample.
A sample is made by a given material (that may have multiple phases), has a name and a position in the experimental local frame. A sample also has a geometry (just a point by default), that may be used to discretize the volume in space or display it in 3D.
Note
For the moment, the material is simply a crystal lattice.
- __init__(name=None, position=None, geo=None, material=None, microstructure=None)¶
- set_name(name)¶
Set the sample name.
- Parameters
name (str) – The sample name.
- set_position(position)¶
Set the sample reference position.
- Parameters
position (tuple) – A vector (tuple or array form) describing the sample position.
- set_geometry(geo)¶
Set the geometry of this sample.
- Parameters
geo (ObjectGeometry) – A vector (tuple or array form) describing the sample position.
- get_material()¶
- set_material(material)¶
- get_microstructure()¶
- set_microstructure(microstructure)¶
- has_grains()¶
Method to see if a sample has at least one grain in the microstructure.
- Returns
True if the sample has at east one grain, False otherwise.
- get_grain_ids()¶
- class pymicro.xray.experiment.Experiment¶
Bases:
objectClass to represent an actual or a virtual X-ray experiment.
A cartesian coordinate system (X, Y, Z) is associated with the experiment. By default X is the direction of X-rays and the sample is placed at the origin (0, 0, 0).
- __init__()¶
- set_sample(sample)¶
- get_sample()¶
- set_source(source)¶
- get_source()¶
- set_slits(slits)¶
- get_slits()¶
- add_detector(detector, set_as_active=True)¶
Add a detector to this experiment.
If this is the first detector, the active detector id is set accordingly.
- Parameters
detector (Detector2d) – an instance of the Detector2d class.
set_as_active (bool) – set this detector as active.
- get_number_of_detectors()¶
Return the number of detector for this experiment.
- get_active_detector()¶
Return the active detector for this experiment.
- forward_simulation(fs, set_result_to_detector=True)¶
Perform a forward simulation of the X-ray experiment onto the active detector.
This typically sets the detector.data field with the computed image.
- Parameters
set_result_to_detector (bool) – if True, the result is assigned to the current detector.
fs – An instance of ForwardSimulation or its derived class.
- save(file_path='experiment.txt')¶
Export the parameters to describe the current experiment to a file using json.
- static load(file_path='experiment.txt')¶
- class pymicro.xray.experiment.ExperimentEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)¶
Bases:
JSONEncoder- default(o)¶
Implement this method in a subclass such that it returns a serializable object for
o, or calls the base implementation (to raise aTypeError).For example, to support arbitrary iterators, you could implement default like this:
def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return JSONEncoder.default(self, o)
fitting Module¶
- The fitting module define some classes to easily perform 1D curve
fitting. This module supports fitting (x, y) data with a general mechanism. Any fitting function can be provided to the fit method and a few general purpose fuctions are predefined:
Gaussian
Lorentzian
Cosine
Voigt
Plot of the main predefined fitting function in the fitting module.¶
- pymicro.xray.fitting.fit(y, x=None, expression=None, nb_params=None, init=None)¶
Static method to perform curve fitting directly.
Parameters
y: the data to match (a 1d numpy array)
x: the corresponding x coordinates (optional, None by default)
expression: can be either a string to select a predefined function or alternatively a user defined function with the signature f(x, p) (in this case you must specify the length of the parameters array p via setting nb_params).
nb_params: the number of parameters of the user defined fitting function (only needed when a custom fitting function is provided, None by default)
init: a sequence (the length must be equal to the number of parameters of the fitting function) used to initialise the fitting function.
- For instance, to fit some (x,y) data with a gaussian function, simply use:
F = fit(y, x, expression='Gaussian')
Alternatively you may specify you own function directly defined with Python, like:
def myf(x, p): return p[0]*x + p[1] F = fit(y, x, expression=myf, nb_params=2)
- pymicro.xray.fitting.lin_reg(xi, yi)¶
Apply linear regression to a series of points.
This function return the best linear fit in the least square sense. :param ndarray xi: a 1D array of the x coordinate. :param ndarray yi: a 1D array of the y coordinate. :return tuple: the linear intercept, slope and correlation coefficient.
- class pymicro.xray.fitting.Parameter(value, name=None)¶
Bases:
objectA class to handle modiable parameters.
- __init__(value, name=None)¶
Create a new parameter with the given value.
- set(value)¶
Set the value of the parameter.
- set_name(name)¶
Set the name of the parameter.
- class pymicro.xray.fitting.FitFunction¶
Bases:
objectThis class provides a basic canvas to define a fit function.
You may subclass it to create your own fitting function just as the predfined fit function do (see Gaussian for instance).
- __init__()¶
- get_parameters()¶
Return the list of parameters of this fit function.
- get_parameter_names()¶
Return the list os parameter names of this fit function.
- add_parameter(value, name)¶
- compute(x)¶
Evaluate the fit function at coordinates x.
- fit(y, x=None, verbose=False)¶
Perform fitting on the given data.
This will adjust the parameters of this fit function to match as well as possible the given data using a least square minimisation.
Parameters
y: the data to match (a 1d numpy array)
x: the corresponding x coordinates (optional, None by default)
verbose: boolean, activate verbose mode
- class pymicro.xray.fitting.SumOfFitFunction(function_list)¶
Bases:
FitFunction- __init__(function_list)¶
- compute(x)¶
Evaluate the fit function at coordinates x.
- class pymicro.xray.fitting.Gaussian(position=0.0, sigma=1.0, height=1.0)¶
Bases:
FitFunctionfirst parameter is position, second is sigma, third is height
- __init__(position=0.0, sigma=1.0, height=1.0)¶
- set_position(position)¶
Set the position (center) of the gauss function.
- set_sigma(sigma)¶
Set the width (variance) of the gauss function.
- set_height(height)¶
Set the maximum (height) of the gauss function.
- fwhm()¶
Compute the full width at half maximum of the gauss function.
- class pymicro.xray.fitting.Lorentzian(position=0.0, gamma=1.0, height_factor=1.0)¶
Bases:
FitFunctionLorentzian funtion.
The first parameter is the position, the second is gamma. The maximum of the function is given by height_factor/(pi*gamma). The FWHM is just 2*gamma.
- __init__(position=0.0, gamma=1.0, height_factor=1.0)¶
- set_position(position)¶
- set_gamma(gamma)¶
- set_height(height)¶
Set the maximum (height) of the Lorentzian function. This actually set the height factor to the value height*pi*gamma.
- fwhm()¶
- class pymicro.xray.fitting.Cosine(position=0.0, width=1.0)¶
Bases:
FitFunctionfirst parameter is position, second is width
- __init__(position=0.0, width=1.0)¶
- set_position(position)¶
- set_width(a)¶
- fwhm()¶
- class pymicro.xray.fitting.Voigt(position=0.0, sigma=1.0, gamma=1.0, height_factor=1.0)¶
Bases:
FitFunctionThe Voigt function is also the real part of w(x) = exp(-x**2) erfc(ix), the Faddeeva function.
Here we use one of the popular implementation which is available in scipy with the wofz function.
- __init__(position=0.0, sigma=1.0, gamma=1.0, height_factor=1.0)¶
- set_position(position)¶
Set the position (center) of the Voigt function.
- set_sigma(sigma)¶
Set the sigma of the Voigt function.
- set_height(height)¶
Set the maximum (height) of the Voigt function. This actually set the height factor to the proper value. Be careful that if you change the other parameters (sigma, gamma) the maximum height will be changed.
- fwhm()¶
Compute the full width at half maximum of the Voigt function.
The height factor does not change the fwhm. The fwhm can be evaluated by the width of the associated Gaussian and Lorentzian functions. The fwhm is approximated by the equation from J. Olivero and R. Longbothum [1977]
laue Module¶
The laue module provide helpers functions to work with polychromatic X-ray diffraction.
- pymicro.xray.laue.select_lambda(hkl, orientation, Xu=(1.0, 0.0, 0.0), verbose=False)¶
Compute the wavelength corresponding to the first order reflection of a given lattice plane in the specified orientation.
- Parameters
hkl – The given lattice plane.
orientation – The orientation of the crystal lattice.
Xu – The unit vector of the incident X-ray beam (default along the X-axis).
verbose (bool) – activate verbose mode (default False).
- Returns tuple
A tuple of the wavelength value (keV) and the corresponding Bragg angle (radian).
- pymicro.xray.laue.build_list(lattice=None, max_miller=3, extinction=None, Laue_extinction=False, max_keV=120.0)¶
- pymicro.xray.laue.compute_ellipsis(orientation, detector, uvw, Xu=(1.0, 0.0, 0.0), n=101, verbose=False)¶
Compute the ellipsis associated with the given zone axis.
The detector is supposed to be normal to the X-axis, but the incident wave vector can be specified. All lattice planes sharing this zone axis will diffract along that ellipse.
- Parameters
orientation – The crystal orientation.
detector – An instance of the Detector2D class.
uvw – An instance of the HklDirection representing the zone axis.
Xu – The unit vector of the incident X-ray beam (default along the X-axis).
n (int) – number of poits used to define the ellipse.
verbose (bool) – activate verbose mode (default False).
- Returns data
the (Y, Z) data (in mm unit) representing the ellipsis.
- pymicro.xray.laue.diffracted_vector(hkl, orientation, Xu=(1.0, 0.0, 0.0), min_theta=0.1, use_friedel_pair=True, verbose=False)¶
Compute the diffraction vector for a given reflection of a crystal.
This method compute the diffraction vector. The incident wave vector is along the X axis by default but can be changed by specifying any unit vector.
- Parameters
hkl – an instance of the HklPlane class.
orientation – an instance of the Orientation class.
Xu – The unit vector of the incident X-ray beam (default along the X-axis).
min_theta (float) – the minimum considered Bragg angle (in radian).
use_friedel_pair (bool) – also consider the Friedel pairs of the lattice plane.
verbose (bool) – a flag to activate verbose mode.
- Returns
the diffraction vector as a numpy array.
- pymicro.xray.laue.diffracted_intensity(hkl, I0=1.0, symbol='Ni', verbose=False)¶
Compute the diffracted intensity.
This compute a number representing the diffracted intensity within the kinematical approximation. This number includes many correction factors:
atomic factor
structural factor
polarisation factor
Debye-Waller factor
absorption factor
…
- Parameters
hkl (HklPlane) – the hkl Bragg reflection.
I0 (float) – intensity of the incident beam.
symbol (str) – string representing the considered lattice.
verbose (bool) – flag to activate verbose mode.
- Returns
the diffracted intensity.
- Return type
float.
- pymicro.xray.laue.compute_Laue_pattern(orientation, detector, hkl_planes=None, Xu=<Mock name='mock.array()' id='139894746709968'>, use_friedel_pair=False, spectrum=None, spectrum_thr=0.0, r_spot=5, color_field='constant', inverted=False, show_direct_beam=False, verbose=False)¶
Compute a transmission Laue pattern. The data array of the given Detector2d instance is initialized with the result.
The incident beam is assumed to be along the X axis: (1, 0, 0) but can be changed to any direction. The crystal can have any orientation using an instance of the Orientation class. The Detector2d instance holds all the geometry (detector size and position).
A parameter controls the meaning of the values in the diffraction spots in the image. It can be just a constant value, the diffracted beam energy (in keV) or the intensity as computed by the
diffracted_intensity()method.- Parameters
orientation – The crystal orientation.
detector – An instance of the Detector2d class.
hkl_planes (list) – A list of the lattice planes to include in the pattern.
Xu – The unit vector of the incident X-ray beam (default along the X-axis).
use_friedel_pair (bool) – also consider the Friedel pair of each lattice plane in the list as candidate for diffraction.
spectrum – A two columns array of the spectrum to use for the calculation.
spectrum_thr (float) – The threshold to use to determine if a wave length is contributing or not.
r_spot (int) – Size of the spots on the detector in pixel (5 by default)
color_field (str) – a traing describing, must be ‘constant’, ‘energy’ or ‘intensity’
inverted (bool) – A flag to control if the pattern needs to be inverted.
show_direct_beam (bool) – A flag to control if the direct beam is shown.
verbose (bool) – activate verbose mode (False by default).
- Returns
the computed pattern as a numpy array.
- pymicro.xray.laue.gnomonic_projection_point(data, OC=None)¶
compute the gnomonic projection of a given point or series of points in the general case.
This methods does not assumes the incident X-ray beam is along (1, 0, 0). This is accounted for with the parameter OC which indicates the center of the projection (the incident beam intersection with the detector). A conditional treatment is done as the projection is is faster to compute in the case of normal incidence.
The points coordinates are passed along with a single array which must be of size (n, 3) where n is the number of points. If a single point is used, the data can indifferently be of size (1, 3) or (3).
- Parameters
data (ndarray) – array of the point(s) coordinates in the laboratory frame, aka OR components.
OC (ndarray) – coordinates of the center of the gnomonic projection in the laboratory frame.
- Return data_gp
array of the projected point(s) coordinates in the laboratory frame.
- pymicro.xray.laue.gnomonic_projection(detector, pixel_size=None, OC=None, verbose=False)¶
This function carries out the gnomonic projection of the detector image.
The data must be of uint8 type (between 0 and 255) with diffraction spots equals to 255. The function create a new detector instance (think of it as a virtual detector) located at the same position as the given detector and with an inverse pixel size. The gnomonic projection is stored into this new detector data. The gnomonic projection of each white pixel (value at 255) is computed. The projection is carried out with respect to the center detector (ucen, vcen) point.
- Parameters
detector (RegArrayDetector2d) – the detector instance with the data from which to compute the projection.
pixel_size (float) – pixel size to use in the virtual detector for the gnomonic projection.
OC (tuple) – coordinates of the center of the gnomonic projection in the laboratory frame.
verbose (bool) – flag to activate verbose mode.
- Returns RegArrayDetector2d gnom
A virtual detector with the gnomonic projection as its data.
- pymicro.xray.laue.identify_hkl_from_list(hkl_list)¶
- pymicro.xray.laue.triplet_indexing(OP, angles_exp, angles_th, tol=1.0, verbose=False)¶
evaluate each triplet composed by 3 diffracted points. the total number of triplet is given by the binomial coefficient (n, 3) = n*(n-1)*(n-2)/6
- pymicro.xray.laue.transformation_matrix(hkl_plane_1, hkl_plane_2, xyz_normal_1, xyz_normal_2)¶
- pymicro.xray.laue.confidence_index(votes)¶
- pymicro.xray.laue.poll_system(g_list, dis_tol=1.0, verbose=False)¶
Poll system to sort a series of orientation matrices determined by the indexation procedure.
For each orientation matrix, check if it corresponds to an existing solution, if so: vote for it, if not add a new solution to the list :param list g_list: the list of orientation matrices (should be in the fz) :param float dis_tol: angular tolerance (degrees) :param bool verbose: activate verbose mode (False by default) :return: a tuple composed by the most popular orientation matrix, the corresponding vote number and the confidence index
- pymicro.xray.laue.index(hkl_normals, hkl_planes, tol_angle=0.5, tol_disorientation=1.0, symmetry=Symmetry.cubic, display=False)¶
- pymicro.xray.laue.zone_axis_list(angle, orientation, lattice, max_miller=5, Xu=<Mock name='mock.array()' id='139894746709968'>, verbose=False)¶
This function allows to get easily the Miller indices of zone axis present in a pattern.
- Parameters
angle (float list) – the angle max or the zone axis angle range admissible around the detector center.
orientation – The orientation of the crystal lattice.
lattice – The corresponding crystal lattice, instance of Lattice.
max_miller (int) – Maximal value allowed of Miller indices direction.
Xu (array) – The unit vector of the incident X-ray beam (default along the X-axis).
verbose (bool) – activate verbose mode (default False).
- Returns
A list of HklDirection instance of all zone axis in the angle range.
- pymicro.xray.laue.get_gnomonic_edges(detector, gnom, OC=None, num_points=21)¶
This function allows to get the blind area of the gnomonic projection.
- Parameters
detector (RegArrayDetector2d) – the detector instance with the data from which to compute the projection.
gnom (RegArrayDetector2d) – A virtual detector with the gnomonic projection as its data.
OC (ndarray) – coordinates of the center of the gnomonic projection in the laboratory frame.
num_points (int) – number of points to describe an edge (minimum 2)
- Returns
ndarray of gnomonic blind area edges coordinates.
- pymicro.xray.laue.diffracting_normals_vector(gnom)¶
This function allows to get easily the diffracting normal vector from the gnomonic projection images.
- Parameters
gnom (RegArrayDetector2d) – A virtual detector with the gnomonic projection as its data.
- Returns
Normalized normal vector of diffracting plane
- class pymicro.xray.laue.LaueForwardSimulation(verbose=False)¶
Bases:
ForwardSimulationClass to represent a Forward Simulation.
- __init__(verbose=False)¶
- set_experiment(experiment)¶
Attach an X-ray experiment to this simulation.
- set_use_energy_limits(use_energy_limits)¶
Activate or deactivate the use of energy limits.
- set_hkl_planes(hkl_planes)¶
- setup(include_grains=None)¶
Setup the forward simulation.
- static fsim_laue(orientation, hkl_planes, positions, source_position)¶
Simulate Laue diffraction conditions based on a crystal orientation, a set of lattice planes and physical positions.
This function is the work horse of the forward model. It uses a set of HklPlane instances and the voxel coordinates to compute the diffraction quantities.
- Parameters
orientation (Orientation) – the crystal orientation.
hkl_planes (list) – a list of HklPlane instances.
positions (list) – a list of (x, y, z) positions.
source_position (tuple) – a (x, y, z) tuple describing the source position.
- fsim_grain(gid=1)¶
- fsim()¶
run the forward simulation.
If the sample has a CAD type of geometry, a single grain (the first from the list) is assumed. In the other cases all the grains from the microstructure are used. In particular, if the microstructure has a grain map, it can be used to carry out an extended sample simulation.
xray_utils Module¶
- class pymicro.xray.xray_utils.Element(name, symbol, density)¶
Bases:
objectA class to represent a chemical element described by its name, symbol and density.
- __init__(name, symbol, density)¶
- property name¶
Returns the name of the Element.
- property symbol¶
Returns the symbol of the Element.
- property density¶
Returns the density of the Element.
- pymicro.xray.xray_utils.density_from_Z(Z)¶
- pymicro.xray.xray_utils.density_from_symbol(symbol)¶
- pymicro.xray.xray_utils.f_atom(q, Z)¶
Empirical function for the atomic form factor f.
This function implements the empirical function from the paper of Muhammad and Lee to provide value of f for elements in the range Z <= 30. doi:10.1371/journal.pone.0069608.t001
- Parameters
q (float) – value or series for the momentum transfer, unit is angstrom^-1
Z (int) – atomic number, must be lower than 30.
- Returns
the atomic form factor value or series corresponding to q.
- pymicro.xray.xray_utils.atom_scat_factor_function(mat='Al', sintheta_lambda_max=12, display=True)¶
Compute and display the fit function of the atomic scattering factor.
- Parameters
mat (string) – A string representing the material (e.g. ‘Al’)
sintheta_lambda_max (float) – maximal value of sin theta / lambda
display (bool) – display an image of the plot
- pymicro.xray.xray_utils.lambda_keV_to_nm(lambda_keV)¶
Change the unit of wavelength from keV to nm.
- Parameters
lambda_keV (float) – the wavelength in keV unit.
- Returns
the wavelength in nm unit.
- pymicro.xray.xray_utils.lambda_keV_to_angstrom(lambda_keV)¶
Change the unit of wavelength from keV to angstrom.
- Parameters
lambda_keV (float) – the wavelength in keV unit.
- Returns
the wavelength in angstrom unit.
- pymicro.xray.xray_utils.lambda_nm_to_keV(lambda_nm)¶
Change the unit of wavelength from nm to keV.
- Parameters
lambda_nm (float) – the wavelength in nm unit.
- Returns
the wavelength in keV unit.
- pymicro.xray.xray_utils.lambda_angstrom_to_keV(lambda_angstrom)¶
Change the unit of wavelength from angstrom to keV.
- Parameters
lambda_angstrom (float) – the wavelength in angstrom unit.
- Returns
the wavelength in keV unit.
- pymicro.xray.xray_utils.plot_xray_trans(mat='Al', ts=[1.0], rho=None, energy_lim=[1, 100], legfmt='%.1f', display=True)¶
Plot the transmitted intensity of a X-ray beam through a given material.
This function compute the transmitted intensity from tabulated data of the mass attenuation coefficient mu_
- ho (between 1 and 100 keV) and
applying Beer’s Lambert law:
\[I/I_0 = \exp(-\mu_\]
ho* ho*t)
The tabulated data is stored in ascii files in the data folder. It has been retrieved from NIST http://physics.nist.gov/cgi-bin/ffast/ffast.pl The density is also tabulated and can be left blanked unless a specific value is to be used.
- param string mat
a string representing the material (must be the atomic symbol if the density is not specified, e.g. ‘Al’)
- param list ts
a list of thickness values of the material in mm ([1.0] by default)
- param float rho
density of the material in g/cm^3 (None by default)
- param list energy_lim
energy bounds in keV for the plot (1, 100 by default)
- param string legfmt
string to format the legend plot
- param bool display
display or save an image of the plot (False by default)
- pymicro.xray.xray_utils.radiograph(data, omega)¶
Compute a single radiograph of a 3D object using the radon transform.
- Parameters
data (np.array) – an array representing the 3D object in (XYZ) form.
omega – the rotation angle value in degrees.
- Returns projection
a 2D array in (Y, Z) form.
- pymicro.xray.xray_utils.radiographs(data, omegas)¶
Compute the radiographs of a 3D object using the radon transform.
The object is represented by a 3D numpy array in (XYZ) form and a series of projection at each omega angle are computed assuming the rotation is along Z in the middle of the data set. Internally this function uses the radon transform from the skimage package.
- Parameters
data (np.array) – an array representing the 3D object in (XYZ) form.
omegas – an array of the rotation values in degrees.
- Returns projections
a 3D array in (Y, Z, omega) form.
Change log history¶
pymicro version 0.5.3¶
formatting according to PEP8 (Henry Proudhon)
Merge branch ‘master’ into devel (Henry Proudhon)
handle the case where the formula in empty when reading ang files (Henry Proudhon)
new test for the ebsd.py module (Henry Proudhon)
make sure we have at least one phase for grain segmentation (Henry Proudhon)
fix the name of the testing class (Henry Proudhon)
simplify try/except blocks (Henry Proudhon)
added new function to import EBSD data from CTF files (Henry Proudhon)
new batch eu2ro method, added non empty grain_map check in remove_small_grains (Henry Proudhon)
new function compute_mean_orientation (Henry Proudhon)
equation for the m factor includes abs (Henry Proudhon)
new test for the compute_m_factor function (Henry Proudhon)
new fonction to compute the m factor (Henry Proudhon)
improved docstrings (Henry Proudhon)
speed up function find_neighbors by using the grain bounding box (Henry Proudhon)
allow to pass a list of HklPlanes to this function (Henry Proudhon)
improve docstrings (Henry Proudhon)
Merge pull request #18 from heprom/basile (Basile Marchand)
Example update (bmarchand)
debugged get xdmf field for time collection grids (Aldo Marano)
Debug tests, updated documentation for new Microstructure class after_file_open method (Aldo Marano)
SampleData object constructor has a new dictionary argument aimed at containing arugments for the after_file_open method (Aldo Marano)
new Euler_rad2Rodrigues method (Henry Proudhon)
Removed automatic building of SampleData User Guide. Notebook are stored with output cells filled again (Aldo Marano)
solved kernel issue (Aldo Marano)
attempt to solve documentation autobuild with absolute pathes to import Pymicro module in Notebooks (Aldo Marano)
new kernel change for doc notebooks (Aldo Marano)
removed kernel changed in Notebook documentation building (Aldo Marano)
new attempt to solve readthedocs/nbsphinx autobuild of Notebooks (Aldo Marano)
Solved some documentation building warnings (Aldo Marano)
Changed SampleData User Guide Notebooks kernel to allow online building (Aldo Marano)
Documentation modified to have automatic execution of Notebooks when building the SampleData User Guide (Aldo Marano)
Microstructure class and SampleData derived classes documentation (Aldo Marano)
fix a bug with symmetry to compute the disorientation (Henry Proudhon)
minor update of test_derived_class in pymicro.core.tests (Aldo Marano)
light additions to SampleData unit test “test_derived_class” (Aldo Marano)
Improved data model handling for derived classes (Aldo Marano)
new example to load EBSD dat from .osc file (Henry Proudhon)
add a test for the Quaternion class (Henry Proudhon)
Documentation update, new example file for tests and doc (Aldo Marano)
new method read_osc (Henry Proudhon)
Remove __main__ test in examples and fix some py2/py3 print syntax error (bmarchand)
First draft for gitlab-ci pipeline (bmarchand)
Update python version to 3.6 in setup.py (bmarchand)
Add docstring (bmarchand)
make sure the scalar part of a Quaternion created from Euler angles is positive (Henry Proudhon)
Test example with data, use pymicro.example.PYMICRO_EXAMPLES_DATA_DIR variable (bmarchand)
Update setup.py to include examples in distribution as pymicro.examples module (bmarchand)
Add conda environment file (bmarchand)
expose more parameters for ebsd grain segmentation (Henry Proudhon)
update path for libUmatAmitex.so for grips and exterior too (Henry Proudhon)
update Ti_ebsd_demo_data.h5 (Henry Proudhon)
small update on printing out progress evolution (Henry Proudhon)
new parameter elasaniso_path for method to_amitex_fftp (Henry Proudhon)
make all unit tests pass (Aldo Marano)
General code improvement (Aldo Marano)
added 5 tutorials of the SampleData User Guide uploaded + update of documentation building files (Aldo Marano)
Added new reference files for tutorials/docs and tests (Aldo Marano)
minor adjustments for the requirements (Henry Proudhon)
updated documentation (Henry Proudhon)
new example with EBSD microstructure (Henry Proudhon)
add a new display argument in view_slice method (Henry Proudhon)
fix radian to degree conversion in change_orientation_reference_frame (Henry Proudhon)
small formatting changes (Henry Proudhon)
added slip traces vizualisation to the view_slice method (Henry Proudhon)
new method to change the orientation reference frame (Henry Proudhon)
improved doctrings (Henry Proudhon)
First attempt to introduce nbsphinx in pymicro’s autobuild, to make pymicro’s documentation from jupyter Notebooks.
Debugued auto mesh scaling in automatic meshers (Aldo Marano)
New initialization method _after_file_open, called when a hdf5 file is opened by the SampleData class (or inherited class) (Aldo Marano)
Minor debug (Aldo Marano)
Cleaned a few # TODO already done (Aldo Marano)
New module SDGridUtils: classes to interact and apply operations on Grids (Images and Meshes) (Aldo Marano)
Debugued elset and nodeset compression and added Skip phase ID 0 to Image mesher (Aldo Marano)
debuged ‘add_grid_time’ –> Now OK – Microstructure method ‘from_amitex_fftp’ now loads Amitex output fields as temporal series (Aldo Marano)
Implemented handling of time series attributes for XDMF grids (fields with values through time) (Aldo Marano)
Implementation of dictionary and attribute like access for SampleData datasets items (Aldo Marano)
Improved SampleData print methods (Aldo Marano)
Creation of a autorepack flag that controls activation of hdf5 file repacking in SampleData class destructor (Aldo Marano)
Added data normalization option for data compression (Aldo Marano)
Pass compression settings for all SampleData methods (Aldo Marano)
Debuged XDMF spacing and origin ordering (Aldo Marano)
Adapted Microstructure.from_amitex_fftp to new methods in SDAmitexUtils (Aldo Marano)
Added loading of finite strain stress/strain and internal variables fields to load_amitex_output_fields method (Aldo Marano)
Implemented loading of .std amitex output files for finite strain simulations (Aldo Marano)
implemented new definitive ordering convention of 2nd order tensors in vector notation (Aldo Marano)
Moved Amitex input/ouput utility methods from pymicro.crystal.microstructure to new module pymicro.core.utils.SDAmitexUtils (Aldo Marano)
Added .ut file argument for SDZset load_output methods (Aldo Marano)
adjust __repr__ method for the Lattice class (Henry Proudhon)
convert lattice constants to nm when importing an EBSD scan (Henry Proudhon)
improved the speed of the create_IPF_map method by using the grain bounding boxes (Henry Proudhon)
improved __repr__methods (Henry Proudhon)
Modified examples SampleData files to adapt tests to new data model (Aldo Marano)
Minor corrections (Aldo Marano)
Merge branch ‘AM_Zset_utils’ into master_test (Aldo Marano)
Orientation repr correction (Aldo Marano)
Added method to compute orthotropic elastic moduli (the 3 Young, 3 Poisson and 3 Shear moduli) from stiffness matrix in Lattice class (Aldo Marano)
fix typos in add_IPF_maps (Henry Proudhon)
new method add_IPF_maps to create and store all 3 IPF maps at once (Henry Proudhon)
data files modified after introducing phaes in Microstructure (Henry Proudhon)
add a new method to compute ipf coloring (Henry Proudhon)
add pyramidal1 <a> slip systems (Henry Proudhon)
fix miller indices in the test_4indices_representation (Henry Proudhon)
new option to use a mask for the tt_rock function (Henry Proudhon)
fixed a small issue in unit_arrow_3d (Henry Proudhon)
now create a default hexagonal lattice when using the 4 indices notation (Henry Proudhon)
add method move_vector_to_FZ to the Symmetry class (Henry Proudhon)
add more cases to test the Miller-Bravais conversion for the hexagonal lattice (Henry Proudhon)
Moved storaged of mesh element/node tags and grid field indexes to String arrays (Aldo Marano)
Added a new data structure for SampleData objects (Aldo Marano)
Adapted to_amitex_fftp to produce Amitex vtk input files from Microstructure ‘mask’ (Aldo Marano)
added generic methods in Microstructure class to read Amitex_fftp output files (Aldo Marano)
Small corrections in lattice and Microstructure class, introduced new method -from_amitex_fftp’ (Aldo Marano)
Bug correction in SampleData method ‘get_description’ (Aldo Marano)
corrected typo after refactoring (Henry Proudhon)
added method compute_ipf_maps (Henry Proudhon)
new method to read EBSD scan from .ang files (Henry Proudhon)
Fixed bug with empty node tags names where stored in node_tag_list (Aldo Marano)
new method get_slip_systems for the Lattice class (Henry Proudhon)
added new option show_grain_ids in the view_slice method (Henry Proudhon)
Merge branch ‘master’ into AM_utils_tmp (Aldo Marano)
Added some control options to Microstructure crop and mesh_grain_map (Aldo Marano)
Added output control for SDMeshers and auto-mesh resize from Image dimensions (Aldo Marano)
Small debugs for pathes in SampleData (Aldo Marano)
improve how phases are handled in Microstructure (Henry Proudhon)
add method set_lattice to CrystallinePhase (Henry Proudhon)
transpose volume data in from_labdct (Henry Proudhon)
added method to remove small grains with a volume threshold (Aldo Marano)
Correction of Microstructure.crop method to crop all fields in CellData group (Aldo Marano)
new from_labdct method (Henry Proudhon)
new test to add phases (Henry Proudhon)
the Microstructure class now has material phases (Henry Proudhon)
OimPhase now extends CrystallinePhase (Henry Proudhon)
new method add_slip_system_to_grain (Henry Proudhon)
new tt_rock method (Henry Proudhon)
add CrystallinePhase tests (Henry Proudhon)
continues implementing the CrystallinePhase class (Henry Proudhon)
Added clean grain map method to Microstructure class (Aldo Marano)
Adapted Microstructure ‘from_ebsd’ to latest changes in SampleData and Microstructure classes (Aldo Marano)
Add print of std output for morphological image cleaner (Aldo Marano)
minor SampleData print method debug (Aldo Marano)
Correct matlab ‘addpath’ command generation for automatic meshing tools (Aldo Marano)
light modifications for F. Nguyen meshing tools integration (Aldo Marano)
logo adjustment (Henry Proudhon)
removed blank spaces at endlines, solved a bug when having a None name entering SampleData._name_or_node_to_path (Aldo Marano)
Complete Integration of F. Nguyen Matlab/Zset image meshing tools in SDMeshers class (Aldo Marano)
new pymicro logo (Henry Proudhon)
start to add support for multi-phase materials (Henry Proudhon)
removed non ascii character in compute_grain_volume (Henry Proudhon)
remove non ascii character in create_orientation_map (Henry Proudhon)
Add automatic transposition of tensor fields components in hdf5 arrays to match Paraview/XDMF ordering convention (Aldo Marano)
Correction of field component names when writing Zset output from SampleData (Aldo Marano)
Bug solving: mesh field padding ignored when using add_field on Image groups (Aldo Marano)
Add methods to write SampleData Image groups as Zset outputs of fields defined on a regular hexaedral mesh (Aldo Marano)
Bug solving: mesh field padding ignored when using add_field on Image groups (Aldo Marano)
updated change log for version 0.5.2 (Henry Proudhon)
Bug corrections (print_index) (Aldo Marano)
SDZset : * method to read Zset output fields with automatic construction of vector and tensor fields (Aldo Marano)
Bug corrections (print_index) (Aldo Marano)
Minor debugs (Aldo Marano)
Added method to write Zset output from SampleData mesh groups and fields datasets (Aldo Marano)
Handling of nodes and elements IDs for returning basictools mesh objects (Aldo Marano)
Add computation of nodal normal fields from element wise normal fields for SampleData mesh groups (Aldo Marano)
New methods for mesh elements and elements tag manipulation (Aldo Marano)
Added a new class to handle Zset post_processing .inp blocks, based on SDZset (parent class) (Aldo Marano)
deform_mesh Zset mesher command interface implemented in SDZsetMesher (Aldo Marano)
Zset command options passing modification
New class SDZsetFieldTransfer to serve as interface between SampleData and Zset transfer_fields scripts (Aldo Marano)
Improved string templates handling in SDZsetUtils
New organization of core.utils.Zsetutils (Aldo Marano)
a few minor changes (Henry Proudhon)
SampleData class update with mesh field padding implementation (Aldo Marano)
Add method to reinitialize mesher commands (SDZsetMesher) (Aldo Marano)
added squeeze for the ids_to_blank array and improved docstrings (Henry Proudhon)
Handling of element wise fields who are defined only on bulk
Creation of multiple SDZsetMesher class methods to use Zset mesher commands (Aldo Marano)
SDZsetMesher class : Add safety check to ensure all mesher template arguments are set before runing mesher script (Aldo Marano)
Add mesh script and files cleaning methods in SDZsetMesher class
First minimal implementation of a working automatic Zset mesher
Debuged Sdmeshers and ScriptTemplate (Aldo Marano)
Creation of a dedicated class to handle script templates ScriptTemplate (Aldo Marano)
Addition of core subpackage utils to implement interfaces between SampleData objects and external tools (Aldo Marano)
pymicro version 0.5.2¶
updated version number to 0.5.2 (Henry Proudhon)
add methods to compute grain aspect ratios and sphericities (Henry Proudhon)
removed unnecessary print statement (Henry Proudhon)
new test for get_grain properties (Henry Proudhon)
fix all the get_grain properties methods, added docstrings (Henry Proudhon)
new method to build a condition to filter the grain data table based on a list of grain ids (Henry Proudhon)
Handling of nodes and elements IDs for returning basictools mesh objects (Aldo Marano)
Travis build bug solving (Aldo Marano)
Change one AssertEqual to AssertAlmostEqual to solve Travis build issues in test_Lattice.py (Aldo Marano)
Debuged test BasicTools_binding in core.tests package (Aldo Marano)
Merge pull request #12 from basileMarchand/master (Aldo Marano)
Disable system_site_packages in travis (because of the previous python version update) (Basile MARCHAND)
Update travis to use python 3.6 instead of python 3.5 neadeed for basic tools dependency (Basile MARCHAND)
Update setup.py and requirements.txt in order to install all required dependency. (Basile MARCHAND)
axes label modification allowed (Alexiane Arnaud)
change the way grain dilation is made replacing min by the most frequent value within the neighbors (Henry Proudhon)
debug print_dataset_content method, add new method to change indexnames in the dataset (set_new_indexname), new field index naming convention: if no indexname is provided indexname for fields is ‘grid_indexname_fieldname’ (Aldo Marano)
updated example for recent changes, fixes #10 (Henry Proudhon)
replaced is not by != to test the length of the elastic_constants list (Henry Proudhon)
SampleData class update -> Mesh field padding implementation to allow field shape compliance with mesh elements number (Aldo Marano)
Merge branch ‘AM_SD_grids’ (Aldo Marano) - Handling of element wise fields who are defined only on bulk (same dimensionality than the mesh) or boundary elements (dimensionality lower than the mesh), bulk and boundary elements count and element wise fields padding (Aldo Marano) - Solved issue with Mesh Geometry groups removal when removing recursively a Mesh Group with SampleData class (Aldo Marano)
Element and Node tags loading is now optional in get_mesh (Aldo Marano)
merged changes from last commit (Henry Proudhon)
fine tune to_amitex method (Henry Proudhon)
new method to create a VTK colormap from pyplot (Henry Proudhon)
more default values to init CrystallinePhase (Henry Proudhon)
added option overwrite_hdf5=True (Henry Proudhon)
corrections to remove documentation build warnings (Aldo Marano)
Merge branch ‘AM_SD_grids’ (Aldo Marano)
solved bug for get_element method
add_mesh and add_image now return mesh and image BasicTools objects (Aldo Marano)
Debuged connectivity shape and nature (numpy table option required to write .geof file) in get_mesh (Aldo Marano)
Add compression of elset indicator fields as defaut setting (1s and 0s => very strong compression ratios and no issue with Paraview visualization) (Aldo Marano)
Add option to dilate_grains, to automatically dilate until ID 0 is removed from grain map (Aldo Marano)
Debug to get passed nosetests –> OK (Aldo Marano)
Changed sample data nosetests reference file due to non retrocompatibility of recent developments (Aldo Marano)
Documentation update (docstrings), Microstructure Class update: renumbering grains consistant with grain map -> methods to recompute grain data table from grain map -> methods to synchronize grain data table with grain map (Aldo Marano)
Solved xdmf node removal bug (Aldo Marano)
Splitting of morphological cleaning and meshing methods (Aldo Marano)
updated example to python3 (Henry Proudhon)
updated change log for version 0.5.1 (Henry Proudhon)
Merge branch ‘master’ into AM_SD_grids (Aldo Marano)
Documentation build correction for Read the Docs online doc (Aldo Marano)
Debug to get passed the nosetests (Aldo Marano)
minor correction (Aldo Marano)
Multi_phase_mesher –> include storage of surface mesh and cleaned multi phase image (Aldo Marano)
Implementation of XDMF Sets format for elements and node sets (Aldo Marano)
SampleData automatic multiphase mesher V0 implementation (Aldo Marano)
Add method to create a grainID or orientation FEM field (resp. scalar/vector). Add ‘is_mesh’ SampleData private method (Aldo Marano)
Utility methods implemented to add a data array or a mesh from a file, create an orientation map in the Microstucture class from grain map and orientation data in GrainDataTable (Aldo Marano)
Add image automatic transposition whith add_field, and get_node, to get (X, Y, Z, Dim) indexing consistent with Paraview rendering (Aldo Marano)
Increased functionalities of print methods for conveniance (Aldo Marano)
New methods to convert an image group into a mesh group, with nodal Field transfer at least (Aldo Marano)
SampleData dev: integration with BasicTools mesh objects -> corrected bugs in nodesID and element tag indexing to allow from geof to geof (Aldo Marano)
SampleData dev: BasicTools mesh object integration (Aldo Marano)
Changed mesh data model and mesh_object support (Aldo Marano)
Debuged add_table_col consequences on table update at SD file opening (Aldo Marano)
Implemented possibility to add new columns to tables in dataset already existing structured storage tables (Aldo Marano)
SampleData dev: Solved initialization bugs with new method add_XXX methods implementation (Aldo Marano)
Moved to BasicTools ConstantRectilinearMesh object as Image based data object (Aldo Marano)
Solved a documentation construction bug, method pause_for_visualization options to open dataset with Paraview and / or Vitables (Aldo Marano)
pymicro version 0.5.1¶
updated version number to 0.5.1 (Henry Proudhon)
use Euler2Rodrigues directly in add_grains method (Henry Proudhon)
corrected examples according to recent changes in the Microstructure class (Henry Proudhon)
Add bibliography file to sphinxcontrib bibtex configuration (Aldo Marano)
Debug documentation to remove warnings and allow Read the Docs automatic doc building (Aldo Marano)
Add bibliography file to sphinxcontrib bibtex configuration (Aldo Marano)
Attempt to solve Travis building problems n2 (Aldo Marano)
Debug for travis build (Aldo Marano)
change the id used to pad arrays when preparing for amitex (Henry Proudhon)
updated test files after changes in the microstructure geometry calculations (Henry Proudhon)
updated the calculation of grain center to place the first voxel at (0.5, 0.5, 0.5) (Henry Proudhon)
Merge pull request #7 from basileMarchand/master (Henry Proudhon)
new method stiffness_matrix for the Symmetry class and new CrystallinePhase class (Henry Proudhon)
Update .travis.yml (Basile Marchand)
Update .travis.yml (Basile Marchand)
Update .travis.yml (Basile Marchand)
Update travis config for hdf5 (Basile MARCHAND)
updated change log for version 0.5.0 (Henry Proudhon)
pymicro version 0.5.0¶
updated version number to 0.5.0 (Henry Proudhon)
add element type c3d8r (Henry Proudhon)
new methods to transform vector/matrix to and from crystal/sample frames (Henry Proudhon)
bug fix in compute_grain_center method (Henry Proudhon)
new test for the renumber_grains function (Henry Proudhon)
new function to renumber the grains consecutively (Henry Proudhon)
try fixing hdf5 build error in travis CI (Henry Proudhon)
mark compute_grains_geometry as needing work (Henry Proudhon)
change how we recompute the grain geometries (Henry Proudhon)
fixed issue in example data file (Henry Proudhon)
improve performances while calculating grain geometry by using bounding boxes (Henry Proudhon)
fixed a performance issue with recompute_grain_bounding_boxes (Henry Proudhon)
fix the id of renumbered grains in merge_mirostructures (Henry Proudhon)
add test for merge_microstructures method (Henry Proudhon)
added test for the crop method (Henry Proudhon)
added a crop_name parameter to the crop method (Henry Proudhon)
a few tweaks to the merge_mirostructures method (Henry Proudhon)
add a few print statements to merge_microstructures (Henry Proudhon)
updated the from_grain_file method with the new SampleData organisation (Henry Proudhon)
fix access to old name attribute (Henry Proudhon)
recompute the geometry of the grains after the merge (Henry Proudhon)
follow up on merge_microstructures (Henry Proudhon)
fixing method merge_microstructures for the new sampleData organisation (Henry Proudhon)
fix the crop method in the Microstructure class (Henry Proudhon)
missing spaces (Henry Proudhon)
fix unit in compute_grain_volume (Henry Proudhon)
new IPF example to plot crystal rotation (Henry Proudhon)
additional fixes in set_map_field and plot (Henry Proudhon)
updated pole figure examples for the texture module changes (Henry Proudhon)
updated docstring (Henry Proudhon)
fixed set_map_field with the new Microstructure organization (Henry Proudhon)
new test for add_grains method (Henry Proudhon)
updated code to the new Mirostructure organisation inherited from SampleData (Henry Proudhon)
new function to add a list of grains to the microstructure (Henry Proudhon)
improved function to_amitex_fftp (Henry Proudhon)
Solve non-regression test error (Aldo Marano)
Bug introduced in the last commit solved –> non regression test OK (Aldo Marano)
improved from_indices method in the SlipSystem class (Henry Proudhon)
new parameters in to_amitex_fftp to add a buffer layer (Henry Proudhon)
(SampleData dev) Externalization of global variables of the core package in a specific module (Aldo Marano)
fixed grain map test in recompute_grain_bounding_boxes (Henry Proudhon)
added slip systems for hexagonal lattice (Henry Proudhon)
new method from_ebsd (Henry Proudhon)
reshape grain_map and mask to 3D if only 2D (Henry Proudhon)
new segment_grains method, transposed results to match pymicro’s convention, various improvements (Henry Proudhon)
Revert “Revert “new files for new test in Microstructure class”” (Alexiane Arnaud)
added overwrite_hdf5=True when creating microstructures from existing data (Henry Proudhon)
Revert “new files for new test in Microstructure class” (Alexiane Arnaud)
allow zero padding when loading the mask (Henry Proudhon)
Bug fixes in non regression tests. All OK now (Aldo Marano)
Bug fix –> Initialization bug with SampleData attribute Filters and with some methods argument name changes (Aldo Marano)
SampleData documentation corrections (Aldo Marano)
Revert “Test to solve read the doc compilation errors” (Aldo Marano)
New test to solve doc building issues on ReadThedocs (Aldo Marano)
Test to solve read the doc compilation errors (Aldo Marano)
Splitted crystal documentation into three pages, one for each module (lattice, microstructure, texture) (Aldo Marano)
Resolved most of documentation building Warnings and Errors (Aldo Marano)
Updated documentation of the samples modules and the core package (Aldo Marano)
Introduction of pymicro.core package into documentation and core.samples module (Aldo Marano)
end of code style modifications (Henry Proudhon)
removed extra parentheses (Henry Proudhon)
simpler Microstructure constructor by removing file_path (Henry Proudhon)
fix for Python3 (Henry Proudhon)
Correction of some bugs in documentation buildings, in progress. (Aldo Marano)
continuing to improve code style (Henry Proudhon)
improved docstring in __contains__ method (Henry Proudhon)
fixed typos and lines longer than 80 characters, improved documentation (Henry Proudhon)
fix pb with comparing two sequences as boolean (Henry Proudhon)
updates to account that Microstructure does not have a name attribute anymore (Henry Proudhon)
get rid of the grain_ids alias for now (Henry Proudhon)
cosmetic adjustments (Henry Proudhon)
fixed relace=True in set_mask, added a from_legacy_h5 method (Henry Proudhon)
improved getter and setter methods for sample_name and description (Henry Proudhon)
merged the name attribute with the sample_name of class SampleData (Henry Proudhon)
Correction of variable type for spacing attribute of SampleData image nodes (Aldo Marano)
moved code block outside except statement, improved code style (Henry Proudhon)
Merge branch ‘SampleData’ Merge dependency specification for SampleData integration (Aldo Marano)
Merge pull request #5 from Aldo Marano/SampleData (Henry Proudhon)
Added new requirements for SampleData integration (Aldo Marano)
SampleData/Pymicro integration: non-regression tests for core.samples and new methods to get specific data nodes disk size (Aldo Marano)
SampleData/Pymicro integration: new methods to set structured table data and specifics counterparts (Aldo Marano)
SampleData/Pymicro integration: new method copy_sample (Aldo Marano)
Externalization of Image and Mesh handling classes (Aldo Marano)
SampleData integration dev: finalization of the generic data model development (Aldo Marano)
Merge branch ‘master’ into SampleData_merge_master (Aldo Marano)
MAJOR DEVELOPMENT: SampleData and Pymicro merged, SampleData becomes the core package of Pymicro (Aldo Marano)
new files for new test in Microstructure class (Henry Proudhon)
new test to cover from_neper method (Henry Proudhon)
fix axes of the grain_map read from neper, improved slice_view (Henry Proudhon)
new method to_amitex_fftp (Henry Proudhon)
fixed typos in docstring (Henry Proudhon)
SampleData dev: small corrections (Aldo Marano)
added new method to import microstructure from a neper raster tesselation (Henry Proudhon)
added rotate_mesh and translate_mesh functions (Henry Proudhon)
changed the way the region ids are tested in show_boundaries (Henry Proudhon)
SampleData Development: alias name mechanism implementation (Aldo Marano)
SampleData Class Development: Last corrections for add_data_array (Aldo Marano)
SampleData Development: Finalization of add_data_array : xdmf field type detection developed (Aldo Marano)
SampleData Class development (Aldo Marano)
update dilate_labels fro 2D arrays (Henry Proudhon)
updated test for SampleData integration within the Microstructure class (Henry Proudhon)
continuing integration with the Microstructure class (Henry Proudhon)
continuing integration with SampleData class, fixed the set_grain_map and voxel_size issues (Henry Proudhon)
SampleData developments: changed information messages implementation (Aldo Marano)
add docstring to dct_projection method (Henry Proudhon)
change the way the get_grain_ids works (Henry Proudhon)
improve how the depth of the elset_id field is determined (Henry Proudhon)
factorized some code in new grain_projections method, improved docstrings (Henry Proudhon)
forward simulation now account for detector flips, also added an option to limit the grains used in the simulation (Henry Proudhon)
merge last SampleData modifs with Microstructure modifs (Aldo Marano)
(Debug) SampleData: remove errors when adding a node already existing (Aldo Marano)
moving to SampleData for storing image fields in the Microstructure class (Henry Proudhon)
Path update in imports for SampleData code in package core (Aldo Marano)
Introduction of SampleData source code for integration with Pymicro (Aldo Marano)
new class to handle chemical elements and their densities (Henry Proudhon)
added new example to plot atomic structure factors (Henry Proudhon)
corrected print statements for Python 3 (Henry Proudhon)
updated change log for version 0.4.5 (Henry Proudhon)
pymicro version 0.4.5¶
updated version number to 0.4.5 (Henry Proudhon)
view_slice now allow to set the colormap (Henry Proudhon)
new method to crop a microstructure (Henry Proudhon)
factorized dilation code to create a static method just working on a numpy array (Henry Proudhon)
updated copyright year (Henry Proudhon)
new method to discretize a non punctual source (Henry Proudhon)
added missing file containing the parameters for the atomic form factor calculations (Henry Proudhon)
fix path issue when running from outside pymicro (Henry Proudhon)
factorized code into fsim_laue method (Henry Proudhon)
new f_atom function to compute the atomic for factor for Z<=30 (Henry Proudhon)
added a new view_slice method (Henry Proudhon)
fall back on matlab format when loading mask from hdf5 fails (Henry Proudhon)
added data for Silicium material (Henry Proudhon)
cleanup __init__.py (Henry Proudhon)
removed old deprecated wx apps (Henry Proudhon)
added check boxes to handle image flips (Henry Proudhon)
updated for PyQt5 (Henry Proudhon)
added cases to numpy_to_esrf_datatype to behave properly in Python3 (Henry Proudhon)
add show_intersection args to add_hkl_plane_to_grain method and some cleanup (Henry Proudhon)
added new method delete_orphan_nodes (Henry Proudhon)
new methods get_bounds and save_to_geof (Henry Proudhon)
added a parameter in tt_stack to control the number of projections to sum (Henry Proudhon)
new test for find_neighbors function in Microstructure class (Henry Proudhon)
new function find_neighbors (Henry Proudhon)
fixed typo in docstring (Henry Proudhon)
added method compute_elset_center_of_mass (Henry Proudhon)
new methods get_grain_positions and from_grain_file in the Microstructure class (Henry Proudhon)
added coverage.xml (Henry Proudhon)
cleaning up plotting examples (Henry Proudhon)
fixed legend when using grain_id field (Henry Proudhon)
changed to scatter plot mode and fully moved to a kwargs mode (Henry Proudhon)
add new get_grain_volume_fractions method to te grain class (Henry Proudhon)
fixed missing data array (Henry Proudhon)
fixed issue with data_type in edf_write (Henry Proudhon)
add a new tt_stack method (Henry Proudhon)
fix problem with mask (Henry Proudhon)
new methode dilate_grain (Henry Proudhon)
added new option to dilate only selected grains in a microstructure (Henry Proudhon)
new function to create a random orientation (Henry Proudhon)
changed package name in setup.py (Henry Proudhon)
small fixes in from_dct method (Henry Proudhon)
fixed probleme with grain ids being stored as none (Henry Proudhon)
added verbose mode in grain_3d method (Henry Proudhon)
new method get_frame_as_array (Henry Proudhon)
added __pycache__ (Henry Proudhon)
single sourced version number (Henry Proudhon)
new indexed DCT file for examples (Henry Proudhon)
remove all the deleted element ids from other elset (Henry Proudhon)
update with voxel_size being an attribute of Microstructure (Henry Proudhon)
polishing the new voxel_size attribute (Henry Proudhon)
fine tuning packaging (Henry Proudhon)
fixed bug in vtkRotateActorAroundAxis, closes #4 (Henry Proudhon)
added an exception when a grain from the microstructure is not present in the grain map (Henry Proudhon)
new methods compute_grain_center and recompute_grain_centers (Henry Proudhon)
Forward simulation now support point, array and CAD geometry. (Henry Proudhon)
Sample Geometry now supports point, array and cad properly (Henry Proudhon)
new method is_in_array (Henry Proudhon)
updated for new attribute voxel_size (Henry Proudhon)
renamed grain attribute position into center (Henry Proudhon)
added new voxel_size attribute to the Microstructure class (Henry Proudhon)
quick fixes after renamning grain attribute position into center (Henry Proudhon)
new test microstructure (slice from a DCT volume) (Henry Proudhon)
use the new config file to determine the path to the data folder (Henry Proudhon)
added test for Microstructure.from_h5 function (Henry Proudhon)
renamed grain attribute position into center (Henry Proudhon)
add a new config file at the project root (Henry Proudhon)
new method to delete an elset from a FE_Mesh instance. (Henry Proudhon)
new dilate_grains function for the microstructure module (Henry Proudhon)
updated code with the new function from_dct to build the two microstructures (Henry Proudhon)
quick bug fix to load the mask from a DCT reconstruction (Henry Proudhon)
updated from_dct function in the Microstructure module (now uses the index.mat file) and new from_dct function in the Grain module (Henry Proudhon)
added docstrings to has_grain (Henry Proudhon)
new function merge_dct_scans (Henry Proudhon)
added docstrings for merge_microstructures (Henry Proudhon)
docstring fix (Henry Proudhon)
new to_h5 and from_h5 functions (Henry Proudhon)
new functions to create a Lattice instance from its symmetry and the list of lattice parameters (Henry Proudhon)
replaced function names from_h5 / to_h5 by from_dream3d / to_dream3d (Henry Proudhon)
added a crystal lattice attribute to the Microstructure class (Henry Proudhon)
few tweaks to the new match_grains function (Henry Proudhon)
added new match_grain function in class Microstructure (Henry Proudhon)
replaced ‘’’ by “”” throughout the file (Henry Proudhon)
removed unnecessary AxShowPixelValue class (Henry Proudhon)
refactored forward simulation classes (Henry Proudhon)
added circle=False parameter by default when computing radiographs with the radon transform (Henry Proudhon)
made h, k and l properties in HklObject (Henry Proudhon)
removed unused color_by_grain_id attribute (Henry Proudhon)
removed print statement (Henry Proudhon)
made a new class DctForwardSimulation, and updated Experiment code (Henry Proudhon)
added circle=False parameter by default when computing radiographs with the radon transform (Henry Proudhon)
fixed the show_array method that was broken for VTK > 6.2 (Henry Proudhon)
resolved merge confict (Henry Proudhon)
replaced has_key by in for python3 (Henry Proudhon)
Add ‘hkl_planes’ in experiment file (Alexiane)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
add verbose mode to edf_read and edf_info (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane)
Remove unecessary print (Alexiane)
comment out deprecated code (Alexiane)
update save_vtk_repr method for VTK version > 5 (Henry Proudhon)
update get_frame method for Python3 (Henry Proudhon)
add edf file for tests (Henry Proudhon)
also fixed edf_read and added corresponding test (Henry Proudhon)
improved edf_info and unpack_header functions for both Python 2 and 3, added test (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
new function to compute the diffractometer configuration and new test case (Henry Proudhon)
now account for specific configuration of the instrument in topotomo_tilts function (Henry Proudhon)
New Quaternion functions (Alexiane)
normalize quaternion in __init__ (Henry Proudhon)
increased tolerance on misorientation_from_delta for extremely low misorientation that may lead to a traceslightly larger than 3.O (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane)
New fonction for nomalization (Alexiane)
add uv_exp field to Grain in experiment (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
First implementation of quaternions (Henry Proudhon)
New functions for quaternions (Alexiane)
Merge pull request #2 from tobyfrancis/master (Henry Proudhon)
New functions with quaternions (Alexiane)
New class for slits geometry (Alexiane)
Merge remote-tracking branch ‘origin/master’ (Alexiane)
fixed issue with 4x4 symmetry operators for hexagonal lattices (Henry Proudhon)
added support to read EBSD scan from Oim (Henry Proudhon)
new class for Slits description (Alexiane)
new class for Slits description (Alexiane)
python3 support (toby)
added .coverage to .gitignore and removed from repository (Henry Proudhon)
fixing source error in .coveragerc (Henry Proudhon)
added coveralls to the build install requirements (Henry Proudhon)
added code coverage for the tests (Henry Proudhon)
added setup.py to package pymicro (Henry Proudhon)
updated docsting for multiplicity method (Henry Proudhon)
modified index.rst to include the README file (Henry Proudhon)
updated .gitignore to ignore distribution files (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
new method to create a detector from a poni file (Henry Proudhon)
removed python 2.7 build for now (Henry Proudhon)
added MIT license badge (Henry Proudhon)
added testing with nosetests to travis (Henry Proudhon)
changed dependency from skimage to scikit-image (Henry Proudhon)
fixed travis badge path (Henry Proudhon)
added pip instruction to travis build (Henry Proudhon)
adding required libraries file (Henry Proudhon)
fixing badge paths (Henry Proudhon)
added travis file (Henry Proudhon)
added README file (Henry Proudhon)
reran cubic example (Henry Proudhon)
fixed doc build (Henry Proudhon)
added h5py as a dependency (Henry Proudhon)
finally updated the installation section (Henry Proudhon)
added Rodrigues2Axis method (Henry Proudhon)
simplified a bit the View module and corrected a fex typos (Henry Proudhon)
completed docstring of HST_read function (Henry Proudhon)
New expression of the major axis factor (see article) (Alexiane Arnaud)
fixed old import statement (Henry Proudhon)
allow uppercase data type from HST_info (Henry Proudhon)
fixed Binning key in dictionnary (Henry Proudhon)
now save binning and source energy range (Henry Proudhon)
enforce default plane color to grey (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
fixed pixel size in forward simulation (Alexiane Arnaud)
added __pycache__ to .gitignore (Henry Proudhon)
added support for python3 (Henry Proudhon)
updated change log for version 0.4.4 (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
solve version conflit (Alexiane Arnaud)
pymicro version 0.4.4¶
updated version number to 0.4.4, last in Python2 (Henry Proudhon)
added some docstrings (Henry Proudhon)
added extended sample geometry (Henry Proudhon)
add binning support for RegArrayDetector2d (Henry Proudhon)
fix no diffraction if no energy range is present (Henry Proudhon)
renamed parameter fs_type into sim_type (Henry Proudhon)
initial code for Forward Simulation of a complete X-ray experiment (Henry Proudhon)
corrected typo (Henry Proudhon)
added clear_data method and call it when creating a new detector (Henry Proudhon)
added to_string method for Symmetry (Henry Proudhon)
a Scene3d can now be automaticcaly created from an Experiment instance (Henry Proudhon)
added sample and geometry to an Experiment (Henry Proudhon)
modified the box_3d function to use an origin (Henry Proudhon)
new Experiment class, with some tests (Henry Proudhon)
start developing code to handle a complete experiment (Henry Proudhon)
modified the way the detector tilts are handled (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
added Co and Mn to xray data (Henry Proudhon)
added fzDihedral function and load label grain volume when importing from dct (Henry Proudhon)
fixed get_family calculations for hexagonal symmetry and added tests (Henry Proudhon)
Solved version of detector (Alexiane Arnaud)
Merge remote-tracking branch ‘origin/master’ (Alexiane Arnaud)
Update verbose (Alexiane Arnaud)
New detector definition (Alexiane Arnaud)
added simple region growing algorithm (Henry Proudhon)
fixed bug after HST_info update (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
added support to load a microstructure from a DCT reconstruction (Henry Proudhon)
Update LICENSE.md (Henry Proudhon)
new grain_3d_planes example (Henry Proudhon)
added sst_symmetry function so that ipf plotting now supports both cubic and hexagonal symmetries (Henry Proudhon)
cosmetic change from B to g (Henry Proudhon)
removed labels from ipf plot (Henry Proudhon)
added a plot_ipf_symmetry method (Henry Proudhon)
get_family now supports 4 indices for hexagonal symmetry (Henry Proudhon)
added a new test for the scattering_vector method (Henry Proudhon)
quick fix for new symmetry code (Henry Proudhon)
New folder organisation (Alexiane Arnaud)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
Test on detector tilt (Alexiane Arnaud)
add the colorbar (Henry Proudhon)
make use of the Symmetry class (Henry Proudhon)
updated examples with changes in the code (Henry Proudhon)
add new example using a color field within a IPF plot (Henry Proudhon)
fix an issue when using a color field (Henry Proudhon)
update to use the new Symmetry class (Henry Proudhon)
new Class Symmetry to handle lattice symmetries (Henry Proudhon)
added multiplicity method (Henry Proudhon)
remove unecessary for loops (Henry Proudhon)
generalized get_family method with any hkl triplet and any symmetry (Henry Proudhon)
fix a problem with the size of the arguments in pixel_to_lab (Henry Proudhon)
Get family new item (Alexiane Arnaud)
Build list new item (Alexiane Arnaud)
fixed typo in last commit (Henry Proudhon)
update how negative energy is handled (Henry Proudhon)
changed the way the friedel pairs are handled for diffraction (Henry Proudhon)
added new lattice plane families (Henry Proudhon)
more complete test for project_along_direction (Henry Proudhon)
get_family now accept any hkl combination (Henry Proudhon)
added 133 family to the get_family method (Henry Proudhon)
New Laue functions (Alexiane Arnaud)
New Laue functions (Alexiane Arnaud)
New function to get ellipsis easily (Alexiane Arnaud)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
now use a floating point precsion when testing if points are on the detector (Henry Proudhon)
the gnomonic projection of detecot image now uses the generalized code (Henry Proudhon)
added test for the gnomonic projection (Henry Proudhon)
test data for the gnomonic projection (Henry Proudhon)
small cleanup in lab_to_pixel (Henry Proudhon)
new test_detectors module (Henry Proudhon)
lab_to_pixel can now be used with arrays of points (Henry Proudhon)
FIXME message (Henry Proudhon)
explicitely use the centering parameter when creating a lattice instance (Henry Proudhon)
fixed print bug in print_camera_settings (Henry Proudhon)
fixed missing abs in lab_to_pixel (Henry Proudhon)
pixel_to_lab can now use arrays (Henry Proudhon)
removed unnecessary staticmethod decorator, new test (Henry Proudhon)
updated import after refactoring (Henry Proudhon)
updated import after refactoring (Henry Proudhon)
generalized the gnomonic projection (Henry Proudhon)
test the gnomonic projection with normal and non normal incidence (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
cleanup and new function point_cloud_3d (Henry Proudhon)
renamed gethkl_from_two_directions to indices_from_two_directions plus some cleanup (Henry Proudhon)
added head docstrings (Henry Proudhon)
some cleanup in the laue module (Henry Proudhon)
added a new test with select_lambda (Henry Proudhon)
updated change log for version 0.4.3 (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Alexiane Arnaud)
Practical modification of transmission plot (Alexiane Arnaud)
pymicro version 0.4.3¶
updated version number to 0.4.3 (Henry Proudhon)
much faster version of recad (Henry Proudhon)
small fix in gnomonic_projection_point2 (Henry Proudhon)
force direction to be a numpy array in case a list is given (Henry Proudhon)
updated compute_ellipsis and added gnomonic_projection_point2 for non normal X-ray incidence (Henry Proudhon)
refactoring after renaming four_to_three_indices method (Henry Proudhon)
some cleanup in names (Henry Proudhon)
updated the use of project_along_direction to its new signature (Henry Proudhon)
make the width variable being an integer (Henry Proudhon)
changed the pole list from a vector list to a list of HklPlane instances (Henry Proudhon)
change in from_h5 signature to be more versatile (Henry Proudhon)
added the possibility to directly pass a list of HklPlane instances to define the poles in a PoleFigure (Henry Proudhon)
quick fix on removing test files (Henry Proudhon)
new method to create a slip system from miller indices (Henry Proudhon)
added append to file mode to function HST_write (Henry Proudhon)
gethkl_fromtwo_directions new function (Alexiane Arnaud)
new tests for the vol_utils module (Henry Proudhon)
added new method min_max_cumsum and refactored auto_min_max code (Henry Proudhon)
the X-ray beam is no longer restricted to the X-axis (Henry Proudhon)
new apply_rotation_to_actor method and factorized code to use it (Henry Proudhon)
re-do the poll system update (Alexiane Arnaud)
Global commit (Alexiane Arnaud)
Add extinctions to the build_list function (Alexiane Arnaud)
convert bool arrays to uint8 in HST_write when not using pack_binary option (Henry Proudhon)
added try block to import modules (Henry Proudhon)
added symmetry operators for hexagonal lattice (Henry Proudhon)
added verbose option in poll_system (Henry Proudhon)
changed orthorhombic calls in tests according last commit (Henry Proudhon)
Orthorhombic modification (Alexiane Arnaud)
use the numpy array tye to create the right vtk array type in numpy_array_to_vtk_grid (Henry Proudhon)
fixed issue with Lattice transformation matrix in non cubic case (Henry Proudhon)
added new regression case for topotomo_tilts (Henry Proudhon)
updated slip traces example (Henry Proudhon)
fixed issue with calling slip_trace from plot_slip_traces method (Henry Proudhon)
exposed solve_trig_equation method and added regression tests (Henry Proudhon)
additional test for dct_omega_angles (Henry Proudhon)
added plane_origins option in grain_3d (Henry Proudhon)
added test for HklDirection 4 indices representation (Henry Proudhon)
fixed three_to_four_indices and the like for HklPlane and HklDirection classes (Henry Proudhon)
fixed docstring for hexagonal_lattice_grid (Henry Proudhon)
corrected example description (Henry Proudhon)
added non regression test for .info files without DATA_TYPE entry (Henry Proudhon)
fix data type issue in HST_info (Henry Proudhon)
Merge remote-tracking branch ‘origin/master’ (Alexiane Arnaud)
factorized code for slip plane traces (Henry Proudhon)
added initialisation of U field (Henry Proudhon)
removed print statement in misorientation_angle_from_delta (Henry Proudhon)
added symmetry operators for orthorhombic and an option to keep friedel pairs in skip_higher_order (Henry Proudhon)
update gnomonic projection code with a new gnomonic_projection_point method (Henry Proudhon)
added new test for gnomonic_projection_point method (Henry Proudhon)
fix import of tifffile (Henry Proudhon)
update Image Processing folder (Alexiane Arnaud)
updated cookbook to better explain the orientation matrix (Henry Proudhon)
from_euler now support the Roe convention to compute the orientation matrix (Henry Proudhon)
pymicro version 0.4.2¶
updated version number (Henry Proudhon)
new method pole_figure_3d and updated example (Henry Proudhon)
added new get method to retreive the orientation list (Henry Proudhon)
small code style improvements (Henry Proudhon)
corrected indentation in pointset_registration.rst (Henry Proudhon)
updated examples (Henry Proudhon)
updated .gitignore after changing source path (Henry Proudhon)
added point set registration entry in cookbook (Henry Proudhon)
updated paths to reflect source move to pymicro folder (Henry Proudhon)
removed files from tree corresponding to the previous move (Henry Proudhon)
moved source to a pymicro folder so the docs build properly on rtfd (Henry Proudhon)
try to fix path to build autodoc (Henry Proudhon)
added requirements file to build the documentation (Henry Proudhon)
removed old EBSDMicrostructure class (Henry Proudhon)
changed single quote to double quotes in docstrings (Henry Proudhon)
fixed issue with the no more needed col parameter (Henry Proudhon)
plot_sst now displays the 3 main crystal axes, refactored some code to use get_color_from_field, docstring improvements (Henry Proudhon)
changed a bit how the elset names are handled in compute_elset_id_field, plus docstrings and vtk version specific code (Henry Proudhon)
added new method to select cells in vtkUnstructuredGrid (Henry Proudhon)
corrected type in set_rank (Henry Proudhon)
updated make_vtu to supprt .mesh files and added docstring (Henry Proudhon)
new method load_from_mesh to create FE_Mesh object from .mesh files (Henry Proudhon)
new boundary parameter in extract_poly_data (Henry Proudhon)
factorized code with vtkExtractGeometry in new method extract_poly_data (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
added nodal field support (Henry Proudhon)
Merge remote-tracking branch ‘origin/master’ (Alexiane Arnaud)
Update poll system function to see every solutions (Alexiane Arnaud)
fixed rounding float error in misorientation_angle_from_delta (Henry Proudhon)
added reference in misorientation_MacKenzie docstring (Henry Proudhon)
added sphinx-bibtex extension to handle references and started using it (Henry Proudhon)
added new function poll_system and confidence_index from the work of Wijdene (Henry Proudhon)
cosmetic changes (Henry Proudhon)
added todo to improve FZ computation (Henry Proudhon)
new tests for Rodrigues fundamental zone. (Henry Proudhon)
new methods to move rotations to the fundamental zone of the Rodrigues space. (Henry Proudhon)
updated OrientationMatrix2Euler method and strated FZ code (Henry Proudhon)
fixed plane normal not showing issue and cleaned the code by removing method add_plane_to_grid_with_normal (Henry Proudhon)
remove outdated read_dif method (Henry Proudhon)
change type of energy_lim param to list (support assignment) (Henry Proudhon)
change energy bound to 200 keV (Henry Proudhon)
added docstring for the read_image_sequence method. (Henry Proudhon)
added missing parameter in docstring and fixed print statements (Henry Proudhon)
fixed a small issue with working with a uint8 data array, now the array is only converted at the end of the function to uint8 (Henry Proudhon)
new recipe showing how to perform point set registration (Henry Proudhon)
new line_actor method to factorize some code (Henry Proudhon)
more detailed doctsring for compute_affine_transform (Henry Proudhon)
made line_3d and circle_line_3d use the new build_line_mesh function (Henry Proudhon)
new function build_line_mesh (Henry Proudhon)
new method to compute affine transform by point set registration (Henry Proudhon)
changes teh notation from B to g for the orientation matrix to avoid confusion. (Henry Proudhon)
added more Orientation tests (Henry Proudhon)
removed unnecessary eps parameter and fix docstrings (Henry Proudhon)
make sure origin is set properly in lattice_3d_with_planes (Henry Proudhon)
make sure origin is an array (Henry Proudhon)
Merge branch ‘master’ of https://github.com/heprom/pymicro (Henry Proudhon)
new gnomonic projection function (Henry Proudhon)
set default for origin in project_along_direction and docstring update (Henry Proudhon)
Correction : from compute_ellpisis to compute_ellipsis (Alexiane Arnaud)
cleaning up the new stitch method (Henry Proudhon)
new method to stitch image series (Henry Proudhon)
new method read_image_sequence (Henry Proudhon)
more docstring fixes (Henry Proudhon)
auto_min_max docstring improvements plus minor tweaks (Henry Proudhon)
updated change log for version 0.4.1 (Henry Proudhon)
pymicro version 0.4.1¶
updated version number (Henry Proudhon)
make sure motion components are interpreted as float in vtkMove animation (Henry Proudhon)
corrected typo in print statement (Henry Proudhon)
Merge remote-tracking branch ‘origin/master’ (Henry Proudhon)
updated compute_Laue_pattern method with new parameters such as spectrum and color_spots_by_energy (Henry Proudhon)
update imports to match the refactoring of the fitting module into the xray package (Henry Proudhon)
improved compute_Laue_pattern with new available methods from the Detector2d class (Henry Proudhon)
improved compute_Laue_pattern with new available methods from the Detector2d class (Henry Proudhon)
deleted excluded files in docs/_build directory (Henry Proudhon)
fixed a visibility issue when using an assembly instead of single actor (Henry Proudhon)
added the possibility to use a 4D array (multi-component 3D array) when converting from numpy to vtk (Henry Proudhon)
moved the fitting module to the xray package. This should fix import confict with the math module of python as well as simplify the tree structure. (Henry Proudhon)
new method def get_edges (Henry Proudhon)
temporarily changed from Microstructure object to a set of orientations in dct_projection, added include_direct_beam option. (Henry Proudhon)
added pixel_to_lab and get_origin methods (Henry Proudhon)
added dictionary to the read_orientations method. (Henry Proudhon)
renamed read_euler_txt to read_orientations and added a method for backward compatibility. (Henry Proudhon)
fixed the dct_omega_angle method which assumed a cubic lattice. (Henry Proudhon)
detector now handle a reference position, regular detectors have a better support with u and v directions (Henry Proudhon)
new method lattice_3d_with_plane_series (Henry Proudhon)
add new color parameter to axes_actor function (Henry Proudhon)
untrack files in docs/_build (Henry Proudhon)
completed euler_angle cookbook example with orientation matrix definition (Henry Proudhon)
fixed mathjax_path and updated some docstrings in the microstructure module to have math working. (Henry Proudhon)
add mathjax_path pointing to the cloudflare CDN (Henry Proudhon)
enable the mathjax extension (Henry Proudhon)
added docstring for class View (Henry Proudhon)
now handle instance of vtkActor as an argument (Henry Proudhon)
changes to use the readthedocs html theme (Henry Proudhon)
updated version of the Euler angles animation (Henry Proudhon)
fix issue with gif path in generated rst (Henry Proudhon)
Merge remote-tracking branch ‘origin/master’ (Henry Proudhon)
added missing files for new example (Henry Proudhon)
added missing files for new example (Henry Proudhon)
changed the sign of omega to conform with the passive convention for rotations (Henry Proudhon)
new animation example with cookbook entry (Henry Proudhon)
more mock definitions for skimage (Henry Proudhon)
fix auto generation with animation examples (Henry Proudhon)
removed hot method and some docstring changes (Henry Proudhon)
new method vtkUpdateText (Henry Proudhon)
added new methds Axis2OrientationMatrix and Euler2Axis to use the (axis, angle) representation of rotations. (Henry Proudhon)
added a new method to render animation at a given time. (Henry Proudhon)
added support for viewing 3D numpy arrays. (Henry Proudhon)
make sur we work with an array of angles in the radiographs method (Henry Proudhon)
new methods to compute radiographs of 3D objects. (Henry Proudhon)
new module dct with code to help process, analyse and simulate dct data (Henry Proudhon)
added automated method to find all hkl planes in a given family (plus some testing) (Henry Proudhon)
added module docstring and commented matplotlib rcparams stuff (Henry Proudhon)
docstring fixes (Henry Proudhon)
added verbose mode to diffracted_vector method (Henry Proudhon)
double checked (corrected notations) the orientation matrix in dct_omega_angles (Henry Proudhon)
added lattice parameter to the get_family method (Henry Proudhon)
add (113) hkl plane family (Henry Proudhon)
correct for wrong indentation causing problems to the literalinclude directive (Henry Proudhon)
docstring fixes/tests in the module header (Henry Proudhon)
updated documentation so it work better on readthedocs (Henry Proudhon)
more indentation fixes (Henry Proudhon)
pymicro version 0.4.0¶
updated version number to 0.4.0 (Henry Proudhon)
added license file (Henry Proudhon)
prefer the use of is instead of == testing for None (Henry Proudhon)
adjust assertion after updating Al cif file (Henry Proudhon)
new fil_utils tests (Henry Proudhon)
added memory mapping option to HST_read function (Henry Proudhon)
use the proper fcc Ni cif file (Henry Proudhon)
added Ga and Mn cif files (Henry Proudhon)
cleaening up cif files and minor tweaks (Henry Proudhon)
added lattice arg to angle_between_directions to prevent forcing it to cubic (Henry Proudhon)
added new test for angle between two hkl directions (Henry Proudhon)
reformated all code with a consisten style (Henry Proudhon)
doc changes to reflect new file organisation (Henry Proudhon)
moving a bunch of file for a better organisation, created external folder (Henry Proudhon)
cleaning old stuff, moving tifffile out (Henry Proudhon)
new edge_actor method and updated show_boundaries (Henry Proudhon)
hist now plot in a new figure (Henry Proudhon)
updated xray_trans function, added table for densities (Henry Proudhon)
moved all mass attenuation coefficients to the data folder (Henry Proudhon)
fixed all docstrings (Henry Proudhon)
fixed bug in compute_ellipsis method (Henry Proudhon)
fixed dupplicate link for packages and added xray package (Henry Proudhon)
added a4 paper option (Henry Proudhon)
new example laue_ellipse (Henry Proudhon)
more fixes to the compute_ellpisis method (Henry Proudhon)
new test for select_lambda method (Henry Proudhon)
added docstring + bugfix in compute_ellpisis (Henry Proudhon)
fix small typo with the data_type variable (Henry Proudhon)
make full use of the detector variable in compute_ellpisis (Henry Proudhon)
modified detector classes, added new class PerkinElmer1620 (Henry Proudhon)
added initial code for Laue diffraction calculation and one first unit test (Henry Proudhon)
added a vtkCleanPolyData step for the show_boundaries method (Henry Proudhon)
added docstring for function show_boundaries (Henry Proudhon)
updated changelog (Henry Proudhon)
pymicro version 0.3.3¶
updated version number (Henry Proudhon)
added new example for misorientation (Henry Proudhon)
corrested a few typos (Henry Proudhon)
corrected a few small bugs (Henry Proudhon)
new orientations types and misorientation code (Henry Proudhon)
added symmetry operator for cubic structure (Henry Proudhon)
new tests for misorientation caculations (Henry Proudhon)
make scene3d more simple to use with better defaults (Henry Proudhon)
initial support for volume rendering (Henry Proudhon)
working on ImageViewer (Henry Proudhon)
new xray absorption data (Henry Proudhon)
updated import_modules.py with new functions (Henry Proudhon)
new legend format option in plot_xray_trans (Henry Proudhon)
new absorption data (Henry Proudhon)
added color option for atoms and bond in lattice_3d (Henry Proudhon)
fixed version pb with np.savetxt (Henry Proudhon)
small fix with psi_values (Henry Proudhon)
modified ImageViewer to work with Qt (Henry Proudhon)
improved detector code, both for Mar and Xpad (Henry Proudhon)
fixed fwhm for Voigt function (Henry Proudhon)
removed broken method get_octaedral_slip_systems (Henry Proudhon)
added new sagital_regroup method (Henry Proudhon)
fix data folder (Henry Proudhon)
a first example for finite elements (Henry Proudhon)
__init__ file for fe package and texture rst file (Henry Proudhon)
initial commit of the finite element part of pymicro (Henry Proudhon)
fix typo, remove print output (Henry Proudhon)
small fix in xray_utils (Henry Proudhon)
new method angle_with_direction (Henry Proudhon)
add check for nbParams in init (Henry Proudhon)
added lambda to keV conversion functions (Henry Proudhon)
added installation section (Henry Proudhon)
new example with skimage and radon (Henry Proudhon)
docstrings for contourFilter (Henry Proudhon)
fix orientation_tensor stuff for Taylor calculation (Henry Proudhon)
fix orientation_tensor stuff for Taylor calculation (Henry Proudhon)
added contour plot for pole figures and a demonstrating example (Henry Proudhon)
added more slip planes families and slip systems families, updated testing (Henry Proudhon)
fixed typo in dct_omega_angles (Henry Proudhon)
moved dc_omega_angles to the Orientation class (Henry Proudhon)
fixed colors when plotting ipf (Henry Proudhon)
new options to dsplay mesh edges in show_mesh (Henry Proudhon)
cleanup: moved the testing code of TaylorModel to external files (Henry Proudhon)
fixed a missing Update which caused the actor not to be displayed (Henry Proudhon)
new Taylor model class (experimental) (Henry Proudhon)
updated xray data files for Cu and Al (Henry Proudhon)
new method to compute a slip system orientation strain and rotation tensors (Henry Proudhon)
added change log history to the documentation (Henry Proudhon)
pymicro version 0.3.2¶
updated version number (Henry Proudhon)
docstring changes (Henry Proudhon)
a few tweaks in dct_projection (Henry Proudhon)
changed numpy in np and added a new method circle_line_3d (Henry Proudhon)
update flat field code for xpad (Henry Proudhon)
improved xpad code to hangle multiple images in raw or nxs, added flat field correction (Henry Proudhon)
fixed minus sign in dct_omega_angles (Henry Proudhon)
one more fix to the elevationFilter (Henry Proudhon)
changed the way the elevation axis is handled (Henry Proudhon)
fixed broken docstring in elevationFilter (Henry Proudhon)
added the possibility to use any axis in the elevationFilter (Henry Proudhon)
added new example with segmented crack (Henry Proudhon)
add grain objects to View (Henry Proudhon)
new find_planes_in_zone method (Henry Proudhon)
fixed bug in lambda_nm_to_keV and lambda_angstrom_to_keV (Henry Proudhon)
updated documentation (Henry Proudhon)
recompiled some figures (Henry Proudhon)
new test apply_orientation_to_actor (Henry Proudhon)
added skimage dependency (Henry Proudhon)
fixed vtk version problem for function volren (Henry Proudhon)
fixed vtk version problem for function volren (Henry Proudhon)
small fix with f.tell() (Henry Proudhon)
fix int conversion for numpy version (Henry Proudhon)
imporved View app and added it to the list of imported modules (Henry Proudhon)
modified dct_projection to use radon from skimage (Henry Proudhon)
new methods for rotating crystals and topotomography alignment with tests (Henry Proudhon)
new method to compute the scattering vector (Henry Proudhon)
corrected type in docstring (Henry Proudhon)
changed copyright year (Henry Proudhon)
missing file from previous commit (Henry Proudhon)
increased a bit the resolution for 2 examples (Henry Proudhon)
added example for volume rendering (Henry Proudhon)
added change log history to the documentation (Henry Proudhon)
pymicro version 0.3.1¶
updated version number (Henry Proudhon)
added new example for misorientation (Henry Proudhon)
corrested a few typos (Henry Proudhon)
corrected a few small bugs (Henry Proudhon)
new orientations types and misorientation code (Henry Proudhon)
added symmetry operator for cubic structure (Henry Proudhon)
new tests for misorientation caculations (Henry Proudhon)
make scene3d more simple to use with better defaults (Henry Proudhon)
initial support for volume rendering (Henry Proudhon)
working on ImageViewer (Henry Proudhon)
new xray absorption data (Henry Proudhon)
updated import_modules.py with new functions (Henry Proudhon)
new legend format option in plot_xray_trans (Henry Proudhon)
new absorption data (Henry Proudhon)
added color option for atoms and bond in lattice_3d (Henry Proudhon)
fixed version pb with np.savetxt (Henry Proudhon)
small fix with psi_values (Henry Proudhon)
modified ImageViewer to work with Qt (Henry Proudhon)
improved detector code, both for Mar and Xpad (Henry Proudhon)
fixed fwhm for Voigt function (Henry Proudhon)
removed broken method get_octaedral_slip_systems (Henry Proudhon)
added new sagital_regroup method (Henry Proudhon)
fix data folder (Henry Proudhon)
a first example for finite elements (Henry Proudhon)
__init__ file for fe package and texture rst file (Henry Proudhon)
initial commit of the finite element part of pymicro (Henry Proudhon)
fix typo, remove print output (Henry Proudhon)
small fix in xray_utils (Henry Proudhon)
new method angle_with_direction (Henry Proudhon)
add check for nbParams in init (Henry Proudhon)
added lambda to keV conversion functions (Henry Proudhon)
added installation section (Henry Proudhon)
new example with skimage and radon (Henry Proudhon)
docstrings for contourFilter (Henry Proudhon)
fix orientation_tensor stuff for Taylor calculation (Henry Proudhon)
fix orientation_tensor stuff for Taylor calculation (Henry Proudhon)
added contour plot for pole figures and a demonstrating example (Henry Proudhon)
added more slip planes families and slip systems families, updated testing (Henry Proudhon)
fixed typo in dct_omega_angles (Henry Proudhon)
moved dc_omega_angles to the Orientation class (Henry Proudhon)
fixed colors when plotting ipf (Henry Proudhon)
new options to dsplay mesh edges in show_mesh (Henry Proudhon)
cleanup: moved the testing code of TaylorModel to external files (Henry Proudhon)
fixed a missing Update which caused the actor not to be displayed (Henry Proudhon)
new Taylor model class (experimental) (Henry Proudhon)
updated xray data files for Cu and Al (Henry Proudhon)
new method to compute a slip system orientation strain and rotation tensors (Henry Proudhon)
added change log history to the documentation (Henry Proudhon)
pymicro version 0.3.0¶
32120a1 changed version number to 0.3.0 (Henry Proudhon)
debfbb8 updated pole figure example (Henry Proudhon)
0cdd294 fixed colormap issue when using map_field option (Henry Proudhon)
7323228 changes in examples to account for previous commits (Henry Proudhon)
4a51061 small bounding box fix in the map_data_with_clip method (Henry Proudhon)
3b72591 significant changes to handle field map with pole figures (Henry Proudhon)
267ca37 new OrientationTests class (Henry Proudhon)
667fcaf new test_from_symbol test method (Henry Proudhon)
f94eb22 very small docstring changes (Henry Proudhon)
61e311e show_data has been split in show_array and show_mesh (Henry Proudhon)
b46f7e4 corrected typo (Henry Proudhon)
9a43e04 final docstring changes to the texture module (Henry Proudhon)
cbbe2ae new example to demonstrate field coloring in pole figures (Henry Proudhon)
5d090fe a little more explanations (Henry Proudhon)
2a4e4d2 more docstring fixes in texture.py (Henry Proudhon)
acdaa6e change part of the title for inverse pole figure from family to axis (Henry Proudhon)
64cca29 more docstring formatting (Henry Proudhon)
5c13e80 updated some docs with Info field lists (Henry Proudhon)
667b8f2 Merge branch ‘master’ of vcs:pymicro (Nicolas Gueninchault)
3f705d9 new methods Calculate_Omega_dct, calc_poles_id11, Sam2Lab, Lab2sam, Sam2Sam (Nicolas Gueninchault)
a161f58 new methods plot_ipf_density, Eul2Mat, Write_inp_crystals (Nicolas Gueninchault)
cdba718 improved docstrings in apply_orientation_to_actor (Henry Proudhon)
fc530aa changed the way the rotation is applied in apply_orientation_to_actor (Henry Proudhon)
5c15dda cleaned grain_hkl_anim_3d.py (Henry Proudhon)
6d8f9e0 added options in unit_arrow_3d to display text in 3d aside the arrow (Henry Proudhon)
e119dfe more file for X-ray attenuation coefficients (Henry Proudhon)
24f2700 new code for X-ray detectors with two examples (Henry Proudhon)
cc88b87 added extension option in load_STL_actor function (Henry Proudhon)
8dd35e4 add opacity in alpha_cmap (Henry Proudhon)
6edfd6e small fix with iren.AddObserver (Henry Proudhon)
8f5b9e5 improved handling of Voigt function (Henry Proudhon)
ffdaa04 updated example to use the new animation framework (Henry Proudhon)
9fd27d4 add new plot_pf_hot function (Henry Proudhon)
96bacb8 added x1x2x3 rotation type when importing orientations from z-set (Henry Proudhon)
a141131 small fix to read image with HST_read (Henry Proudhon)
99f276c updated version (Henry Proudhon)
692f89a new dct_projection function and various small fixes (Henry Proudhon)
a987c56 improved anim framework (Henry Proudhon)
31106b5 added new hkl families (Henry Proudhon)
b1623aa new set_opacity for assembly and more parameters in unit_arrow_3d (Henry Proudhon)
dcf3810 improved plot_xray_trans function (Henry Proudhon)
pymicro version 0.2.3¶
c5d5ebb small typo (Henry Proudhon)
eb77e84 new color_bar function and new colormaps (Henry Proudhon)
7c61f57 added class for the Voigt fitting function (Henry Proudhon)
c0f082d new recipe to explein how 3d images are structured and should be read (Henry Proudhon)
22b710d new class to allow pyplot showing the pixel value and associated example (Henry Proudhon)
a3ffd0e small fix to the Gold pole figure example (Henry Proudhon)
4a6984c testing the Orientation class, merged Schmid factor utilities from Nfun into the Orientation class (Henry Proudhon)
6e0e2e1 fidling with no longer supported matplotlib wx backend (Henry Proudhon)
271bbd8 new recipe to explain plotting, reading, writting 2d images with pyplot (Henry Proudhon)
46591a3 fix always plot sst in plot_pole_figures (Henry Proudhon)
31e27a6 added missing files to the tree (Henry Proudhon)
424d22b added a picture for fitting functions (Henry Proudhon)
fdde04a changed doc accordingly to build the new math module (Henry Proudhon)
pymicro version 0.2.2¶
cec8566 changed version number to 0.2.2 (Henry Proudhon)
34f0601 allow to use a custom fit function in the fitting module (Henry Proudhon)
02c1cf9 added a general use fit method (Henry Proudhon)
28fd8d3 added a new example to demonstrate fitting (Henry Proudhon)
af6c1ad new math package with fitting functions (Henry Proudhon)
7b442f7 added a Makefile to run all example at once (Henry Proudhon)
944cb9f modified all examples to use the new scene3d stuff (Henry Proudhon)
c40a540 new file to configure ipython and updated documentation (Henry Proudhon)
a4fc7f5 docstrings updates (Henry Proudhon)
b48f7c5 changed from angstrom to nanometer when loading from CIF (Henry Proudhon)
dc7eacd added a new way to create crystal lattice via CIF files (Henry Proudhon)
28709df further improvements with Scene3d (Henry Proudhon)
5cc5bbc minor changes after last commit (Henry Proudhon)
8e8a353 adapted cubic_crystal example to new scene3d code (Henry Proudhon)
05cd497 new code to simplify building a 3d scene (Henry Proudhon)
3af0c66 improved axes_actor (Henry Proudhon)
1771ee7 corrected small bug in HST_read parameters (Henry Proudhon)
caf7955 new method to generate a microstructure with a random texture (Henry Proudhon)
b9be508 added new elevationFilter (Henry Proudhon)
04a4aa9 added num_color parameter to show_grains (Henry Proudhon)
df2b20d new xray package with a first example (Henry Proudhon)
215f1bf corrected small bug with cut option in auto_min_max (Henry Proudhon)
b5b0c7d new method for inverting a vtk lookup table (Henry Proudhon)
d6d4bce added new vtk stuff for displaying a pin hole and a Fresnel zone plate (Henry Proudhon)
71b7d4d method to show xray arrow and fix to the slits (Henry Proudhon)
4fa08d4 new method read_euler_txt (Henry Proudhon)
f3402b2 new method to display X-ray slits (Henry Proudhon)
a757cda new method to import a list of orientation from a text file and 2 new pole figure examples with 10000 orientations (Henry Proudhon)
cbe74e9 changed default ipf color to black, removed unecessary output (Henry Proudhon)
7b420f4 small docstring fixes (Henry Proudhon)
fb5ae80 Worked on adding the possibility of plot direct and inverse PF not in reference to Z (Nicolas GUENINCHAULT)
836f143 updated my_fun.py, adding a function to plot crystal rotations into ipf, sst ,… (Nicolas GUENINCHAULT)
8e10d45 adding my_fun file containing simple functions using pymicro. For now my_fun contain one class ‘Nfun’ with two functions dedicated to compute Schmid factors (Nicolas GUENINCHAULT)
34d6516 some bugs get fixed … (Nicolas GUENINCHAULT)
1d40e45 test (Nicolas GUENINCHAULT)
cfa3dde Corrected x1, x2, x3 instead of x1, x1, x1 (Erembert Nizery)
076c91f Merge branch ‘master’ of vcs:pymicro (Erembert Nizery)
5da5d5e added normalized vectors to compute B. (thank you Erembert ;-) ) (Nicolas GUENINCHAULT)
861aef1 Merge branch ‘master’ of vcs:pymicro (Erembert Nizery)
2d09cd9 Merge branch ‘master’ of vcs:pymicro (Erembert Nizery)
9d117f3 Merge branch ‘master’ of vcs:pymicro (Nicolas GUENINCHAULT)
83209f3 added the possibility of computiong the orientation matric from rotation like in a .inp file from Zebulon : Zrot2OrientationMatrix (Nicolas GUENINCHAULT)
0004dc5 updated pole figure doc example (Henry Proudhon)
fb8b165 pole figures can now be plotted with respect to X, Y or Z direction (Henry Proudhon)
1d2679a Merge branch ‘master’ of vcs:pymicro (Henry Proudhon)
9574b8d lots of update to handle upgrade to centos7 with backward compatibility with centos5 (Henry Proudhon)
de17c9d added new example with map_data_with_clip (Henry Proudhon)
96b5538 added new test for vtk numpy array (Henry Proudhon)
329321f conf.py now gets the version number in the main __init__.py file (Henry Proudhon)
505ebd6 adding modified symetry function in plot_sst (ss_syletry to sst_symetry_cubic) (Nicolas GUENINCHAULT)
f48dc7a Merge branch ‘master’ of vcs:pymicro (Henry Proudhon)
7b94c4d updated docstrings for dct_omega_angles (Henry Proudhon)
dd5dd78 added reciprocal lattice calcultion (Henry Proudhon)
e9b1daa Bug corrected (function sst_symmetry_cubic called instead of sst_symmetry) (Erembert Nizery)
2ddb457 Merge branch ‘master’ of vcs:pymicro (Henry Proudhon)
58cc9ee new example to display a polycrystal in 3d (Henry Proudhon)
72404c9 new load_STL_actor and show_data methods (Henry Proudhon)
7bce0fb removed .pyc test files from tree (Henry Proudhon)
067e52b moved tests for HklPlane in single file (Henry Proudhon)
adc3ccd First version of fastcrystal.py (Erembert Nizery)
be2673a Plotting two points for directions lying in plane. (Erembert Nizery)
c1be990 Reduced IPF set as default in plot_pole_figures. (Erembert Nizery)
607bd2c sst_symmetry_cubic corrected (used for IPF plot) (Erembert Nizery)
24563a0 No change - only test. (Erembert Nizery)
af9999e Merge branch ‘master’ of vcs:pymicro (Nicolas GUENINCHAULT)
8754ac0 just a test (Nicolas GUENINCHAULT)
e0ed245 fix HST_read while using autoparse_filename option (Henry Proudhon)
acc05b2 added non single atom basis for unit cells (Henry Proudhon)
f1743ca new hcp crystal example (Henry Proudhon)
dca3079 fix show_grains not showing grain 1 (Henry Proudhon)
db72a68 fix print statement in edf_read (Henry Proudhon)
131cc1d fix version number for tagging (Henry Proudhon)
ef62d20 fix version number for tagging (Henry Proudhon)
17cf50e added all the possible lattice centering and subsequent fixes in docs and examples (Henry Proudhon)
0a9ba28 moved wxPlotPanel.py to apps sub-package (Henry Proudhon)
f710e6c added new static method to easily plot a pole figure for a single orientation, docstring fixes (Henry Proudhon)
92a2002 added new method map_data, new options to map_data_with_clip, fixed many docstrings (Henry Proudhon)
937993c added math to Orientation docstring (Henry Proudhon)
19dc758 moved wxPlotPanel.py to apps package (Henry Proudhon)
042faf4 fixed note directives in doctrings (Henry Proudhon)
b427466 fixed vtk.util mock (Henry Proudhon)
5c13de4 small doctring fixes and variable renaming (Henry Proudhon)
272498d fixed import * for vtk colors (Henry Proudhon)
15b8466 several fixes to edf_write, now handle SignedInteger encoding (Henry Proudhon)
94bb046 new method to compute euler angle as in MandelCrystal (Henry Proudhon)
819bf56 added more matplotlib mocks (Henry Proudhon)
76c1543 added more mocks (Henry Proudhon)
ba09f84 removed unecessary toctree maxdepth option (Henry Proudhon)
9264ef0 many docstrings fixes and new function plot_sst (Henry Proudhon)
825052f many docstrings fixes (Henry Proudhon)
ab42227 modified __init__ files (Henry Proudhon)
71cc94d new figure AlLi_sam8_pole_figure for documentation (Henry Proudhon)
3a902f8 small fix in new option autoparse_filename (Henry Proudhon)
8807aa4 added cookbook with first recipe (Henry Proudhon)
d9de338 mock tifffile to build doc (Henry Proudhon)
c428b7b subsequent modifications in edf_read and edf_write (Henry Proudhon)
84f1687 fixed Size field in edf header (Henry Proudhon)
4dc6d2e now using mock module to help build documentation (Henry Proudhon)
f4bf3e9 trying to fix sphinx path on remote server (Henry Proudhon)
36a4eb8 trying to fix sphinx path on remote server (Henry Proudhon)
83d081b removed unused jsMath and MathJax from tree (Henry Proudhon)
05eed8f trying to fix sphinx path on remote server (Henry Proudhon)
3ea96dd trying to fix sphinx path on remote server (Henry Proudhon)
0e25b9a modified files not to depend on matplotlib (Henry Proudhon)
293f4c6 added thumb images files for example gallery (Henry Proudhon)
8d022c8 added schmid factor calculations (for octaedral slip) (Henry Proudhon)
e080880 change number of bytes to 512 to peek in header in edf_info (Henry Proudhon)
18e7bd5 do not include auto_example in the tree (Henry Proudhon)
fd4f935 file changes for the automated gallery of example (Henry Proudhon)
38ea32b added an automated gallery of example (Henry Proudhon)
0c0b002 added max_opacity option in vtkSetVisibility animation (Henry Proudhon)
dc540a9 new function show_grains (Henry Proudhon)
e756429 fixes path to example files (Henry Proudhon)
84b9f42 small fixes to cubic_crystal_3d example (Henry Proudhon)
94259eb very small fix to lattice_3d_with_planes function for opacity (Henry Proudhon)
2802f36 make current image name readonly (StaticText) (Henry Proudhon)
fd7e133 added docstrings for edf_info and unpack_header (Henry Proudhon)
c1a6757 new functions edf_info and esrf_to_numpy_datatype (Henry Proudhon)
bd7cbd7 fixed doctstring for alpha_cmap (Henry Proudhon)
d79ebd4 moved example files (Henry Proudhon)
125f6bf updated documentation with examples (Henry Proudhon)
6e79251 added flat field correction function (Henry Proudhon)
7dbd525 more __init__ stuff (Henry Proudhon)
c1c1557 now load tif files as well (Henry Proudhon)
ff80f6e added recad util functions (Henry Proudhon)
9afc721 documented some functions likre vtk_write (Henry Proudhon)
8a0c100 small fix to the raw_mar_read function (Henry Proudhon)
057911c added/fixed docstrings (Henry Proudhon)
3e592b2 moved all examples to different subfolders (Henry Proudhon)
a4d68a1 new application to view image files in a folder (Henry Proudhon)
dbfa470 small change on how to get the image dim from the header in edf_read (Henry Proudhon)
8f161ae added mousse_3d example (Henry Proudhon)
7b23d95 bug fix in bragg calculation and minor docstring changes (Henry Proudhon)
6b9ead1 added more families in HklPlane get_family method (Henry Proudhon)
dba603f fixed docstrings in the microstructure module (Henry Proudhon)
6910930 added bragg_angle method with unit testing (Henry Proudhon)
85d07ca added image for hist function (Henry Proudhon)
9469a33 small docstrings changes (Henry Proudhon)
8f5499c added new method dct_omega_angles to the Grain class (Henry Proudhon)
80ab017 small docstrings changes (Henry Proudhon)
7f30e18 changed origin to lower in show_and_save function (Henry Proudhon)
0e2d8f1 added save option in render function and fixed the documentation (Henry Proudhon)
7706ffa a few more documentation fixes (Henry Proudhon)
5fa810f fixed the documentation of the hist function (Henry Proudhon)
b9696d9 added examples folder and some documentation of these examples (Henry Proudhon)
2204068 added new method show_and_save for a 2d image (Henry Proudhon)
fd873c9 improve documentation in vtk_utils (Henry Proudhon)
2bf1e91 added new vtk function map_data_with_clip (Henry Proudhon)
64f3871 improved documentation for color maps (Henry Proudhon)
a2b90e2 added test for tif file (Henry Proudhon)
c4d6dce added tifffile module (Henry Proudhon)
945257e improved documentation for vtk_utils (Henry Proudhon)
63465a0 improved the documentation with sphinx (Henry Proudhon)
9e8a10e added pymicro logo (Henry Proudhon)
2dd3e8b added Dependencies section to the documentation (Henry Proudhon)
5f437ac switched to shpinx theme proBlue (Henry Proudhon)
46d3ceb updated documentation (Henry Proudhon)
2d9cb0b remove old rst files from the crystal package (Henry Proudhon)
b9c398b updated documentation (Henry Proudhon)
7b9ddef new method grain_3d with subsequent changes (Henry Proudhon)
bbedd5e added verbose mode to hkl plane normal method (Henry Proudhon)
2fa7585 added modules in sphinx, updated documentation (Henry Proudhon)
5fd4612 deleted old unmaintained stuff (Henry Proudhon)
0a54b50 changed numpy import (Henry Proudhon)
1396e7a added new classes to handle animations through a 3d scene (Henry Proudhon)
3caf836 added support for hkl planes in hexagonal lattices (Henry Proudhon)
fec27d8 fixed a small issue with dot product for python2.6 at esrf (Henry Proudhon)
4c48986 added opacity control in add_plane_to_grid function (Henry Proudhon)
69fb39c bug correction in HklPlane normal (Henry Proudhon)
b66f374 removed printed output (Henry Proudhon)
8f3cbb8 several corrections + added hexagonal 3d lattice handling (Henry Proudhon)
3899715 remove old diffract.py file (Henry Proudhon)
af2c658 remove old grains.py file (Henry Proudhon)
7747c77 remove old grain_conn.py file (Henry Proudhon)
e474f65 remove old chg_label.py file (Henry Proudhon)
78e094b remove old grain_53.py file (Henry Proudhon)
539c61b remove old animp.py file (Henry Proudhon)
8ec9806 added a grain_ids filter to load a microstructure from an XML file (Henry Proudhon)
fd20143 added .gitignore file (Henry Proudhon)
cc51795 new file to handle animation (rotation around Z axis for now) (Henry Proudhon)
1ac249b added new add_grain_to_3d_scene method (Henry Proudhon)
fcab593 added new method: lattice_3d_with_planes (Henry Proudhon)
56858bc bug correction in slip plane rotation (Henry Proudhon)
370a5cd added a way to control which planes are used in add_HklPlanes_with_orientation_in_grain (Henry Proudhon)
5798f2d added a box_3d method (Henry Proudhon)
9a98c38 improve consistency in file names when saving microstructure in xml format (Henry Proudhon)
54c0693 added verbose mode to add_vtk_mesh in class Grain (Henry Proudhon)
c9c5758 small corrections like phi1 in phi1() (Henry Proudhon)
4d995ed added scaling possibility to unit_arrow_3d (Henry Proudhon)
622e39d removed shpinx doc build from tree (Henry Proudhon)
50d2923 removed .pyc files rom tree (Henry Proudhon)
5cfdc17 started to remove .pyc files (Henry Proudhon)
25629f0 small changes in pole figure legend handling (Henry Proudhon)
d683db9 small update of slip_traces doc (Henry Proudhon)
2f6ae4e added slip_traces methods for HklPlane class (Henry Proudhon)
b905bed added interplanar_spacing calculation for HklPlane class (Henry Proudhon)
15f28fc remove attribute normal from the HklPlane class (a method exists) (Henry Proudhon)
f90f19c added lattice attribute to the HklPlane class (Henry Proudhon)
5c64e60 corrected a small bug in get_family (Henry Proudhon)
245211d added doc to the get_family method (Henry Proudhon)
0fe48e5 added a static get_family method to the HklPlane class (Henry Proudhon)
4da82b5 changed default clipping range in setup_camera (Henry Proudhon)
6bfc2b5 added verbose option to read_image_data, changed setupCamera name to setup_camera (Henry Proudhon)
0ed75d4 fix header in edf_write according to data type (Henry Proudhon)
e5f61e8 added alpha_cmap method (Henry Proudhon)
666ed65 added a method to automatically setup the vtk camera (Henry Proudhon)
9cc44b5 fixed issue with .info file in HST_write (Henry Proudhon)
5732820 added VTK_UNSIGNED_INT to uint32 equivalence (Henry Proudhon)
b6aa3e6 added density option in grey level histogram plotting (Henry Proudhon)
ed6ad97 corrected prefix variable in grey level histogram plotting (Henry Proudhon)
d4e321d added new grey level histogram plotting (Henry Proudhon)
6fc4d64 add variable header in read_image_data (Henry Proudhon)
a5c51f7 fixed orientation issue in add_hklplane_to_grain (Henry Proudhon)
d873f01 added float and double conversion from numpy to vtk (Henry Proudhon)
b266ec4 merged contourByDiscreteMarchingCubes intour contour filter (Henry Proudhon)
d6c6735 removed crystal/microstructure.pyc (Henry Proudhon)
9b985d3 added contour filter method (Henry Proudhon)
fdbb00b corrected bug in OrientationMatrix2Euler when Phi=0 (Henry Proudhon)
230f395 changed from PyMicro to pymicro (Henry Proudhon)
3a7e3bf Show only one point per grain in legend in direct pole figures (Henry Proudhon)
a733719 added custom legend for direct pole figure (Henry Proudhon)
d42c915 updated apply_orientation_to_actor for new Orientation class syntax, added custom color for unit_arrow_3d (Henry Proudhon)
364d72b added euler angle corrections from orientation matrix (Henry Proudhon)
d2e754d improved pole figures (Henry Proudhon)
e23c32b added project documentation through sphinx (Henry Proudhon)
pymicro version 0.1.0¶
ce0ce0b changed from white color to (1,1,1) (Henry Proudhon)
008a44c cleaned up crystal_lattice_3d (Henry Proudhon)
a12ee2f documentation small correction (Henry Proudhon)
0cadf68 documentation small corrections (Henry Proudhon)
d86433a added several helper vtk functions (read, outline, render, contour) (Henry Proudhon)
b028bd2 added add_outline method in vtk_utils (Henry Proudhon)
9dac6e3 update happy new year (Henry Proudhon)
d8eb5f3 initial project version (Henry Proudhon)
References¶
- 1
J K MacKenzie. Second paper on statistics associated with the random distribution of cubes. Biometrika, 45:229–240, 1958.
- 2
D Rowenhorst, A D Rollett, G S Rohrer, M Groeber, M Jackson, P J Konijnenberg, and M De Graef. Consistent representations of and conversions between 3D rotations. Model. Simul. Mater. Sci. Eng., 23(8):083501, dec 2015. doi:10.1088/0965-0393/23/8/083501.