Pymicro Data Platform: The Sample Data and Microstructure Classes User Guide

The pymicro package is based on a data format designed to create, organize and manage efficiently complex multi-modal datasets, used in material science, with a particular focus on the mechanics of microstructures. These data sets are said to be multi-modal because they bring together data from various measurement and numerical simulation techniques, originally produced in very different formats.

The pymicro.core package implements a data platform class allowing to meet the numerous challenges raised by the complexity and size of these datasets. Pymicro’s main class Microstructure is now based on this data format. The data platform is based on a central class, SampleData, that implements the backend and user interface with the datasets.

This User Guide objective is to serve as a documentation of the classe, and as a guide to learn step by step the SampleData file format, data model, and the use of the class to manipulate complex and multimodal datasets. It is composed of a serie of Jupyter notebooks, that have been integrated into the Pymicro package documentation.

The first documentation page/notebook is an introduction, and does not contain Python code. It is dedicated to the presentation of the challenges that motivated the implementation of this data platform, and of its main features.

The introduction is not yet part of the documentation, it will be as soons as the presenting publication of the code will be released.

The other pages/notebooks are tutorials that will introduce you to the various functionnalites of the platform, the different data item types that can be stored into dataset, and their data model. They are introduced from the simplest to the most complex. Hence, it is adviced to read through them in the indicated order. You will find at the end of each page/notebook a short summary of the functionalities and code lines presented. All these summary have been gathered in a complete quick reference sheet, in the last page/notebook of this guide.

If you are reading through this on the on-line documentation, please note that the associated notebooks can be found in the pymicro package directories, under the path pymicro_package_path/examples/SampleDataUserGuide/. You can open these files with the jupyter-notebook program so that you can run the code as you read this documentation, and get familiar with the use of SampleData. If you do not have jupyter installed in your python environement, you may consult the dedicated webpage of the Jupyter project to help you get it: https://jupyter.org/install.

Even without jupyter-notebooks, you may reproduce the code content of this guide within any Python interactive console (like ipython). We strongly encourage you to do so while reading through this user guide !

• 1 - Introduction to the SampleData data platform
  • I - The SampleData platform
  • II - SampleData dependencies and pre-requisites
• 2 - Getting started with SampleData : Exploring dataset contents
  • I - Create and Open datasets with the SampleData class
  • II - The SampleData Naming system
  • III- Interactively get information on datasets content
  • IV - Visualize dataset contents with Vitables
  • V - Visualize datasets grids and fields with Paraview
  • VI - Using command line tools
• 3 - Creating and retrieving basic data items
  • I - Interactively get data from data items
  • II - HDF5 Groups
  • III - HDF5 attributes
  • IV - Data arrays
  • V - String arrays
  • VI - Structured arrays
  • VII - Removing data items from datasets
• 4 - Creating, getting and visualizing Image Groups and Image Fields
  • I - SampleData Image Groups
  • II - Image Groups Data Model
  • III - Creating Image Groups
  • IV - Image Fields data model
  • V - Adding Fields to Image Groups
  • VI - Getting Images and Fields
• 5 - Creating, getting and visualizing Mesh groups and Mesh Fields
  • I - SampleData Mesh Groups
  • II - Mesh Groups Data Model
  • III - Creating Mesh Groups
  • IV - Mesh Field data model
  • V - Adding Fields to Image Groups
  • VI - Getting Mesh Groups and Mesh Fields
• 6 - Data Compression
  • Data compression with HDF5 and Pytables
  • Compressing your datasets with SampleData
• 7 - Sample Data Inheritance - The Microstructure Class
  • SampleData Inheritance
  • The Microstructure Class
  • Microstructure Data Model & Getting Microstructure data
  • Creating and setting Microstructures
• SampleData Quick Reference Sheet
  • Notation
  • Creating, Opening datasets, and Exploring their content
  • Basic data items: creating and getting them
  • Image Groups and Image fields: creating and getting them
  • Mesh Groups and Mesh Fields: creating and getting them
  • Data Compression