{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "23081c10-5037-4d27-9b72-acb4bd915d1a",
   "metadata": {},
   "source": [
    "# Polycrystalline Datasets -- The Microstructure class"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "408706ec-745d-4bac-b01a-db50365affd0",
   "metadata": {},
   "source": [
    "This tutorial focus on the polycrystalline datasets that can be built and processed with Pymicro, and on the data model and content of these datasets.\n",
    "\n",
    "The interface with these datasets is the `Microstructure` class of the `pymicro.crystal.microstructure` module. This class is a derived from the `SampleData` class, that implements a generic interface for complex multimodal HDF5 scientific datasets. As such, the `Microstructure` class and its datasets inherits all the features of the `SampleData` class. Hence, a `Microstructure` object will be automatically associated with a HDF5 file that will contain the dataset associated to this instance. \n",
    "\n",
    "A practical knowledge of the manipulation of `SampleData` and associated datasets is strongly recommended to go through this tutorial. For a detailed review of the features related to the `SampleData` class, please refer to the [data management tutorials](./Data_Management_tutorial.rst). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b708782c-cb3d-423e-b6d2-2d07b05f7b39",
   "metadata": {},
   "source": [
    "## Microstructure datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e9180e5-7b7f-448d-a71f-baa34061b3c5",
   "metadata": {},
   "source": [
    "*Pymicro* examples data base contains several polycrystalline samples datasets built with the `Microstructure` class. We will observe these datasets to introduce them.\n",
    "\n",
    "This file is zipped in the package to reduce its size. If you are reading through this tutorial as a Notebook, you will first have to unzip the file: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7dbc1060-080b-4eff-9132-8e6336221672",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pymicro import get_examples_data_dir # import file directory path\n",
    "PYMICRO_EXAMPLES_DATA_DIR = get_examples_data_dir() # get the file directory path\n",
    "import os \n",
    "dataset_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure') # test dataset desired file path\n",
    "tar_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR, 'example_microstructure.tar.gz') # dataset zipped archive path\n",
    "\n",
    "# Save current directory\n",
    "cwd = os.getcwd()\n",
    "# move to example data directory\n",
    "os.chdir(PYMICRO_EXAMPLES_DATA_DIR)\n",
    "# unarchive the dataset\n",
    "os.system(f'tar -xvf {tar_file}')\n",
    "# get back to UserGuide directory\n",
    "os.chdir(cwd)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee601e5-bccb-4fed-a64a-64d3adb0c740",
   "metadata": {},
   "source": [
    "Let us now open the dataset, using the `Microstructure`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87f53ce2-2ace-4c89-bec6-3fee7a4358a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import SampleData class\n",
    "from pymicro.crystal.microstructure import Microstructure "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9072b38a-b66e-49e4-9a05-61158844d150",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open Microstructure dataset\n",
    "micro = Microstructure(filename=dataset_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7867f62f-b24e-4d13-a221-c9476f04c323",
   "metadata": {},
   "source": [
    "The dataset is now open, we can now look at its content:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19afac89-5f64-4dbe-ae86-4f82c84ff65c",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro.print_dataset_content(short=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a8008d-2d1d-4ecd-88bd-f80b3a1a93ba",
   "metadata": {},
   "source": [
    "We can also print the description attribute of the dataset to read information written by its creators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55379216-777d-4274-9274-c9ba8f0be8b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(micro.get_description())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a887f9e0-0258-4f3c-be25-a8ee83001631",
   "metadata": {},
   "source": [
    "This second print inform us that this dataset refer to a polycristalline microstructure of a grade 2 Titanium sample, gathering both experimental and numerical data obtained through Diffraction Contrast Tomography imaging, and FFT-based mechanical simulation. \n",
    "\n",
    "The first print shows that the dataset contains various groups of data:\n",
    "* a group `PhaseData` that contains information on the sample phases (a single phase in this case)\n",
    "* a group `CellData` that contains maps (*fields*) of the grains, *i.e.* a mapping of the microstructure, and a stress field obtained via numerical simulation. This group indicates that the mapping of the phases in the sample is not provided.\n",
    "* a group `MeshData` that contains a mesh of the grains in the sample\n",
    "* a group `AmitexResults` that contains simulation results \n",
    "* a group `GrainData` that contains statistical data for each grain of the microstructure, stored in a structured table."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72deedef-d20c-4d65-b257-be4c9cb7cbf8",
   "metadata": {},
   "source": [
    "These various elements are **data items** introduced by the `SampleData` class (Groups, Image/Mesh groups, structured tables, arrays, field arrays...). Again, to get a detailed tutorial on these object and how to interact with them, please refer to the [data management tutorials](./Data_Management_tutorial.rst). \n",
    "\n",
    "To get a better idea of the microstructure, you can observe the following slice of the mapping of the grains stored in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a953e85-9870-45aa-bb1f-4df359569f3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro.view_slice(display=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2c0cd39-732b-49d8-b46d-1c760da5b313",
   "metadata": {},
   "source": [
    "We can now close this example and see how to create datasets with the class. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f680c67f-c81f-4eb0-be15-fe6cb6187362",
   "metadata": {},
   "outputs": [],
   "source": [
    "del micro"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "453bc24a-1953-4a91-b5d2-d5fb877fe68d",
   "metadata": {},
   "source": [
    "## Creating Microstructures"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91e3daa5-d457-40f3-b74b-318b82b17026",
   "metadata": {},
   "source": [
    "There are four ways to create a *Microstructure* dataset:\n",
    "\n",
    "1. Creating an empty microstructure dataset \n",
    "2. Copying an existing microstructure dataset \n",
    "3. Crop a region from an existing microstructure dataset\n",
    "3. Create a microstructure from data files, such as outputs of imaging techniques (reconstruction of DCT or EBSD scans for instance) or microstructure generation tools (such as Neper)\n",
    "\n",
    "Lets start by creating a empty microstructure dataset. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47ce953b-aca2-4887-9aae-0b9ce0a0e198",
   "metadata": {},
   "source": [
    "### Creating an empty microstructure dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc6e4c55-1849-4c34-a204-bf5155a8e84e",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro = Microstructure(filename='test_microstructure', overwrite_hdf5=True, autodelete=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13fdc42c-364b-442d-8ac5-17c94de5d237",
   "metadata": {},
   "source": [
    "The `overwrite_hdf5` instruction is used to overwrite any pre-existing dataset with the same filename, and the `autodelete` instruction ensures that the dataset will be removed when we close the class instance. The datasets created with the `Microstructure` have a predefined data model that is automatically generated (see the tutorial on [custom data models](./Curstom_Data_Models.ipynb)) : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fa2d2d7-509d-4574-a69f-a9bb17201146",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro.print_dataset_content(short=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09efeba1-a132-4379-ad25-40d892d9dbd4",
   "metadata": {
    "tags": []
   },
   "source": [
    "This data model is composed of the `CellData` group to store maps of the microstructure, the `GrainData` group to store information on grains, and the `PhaseData` group to store information on the sample phases. Many methods of the `Microstructure` class interact with these data items, that are the most common and important information used to study a material sample. Naturally, all these data items are empty in this just created dataset. The next tutorials will detail how to store actual data into a microstructure object.\n",
    "\n",
    "We will now close the dataset and see how to create a dataset from an existing one. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fcdadbd-0afc-4c14-9b78-9d6e2a9f4418",
   "metadata": {},
   "outputs": [],
   "source": [
    "del micro"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25366e9b-e1e4-4c5a-8d5f-7b7251695597",
   "metadata": {},
   "source": [
    "### Copy a microstructure dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f030c52c-ffa4-4339-8fa1-1c053255fe0b",
   "metadata": {},
   "source": [
    "As for *SampleData* datasets, *Microstructure* datasets can be copied from already existing one, using the `copy_sample` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed008453-cbcb-4745-bae8-43deb246fed4",
   "metadata": {},
   "outputs": [],
   "source": [
    "original_file = os.path.join(PYMICRO_EXAMPLES_DATA_DIR,'t5_dct_slice_data.h5')\n",
    "micro_copy = Microstructure.copy_sample(src_micro_file=original_file, dst_micro_file='test_microstructure',\n",
    "                                       get_object=True, autodelete=True, overwrite=True)\n",
    "print(micro_copy)\n",
    "micro_copy.view_slice(display=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c22d3c2-5b0a-400c-9920-6688eefb3bf7",
   "metadata": {
    "tags": []
   },
   "source": [
    "The new dataset file is created with the same content as the copied dataset. This process can be used to keep a reference dataset unchanged and work on its data on a separate dataset with no risk of corrupting the original data. \n",
    "\n",
    "Sometimes, it is relevant to work only on a small region of interest of a dataset. In that case, dealing with a large dataset can be a burden. To solve this issue, you can create a dataset of the region of interest by using the `crop` method. We will now close the copied dataset and see how to create such a cropped dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3056f2e-ca93-48c1-97dd-88c6ac9b7d29",
   "metadata": {},
   "outputs": [],
   "source": [
    "del micro_copy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0ded2a6-89c7-4ecb-9e95-e2df540edec2",
   "metadata": {},
   "source": [
    "### Crop an existing Microstructure"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13b12e4-8eb9-4698-aa87-0235626ae886",
   "metadata": {},
   "source": [
    "The `crop` method:\n",
    "\n",
    "* creates a new Microstructure dataset, with the same name plus the suffix `_crop`, or the name specified by the optional argument `crop_name`\n",
    "* crops all fields of the `CellData` group of the original Microstructure, by extracting the subregion indicated by the `x_start, x_end, y_start, y_end, z_start, z_end` arguments (bounds indices of the cropped region). Then, it adds them to the `CellData` group of the new Microstructure.\n",
    "* fills the *GrainDataTable* of the new microstructure with only the grains contained in the cropped region, and recomputes the grains geometric data for the new grain map, unless argument `recompute_geometry` is set to `False`.\n",
    "\n",
    "Let us try to crop a small region of our first example microstructure dataset. First we need to open again the first microstructure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f8481ef-b630-458d-aca0-5b71234606b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro = Microstructure(filename=dataset_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91bed0e2-0508-4e8c-a30a-43f53de14957",
   "metadata": {},
   "source": [
    "and then use its `crop` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dcf19ef-4225-4f15-b618-8d9d9d20cc17",
   "metadata": {},
   "outputs": [],
   "source": [
    "micro_crop = micro.crop(x_start=30, x_end=70, y_start=30, y_end=70, z_start=30, z_end=70, crop_name='test_crop',\n",
    "                       autodelete=True)\n",
    "micro_crop.print_dataset_content(short=True)\n",
    "micro_crop.view_slice(display=True) # set to True to try reproducing the figure below !"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2e9131a-fa47-44ca-9acc-87732f31478f",
   "metadata": {},
   "source": [
    "         \n",
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "**Warning** \n",
    "    \n",
    "Cropping a microstructure can be long if the original microstructure is heavy and has a lot a fields for the `CellData` group. If you only want to crop some of these fields, you may want to create a new microstructure, add to its `CellData` group only the fields you want to crop, and then create your crop from this new instance.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b3caaf-21bb-41e1-ad77-5002baf188a6",
   "metadata": {},
   "source": [
    "We have now reviewed three methods to create a polycrystalline dataset with the `Microstructure` class. Specific tutorials are dedicated to the creation of datasets from external data files. We can now close our datasets, and move the next tutorials, that will teach you how to interact with the different type of data stored into a microstructure ! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3fa74932-0d1a-4aa2-95dc-c545675b429e",
   "metadata": {},
   "outputs": [],
   "source": [
    "del micro\n",
    "del micro_crop "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env_dev",
   "language": "python",
   "name": "env_dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}