{ "cells": [ { "cell_type": "markdown", "id": "b57c8ba2", "metadata": {}, "source": [ "# Image based data handling with Pymicro " ] }, { "cell_type": "markdown", "id": "4297f61d", "metadata": {}, "source": [ "This Notebook will introduce you to the creation and handling of image based data, *i.e.* a set of data arrays that define fields on a regular grid, and the associated metadata that defines the grid topology. " ] }, { "cell_type": "markdown", "id": "0c448efa", "metadata": {}, "source": [ "## I - SampleData Image Groups" ] }, { "cell_type": "markdown", "id": "ca41bfde", "metadata": {}, "source": [ "### SampleData Grids" ] }, { "cell_type": "markdown", "id": "926b0489", "metadata": {}, "source": [ "Spatially organized data (2D or 3D data), which includes field measurements, imaging techniques and numerical simulation outputs, is central to materials science. Each of those outputs can be seen as a set of fields supported by a grid. \n", "\n", "A **grid** is a set of nodes and elements that define a discretization of a geometrical domain. A **field** is a scalar or tensorial function defined on this domain, defined by an array containing the values that it takes at grid nodes or grid elements. A **field time serie** is a set of numerical arrays that define the values of a field at different times.\n", "\n", "Hence, those three types of information (geometry, time and field values) must be stored in the dataset for spatially organized data. To this end, `SampleData` defines **Grid groups**. The specific structure and metadata of these groups is used to automatically generate an associated XDMF file to ensure visualization with the *Paraview* software of all spatially organized data in the dataset on the relevant grid and time value. \n", "\n", "There are two types of **Grid groups**: *Image* groups, and *Mesh* groups. This tutorial focuses on Image groups." ] }, { "cell_type": "markdown", "id": "47dbc292", "metadata": {}, "source": [ "### SampleData Images " ] }, { "cell_type": "markdown", "id": "493b6d54", "metadata": {}, "source": [ "As indicated by their name, **Image Groups** are a specific type of Grid group whose data model is adapted to store images. Images are 2D or 3D regular grids of pixels(2D)/voxels(3D), all having the same dimensions. To this pixel/voxel grid is also associated a node grid, composed by the vertexes of the pixels/voxels. Three types of *Image Groups* can be handled by *SampleData*:\n", "\n", "\n", "1. `emptyImage`: an empty group that can be used to set the organization and metadata of the dataset before adding actual data to it (similar to [empty data arrays, see last tutorial, section IV](./Data_Items.ipynb))\n", "2. `2DImage`: a regular grid of pixels \n", "3. `3DImage`: a regular grid of voxels\n", "\n", "An Image field is a data array that has one value associated to each pixel/voxel or each node of the image. " ] }, { "cell_type": "markdown", "id": "4ec59daa", "metadata": {}, "source": [ "Hence, *Image Groups* contain metadata indicating the image topology, data arrays containing the image fields values, and metadata to provide time values associated to the grid and the stored field values. To explore in details this data model, we will once again open the reference test dataset of the *SampleData* unit tests, in the next section." ] }, { "cell_type": "markdown", "id": "53400502", "metadata": {}, "source": [ "## II - Image Groups Data Model" ] }, { "cell_type": "code", "execution_count": null, "id": "9f5caf21", "metadata": {}, "outputs": [], "source": [ "from pymicro.core.samples import SampleData as SD" ] }, { "cell_type": "code", "execution_count": null, "id": "f4c9c4ac", "metadata": {}, "outputs": [], "source": [ "from pymicro import get_examples_data_dir # import file directory path\n", "import os\n", "dataset_file = os.path.join(get_examples_data_dir(), 'test_sampledata_ref') # test dataset file path\n", "data = SD(filename=dataset_file)" ] }, { "cell_type": "markdown", "id": "e292c879", "metadata": {}, "source": [ "Let us print the content of the dataset to remember its composition:" ] }, { "cell_type": "code", "execution_count": null, "id": "36dde027", "metadata": {}, "outputs": [], "source": [ "print(data)" ] }, { "cell_type": "markdown", "id": "a901d1c0", "metadata": {}, "source": [ "The test dataset contains one *3DImage* Group, `test_image`, with indexname `image`. Let us print more information about this Group:" ] }, { "cell_type": "code", "execution_count": null, "id": "2084c82f", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image')" ] }, { "cell_type": "markdown", "id": "30c7ff29", "metadata": {}, "source": [ "As you can observe, this 3DImage group already contains a lot of metadata. All of them are automatically created when adding a new Image to a *SampleData* dataset, and are part of the *SampleData* Image data model. We will review the most important ones in this paragraph.\n", "\n", "First, the `group_type` attribute informs us the the Group is a *3DImage* group. In the case of a 2D image, it would have the value *2DImage*. The `empty` attribute is False, indicating that the image contains fields data. They are indicated in the `Childrens` section of the node information print. We will examine them later on in this section. \n", "\n", "For now, let us focus on the attributes linked to the topology of the image:\n", "\n", "1. `dimension`: the number of pixels/voxels in each direction of the grid\n", "2. `nodes_dimension`: the number of nodes in each direction of the grid (always `dimension` + 1)\n", "3. `origin`: the coordinate of the first node of the grid\n", "4. `spacing`: the size along each direction of each pixel/voxel of the grid\n", "\n", "**For all those attributes, the order of the values in the arrays correspond to the directions (X,Y) or (X,Y,Z)**. \n", "\n", "**The user as to ensure that the scale of geometrical data provided for various grid groups are consistent with each other, no length unit is assumed for grid dimension attributes.** However, user are free to create attribute to specify the unit length corresponding to the numbers in the dataset.\n", "\n", "The `time_list` attribute contains three time values. This indicates the possible time values at which the values of the `test_image` fields are associated to. " ] }, { "cell_type": "markdown", "id": "49a4e49e", "metadata": {}, "source": [ "Before closing our reference dataset, Let us try to visualize its image Group, with the `pause_for_visualization` method and the Paraview software. You have to choose the **XdmfReader** to open the file, so that Paraview may properly read the data. You will see in the **Hierarchy** panel, the presence of two **Blocks** of data, one for each grid of the dataset, divided in subgrids for each time value stored in the dataset:\n", "\n", "\n", "\n", "To easily visualize the Image group, untick the mesh group box in the appropriate panel. Now you can click on the `Apply` button: you should now see the 3D image. You can choose to plot `test_image_field`, which takes the value 1 over half of the image nodes, and 0 on the other half. It should render like this: \n", "\n", "\n", "\n", "As suggested by the **image group** children's names, this field has only one time value. On the contrary, you may also visualize the values of the field `test_tensor` for the three time values associated to the **image group**." ] }, { "cell_type": "code", "execution_count": null, "id": "020f8359", "metadata": {}, "outputs": [], "source": [ "# Use the second code line if you want to specify the path of the paraview executable you want to use\n", "# otherwise use the first line \n", "#data.pause_for_visualization(Paraview=True)\n", "#data.pause_for_visualization(Paraview=True, Paraview_path='/home/amarano/Sources/ParaView-5.9.0-RC1-MPI-Linux-Python3.8-64bit/bin/paraview')" ] }, { "cell_type": "markdown", "id": "48885c63", "metadata": {}, "source": [ "We will print again the information on the Image Group, to focus on the childrens off the **image group**:" ] }, { "cell_type": "code", "execution_count": null, "id": "eb4e032c", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image')" ] }, { "cell_type": "markdown", "id": "d67c33d7", "metadata": {}, "source": [ "Let us look at the content of the `test_image_field` node:" ] }, { "cell_type": "code", "execution_count": null, "id": "b7b19bce", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('test_image_field')" ] }, { "cell_type": "markdown", "id": "2e9969d3", "metadata": {}, "source": [ "This node is a field data item, as indicated by the `node_type` attribute. " ] }, { "cell_type": "code", "execution_count": null, "id": "e625b416", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image_Field_index')" ] }, { "cell_type": "markdown", "id": "bbdc78af", "metadata": {}, "source": [ "The Field Index is a *String Array* (see [previous tutorial (section IV)](./Data_Items.ipynb)) that stores a list of strings that are the names of the fields stored in this Image group. You can get the list of field stored on the group by looking at this string array content:" ] }, { "cell_type": "code", "execution_count": null, "id": "bd0e5c95", "metadata": { "scrolled": true }, "outputs": [], "source": [ "data['image_Field_index'][2].decode('utf-8')" ] }, { "cell_type": "markdown", "id": "01697367", "metadata": {}, "source": [ "It can also be accessed easily using the `get_grid_field_list` method:" ] }, { "cell_type": "code", "execution_count": null, "id": "45fb536f", "metadata": {}, "outputs": [], "source": [ "data.get_grid_field_list('image')" ] }, { "cell_type": "markdown", "id": "5ec8c399", "metadata": {}, "source": [ "The `Field_index` array is stored in each **Grid** group, and like the dataset Index, contains the **indexnames** of the fields stored in the **Grid** group. \n", "\n", "Now that we know what composes an Image Group, and how to visualize it, we will close the reference dataset and study how to create image groups." ] }, { "cell_type": "code", "execution_count": null, "id": "b602e620", "metadata": {}, "outputs": [], "source": [ "del data" ] }, { "cell_type": "markdown", "id": "fd4c269a", "metadata": {}, "source": [ "## III - Creating Image Groups " ] }, { "cell_type": "markdown", "id": "dc5c7d88", "metadata": {}, "source": [ "There are three ways to create an Image group in a *SampleData* dataset:\n", "1. creating an image from a data array\n", "2. creating an image from an image object indicating its topology\n", "3. creating an empty image\n", "\n", "We will try all three. But first, we will once again create a temporary dataset, to try the examples of this tutorial, with verbose mode on:" ] }, { "cell_type": "code", "execution_count": null, "id": "5745a5f9", "metadata": {}, "outputs": [], "source": [ "data = SD(filename='tutorial_dataset', sample_name='test_sample', verbose = True, autodelete=True, overwrite_hdf5=True)" ] }, { "cell_type": "markdown", "id": "e0252eec", "metadata": {}, "source": [ "### Creation an image from a data array" ] }, { "cell_type": "markdown", "id": "111774e8", "metadata": {}, "source": [ "We will start by creating two *numpy* arrays that will be image fields. Their dimension will define the topology of the image. To have simple and visual examples, we will create:\n", "1. a 2D field of dimension 51x51 representing a matrix with a circular fiber at its center, with a diameter of half the image\n", "2. a random 3D field of dimension 50x50x50x3 \n", "\n", "We will start with the bidimensional field." ] }, { "cell_type": "code", "execution_count": null, "id": "db06edb6", "metadata": {}, "outputs": [], "source": [ "# first we need to import the Numpy package\n", "import numpy as np" ] }, { "cell_type": "code", "execution_count": null, "id": "0897f2c0", "metadata": {}, "outputs": [], "source": [ "dX = 1/101\n", "# we create a grid of coordinates to compute the distance to the center of the image\n", "X = np.linspace(dX, 1-dX,101)\n", "Y = np.linspace(dX, 1-dX,101)\n", "XX, YY = np.meshgrid(X,Y)\n", "# compute the distance function\n", "Dist = np.sqrt(np.square(XX - 0.5) + np.square(YY - 0.5))\n", "# creation of the 2D array: 0 indicate the matrix, 1 the fiber\n", "field_2D = np.int16(Dist <= 0.25)" ] }, { "cell_type": "markdown", "id": "af868ece", "metadata": {}, "source": [ "#### The add_image_from_field method" ] }, { "cell_type": "markdown", "id": "b89b5311", "metadata": {}, "source": [ "Now that our field is created, we can introduce the method `add_image_from_field`. It will create an appropriate *Image* group and store the data array inputed as an image field. " ] }, { "cell_type": "code", "execution_count": null, "id": "e0712bd6", "metadata": {}, "outputs": [], "source": [ "data.add_image_from_field(field_array=field_2D, fieldname='test_2D_field', imagename='first_2D_image',\n", " indexname='image2D', location='/', \n", " description=\"Test image group created from a bidimensional Numpy array.\")" ] }, { "cell_type": "markdown", "id": "9a378b1c", "metadata": {}, "source": [ "The verbose mode of the class instructed us that the image group has been created, along with a field/array object. You see that the method accepts similar arguments that the ones studied in the [previous tutorial](./2_SampleData_basic_data_items.ipynb). Note that here the `indexname` argument is for the image group indexname, the field indexname being constructed from the `field_name` argument (see below). To add another name to the field, you may use the `add_alias` method. Note also that the `description` argument here allows to set the `description` attribute of the created image group.\n", "\n", "We will now print the content of our dataset to see the results of the image group creation:" ] }, { "cell_type": "code", "execution_count": null, "id": "63c13f8d", "metadata": {}, "outputs": [], "source": [ "data.print_dataset_content()" ] }, { "cell_type": "markdown", "id": "ac98541c", "metadata": {}, "source": [ "Reading its attributes, we can observe that a *2DImage* group has been created, with a dimension of 51x51 pixels, in accordance with the inputed numpy array shape. We also can see that two childrens have been created for this group, a `Field_index` children, and a `test_2D_field` containing our data array (51x51 Carray). \n", "\n", "If we look at the dataset Index, we obtain:" ] }, { "cell_type": "code", "execution_count": null, "id": "1ae1fbae", "metadata": {}, "outputs": [], "source": [ "data.print_index()" ] }, { "cell_type": "markdown", "id": "dc5a43b6", "metadata": {}, "source": [ "You can see that the *indexnames* of the field and Field Index data items have been automatically constructed, following the rule: \n", "\n", "`indexname = grid_name + '_' + node_name`.\n", "\n", "We will look now at the other topology attributes of our *Image group*:" ] }, { "cell_type": "code", "execution_count": null, "id": "c2c6933e", "metadata": {}, "outputs": [], "source": [ "data.print_node_attributes('image2D')" ] }, { "cell_type": "markdown", "id": "b0f45025", "metadata": {}, "source": [ "You can observe that the `origin` and `spacing` attributes have been set to their default value (`[0.,0.]` and `[1.,1.]`), as they have not been specified in `add_image_from_field` arguments. We can also note that no time values have been added to the group. \n", "We will see how to modify values for these attributes in the next subsection of this tutorial." ] }, { "cell_type": "markdown", "id": "42c35e29", "metadata": {}, "source": [ "#### Create an image with specific pixel/voxel size and origin" ] }, { "cell_type": "markdown", "id": "cc42b91b", "metadata": {}, "source": [ "We will now use again the image group add method to recreate our image, this time specifying the value of the grid spacing and origin:" ] }, { "cell_type": "code", "execution_count": null, "id": "f19f8931", "metadata": {}, "outputs": [], "source": [ "data.add_image_from_field(field_array=field_2D, fieldname='test_2D_field', imagename='first_2D_image',\n", " indexname='image2D', location='/', replace=True,\n", " description=\"Test image group created from a bidimensional Numpy array.\",\n", " origin=[0.,10.], spacing=[2.,2.])" ] }, { "cell_type": "markdown", "id": "619861e8", "metadata": {}, "source": [ "**Note that we had to set the `replace` argument to `True` to be able to overwrite our Image group in the dataset.**" ] }, { "cell_type": "code", "execution_count": null, "id": "8f995778", "metadata": {}, "outputs": [], "source": [ "data.print_node_attributes('image2D')" ] }, { "cell_type": "markdown", "id": "813a76f1", "metadata": {}, "source": [ "This time, the `origin` and `spacing` attributes have been set accordingly to our choice. " ] }, { "cell_type": "code", "execution_count": null, "id": "3f7a3ddd", "metadata": {}, "outputs": [], "source": [ "data.print_xdmf()" ] }, { "cell_type": "markdown", "id": "a7f92697", "metadata": {}, "source": [ "**These values are used to build the XDMF file, and, in addition to being important grid metadata, will thus determine the scale of the 3D rendering of data in Paraview.**\n", "\n", "*You may note that in the XDMF file, the grid origin is stored as* $[10, 0]$ *and not* $[0, 10]$, *as it was inputed. This is due to the inverse interpretation of coordinate order by Paraview when visually rendering data. Hence, the values of the various data items in the XDMF file are written with an inverse coordinate order.*" ] }, { "cell_type": "markdown", "id": "83b835ce", "metadata": {}, "source": [ "To modify an image spacing or origin, you can use the `set_voxel_size` and `set_origin` methods:" ] }, { "cell_type": "code", "execution_count": null, "id": "579dcbd8", "metadata": {}, "outputs": [], "source": [ "data.set_voxel_size(image_group='image2D', voxel_size=np.array([4.,4.]))\n", "data.set_origin(image_group='image2D', origin=np.array([10.,0.]))\n", "data.print_node_attributes('image2D')\n", "data.print_xdmf()" ] }, { "cell_type": "markdown", "id": "e744a391", "metadata": {}, "source": [ "The *Image Group* spacing and origin have indeed been modified in the group attributes. \n", "\n", "**You can use those methods to translate and dilate your image group in the visualization rendered by Paraview, which can be usefull to avoid grid superposition if you want to visualize them in the same RenderView, or, on the contrary, to force visual superposition of grids that had not the same position/scale when added to the dataset.** " ] }, { "cell_type": "markdown", "id": "267cb945", "metadata": {}, "source": [ "#### Creating images from pixel wise or nodal value defined fields" ] }, { "cell_type": "markdown", "id": "02213b18", "metadata": {}, "source": [ "By using the `is_elem_Field` argument of `add_image_from_field`, we can change this standard behavior. By setting it to `False`, the method will consider the inputed array as a nodal value field: each value will be associated with a node of the image regular grid. We can test it by creating another image group from the same array:" ] }, { "cell_type": "code", "execution_count": null, "id": "a82296b0", "metadata": {}, "outputs": [], "source": [ "data.add_image_from_field(field_array=field_2D, fieldname='field_nodes_2D', imagename='image_2D_bis',\n", " indexname='image2D_bis', location='/', \n", " description=\"Test image group created from a bidimensional Numpy array.\",\n", " origin=[0.,10.], spacing=[2.,2.], is_elem_field=False)" ] }, { "cell_type": "code", "execution_count": null, "id": "64e0314a", "metadata": {}, "outputs": [], "source": [ "data.print_xdmf()\n", "data.print_node_info('image2D_bis')" ] }, { "cell_type": "markdown", "id": "aad32440", "metadata": {}, "source": [ "As you can see, the second image group has now dimensions reduced by one for each coordinate compared to the first image group, and the associated xdmf Grid node has an attribute that is defined on **Nodes**.\n", "\n", "We can now visualize the fields to see what differencies does it make. Use the code of the following cell to open the dataset with Paraview. Then, you can open again the same file in Paraview, and only activate the rendering of one image group for each file, by ticking it in the property panel, as shown on the image below: \n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "id": "e0237c2a", "metadata": {}, "outputs": [], "source": [ "# Use the second code line if you want to specify the path of the paraview executable you want to use\n", "# otherwise use the first line \n", "#data.pause_for_visualization(Paraview=True)\n", "#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')" ] }, { "cell_type": "markdown", "id": "2a3cd37d", "metadata": {}, "source": [ "What you should see, is, for the pixel wise field, an rendering close to this:\n", "\n", "\n", "\n", "And, for the nodal value field, you should have a rendering close to this:\n", "\n", "\n", "\n", "Both images display the exact same array. In the first case, with **Cell centered** data, Paraview interprets the data as a field constant within each element. On the contrary, in the second case, with **Node centered** data, Paraview interprets the data as values that the field takes at grid vertexes, and interpolate linearly field values in the pixels (the grid elements) to render the visualization. That is why the field appears continuous at the interface between the fiber and the matrix. " ] }, { "cell_type": "markdown", "id": "1e96deab", "metadata": {}, "source": [ "#### Creating images from scalar, vector or tensor fields" ] }, { "cell_type": "markdown", "id": "2f910bbb", "metadata": {}, "source": [ "There is another argument that changes the way `add_image_from_field` interprets the field data array: the `is_scalar` argument. Its default value is `True`. In this case the shape of the field is interpreted as a the dimension of the image to create. In the case just above, we inputed a (101x101) field, and the method created an image group of 101x101 pixels, containing a **scalar field**, *i.e.* a field with only one value per pixel.\n", "\n", "However, we may want to create an image from a field whose values are vector or tensors. In that case, there is an ambiguity for arrays with 3 dimensions. They can either represent a $(x,y,z)$ array, or a $(x,y,i)$ array, where $i$ denotes the field component index. \n", "\n", "**The `is_scalar` argument role is to avoid this ambiguity. If it is `True` (default) value, a 3D array will be interpreted as a 3D $(x,y,z)$ scalar field, and if it is `False`, it will be interpreted as a $(x,y,i)$ 2D vector or tensor field.**\n", "\n", "Time to test this feature:" ] }, { "cell_type": "code", "execution_count": null, "id": "6a4edbf8", "metadata": {}, "outputs": [], "source": [ "# creation of a random 3D field :\n", "field_3D = np.random.rand(50,50,3)\n", "\n", "# creation of a 3D image group --> is_scalar set to True (default value)\n", "data.add_image_from_field(field_array=field_3D, fieldname='field_3D', imagename='image_3D',\n", " indexname='image3D', location='/', \n", " description=\"\"\"\n", " Test 3D image group created from a tridimensional Numpy array with `is_scalar` = True.\"\"\")\n", "\n", "# creation of a 2D image group --> is_scalar set to False\n", "data.add_image_from_field(field_array=field_3D, fieldname='field_vect', imagename='image_2D_3',\n", " indexname='image2D_3', location='/', is_scalar=False,\n", " description=\"\"\"\n", " Test 2D image group created from a tridimensional Numpy array with `is_scalar` = False.\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "id": "91602a08", "metadata": {}, "outputs": [], "source": [ "# Getting information on the dataset and the created image groups\n", "print(data)\n", "data.print_node_info('image_3D')\n", "data.print_node_info('image_2D_3')" ] }, { "cell_type": "code", "execution_count": null, "id": "07c237e1", "metadata": {}, "outputs": [], "source": [ "# Getting information on created image fields\n", "data.print_node_info('image_3D_field_3D')\n", "data.print_node_info('image_2D_3_field_vect')" ] }, { "cell_type": "markdown", "id": "4edb109a", "metadata": {}, "source": [ "If you look at the two created image groups, you see indeed that one has been created as a *3DImage* group (with `is_scalar=True`), and one as a *2DImage* Group (with `is_scalar=False`). Moreover, the `field_dimensionality` attribute of the two created image fields state clearly that the field of the 3D image is a scalar field, whereas the other one is a vector field. \n", "\n", "We can now visualize them:" ] }, { "cell_type": "code", "execution_count": null, "id": "6a512b89", "metadata": {}, "outputs": [], "source": [ "# Use the second code line if you want to specify the path of the paraview executable you want to use\n", "# otherwise use the first line \n", "#data.pause_for_visualization(Paraview=True)\n", "#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')" ] }, { "cell_type": "markdown", "id": "e5f4b109", "metadata": {}, "source": [ "The rendering of the 3D image group with Paraview, containing a random 3D scalar field, should look the image below:\n", "\n", "\n", "\n", "*To obtain the rendering above, we only activated the rendering of the `image_3D` block in Paraview's window Property Panel, and plotted it with the Surface renderer.* \n", "\n", "The rendering of the 2D image group with Paraview, containing a random 2D vector field, should look the this:\n", "\n", "\n", "\n", "*To obtain the rendering above, we only activated the rendering of the `image_2D_3` block in Paraview's window Property Panel, added a Glyph filter in Arrow mode (that is how to plot vector field with Paraview) and colored the arrows with the vector field magnitude. We also bounded the maximum number of plotted arrows to 500.*\n", "\n", "**Again, those two 3D rendering where obtained with stricly the same input data array.** This illustrate the formatting possibilities offered by the *SampleData* class, but also highlight the need to be familiar with them, and to clearly specify in what form the data must be stored, to avoid unwanted behaviors. " ] }, { "cell_type": "markdown", "id": "2b09bcac", "metadata": {}, "source": [ "Note that we could have executed the same lines with an array of shape $(50,50,6)$ or $(50,50,9)$. The resulting 2D image would have in those cases supported a symetric tensor or tensor field. \n", "\n", "This closes the presentation of the first and more straightforward way to create an Image group in a *SampleData* dataset. " ] }, { "cell_type": "markdown", "id": "2756b3b1", "metadata": {}, "source": [ "### Creating images from image objects" ] }, { "cell_type": "markdown", "id": "febf5e96", "metadata": {}, "source": [ "The second way to create Image groups, requires the creation of an image object. To handle grids, the *SampleData* class relies on the *BasicTools* Python package, developed by the industrial group Safran. This package offers some classes to represent meshes, and in particular rectilinear meshes, *i.e.* regular grids. What whe refer to as an *image object*, is in fact a *BasicTools* rectilinear mesh object. Let us see how it works with an example." ] }, { "cell_type": "markdown", "id": "80eca186", "metadata": {}, "source": [ "First, we need to import the rectilinear mesh class from Basictools, that should be accessible in your python environment as it is a SampleData dependence:" ] }, { "cell_type": "code", "execution_count": null, "id": "23c48eb4", "metadata": {}, "outputs": [], "source": [ "from BasicTools.Containers.ConstantRectilinearMesh import ConstantRectilinearMesh" ] }, { "cell_type": "markdown", "id": "e2e2da30", "metadata": {}, "source": [ "Then, we will create our image object as an instance of this class. For that, we need to set its dimenion (2D or 3D) image. For this example, we will try to recreate the 3D image with the random scalar field:" ] }, { "cell_type": "code", "execution_count": null, "id": "0ab4f9db", "metadata": {}, "outputs": [], "source": [ "image_object = ConstantRectilinearMesh(dim=3)\n", "print(type(image_object))" ] }, { "cell_type": "markdown", "id": "6f9368b6", "metadata": {}, "source": [ "We can now set the topology of the image, by using the following methods:" ] }, { "cell_type": "code", "execution_count": null, "id": "dd38f484", "metadata": {}, "outputs": [], "source": [ "image_object.SetDimensions((50,50,3))\n", "image_object.SetOrigin([0.,0.,0.])\n", "image_object.SetSpacing([1.,1.,1.])" ] }, { "cell_type": "markdown", "id": "5bb49205", "metadata": {}, "source": [ "Finally, image objects have two attributes that are dictionaries, the `elemFields` and the `nodeFields`, that allow to add fields to the image. Of course, these fields should have dimensions that match the image object dimensions. We will then add our random 3D array as a field of this image:" ] }, { "cell_type": "code", "execution_count": null, "id": "369e59d7", "metadata": {}, "outputs": [], "source": [ "image_object.elemFields['im_object_field'] = field_3D " ] }, { "cell_type": "markdown", "id": "1f9754ad", "metadata": {}, "source": [ "We can print our image object to get information on it:" ] }, { "cell_type": "code", "execution_count": null, "id": "b964480f", "metadata": {}, "outputs": [], "source": [ "print(image_object)" ] }, { "cell_type": "markdown", "id": "18e92588", "metadata": {}, "source": [ "You can see that it has an attribute that is an Element container, that has 4802 elements of type `hex8`, *i.e.* linear hexaedra, which is consistent with a grid of voxels. " ] }, { "cell_type": "markdown", "id": "821a5609", "metadata": {}, "source": [ "Now we can add the image into the dataset with the `add_image` method. Its arguments are exactly the same as those of the `add_image_from_field` method, without the `field_array`, `fieldname`, `is_scalar`, `elemField` arguments:" ] }, { "cell_type": "code", "execution_count": null, "id": "4f2b000c", "metadata": {}, "outputs": [], "source": [ "data.add_image(image_object, imagename='image_from_object', indexname='imageO', location='/', \n", " description=\"\"\"Test 3D image group created from an image object.\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e8dbde2c", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('imageO')" ] }, { "cell_type": "markdown", "id": "5a218428", "metadata": {}, "source": [ "The image group has been created. Though, you can see that the dimension we indicated, actually prescribed the image `nodes_dimension`. Hence, remember that **the `SetDimensions` method of image objects set the dimension of the node grid. The pixel/voxel grid has the same dimensions minus one along each direction**.\n", "\n", "As a result, we should expect here that the field loaded into the image object has been added to the image group as a nodal field. We can verify it by printing its attributes:" ] }, { "cell_type": "code", "execution_count": null, "id": "0888295c", "metadata": {}, "outputs": [], "source": [ "data.print_node_attributes('im_object_field')" ] }, { "cell_type": "markdown", "id": "9d89ef56", "metadata": {}, "source": [ "All the variations linked to the fields presented in above can still be achieved with this way of adding images in the datasets. For instance, if you passed to `SetDimension` the tuple $(N_x,N_y,N_z)$, to add a nodal vector field to your image group add a $(N_x,N_y,N_z,3)$ array to the image object `elemFields` list. " ] }, { "cell_type": "markdown", "id": "7149e652", "metadata": {}, "source": [ "### Creating empty images" ] }, { "cell_type": "markdown", "id": "e3f45b90", "metadata": {}, "source": [ "*SampleData* also offers the possibility to create empty image objects. To do this, you proceed also with the `add_image` method, but without providing any *image object*:" ] }, { "cell_type": "code", "execution_count": null, "id": "d18a7073", "metadata": {}, "outputs": [], "source": [ "data.add_image(imagename='image_empty', indexname='imageE', location='/', \n", " description=\"\"\"Test empty image group.\"\"\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e960c304", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('imageE')" ] }, { "cell_type": "markdown", "id": "a812d9dc", "metadata": {}, "source": [ "Like empty data arrays presented in the [tutorial 2 (section IV)](./Data_Items.ipynb), empty images are used to create the internal organization of your dataset without having to add any data to it. It allows for instance to preempt some data item names, indexnames, and to already add metadata. When you have data to add, you can then create an image group with the same name/location, the empty image will be replace by an actual image group, but all metadata that was added to it before will be preserved.\n", "\n", "To test this, we start by adding metadata to our empty image group:" ] }, { "cell_type": "code", "execution_count": null, "id": "267a2dec", "metadata": {}, "outputs": [], "source": [ "data.add_attributes({'tutorial_file':'Image_data.ipynb', 'tutorial_section':'III'}, 'imageE')\n", "data.print_node_attributes('imageE')" ] }, { "cell_type": "markdown", "id": "27624711", "metadata": {}, "source": [ "Now we will overwrite the empty image group by creating a 3D image group (same as in last subsection). In this case we do not need to set the `replace` argument to `True`. " ] }, { "cell_type": "code", "execution_count": null, "id": "61666555-7392-454f-9568-b509f1a535d2", "metadata": {}, "outputs": [], "source": [ "data.print_index()" ] }, { "cell_type": "code", "execution_count": null, "id": "77a22940", "metadata": {}, "outputs": [], "source": [ "# first, we will change the name of the field stored in the image object to avoid duplicates in the dataset \n", "image_object.elemFields['Im_field'] = image_object.elemFields.pop('im_object_field')\n", "\n", "# Now we can create the new image group to replace the empty one\n", "data.add_image(image_object, imagename='image_empty', indexname='imageE', location='/', \n", " description=\"\"\"Test empty image group overwritten with actual data.\"\"\")\n", "data.print_node_info('imageE')" ] }, { "cell_type": "markdown", "id": "21314bb8", "metadata": {}, "source": [ "The creation of the image group with actual data has indeed preserved the attributes attached to the former empty node." ] }, { "cell_type": "markdown", "id": "23f2a947", "metadata": {}, "source": [ "Now, you know all about the three methods to create image groups in SampleData datasets. " ] }, { "cell_type": "markdown", "id": "8ac19142", "metadata": {}, "source": [ "## IV - Image Fields data model" ] }, { "cell_type": "markdown", "id": "d9e3a98d", "metadata": {}, "source": [ "In this new section, we will now focus on the definition of **image fields** data items. Fields are data arrays with additional metadata allowing to interpret them as spatially organized data. For the *SampleData* class, these metadata main roles are to ensure that:\n", "1. the correspondance between dimensions of the field array values and their physical meaning is preserved from the user perspective\n", "2. that the fields visualization with Paraview through the XDMF format properly renders the geometry and dimensionality of fields values\n", "\n", "To that, some conventions have been defined." ] }, { "cell_type": "markdown", "id": "e3fbadad", "metadata": {}, "source": [ "### SampleData Image fields conventions" ] }, { "cell_type": "markdown", "id": "8951381d", "metadata": {}, "source": [ "#### Image grids coordinates" ] }, { "cell_type": "markdown", "id": "8f2d4fc6", "metadata": {}, "source": [ "Two grids are associated to *Image Groups*:\n", "\n", "\n", "* a grid of pixel/voxel centers, of dimensions $(N_x, N_y)$ or $(N_x, N_y, N_z)$, that can be seen as a set of points indentified by a doublet/triplet index: \n", " - $P^c (i,j) = (x^c_i,y^c_j)$ (2D images) \n", " - $ P^c (i,j,k) = (x^c_i,y^c_j,z^c_k)$ (3D images) \n", " \n", " \n", "* a grid of nodes (pixel/voxel vertexes), of dimensions $(N_x+1, N_y+1)$ or $(N_x+1, N_y+1, N_z+1)$, similarly defined:\n", " - $P^n (i,j) = (x^n_i, y^n_ j)$ (2D images)\n", " - $P^n (i,j,k) = (x^n_i, y^n_ j, z^n_ k)$ (3D images) \n", "\n", "with $x^c,y^c$ and $z^c$ representing the coordinates or pixel/voxel centers and $x^n,y^n$ and $z^n$ representing the coordinates or pixel/voxel centers. " ] }, { "cell_type": "markdown", "id": "29556fb7", "metadata": {}, "source": [ "#### Fields array possible shapes" ] }, { "cell_type": "markdown", "id": "3d3b7ae4", "metadata": {}, "source": [ "You can add *numpy* arrays as fields to *SampleData* image groups. This array specifies the data of the field for each point of one of those two grids. It must have at least as many dimensions as the grid. **Fields array are only allowed to have one more dimension that the grid they are defined on, and their first dimension sizes must comply with the grid dimensions.** This implies that field arrays can only have 8 different types of shapes:\n", "\n", "\n", "1. **2D grids:**\n", " 1. $F(i,j) \\longrightarrow$ size $(N_x,N_y)$\n", " 2. $F(i,j,c) \\longrightarrow$ size $(N_x,N_y,N_c)$\n", " 3. $F(i,j) \\longrightarrow$ size $(N_x+1,N_y+1)$\n", " 4. $F(i,j,c) \\longrightarrow$ size $(N_x+1,N_y+1,N_c+1)$\n", " \n", " \n", "2. **3D grids:**\n", " 1. $F(i,j,k) \\longrightarrow$ size $(N_x,N_y, N_z)$\n", " 2. $F(i,j,k,c) \\longrightarrow$ size $(N_x,N_y, N_c)$ \n", " 3. $F(i,j,k) \\longrightarrow$ size $(N_x +1,N_y +1, N_z +1)$\n", " 4. $F(i,j,k,c) \\longrightarrow$ size $(N_x +1,N_y +1, N_c +1)$ \n", " " ] }, { "cell_type": "markdown", "id": "06d266f7", "metadata": {}, "source": [ "#### Field dimensionality" ] }, { "cell_type": "markdown", "id": "f8a27abd", "metadata": {}, "source": [ "The size of the last dimension of the array determines the **number of field components** $N_c$, which defines the **field dimensionality**. If the array has no supplementary dimension with respect to the grid, *i.e.* no $c$ index, then $N_c=1$.\n", "\n", "*SampleData* will interpret the field dimensionality (scalar, vector, or tensor) depending on the grid topology and the array shape. All possibilities are listed hereafter:\n", "* $N_c=1$: **scalar field**\n", "* $N_c=3$: **vector field**\n", "* $N_c=6$: **symetric tensor field** (*2nd order tensor*)\n", "* $N_c=9$: **tensor field** (*2nd order tensor*)\n", "\n", "The dimensionality of the field has the following implications:\n", "* **XDMF**: the dimensionality of the field is one of the metadata stored in the XDMF file, for Fields (`Attribute` nodes)\n", "* **visualization**: as it is stored in the XDMF file, Paraview can correctly interpret the fields according to their dimensionality. Il allows to plot separately each field component, and the field norm (magnitude). It also allows to use the Paraview *Glyph* filter to plot vector fields with arrows\n", "* **indexing**: The order of the value in the last dimension for non-scalar fields correspond to a specific order of the field components, according to a specific convention. These conventions will be detailed in the next subsection. \n", "* **compression**: Specific lossy compression options exist for fields. See dedicated tutorial.\n", "* **interface with external tools**: when interfacing SampleData with external softwares, such as numerical simulation tools, it is very practical to have fields with appropriate dimensionality accounted for (for instance to use a displacement or temperature gradient vector field stored in a *SampleData* dataset as input for a finite element simulation). " ] }, { "cell_type": "markdown", "id": "49179f46", "metadata": {}, "source": [ "#### Field components indexing" ] }, { "cell_type": "markdown", "id": "6f9103e5", "metadata": {}, "source": [ "We will now review the conventions for the correspondance between field data arrays indexing and field components, hence field arrays whose indexing is $F[i,j,c]$ or $F[i,j,k,c]$. To detail the convention, we will omit the spatial indexes $i,j$ or $k$ and only consider the last dimension of the field: $F[i,j,k,c] = F[c]$. We additionally use the subscript notation $F[c] = F_c$.\n", "\n", "The indexing conventions are:\n", "\n", "\n", "* For **vector fields** (3 components), the convention is $[F_0,F_1,F_2] = [F_x,F_y,F_z]$\n", "* For **symetric tensor fields** (2nd order, 6 components), the convention is\n", " $[F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx}]$\n", "* For **tensor fields** (2nd order, 9 components), the convention is \n", " $[F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{yy},F_{zz},F_{xy},F_{yz},F_{zx},F_{yx},F_{zy},F_{xz}]$\n", "\n", "For the first indices, the convention is $F[i,j,c] = F(x_i,y_i)[c]$ (2D) and $F[i,j,k,c] = F(x_i,y_j,z_k)[c]$ (3D)\n", "\n", "Here are a few examples to illustrate those conventions:\n", "\n", "\n", "* $F[10,23]$ can only be interpreted as:\n", "\n", " - the field value at the grid point $(x_{10},y_{23})$\n", "\n", "\n", "\n", "* $F[10,23,2]$ can be interpreted as:\n", "\n", " - if $F$ is a *scalar field*: the field value at the grid point $(x_{10},y_{23}, z_2)$\n", " - if $F$ is a *vector field*: the $F_z$ component value at the grid point $(x_{10},y_{23})$\n", " - if $F$ is a *tensor field*: the $F_{zz}$ component value at the grid point $(x_{10},y_{23})$\n", "\n", "\n", "\n", "* $F[10,23,7]$ can be interpreted as:\n", " - if $F$ is a *scalar field*: the field value at the grid point $(x_{10},y_{23}, z_{7})$\n", " - if $F$ is a *tensor field* (non-symetric): the $F_{zy}$ component value at the grid point $(x_{10},y_{23})$\n", "\n", "\n", "\n", "* $F[10,23,24,1]$ can be interpreted as:\n", " - if $F$ is a *vector field*: the $F_y$ component value at the grid point $(x_{10},y_{23},z_{24})$\n", " - if $F$ is a *tensor field*: the $F_{yy}$ component value at the grid point $(x_{10},y_{23},z_{24})$\n", "\n", "\n", "\n", "* $F[10,23,24,5]$ can only be interpreted as:\n", "\n", " - the $F_{zy}$ component value (tensor field) at the grid point $(x_{10},y_{23},z_{24})$\n", "\n", "
\n", "\n", "**Warning** \n", " \n", "Field components (example: $x,y,xx,zy$...) are assumed to be defined in the same frame as the grid. However, that cannot be ensured simply by providing a data array. Hence, the user must ensure to have this coincidence between the grid and the data. It is not mandatory, but not respecting this implicit convention may lead to confusions and geometrical misinterpretation of the data by other users or software. If you want to do it, a good idea would be to add attributes to the field to specify and explain the nature of the field components. \n", "\n", "
" ] }, { "cell_type": "markdown", "id": "734f733a", "metadata": {}, "source": [ "#### Field indices in-memory transposition " ] }, { "cell_type": "markdown", "id": "2b1dbdf5", "metadata": {}, "source": [ "The indexing convention presented in the last subsection apply to data array that we add to *Image groups* as fields, and to data array that are retrieved from dataset *Image group* fields. However, you will see below that the fields are not stored in the HDF5 file with the same ordering. In practice, two transpositions are applied to the data arrays before loading them into the dataset, to ensure proper visualization with the Paraview software. There are two reasons for it, linked to Paraview indexing conventions:\n", "\n", "1. **Paraview convention for spatial indices is inverse to *SampleData* condition:**\n", " * 2D: the dimensions $0$ and $1$ must be inverted: $F[i,j,...] \\longrightarrow F[j,i,...]$ \n", " * 3D: the dimensions $0$ and $2$ must be inverted: $F[i,j,k,...] \\longrightarrow F[k,j,i,...]$\n", " \n", "\n", "For these reason, fields are stored with indices transposition in *SampleData* datasets so that their visualization with Paraview yield a rendering that is consistent with the ordering convention presented in the previous subsection. These transposition are automatically handled, as well as the back transposition when a field is retrieved, as you will see in the last section of this tutorial. \n", "\n", "**An attribute `transpose_indices` is added to field data items. Its values represent the order in the first columns of the original array are stored in memory** (see examples below). \n", " \n", "2. **Paraview component order convention is different from *SampleData* condition for tensors.** The convention in Paraview is:\n", " * $[F_0,F_1,F_2,F_3,F_4,F_5] = [F_{xx},F_{xy},F_{xz},F_{yy},F_{yz},F_{zz}]$ for symetric tensors\n", " * $[F_0,F_1,F_2,F_3,F_4,F_5,F_6,F_7,F_8] = [F_{xx},F_{xy},F_{xz},F_{yx},F_{yy},F_{yz},F_{zx},F_{zy},F_{zz}]$ for non symetric tensors\n", " \n", "For these reason, fields are stored with indices transposition in *SampleData* datasets so that their visualization with Paraview yield a rendering that is consistent with the ordering convention presented in the previous subsection. These transposition are automatically handled, as well as the back transposition when a field is retrieved, as you will see in the last section of this tutorial. \n", "\n", "**An attribute `transpose_components` is added to field data items. Its values represent the order in which components or the original array are stored in memory** (see examples below).\n", "\n", "\n", "
\n", "\n", "**Note** \n", " \n", "When visualizing a field in paraview, you may choose which component (including field magnitude) you are plotting in the box (*highlighted in red in the image below*) located in between the boxes for the choice of the visualization mode (*Surface in the image below*) and the data item choice (*tensor_field2D in the image below*). \n", " \n", "In this box, you will have to choose between 9 (0 to 8) components, even for symetric tensors. In the later case, the corresponding equal components (1 and 3: $xy$&$yx$, 2 and 6: $xz$&$zx$, 5 and 7: $yz$&$zy$) will yield the same visualization. \n", "\n", "\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "9808bedf", "metadata": {}, "source": [ "#### Field type" ] }, { "cell_type": "markdown", "id": "282547c2", "metadata": {}, "source": [ "In addition to their dimensionality, **Image Fields** data items also have a field type. You already encountered the two field types that can be supported by image groups in section III:\n", "\n", "* **element field**: \n", " - described by an array of values defined at pixel/voxel centers $P^c$ (dimension $(N_x,N_y,N_c)$ or $(N_x,N_y,N_z,N_c)$ )\n", " - visualization: a pixel/voxel wise constant fields\n", "\n", "\n", "* **nodal field**: \n", " - described by an array of values defined at grid nodes (pixel/voxel vertexes) $P^n$ (dimension $(N_x+1,N_y+1, N_c)$ or $(N_x+1,N_y+1,N_z+1,N_c)$ )\n", " - visualization: field linearly interpolated within each pixel/voxel from values at nodes\n", " \n", "This feature is stored in the field data item attribute `field_type` (see examples above in section III). \n", "\n", "\n", "
\n", "\n", "**Warning** \n", "\n", "Paraview seems to be unable to read non-scalar nodal fields XDMF Attributes for nodal regular grids. However, element vector or tensor fields can be visualized normally. \n", "\n", "
" ] }, { "cell_type": "markdown", "id": "35fb4848", "metadata": {}, "source": [ "#### Field attributes" ] }, { "cell_type": "markdown", "id": "2580f490", "metadata": {}, "source": [ "For each field, the value of all features presented above are stored as attributes. Let us see an example: " ] }, { "cell_type": "code", "execution_count": null, "id": "30aa4e98", "metadata": {}, "outputs": [], "source": [ "data.print_index()" ] }, { "cell_type": "code", "execution_count": null, "id": "6c5a54ad", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image_2D_3_field_vect')" ] }, { "cell_type": "markdown", "id": "cae38d86", "metadata": {}, "source": [ "The attribute printed above inform us on the field data item features. The `field_dimensionality` attribute indicates that the field is a vector field. Has its data array has a dimension of 3 (see the `content` line in the output), it is a scalar field of a 2D image. The `field_type` attribute indicates that the field is defined at pixel centers.\n", "\n", "The `transpose_indices` show that the first and second column of the original inputed array have been inverted in memory (inversion of $X$ and $Y$ direction between SampleData and Paraview conventions).\n", "\n", "The `parent_grid_path` and `xdmf_gridname` attribute provides the path of the *Image Group* and the XDMF Grid Node name to which this field belongs. \n", "The `xdmf_fieldname` provides the name of the XDMF Attribute Node associated to the field.\n", "The `padding` attribute is of no use for Image fields. It will be presented in the next tutorial for mesh fields." ] }, { "cell_type": "markdown", "id": "c695c913", "metadata": {}, "source": [ "## V - Adding Fields to Image Groups" ] }, { "cell_type": "markdown", "id": "bf9e763f", "metadata": {}, "source": [ "Now that we know all elements that compose and define a field data item in a *SampleData* dataset, we will see how to create fields. As for other creation tasks, the *SampleData* class provides a method to do it with an explicit name: the `add_field` method. It is very similar to the `add_data_array` method, with two main additions presented below." ] }, { "cell_type": "markdown", "id": "90428ca9", "metadata": {}, "source": [ "### Adding a field to a grid" ] }, { "cell_type": "markdown", "id": "53ac5ffd", "metadata": {}, "source": [ "To add a field to a grid from a *numpy* array, you just have to call `add_field` like you would have called `add_data_array`, with an additional argument `gridname`, that indicates the grid group to which you want to add the field. In the case of the present tutorial, we can hence provide the Name, Indexname, Path or Alias of an image group to add a field. \n", "\n", "The analysis of the field dimensionality, nature and required transpositions is automatically handled by the class." ] }, { "cell_type": "markdown", "id": "c568e969", "metadata": {}, "source": [ "We will try to add a symetric tensor element field to the first 2D image that we have created, at the beginning of section III. Let us recall the image group features:" ] }, { "cell_type": "code", "execution_count": null, "id": "386e4bee", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image2D')" ] }, { "cell_type": "markdown", "id": "f807d303", "metadata": {}, "source": [ "The image has a `nodes_dimension` of 102x102. We can retrieve this attribute to presscribe the shape of the array that we have to create. As we want to create a symetric tensor field, we also need to add a last dimension of size 6. \n", "\n", "We will create a very simple field, whose values are defined by $F(i,j,c) = c$ for $y <= 51$ and $F(i,j,c) = 2c$ for $y > 51$:" ] }, { "cell_type": "code", "execution_count": null, "id": "1bb13b27", "metadata": {}, "outputs": [], "source": [ "# first we get the image nodal grid dimensions\n", "dim = data.get_attribute('dimension','image2D')\n", "\n", "# then, we create a field of 0 with the right dimensions\n", "Nc = 6\n", "tensor_field = np.zeros(shape=(dim[0],dim[1],Nc))\n", "\n", "# finally we set the values of our tensor field\n", "for c in range(Nc):\n", " tensor_field[:,:52,c] = c\n", " tensor_field[:,52:,c] = 2*c" ] }, { "cell_type": "code", "execution_count": null, "id": "6d5b0fbe", "metadata": {}, "outputs": [], "source": [ "print(tensor_field)" ] }, { "cell_type": "code", "execution_count": null, "id": "0e6ae473", "metadata": {}, "outputs": [], "source": [ "# now we can add our field to the image group\n", "data.add_field(gridname='image2D', fieldname='tensor_field2D', location='image2D', indexname='tensorF',\n", " array=tensor_field, replace=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "e3126e6a", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('tensorF')" ] }, { "cell_type": "markdown", "id": "b5a8b093", "metadata": {}, "source": [ "As you can observe, the field has automatically been created as a symetric tensor (Tensor6) nodal field, with the transposition of its first and second column (see `transpose_indices` value) and of its components (see `transpose_indices` value).\n", "\n", "We can already visualize our created field:" ] }, { "cell_type": "code", "execution_count": null, "id": "626952b2", "metadata": {}, "outputs": [], "source": [ "# Use the second code line if you want to specify the path of the paraview executable you want to use\n", "# otherwise use the first line \n", "#data.pause_for_visualization(Paraview=True)\n", "#data.pause_for_visualization(Paraview=True, Paraview_path='path to paraview executable')" ] }, { "cell_type": "markdown", "id": "a11bf1d0", "metadata": {}, "source": [ "You should be able to visualization the tensor field magnitude and components individually: \n", "\n", "" ] }, { "cell_type": "markdown", "id": "ca217f87", "metadata": {}, "source": [ "You can try to redo these last cells by changing the dimension of the field, to try to change its type, its dimensionality, or the image group on which you add it." ] }, { "cell_type": "markdown", "id": "e879d474", "metadata": {}, "source": [ "### Adding time serie of fields" ] }, { "cell_type": "markdown", "id": "1c33305d", "metadata": {}, "source": [ "*SampleData* allows to create a set of fields array that are related in time. Each of them will represent the value of one field at different instants. To do that, you just have to set the value of the `time` argument when calling the `add_field` method. We will illustrate this possibility with an example. " ] }, { "cell_type": "markdown", "id": "52fdfedf", "metadata": {}, "source": [ "We will create a 2D vector field that will take different values at 3 different instants, and add it to same image to which we added the previous field. At each instant, the whole field will be uniform and have the value of one of the 3 base vectors. As we are providing different time values of the same field, we should provide the same Name each time we provide an time value of this field. " ] }, { "cell_type": "code", "execution_count": null, "id": "b287de68", "metadata": {}, "outputs": [], "source": [ "# get the image nodal grid dimensions\n", "dim = data.get_attribute('dimension','image2D')\n", "\n", "# create of a data array of 0 of dimension (Nx,Ny,3,Ninstants)\n", "temporal_field = np.zeros((dim[0],dim[1],3,3))\n", "\n", "# Set the value of the field to the first unit vector at instant 0\n", "temporal_field[:,:,0,0] = 1\n", "\n", "# Set the value of the field to the second unit vector at instant 1\n", "temporal_field[:,:,1,1] = 1\n", "\n", "# Set the value of the field to the third unit vector at instant 2\n", "temporal_field[:,:,2,2] = 1\n", "\n", "# Set the value of time instants:\n", "instants = [1.,10., 100.]" ] }, { "cell_type": "code", "execution_count": null, "id": "fb95dcdc", "metadata": {}, "outputs": [], "source": [ "# now we can add for each instant the right slice of the array as a field to the image group\n", "# instant 0\n", "data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',\n", " array=temporal_field[:,:,:,0], time=instants[0])\n", "# instant 1\n", "data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',\n", " array=temporal_field[:,:,:,1], time=instants[1])\n", "# instant 2\n", "data.add_field(gridname='image2D', fieldname='time_field', location='image2D', indexname='Field',\n", " array=temporal_field[:,:,:,1], time=instants[2])" ] }, { "cell_type": "markdown", "id": "f1914f3f", "metadata": {}, "source": [ "### Time series Grids and time series attributes" ] }, { "cell_type": "markdown", "id": "201c09b0", "metadata": {}, "source": [ "You will see that creating this time serie of fields has slighlty modified the dataset. We will explore these modifications, and start by looking at our dataset organization to see if it has changed:" ] }, { "cell_type": "code", "execution_count": null, "id": "569499dd", "metadata": { "scrolled": true }, "outputs": [], "source": [ "print(data)" ] }, { "cell_type": "markdown", "id": "b488510c", "metadata": {}, "source": [ "The three fields have been added as arrays under the Image Group, and added to the index. As you can observe, the names and indexnames of the fields have automatically been completed with a suffix indicating the instant they represent: `time_field_T1_0`, `time_field_T10_0` etc...\n", "\n", "We can now look at the associated XDMF tree structure:" ] }, { "cell_type": "code", "execution_count": null, "id": "605a0b1f", "metadata": {}, "outputs": [], "source": [ "data.print_xdmf()" ] }, { "cell_type": "markdown", "id": "6e24bebf", "metadata": {}, "source": [ "The XDMF file of the dataset has become rather large now. As the creation of a time serie of grids requires to rewrite the 2D image XDMF grid node, you will find it at the end of the file. You see that the Grid node named `first_2D_image` is now a `Collection` of grids, of type `Temporal`. Three sub grid nodes that are `Uniform` have been created, each with a Time element, corresponding to one of the instant values that we provided. \n", "\n", "Ultimately, note that the grids of instants 2 and 3 only have one *Attribute* field. **The fields that were attached to the image group before adding this time serie have been automatically attached to the grid of the first instant,** to ensure they visualization with Paraview (*Attributes* of *Collection* grids are not displayed by the software)." ] }, { "cell_type": "markdown", "id": "3e948c5c", "metadata": {}, "source": [ "We can now look at our image group, to find out how it has changed:" ] }, { "cell_type": "code", "execution_count": null, "id": "f78797c0", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('image2D')" ] }, { "cell_type": "markdown", "id": "20ac3448", "metadata": {}, "source": [ "The only difference that we can see, is the creation of a `time_list` attribute, that gather all the time instants that have been defined for the fields of this image group. \n", "\n", "To conclude this subsection, we will look at the content of one of the field data items added in the time serie:" ] }, { "cell_type": "code", "execution_count": null, "id": "362b1c4b", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('time_field_T1_0')" ] }, { "cell_type": "markdown", "id": "2fb176d5", "metadata": {}, "source": [ "This field node is absolutely similar to the field data items that we created earlier without time value. The only difference is that it has two additional attributes `time` and `time_serie_name` that indicate at which instant its values correspond, and was is the name of the time serie field." ] }, { "cell_type": "markdown", "id": "3758789d", "metadata": {}, "source": [ "## VI - Getting Images and Fields " ] }, { "cell_type": "markdown", "id": "9c912863", "metadata": {}, "source": [ "To conclude this tutorial, we will now try to retrieve the data that we created, in the form of field arrays or image objects. " ] }, { "cell_type": "markdown", "id": "3635aee3", "metadata": {}, "source": [ "### Getting image objects from datasets" ] }, { "cell_type": "markdown", "id": "80bcc650", "metadata": {}, "source": [ "Getting an image object from a dataset image Group is very straightforward with the *SampleData* class. You just have to use the `get_image` method. It takes two arguments, the name (or indexname etc...) of the image Group, and a `with_fields` boolean argument, that allow you to chooe if you want or not to load the field data arrays into the image object.\n", "\n", "We will start by printing again the content of our dataset, to remember the image groups and fields stored in it:" ] }, { "cell_type": "code", "execution_count": null, "id": "653c3ade", "metadata": {}, "outputs": [], "source": [ "print(data)" ] }, { "cell_type": "markdown", "id": "b212c4f7", "metadata": {}, "source": [ "Let us retrieve the image group `image_from_object`, with its internal fields:" ] }, { "cell_type": "code", "execution_count": null, "id": "13797a94", "metadata": { "scrolled": true }, "outputs": [], "source": [ "im_object = data.get_image('imageO', with_fields=True)\n", "print(im_object)" ] }, { "cell_type": "markdown", "id": "c511c1c4", "metadata": {}, "source": [ "It is as simple as that. Setting the `with_fields` argument to `False` would have resulted in the same image object, but with en empty `nodeFields` attribute.\n", "\n", "Now you have an image object that contains all the relevant information to recreate a *SampleData* image group with the same fields, and the same topology as the retrieved group `image0`. This is particularly usefull to transfer an image group from a dataset to another, as illustrated in the next cell:" ] }, { "cell_type": "code", "execution_count": null, "id": "12c90b14", "metadata": {}, "outputs": [], "source": [ "# We want to transfer our image group to a new dataset --> here we create a new dataset \n", "# (we could have also opened a pre-existing one)\n", "data2 = SD(filename='tmp_dataset', autodelete=True)\n", "\n", "# We add our image object\n", "data2.add_image(im_object, imagename='transfered_image', indexname='imageTr', location='/')\n", "\n", "# We print the data set content to verify that our image group has been created\n", "print(data2)\n", "\n", "# We compare information about the two images group (the one from the original dataset and the created one)\n", "data.print_node_info('imageO')\n", "data2.print_node_info('imageTr')\n", "\n", "# We close our new dataset \n", "del data2" ] }, { "cell_type": "markdown", "id": "e04297f4", "metadata": {}, "source": [ "You see that the image group has been easily transfered too another dataset, with its fields and metadata linked to the Image group and Image field data models. \n", "\n", "\n", "
\n", "\n", "**Note** \n", " \n", "Other attributes that belonged to the original image group have not been transfered (see for instance the `description` attribute). Transfering these attributes between the two datasets would require to use the `get_dic_from_attribute` and `add_attributes` methods.\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "3e64cf99", "metadata": {}, "source": [ "The `get_node` method has been presented in the [last tutorial (section IV)](./2_SampleData_basic_data_items.ipynb), which is the generic *SampleData* method to retrieve data. What happens if we use it on an Image Group ?" ] }, { "cell_type": "code", "execution_count": null, "id": "3b23b3a3", "metadata": {}, "outputs": [], "source": [ "image = data.get_node('imageO')\n", "print(image)\n", "print(type(image))" ] }, { "cell_type": "markdown", "id": "2e9b91a9", "metadata": {}, "source": [ "What we get is simply a *Pytables* HDF5 Group object, that contains no data or metadata stored into the Image object that we tried to retrieve. Trying to get the image node with the dictionary or attribute like access would yield the same behavior." ] }, { "cell_type": "markdown", "id": "493050bf", "metadata": {}, "source": [ "### Getting field arrays" ] }, { "cell_type": "markdown", "id": "1edb2d6f", "metadata": {}, "source": [ "As for images, it is straightforward to retrieve image fields stored in a *SampleData* dataset. The associated method is `get_field`, and requires only the name of the field (or indexname etc...)." ] }, { "cell_type": "code", "execution_count": null, "id": "9139afc9", "metadata": {}, "outputs": [], "source": [ "data.print_node_info('tensorF')\n", "field_tmp = data.get_field('tensorF')\n", "print(type(field_tmp))\n", "print(field_tmp.shape,'\\n')\n", "\n", "print(f' Is the field the same as the one used to create the field data item ? {np.all(field_tmp == tensor_field)}')" ] }, { "cell_type": "markdown", "id": "8023e88e", "metadata": {}, "source": [ "You see that all the transformation applied to the data before in-memory storage (transposition of columns, of the field component indices) have been transformed back automatically: the method returned exactly the same *numpy* array passed as argument of `add_field` to create the field data item in the dataset.\n", "\n", "What would the `get_node` method return here ? That depends on the value of its `as_numpy` argument. What about the attribute or dictionary like access ? Here follow some examples:" ] }, { "cell_type": "code", "execution_count": null, "id": "118c97c5", "metadata": {}, "outputs": [], "source": [ "test_field1 = data.get_node('tensorF')\n", "test_field2 = data.get_node('tensorF', as_numpy=True)\n", "test_field3 = data['tensorF']\n", "\n", "print('test 1 : ', type(test_field1))\n", "print('test 2 : ', type(test_field2))\n", "print('test 3 : ', type(test_field3))" ] }, { "cell_type": "markdown", "id": "529e4e0c", "metadata": {}, "source": [ "We see that the `get_node`method returned a *Pytables* node object with no option specified, and a *numpy* array with the `as_numpy=True` option. \n", "\n", "As you can see below, the dictionary like access method returned the same results as the `get_node` method with `as_numpy=True` option: " ] }, { "cell_type": "code", "execution_count": null, "id": "8cb3ba60", "metadata": {}, "outputs": [], "source": [ "np.all(test_field2 == test_field3)" ] }, { "cell_type": "markdown", "id": "b82583cc", "metadata": {}, "source": [ "Are those outputs equivalent to the original field ?" ] }, { "cell_type": "code", "execution_count": null, "id": "9768653d", "metadata": {}, "outputs": [], "source": [ "np.all(test_field2 == tensor_field) " ] }, { "cell_type": "markdown", "id": "0a16ae13", "metadata": {}, "source": [ "**You see that the `get_field(...)`, `get_node(..., as_numpy=True)` and dictionary like access to fields data items methods are strictly equivalent**.\n", "\n", "Now, note that *Pytables* node objects behave in some ways as numpy arrays. Then, are there equivalent to the original data array ?" ] }, { "cell_type": "code", "execution_count": null, "id": "533d2e53", "metadata": {}, "outputs": [], "source": [ "np.all(test_field1 == tensor_field)" ] }, { "cell_type": "markdown", "id": "b794c826", "metadata": {}, "source": [ "No, they are not ! The reason behind that is that the **`get_node(..., as_numpy=False)` method returns a Node object that is a link to the data stored in memory in the HDF5 dataset**. As explained before, this data has been transformed in memory to comply with indexing conventions, that is why the arrays are no longer equivalent !\n", "\n", "You can remember that:\n", "\n", "\n", "* **To access field array values as they have been added to the dataset, you need to retrieve them with one of the following methods:**\n", "\n", " - `add_field` method\n", " - `get_node(..., as_numpy=True)` method\n", " - dictionary or attribute like access: `data['fieldname']` or `data.fieldname`\n", " \n", " \n", "* **To access field array values are they are stored in memory in the HDF5 file, you must use the `get_node(...,as_numpy=False)` method.**" ] }, { "cell_type": "markdown", "id": "668e5f41", "metadata": {}, "source": [ "*****\n", "This is the end of this tutorial on *SampleData Image Groups and Image Fields*. We can now close our dataset.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "42079ab0", "metadata": {}, "outputs": [], "source": [ "del data" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.17" }, "vscode": { "interpreter": { "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1" } } }, "nbformat": 4, "nbformat_minor": 5 }